Model QualityBeyond model accuracy and its pitfalls, we discuss emerging behavioral evaluation strategies inspired by traditional software testingFeb 11, 2024Feb 11, 2024
Security and Privacy in ML-Enabled SystemsA brief overview of common security and privacy concerns, new challenges introduced with machine-learning components, and common design s…Dec 20, 20221Dec 20, 20221
Integration and System TestingAfter testing models, pipelines, and other components, we finally that the integration and the system as a whole work…Nov 17, 2022Nov 17, 2022
Quality Assurance for Machine-Learning PipelinesBeyond model quality and data quality, the quality of the pipeline matters. We discuss testability and testing, code review, static…Nov 14, 2022Nov 14, 2022
Quality Assurance BasicsThere are many strategies to evaluate software with and without machine learning, not just testing. This chapter provides an overview.Nov 3, 2022Nov 3, 2022
Safety in ML-Enabled SystemsMachine learning tends to complicate safety considerations since machine-learned models always may make mistakes. Hazard analysis helps…Oct 27, 2022Oct 27, 2022
Planning for Machine-Learning MistakesMachine-learned models will always make mistakes, but with some planning we can anticipate and mitigate many by designing for failures…Sep 13, 2022Sep 13, 2022
Gathering Requirements for ML -Enabled SystemsMany developers building products with ML components will benefit from taking requirements engineering more seriously, checking…Sep 12, 2022Sep 12, 2022
Setting and Measuring Goals for Machine Learning ProjectsTo build products successfully, it is important to understand both the goals of the entire system and the goals of the model within…Sep 2, 2022Sep 2, 2022
When to use Machine LearningMachine learning is often pursued because of hype rather than to solve an actual goal. When is it really appropriate? This chapter…Sep 1, 2022Sep 1, 2022
Fairness in Machine Learning and ML-Enabled ProductsFairness needs to go beyond measurement at the model level and consider the entire system. This chapter surveys fairness notions, sources…Jul 11, 20221Jul 11, 20221
Building Machine Learning Products with Interdisciplinary TeamsConflicts between team members with different backgrounds and roles are common and found in many projects, including between SE and ML…Mar 16, 2022Mar 16, 2022
Data Science and Software Engineering Process ModelsThe processes used by data scientists and software engineers differ quite significantly, leading to conflicts in interdisciplinary teams…Mar 4, 2022Mar 4, 2022
Planning for Operations of ML-Enabled SystemsBuilding observability into the system’s infrastructure early on and virtualizing the software, the operations team can focus on value…Mar 1, 2022Mar 1, 2022
Scaling Data Storage and Data Processing and Machine Learning in Production SystemsThe key principles of how to design scalable systems are fairly well understood. When building software systems, developers will …Feb 24, 2022Feb 24, 2022
Automating the ML PipelineThis chapter discusses automating some or all steps of ML pipelines in production projects with machine learning to make it easier to…Feb 17, 2022Feb 17, 2022
Deploying a ModelWhile seemingly simple, there are lots of design decisions and tradeoffs when considering how to deploy a model, depending on …Feb 16, 2022Feb 16, 2022
Quality Attributes of ML ComponentsWe survey common qualities of interest and how to identify constraints and negotiate tradeoffs, both for the ML component and the system…Feb 15, 2022Feb 15, 2022
Thinking like a Software ArchitectInvesting in some planning before diving into implementation details helps to build systems that better meet requirements, especially …Feb 14, 2022Feb 14, 2022
Architectural Components in ML-Enabled SystemsThis chapter introduces and organizes the various considerations that go into the design of ML-enabled systems and all their components. It…Feb 11, 2022Feb 11, 2022
