(Preliminary schedule, subject to change) Date Topic Reading Assignment due Mon, Feb 01 Motivation, Syllabus, and Introductions Wed, Feb 03 From Models to AI-Enabled Systems (Systems Thinking) Building Intelligent Systems, Ch. 5, 7, 8 Fri, Feb 05 Stream processing: Apache Kafka Mon, Feb 08 Model Quality (blog post, lecture notes) Building Intelligent Systems, Ch. 19 Wed, Feb 10 Model Quality (continued) and Teamwork Behavioral Testing of NLP Models with CheckList I1: Case Study Fri, Feb 12 Remote work and collaboration: Slack, Git, issue trackers Mon, Feb 15 Goals and Success Measures for AI-Enabled Systems Building Intelligent Systems, Ch. 2, 4 Wed, Feb 17 Quality Assessment in Production (lecture notes) Building Intelligent Systems, Ch. 14, 15 Fri, Feb 19 Measurement Mon, Feb 22 Risk and Planning for Mistakes 1 (blog post) The World and the Machine Wed, Feb 24 Risk and Planning for Mistakes 2 Building Intelligent Systems, Ch. 6, 7, 24 M1: Modeling and First Deployment Fri, Feb 26 Requirements/Risk analysis Mon, Mar 01 Tradeoffs among Modeling Techniques Building Intelligent Systems, Ch. 17 and 18 Wed, Mar 03 Software Architecture of AI-Enabled Systems Building Intelligent Systems, Ch. 13 and Exploring Development Patterns in Data Science Fri, Mar 05 Architecture Mon, Mar 08 Data Quality (lecture notes) Automating large-scale data quality verification and The Data Linter Wed, Mar 10 Infrastructure Quality, Deployment, and Operations The ML Test Score I2: Requirements and Architecture Fri, Mar 12 Midterm Review Session Mon, Mar 15 Managing and Processing Large Datasets Business Systems with Machine Learning Wed, Mar 17 Midterm Fri, Mar 19 Midsemester break, no recitation Mon, Mar 22 Process & Technical Debt (blog post) Hidden Technical Debt in Machine Learning Systems Wed, Mar 24 Human AI Interaction Building Intelligent Systems, Ch. 8 and Guidelines for Human-AI Interaction I3: Open Source Tools (submissions: Algorithmia, Amazon Elastic MapReduce, Apache Flink, Azure ML, Dask, Databricks, DataRobot, Google Cloud AutoML, IBM Watson Studio, LaunchDarkly, Metaflow, Pycaret, Split.io, TensorBoard, Weights and Biases) Fri, Mar 26 Continuous Integration Mon, Mar 29 Intro to Ethics & Fairness Algorithmic Accountability: A Primer Wed, Mar 31 Building Fairer AI-Enabled System 1 Improving Fairness in Machine Learning Systems Fri, Apr 02 Containers: Docker Mon, Apr 05 CMU Break day, no class Wed, Apr 07 Building Fairer AI-Enabled System 2 A Mulching Proposal M2: Infrastructure Quality Fri, Apr 09 Monitoring: Prometheus, Grafana Mon, Apr 12 Explainability & Interpretability Black boxes not required or Stop Explaining Black Box ML Models… Wed, Apr 14 Explainability & Interpretability (continued) People + AI, Ch. “Explainability and Trust” Fri, Apr 16 No recitation (spring carnival) Mon, Apr 19 Versioning, Provenance, and Reproducability (lecture notes) Building Intelligent Systems, Ch. 21 & Goods: Organizing Google's Datasets Wed, Apr 21 Security and Privacy Building Intelligent Systems, Ch. 25 & The Top 10 Risks of Machine Learning Security M3: Monitoring and CD Fri, Apr 23 Threat modeling Mon, Apr 26 Safety Practical Solutions for Machine Learning Safety in Autonomous Vehicles Wed, Apr 28 Safety (continued) and a conversation with Xenophon Papademetris on ML and Safety in Medical Systems The need for a system view to regulate artificial intelligence/machine learning-based software as medical device I4: Fairness Fri, Apr 30 No recitation Mon, May 03 Fostering Interdisciplinary Teams Data scientists in software teams Wed, May 05 Summary and Review and Closing discussion M4: Security and Feedback Loops Thu, May 13, 5:30-8:30 PM Final Project Presentations Final report
