Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values

crown jewel figure
GAM Changer empowers domain experts and data scientists to easily and responsibly align model behaviors with their domain knowledge and values, via direct manipulation of GAM model weights. For example, (A) the GAM Canvas enables doctors to interpolate the predicted risk of dying from pneumonia to match their domain knowledge of a gradual risk increase from age 81 to age 87. GAM Changer promotes accountable editing and elucidates potential tradeoffs induced by the edits. (B1) The Metric Panel provides real time feedback on model performance. (B2) The Feature Panel helps users identify characteristics of affected samples and promotes awareness of fairness issues. To enable reversible transparent model edits, (B3) the History Panel allows the doctor to compare and revert changes, as well as document their motivations and editing contexts.
Demo Video
Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions-potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collaboration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action-empowering users to analyze, validate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneumonia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users' web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the following public demo link:
Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
  title = {Interpretability, {{Then What}}? {{Editing Machine Learning Models}} to {{Reflect Human Knowledge}} and {{Values}}},
  booktitle = {Proceedings of the 28th {{ACM SIGKDD Conference}} on {{Knowledge Discovery}} and {{Data Mining}}},
  author = {Wang, Zijie J. and Kale, Alex and Nori, Harsha and Stella, Peter and Nunnally, Mark E. and Chau, Duen Horng and Vorvoreanu, Mihaela and Wortman Vaughan, Jennifer and Caruana, Rich},
  year = {2022},
  series = {{{KDD}} '22}