Monitoring, Oversight, and Learning in Medical AI

Abstract

When medical AI errs, it often goes unnoticed.  If there’s a specific patient injury, and the link to AI is obvious, that problem might be reported to the Food and Drug Administration (FDA), but not always.  And many other types of problems, like worse performance on specific groups or ineffective integration into health system workflows, simply don’t fall within the contours of regularized reporting.  Even if they are noticed by the health system—far from a given—there’s no obvious way to share that information more broadly.  Against this backdrop, there are justified calls for better oversight and reporting.  But there’s the opportunity to do more.  If now is the time to build more robust surveillance systems and standards for sharing that information, it should also be the time to build systems to share information about positive performance and learning, so that AI can help enable the vision of a learning health system that not only fixes mistakes but also constantly improves.