As part of the Artificial Intelligence (AI) Program in the FDA’s Center for Devices and Radiological Health (CDRH), the goal of this regulatory science research is to develop methods and practical tools that detect changes to the inputs of AI-enabled medical devices, monitor the performance of their outputs, and understand the causes of performance variations. 

Overview

Artificial intelligence (AI) models are highly data dependent. Data acquisition systems, protocols, and patient populations change over time and across clinical sites. In addition, out-of-distribution data that a model has not encountered during model development can lead to unexpected outputs. As a result, the clinical utility of AI models may change between their development phases and actual clinical use. Such discrepancies can impact the safety and effectiveness of AI-enabled medical devices. 

Tools that detect changes to the inputs of AI-enabled medical devices, monitor the performance of their outputs, and understand the causes of performance variations aid in quality assurance through monitor and audit data and outputs of AI-enabled medical devices, as well as enable evaluations using patient data across multiple clinical sites. These tools will benefit both device users, such as clinicians and patients seeking to understand the accuracy and reliability of AI-enabled devices as practice conditions and patient populations evolve, and device sponsors aiming to maintain or enhance their devices' performance under clinical use conditions. Additionally, this effort will enable a dynamic cycle of innovation, allowing for continuous improvement. 

Projects

• Detection of Out-of-distribution Inputs for Artificial Intelligence/Machine Learning (AI/ML) Models 
• Proactive Monitoring of Data Drift and AI Model Performance
• Real-world Monitoring of AI models using Federated Evaluation

Resources

• Prathapan S, Samala RK, Hadjiyski N, D’Haese P-F, Maldonado F, Nguyen P, Yesha Y, Sahiner B, “Quantifying input data drift in medical machine learning models by detecting change-points in time-series data,” SPIE Medical Imaging Symposium 2024. 
• Zamzmi G, Venkatesh K, Nelson B, Prathapan S, Sahiner B, Yi P, and Delfino J, “Out-of-Distribution Detection and Radiological Data Monitoring Using Statistical Process Control,” Journal of Imaging Informatics in Medicine, 2024 (under review). 

For more information, email OSEL_AI@fda.hhs.gov.