Workshop
Event Title
Virtual Public Workshop - Transparency of Artificial Intelligence/Machine Learning-enabled Medical Devices
October 14, 2021
- Date:
- October 14, 2021
The Food and Drug Administration (FDA) announced a virtual public workshop on transparency of Artificial Intelligence/Machine Learning (AI/ML)-enabled medical devices to patients, caregivers, and providers. The role of transparency that enhances safe and effective use of AI/ML-enabled medical devices was discussed, with an emphasis on information sharing methods such as labeling.
The purpose of the workshop was to 1) identify unique considerations in achieving transparency for users of AI/ML-enabled medical devices and ways in which transparency might enhance the safety and effectiveness of these devices; and 2) gather input from various stakeholders on the types of information that would be helpful for a manufacturer to include in the labeling of and public facing information of AI/ML-enabled medical devices, as well as other potential mechanisms for information sharing.
Background
This workshop built on the Agency's longstanding commitment to support innovative work in the regulation of digital health technologies, and was a direct response to stakeholder feedback, including feedback from the Agency's Patient Engagement Advisory Committee on AI/ML-enabled medical devices held in October 2020. In January 2021, FDA presented a five-part Action Plan focused on AI/ML-enabled medical devices, which included a proposal to hold a public workshop on how information sharing about a device supports transparency to all users of AI/ML-enabled medical devices. This workshop fulfilled that proposal, an important aspect of the Action Plan.
DATE and TIME
This workshop was held on October 14, 2021, from 10:00 a.m. to 3:30 p.m. (ET) by webcast only.
WEBCAST
TRANSCRIPT
AGENDA
Welcome and Opening Remarks
10:00 AM - 10:30 AM
- Nooshin Kiarashi, PhD - Lead Reviewer, Digital Health Engineer - FDA/CDRH (Presentation)
- Jeff Shuren, MD, JD - Center Director - FDA/CDRH
- Bakul Patel, MSc, MBA - Director - FDA/CDRH/Digital Health Center of Excellence (Presentation)
- Matthew Diamond, MD, PhD - Chief Medical Officer - FDA/CDRH/Digital Health Center of Excellence (Presentation)
Session I. The Meaning and Role of Transparency
10:30 AM - 12:00 PM
Moderator: Anindita Saha - Assistant Director - FDA/CDRH/Digital Health Center of Excellence
- Patient Impressions of AI/ML-enabled Medical Devices
Jessica Weinberg, MPP - Social Science Analyst - FDA/CDRH/OST/DARSS/Patient Science and Engagement
Aubrey Shick, MS - Digital Health Specialist - FDA/CDRH/Digital Health Center of Excellence - Accountability in the use of AI/ML technologies
Monica W. Parker, MD - Director, Minority Engagement Core Goizueta Alzheimer’s Disease Research Center - Emory University - Physician Perspectives on Transparency in Augmented Intelligence
Jack Resneck, MD - President-Elect - American Medical Association, Professor and Vice-Chair of Dermatology - UCSF School of Medicine - The Meaning and Role of Transparency - A Manufacturer’s Viewpoint
Pat Baird, BS, MS, MBA - Sr. Regulatory Specialist - Philips - Nutrition Facts Labels for AI/ML Transparency and Trust?
Barbara Barry, PhD - Assistant Professor / Collaborative Scientist - Mayo Clinic, Division of Health Care Delivery Research
Panelists:
- Aubrey Shick, MS - Digital Health Specialist - FDA/CDRH/Digital Health Center of Excellence
- Monica W. Parker, MD - Director, Minority Engagement Core Goizueta Alzheimer’s Disease Research Center - Emory University
- Jack Resneck, MD - President-Elect - American Medical Association, Professor and Vice-Chair of Dermatology - UCSF School of Medicine
- Pat Baird, BS, MS, MBA - Sr. Regulatory Specialist - Philips
- Barbara Barry, PhD - Assistant Professor / Collaborative Scientist - Mayo Clinic, Division of Health Care Delivery Research
- Kenneth Goodman, PhD - Professor and Director - Institute for Bioethics and Health Policy, University of Miami Miller School of Medicine
- Naomi Aronson, PhD - Executive Director, Clinical Evaluation and Innovation - Blue Cross Blue Shield Association
- Berkman Sahiner, PhD - Electrical Engineer - FDA/CDRH/OSEL/DIDSR
Lunch
12:00 PM - 12:30 PM
Open Public Comment Session
12:30 PM - 1:20 PM
Moderator: Aneesh Deoras - Assistant Director - FDA/CDRH/Office of Cardiovascular Devices, Division of Cardiac Electrophysiology, Diagnostics, and Monitoring Devices
- Building Transparency & Trust in AI/ML-enabled Medical Devices: The Patient & Carepartner Perspective
Grace Cordovano, PhD, BCPA - Board Certified Patient Advocate - Enlightening Results - Reducing Bias in AI/ML Research: Standardization, Harmonization, and Interoperability of Multi-Omic Data and EHR for Marginalized or Underrepresented Community Subsets
The Reverend James Leslie Burroughs, III, BS, Mdiv (candidate) - Chief Executive Officer - North Carolina Health Equity Project, Inc. - AI/ML Transparency: More Than Process-Based Pre-Certification
Charlotte Tschider, MA, JD, CISSP, CIPP/US, CIPP/E-Assistant Professor - Loyola University Chicago School of Law, Beazley Institute for Health Law & Policy - Importance of a Patient-Centered and Flexible Approach for Transparency in AI
Yarmela Pavlovic, BA, JD - VP Regulatory Strategy - Medtronic - An Industry Perspective on Transparency
Zack Hornberger - Director of Cybersecurity & Informatics - Medical Imaging & Technology Alliance - Engineered Explainability in CAD Devices
Robert Lindsey, PhD - Chief Science Officer, Co-Founder - Imagen Technologies - Building Responsible AI with ML Monitoring and Explainability
Amit Paka, MS, MBA, BE - Chief Product Officer - Fiddler AI - AIMIC Key AI Concepts
Ralph Hall, JD - Principal, Leavitt Partners - Professor of Practice, University of Minnesota Law School - Digital Health Community Consensus Recommendations on Advancing Transparency & Explainability of Artificial Intelligence in Healthcare
Brian Scarpelli, JD- Senior Global Policy Counsel - Connected Health Initiative - Open Digital Biomarker Development
Jessilyn Dunn, PhD - Assistant Professor, Biomedical Engineering and Biostatistics & Bioinformatics - Duke University - Comparative Approaches to AI Transparency: Lessons from Recent Regulatory Efforts
Anna R. Gressel, JD - Associate - Debevoise & Plimpton LLP - Transparency of Artificial Intelligence and Machine Learning Devices in Advancing Healthcare Quality and Value-Based Care
Garry Choy, MD, MBA - SVP, Chief Clinical Technology and Innovation Officer, Medical Affairs - UnitedHealth Group - Medical Device Context and State Information can support AI/ML Transparency
Julian M. Goldman, MD
Anesthesiologist - Massachusetts General Hospital,
Medical Director - Mass General Brigham Biomedical Engineering,
Director - MGH Smart and Autonomous Medical Systems (SaAMS) initiative of the MD PnP Program
Break
1:20PM - 1:30PM
Session II. Promoting Transparency
1:30 PM - 3:15 PM
Moderator: Nicholas Petrick - Deputy Director - FDA/CDRH/OSEL/DIDSR
- FDA’s Role in Promoting Transparency Through Labeling and Public Facing Documents
Robert Ochs, PhD - Deputy Office Director - FDA/CDRH/OHT7 - Lived Experience with Diabetes and Technology
Cherise Shockley, BA - Patient Representative - AI Medical Device Transparency
Keith Dreyer, DO, PhD, FACR, FSIIM - Chief Data Science Officer and Chief Imaging Information Officer - Mass General Brigham, Associate Professor of Radiology - Harvard Medical School, Chief Science Officer - American College of Radiology DSI - Promoting Transparency - A Developer's Perspective
Nathan Carrington, PhD - Head of Digital Health and Innovation, Global Regulatory Policy and Intelligence - Roche Diagnostics - Participatory design of transparent and understandable AI/ML-enabled medical devices
Melanie Wright, PhD - Associate Professor - Idaho State University - Designing for Transparency: Consumer-oriented AI
Samantha Winter, PhD - User Experience Researcher - Google
Panelists:
- Robert Ochs, PhD - Deputy Office Director - FDA/CDRH/OHT7
- Cherise Shockley, BA - Patient Representative
- Keith J. Dreyer, DO, PhD, FACR, FSIIM - Chief Data Science Officer and Chief Imaging Information Officer - Mass General Brigham, Associate Professor of Radiology - Harvard Medical School, Chief Science Officer - American College of Radiology DSI
- Nathan Carrington, PhD - Head of Digital Health and Innovation, Global Regulatory Policy and Intelligence - Roche Diagnostics
- Melanie C. Wright, PhD - Associate Professor - Idaho State University
- Lily Peng, MD, PhD - Product Manager - Google
- Savvas Pavlides, PhD - Senior Manager, Scientific Quality, Clinical Evidence Assessment - ECRI
- Cynthia Chauhan, MSW - Patient Representative
Closing Remarks and Adjourn
3:15 PM - 3:30 PM
- Aubrey Shick, MS - Digital Health Specialist - FDA/CDRH/Digital Health Center of Excellence (Presentation)
REGISTRATION
Registration is closed.
COMMENTS
Please submit your comments regarding the workshop to https://www.regulations.gov/, Docket No. FDA-2019-N-1185 by November 15, 2021.
Please refer to the Instructions for submitting comments to the docket to ensure that your feedback is received.
The resulting discussions from the workshop and comments received in the docket will be taken into consideration.
Please be advised that as soon as a transcript is available, it will be posted in the Dockets and accessible at http://www.regulations.gov.
CONTACT
For questions regarding workshop content please contact:
Nooshin Kiarashi, PhD
Center for Devices and Radiological Health
U.S. Food and Drug Administration
10903 New Hampshire Ave.
Silver Spring, MD 20993
Email: AI-MLTransparencyWorkshop2021@fda.hhs.gov
RESOURCES
- Digital Health Center of Excellence
- FDA AI/ML Action Plan
- Discussion Paper: Proposed Regulatory Framework for Modification to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)
- FDA Patient Engagement Advisory Committee (PEAC)
- 2020 PEAC Meeting Materials
- Device Labeling