CERSI Collaborators: Zeyan Liew, PhD, MPH (Yale) (Co-PI), Timothy Lash (Emory), Ashley Naimi (Emory), Joseph Ross, MD, MHS (Yale), Xiaoting Shi, MPhil (Yale), Joshua Wallach, PhD, MS (Emory) (Co-PI)

FDA Collaborators: Sai Dharmaraj, PhD (Formerly of CDER), Wei Hua MD, PhD, Tae Hyun Jung, PhD, Joo-Yeon Lee, PhD, Jie Li, PhD, Mingfeng Zhang, MD, PhD

Project Start: October 22, 2021

Regulatory Science Challenge

For drug approval, FDA typically requires two adequate and well-controlled clinical trials that establish evidence of efficacy and safety. Over the past decade, FDA has developed guidance documents for utilizing observational studies and real-world data (RWD) to inform regulatory decision-making. However, unlike randomized controlled trials, observational studies may have methodological limitations that can generate biases. Quantitative bias analysis (QBA) methods have been developed to evaluate the potential impact of the biases arising from systematic errors in observational studies. Yet, a number of challenges have undermined the widespread adoption of QBA in observational studies, including analytical complexities, difficulties establishing bias parameters, and concerns about the potential misuse/misinterpretation of methods and results. For observational studies to inform regulatory decision-making, it is necessary to have a comprehensive understanding of QBA methods that can be used to evaluate the robustness of observed associations.

Project Description and Goals

The project’s goals are to: (1) systematically identify, summarize, and compare QBA approaches, with a focus on describing the applicable study designs, types of biases addressed, mathematical formulas and bias parameters, data formats, and model assumptions; (2) develop a user-friendly decision tree which will allow researchers to identify QBA methods based on different study characteristics; and (3) test the feasibility of using the QBA decision tree to select appropriate QBA methods using published observational studies. (4) In-depth statistical review of the QBA methods identified will be used to modify a summary table and refine a decision tree. Overall, these evaluations will provide a robust understanding of the types of analytical methods of QBA that can be utilized when conducting observational studies.