RWE and AI: Hand in Hand in the Future of Regulatory Decision Making

As we previously discussed, FDA recently held two meetings that, while separate, provided a cohesive discussion of the use of Real World Evidence (RWE) and Artificial Intelligence (AI) in regulatory decision making.  The discussions during the Artificial Intelligence in Drug & Biological Product Development meeting (AI Meeting) are also relevant to device programs and it was noted that FDA leadership is pushing for a coordinated approach to AI across the centers.

The AI Meeting included panels on the current state of AI in development programs; data quality, reliability, representativeness, and access; model performance, explainability, transparency, and interpretability; and navigating the future.  AI was described as a means to allow for reinvention – not just a faster horse, but a car – and pairing of AI with RWE came up frequently across the panels.

In opening remarks, FDA noted that it takes a risk-based approach when considering AI, not just in looking at safety and efficacy, but in promoting innovation, and also highlighted that innovation doesn’t automatically mean increased risk.  CDER discussed its January 2025 draft guidance, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision Making for Drug and Biological Products, and noted that it is processing over 1400 comments from a variety of perspectives.  FDA also recognized the rapidly changing technology and noted internal and external training being provided to ensure that review staff are familiar with new technology when it is used.  FDA also recognized the need for infrastructure to enable better data sharing between sponsors and the Agency.

The panels noted that AI can be used in many ways across the total product lifecycle.  Examples of areas where AI is used now and can be expected in the future included:

• Indication selection
• Portfolio positioning
• Dose finding
• Protocol design
• Comparator arms for standard of care and disease evolution
• Inclusion/exclusion criteria
• Endpoint optimization
• Digital biomarkers
• Recruiting for studies
• Adaptive trial design
• Digital twins (in silico representations of a complex system, which can include an individual person)
• Agentic AI acting as a Clinical Research Associate (CRA) agent for tactical tasks, allowing humans to focus on strategic work
• Maximizing yield in manufacturing
• Personal use of an app utilizing AI to help a patient titrate dose when there are no data and no studies.

Discussion of data recognized four pillars:   data quality (as the saying goes, garbage in garbage out); data reliability (ensuring data are accurate, complete, consistent); data representativeness (to prevent bias and promote fairness); and data access (data can’t be used if siloed).  Foundation models were identified as a means to address many issues seen in narrow AI models that don’t generalize well across diverse populations.

There was also discussion of data sharing, with recognition of challenges due to the competitive nature of industry and intellectual property concerns.  The importance for both RWE and AI to share not only the successes, but also the failures, so that others can learn from them, was also a frequent point of discussion.

Big promise for AI was seen in collaboration and being able to bring together information that is in isolation across many systems today.  Another area of promise was noted to be in rare disease, where AI can have a big advantage since it is an area without a lot of training data, and AI is increasingly good and getting better where disease is not well characterized.  Speakers noted that in order for these promises to be achieved, the use of AI needs to be transparent, interpretable, and explainable.

In addition to the promise, speakers also cautioned against AI hype and noted many hurdles that need to be cleared in its use.  Throughout the sessions, the need for data standardization, especially for RWD, was discussed as a means of improving AI. Establishing guardrails to prevent misinforming the models was also noted as essential for building trustworthy AI.  Several speakers also mentioned establishing ground truth measures for comparison, especially with generative AI, as an area for thoughtful consideration.  Another hurdle is people themselves, with many being cautious to use AI or accept information generated by AI.  While some of the caution is warranted, the field is moving so quickly that in many cases issues that were bigger concerns in the past have been addressed with more recent technology.  For example, speakers noted that rates of hallucinations in AI models are much lower today than they were two years ago.

For FDA, speakers noted that technology is moving much too quickly for the standard process for release of guidance documents and encouraged more interactive collaboration between the Agency and sponsors to ensure up to date information on acceptance of AI models in applications is available.  It was also emphasized that FDA should consider the patient perspective and ensure policies do not penalize RWE.  Speakers encouraged FDA to not expect perfect data, but transparency and early engagement, to move forward.

As one who is cautious with use of AI, this blogger decided to test the waters.  While an initial draft of these posts, based on notes from the RWE Meeting and AI Meeting, was unclear, and none of the AI-generated text was used, the following concluding paragraph didn’t seem too bad:

As FDA continues to embrace the evolving landscape of RWE and AI, the message from both meetings is clear: collaboration, transparency, and adaptability are essential. These tools are not just technical innovations—they are catalysts for smarter, more inclusive, and more responsive regulatory decision making. By fostering early engagement, prioritizing data fitness, and encouraging shared learning across sectors, FDA is laying the groundwork for a future where RWE and AI work hand in hand to deliver better outcomes for patients and more efficient pathways for innovation.