The Flexibility Papers No. 4: Rare Diseases Can’t Wait for Perfect Data – The Case for Adequately Documented Natural History Evidence for Historical Controls
November 18, 2026In the first installment of The Flexibility Papers, we proposed a different starting point for rare disease development: rather than beginning with a default expectation of large randomized trials, start with the characteristics of the disease, the evidence that is realistically available, the unmet need, and the perspectives of patients, and work toward a fit-for-purpose pathway. Our second installment considered how patient perspectives, including patients’ tolerance for uncertainty, should meaningfully inform that process, and our third looked at how existing knowledge can be put to work through drug repurposing. This post takes up a question that runs through all three: when a randomized, concurrently controlled trial is not feasible, how should FDA evaluate natural history data offered as the control? [Authors’ note: this post is not intended to discuss baseline-controlled studies (also known as patient-as-own-control studies), which we intend to address in a future edition of the Flexibility Papers.]
The question is a practical one. In many rare diseases, natural history data are the most important evidence available, and FDA’s regulations expressly contemplate their use as a control. Yet in practice, natural history rarely carries the weight the regulations appear to allow.
FDA maintains webpages documenting the use of Real-World Data (RWD) and Real-World Evidence (RWE) in regulatory decision-making. For CDER, this contains 13 entries dating back to 2021. For CBER, this shows nine entries, dating back to 2018. Roughly ten of these 22 entries relied on an external or natural history control, nearly all in rare diseases. On the CDER side, only three of the 13 entries (Nulibry, Voxzogo, and Kygevvi) supported approval of a new molecular entity; the rest supported supplemental indications, new formulations, labeling changes, or served as supportive evidence. By FDA’s own count, only 35 drug and biologic product applications have included RWE since 2016, and FDA has acknowledged that “past agency expectations have meant that most RWE could not be considered in product applications.” For perspective, CDER alone approved 50 novel drugs in 2024 and 54 to date in 2026. These lists are not necessarily comprehensive, but they demonstrate how infrequently RWE is successfully relied upon, and they reflect the challenge sponsors face in seeking to leverage natural history.
What Does Natural History Evidence Look Like in a Rare Disease Setting?
For prevalent diseases, there is often less need to rely on natural history for regulatory decision-making. A randomized study is much more feasible, endpoints tend to be more well-known and accepted, and powering for statistical significance tends to be less of an issue. In rare diseases, these things are less likely to be true so relying on natural history data as an external control becomes much more important. However, a search of the medical literature for rare disease natural histories often turns up a fairly bare cupboard: perhaps some case reports or some small uncontrolled studies, each with varying degrees of follow-up, evaluating different endpoints, and with missing information. But perhaps there are one or two longitudinal studies or registries measuring outcomes of interest; they are not data from clinical trials, but they are still useful data. So how can we harness these data to serve as a historical control when a randomized trial in a rare disease setting, particularly one with serious outcomes, may be so challenging as to render it effectively impossible?
What Is a “Well-Defined” Natural History in a Rare Disease Setting?
FDA’s regulations expressly recognize a historical control as one of the types of control that can support an adequate and well-controlled study. Under 21 C.F.R. § 314.126(b)(2)(v), “[t]he results of treatment with the test drug are compared with experience historically derived from the adequately documented natural history of the disease or condition, or from the results of active treatment, in comparable patients or populations.” The regulation cautions that historical controls “usually cannot be as well assessed with respect to pertinent variables as can concurrent control populations” and are therefore “usually reserved for special circumstances,” such as diseases “with high and predictable mortality” or drugs whose effect is “self-evident.” But it asks that the natural history be adequately documented; it does not require that it be “well-defined” (a distinction we return to below).
Nonetheless, when exploring the use of a historical control, one question that inevitably arises is whether the natural history is “well-defined.” For example, FDA’s Draft Guidance, Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products (Feb. 2023) (the “Externally Controlled Trials Guidance”) states “[o]f note, if the natural history of a disease is well-defined and the disease is known not to improve in the absence of an intervention or with available therapies, historical information can potentially serve as the control group.” Likewise, FDA’s Draft Guidance, Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products (June 2026) (the “SEE Guidance”), describes a “well-defined natural history” as an “expectation[]” for a historically controlled trial. Neither guidance explains what makes a natural history “well-defined,” how much missing data or variability is tolerable, or how that expectation should scale to a disease that may affect tens of thousands, a few thousand, or only a few hundred patients worldwide.
- Challenges in having a “well-defined” natural history
In a rare disease setting, a “well-defined” natural history generally does not exist. When utilizing a historical control, there are almost always questions about unknowns, whether they are known unknowns (missing data like lab values that may not have been collected or collected inconsistently) or unknown unknowns (unknown covariates, meaning patient characteristics that affect outcomes but have not been identified or measured, like an unidentified genetic modifier that makes the disease progress faster in some patients than in others, or an unrecorded change in standard supportive care, such as respiratory or nutritional support, that improved outcomes over time). Selection bias (that is, differences that arise from how patients come to be in each group) is a concern since patients captured in a registry or natural history study may differ systematically from those who would enroll in a trial. For example, natural history patients are often identified retrospectively and may be sicker or diagnosed later, while trial participants must meet stricter eligibility criteria. Untreated historical controls have long been observed to fare worse than control groups in randomized trials, which can make a drug look more effective than it is. Systematic differences can also make trial participants appear to fare better: for example, when trial participants are diagnosed earlier (e.g., through genetic testing), they may appear to do better regardless of treatment. Establishing an appropriate and comparable baseline to a clinical trial is a substantial challenge.
When viewed through the lens of what can be controlled for in a randomized study, historically controlled evidence almost never meets that standard. The practical result is that a guidance-level expectation, rather than the regulatory standard, often becomes the barrier, and it falls hardest on the very patients for whom flexibility matters most.
- High bar for historically controlled studies
Paradoxically, the criteria for accepting a historically controlled study appear to be even higher than those for of a randomized study (see, e.g., FDA’s Externally Controlled Trials Guidance: “externally controlled trials are more likely to provide convincing results when the effect size on a well-characterized outcome of interest is anticipated to be large”). Put simply, a randomized trial generally succeeds if it shows a clinically meaningful effect that is statistically significant (conventionally, a two-sided significance level of 0.05), and even a modest effect can meet that test. For an externally controlled trial, FDA looks for considerably more, even if exceeding what is clinically meaningful; the SEE Guidance describes an estimated treatment effect “so large that it is unlikely to be fully attributable to bias,” and ICH E10 explains that the persuasiveness of an externally controlled trial depends on “much more extreme levels of statistical significance and much larger estimated differences between treatments than would be considered necessary in concurrently controlled trials.” This standard is not easily met in a rare disease setting where the effect size may not be large (for example, in a slowly progressive disease, and/or due to challenges related to sufficient powering) and there are often no well-characterized outcomes of interest. This means that although regulations specifically endorse the use of a historical control, when appropriate, it is, in practice, rarely accepted. We do not mean to say it is never accepted; for example, see the recent approvals for recurrent respiratory papillomatosis (RRP), the earlier of which we blogged about here.
- Randomization is not a panacea
While we do not mean to minimize the value of randomization, we also do not wish to overstate its importance. Randomization is not an assurance of comparability. For example, the Externally Controlled Trials Guidance, in describing why a “thorough understanding” of the natural history is needed, states that because “important prognostic factors for an outcome may not be known,” an externally controlled design may not be suitable. However, this same issue may impact a randomized trial; if there are unknown variables affecting disease progression, randomization without the awareness to stratify for such a variable could result in an imbalance of this factor between groups. This is particularly true in a rare disease where individual patients have greater influence on overall outcomes.
Randomization seeks to balance known and unknown factors on average, which is a key strength, but it does not guarantee balance in any single small trial. This is a matter of simple probability. Take a single prognostic factor present in half of patients (for example, a more severe genotype). In a trial with 10 patients per arm, there is roughly a 50% chance that the two arms will differ by 20 percentage points or more on that factor purely by chance; with 5 patients per arm, the chance is about 75%. It is only when you approach 100 patients per arm it falls below 1% — a sample size that exceeds the size of most rare disease trials. And because every baseline factor is subject to the same chance, the more factors that matter, the more likely it is that at least one will be imbalanced: if 10 independent factors each have a 5% chance of a statistically significant imbalance, the odds that at least one does are about 40%.
Stratification can address factors that are known, but by definition, it cannot address those that are unknown. Moreover, there are limitations in small studies to the feasible number of strata. Yet while unknown prognostic factors are routinely cited as a reason to doubt historical controls, the same limitation in small randomized trials receives far less attention. The very challenge that is often raised with historically controlled studies is actually shared with randomized controlled studies. Residual uncertainty about unmeasured factors is essentially unavoidable in rare disease regardless of design and should be weighed in context rather than treated as disqualifying only when the control is external.
A Proposal for FDA’s New Rare Disease Framework Initiative
- Fit-for-purpose demonstration of an “adequately documented” historical control in lieu of “well-defined”
As noted above, the regulations describe the requirement for the historical control to be “adequately documented” (21 C.F.R. § 314.126(b)(2)(v)) and the term “well-defined” is notably absent. In fact, it is worth noting that the only use of the term “well-defined” in the adequate and well-controlled study regulation refers to the “method of assessment of subjects’ response” as an element of an adequate and well-controlled study.
“Adequately documented,” although not explicitly defined, would appear to be a more fit-for-purpose standard than “well-defined.” This distinction is important because it focuses the question on whether the historical control can be used to produce interpretable evidence, even if there are residual uncertainties that may prevent it from being considered “well-defined.”
What would “adequately documented” mean in practice? Much of the answer already exists in FDA’s own guidance, which identifies the factors that bear on whether a historical control can produce interpretable evidence. Rather than asking whether a natural history is “well-defined” in the abstract, FDA could ask whether the sponsor has adequately documented the following, to a degree commensurate with the disease and the decision at hand, and should say so in guidance and reviewer training. In each case, the question should not be whether uncertainty remains, but how much uncertainty remains and whether that level of uncertainty is acceptable in context:
Source and comparability.
Where the data came from (for example, a prospective natural history study, a registry, chart review, or published literature), how patients were identified, and how the historical population compares with the treated population on the prognostic factors that are known, such as age at onset, genotype, disease severity, and prior or supportive treatments. The Externally Controlled Trials Guidance already lists many of these comparability considerations, including time period, geography, diagnostic criteria, and follow-up. Where an important prognostic factor was not captured, the question should be how much that gap could plausibly affect the comparison, and whether the effect exceeds the uncertainty acceptable in the disease setting, not simply whether the gap exists.
Outcome ascertainment.
Whether the endpoint was defined and measured in a sufficiently similar way, at comparable time points, in both groups, with particular weight given to objective endpoints (such as survival or need for permanent ventilation) that are less susceptible to differences in how they are recorded. Where endpoint definitions, assessment schedules, or measurement tools differ, the sponsor should show the likely direction and size of any resulting bias, and that residual difference should be weighed rather than treated as disqualifying.
Transparency about gaps and a prespecified analysis.
Candid documentation of missing data and how they are handled, including with conservative assumptions; selection of the historical control and the analysis plan before treated outcomes are compared; and sensitivity and quantitative bias analyses showing how large an unmeasured bias would have to be to explain away the observed effect. The goal is not to eliminate missing data or unmeasured confounding, which is rarely possible in a rare disease, but to show how much uncertainty remains and whether the observed effect reasonably survives it.
An effect size proportionate to the residual uncertainty.
Rather than a universal expectation of a “dramatic” effect, the effect needed should depend on how much uncertainty remains after the steps above, the severity of the disease and the unmet need, and the degree of uncertainty patients have told us they are willing to accept. The SEE Guidance itself acknowledges that, even when its expectations for an externally controlled trial are not fully met, “it may sometimes be possible to conclude that substantial evidence of effectiveness has been demonstrated despite residual uncertainty.” An “adequately documented” standard would give that acknowledgment a structure, so that a well-documented control could support a meaningful, if not dramatic, effect.
As we return to our initial question, how can we make the best use of natural history data in a rare disease setting? We propose an approach that hews close to the regulatory language, and thus reflects the specific considerations at issue, rather than a universal “well-defined” expectation. Is the natural history adequately documented to serve as an acceptable control in the context of the indication to provide interpretable evidence? This requires accepting some uncertainty in a fit-for-purpose approach that also considers the patient voice and acceptable benefit-risk. It is not a blanket acceptance of comparability without evidence but maintains the requisite rigor to draw conclusions. There will always be weaknesses to identify in historical comparisons, but we cannot let perfect be the enemy of good when patients are in need.
- Permit use of summary-level data in lieu of patient-level data
One of the key challenges in seeking to utilize real-world data and real-world evidence is that patient-level data are not always available. In the 2023 Externally Controlled Trials Guidance, FDA stated that “Sponsors must include in their marketing applications relevant patient-level data (i.e., data on each participant and patient in the externally controlled trial), as required under FDA regulations, for both the treatment and external control arms.” However, as explicitly noted, “[t]his guidance does not address other types of external controls, such as using summary-level estimates instead of patient-level data.” In our blog about this guidance, we noted that this guidance left unanswered how summary-level estimates could be used as an external control if patient-level data were required.
The cited regulation in this guidance refers to tabulations of “data on each patient in each study” unless determined not to be pertinent to FDA’s review (21 C.F.R. § 314.50(f); 21 C.F.R. § 601.2). Notably, that regulation contemplates that FDA and the applicant may agree “in advance” that certain information is “not pertinent to a review of the drug’s safety or effectiveness,” a topic that can be discussed with the Agency during development, and these discussions would benefit from FDA articulating when historical control data might fall into that category.
For example, patient-level data for a historical control may not be pertinent where:
- the expected course of the disease is so consistent and well documented (e.g., near-uniform mortality, or loss of a motor milestone by a predictable age) that the treatment effect is evident against summary estimates, and no plausible re-analysis of individual patient data would change the conclusion (as with Cresemba, discussed below);
- the outcome is objective and routinely recorded the same way in clinical practice (e.g., death, need for permanent ventilation, organ transplantation, or laboratory biomarker), so that summary-level results from a registry or database are reliable on their own; or
- natural history findings are consistent across multiple independent sources (e.g., several published cohorts or registries), so that the comparison does not hinge on any single dataset. In each case, the sponsor would still submit patient-level data for its own treated patients and would explain why summary-level historical data are sufficient, for example through sensitivity analyses showing that the conclusion holds even under conservative assumptions about the historical control.
On December 15, 2025, FDA announced that it would no longer always require patient-level data to be submitted in certain types of medical device submissions. Although the headline of the announcement stated that “FDA Eliminates Major Barrier to Using Real-World Evidence in Drug and Device Application Reviews,” the fine print made clear this did not yet apply to drugs, with FDA stating that it “similarly intends to consider updating its guidance for drugs and biologics.”
Without a change in the regulatory language or FDA’s interpretation of such, this patient-level data requirement is likely to remain a significant barrier to the acceptance of real-world evidence. We acknowledge that this barrier has not always been insurmountable. For example, in 2015, FDA approved Cresemba (isavuconazonium sulfate) for the treatment of both invasive aspergillosis and invasive mucormycosis. The rare invasive mucormycosis indication was based on benefits in all-cause mortality shown in a single-arm trial as compared to natural history. The publication identified as the closest to approximating a natural history study in a modern population did not have patient-level information (Cresemba Medical Review: “We contacted Dr. Kontoyiannis [Chamilos et al] for subject level information, but the data were not readily available”). Yet this study was used for comparisons to support the efficacy of the drug, in part because it represented the “best available data” and the difference in mortality rate was so clear.
Serendipitously for this already-planned blog post, FDA just announced in the Federal Register (published on Nov. 12, 2026) a “request for information (RFI) from interested parties to share perspectives on patient-level data access related to the use of real-world evidence (RWE) in regulatory decision-making for drugs and biological product submissions.” The RFI is intended to solicit input regarding how challenges related to submitting such data may be addressed. Notably, the RFI largely frames the issue as a logistical one (how to get patient-level data to FDA, given data ownership, privacy, and data-use agreement constraints) rather than whether summary-level data can sometimes suffice. We encourage sponsors to submit feedback to this RFI regarding their experiences and insights, including whether an alternative approach should be considered as part of the new Rare Disease Framework initiative. In particular, we encourage commenters to ask FDA to articulate when patient-level data are not “pertinent” to its review and when summary-level data may suffice. Comments are due January 11, 2027, under Docket No. FDA-2026-N-10742.
Conclusion
Historical control data will rarely be perfect in a rare disease, and FDA’s regulations do not ask that they be; they ask that a historical control be adequately documented. Returning to that standard, explaining what it requires, scaling the expected effect size to the residual uncertainty, and accepting summary-level data where patient-level data are unavailable and not pertinent would let FDA use the best evidence that realistically exists for these patients without abandoning rigor. Written comments on the Rare Disease Framework initiative are due December 11, 2026 (Docket No. FDA-2026-N-12214), and we encourage stakeholders to raise these issues there as well.