Updates to FDA’s Software Pre-Certification Program

On January 7, 2019, FDA released new documents related to its Software Pre-Certification (Pre-Cert) Program:

• Developing a Software Precertification Program: A Working Model (Working Model)
• Software Precertification Program: Regulatory Framework for Conducting the Pilot Program within Current Authorities (Regulatory Framework)
• Software Precertification Program: 2019 Test Plan (Test Plan)

In this post, we will cover the Working Model, which is in its third revision, and the Test Plan, which is a new document.  We will discuss the Regulatory Framework on a follow-up post.

The Pre-Cert Program is intended to create a new streamlined regulatory process for software as a medical device (SaMD) (see our earlier blog posts on the program here, here, and here).  The program focuses primarily on the software developer and its processes.  For lower risk software, there may be no premarket product review while for moderate and higher risk software, the product-specific information would be submitted in a streamlined premarket application.  FDA has stated that the goal of the program is to “have tailored, pragmatic, and least burdensome regulatory oversight that assesses organizations (large and small) to establish trust that they have a culture of quality and organizational excellence such that they can develop high quality SaMD products, leverages transparency of organizational excellence and product performance across the entire lifecycle of SaMD, uses a tailored streamlined premarket review, and leverages unique postmarket opportunities available in software to verify the continued safety, effectiveness and performance of SaMD in the real world.”  Working Model at 7.

The details of the Pre-Cert Program are outlined in the Working Model.  The first notable change to the Working Model is the introduction of the Total Product Lifecycle.  While previous versions described each of the steps, this version clarifies that the program has four key components following a Total Product Lifecycle (TPLC) approach and describes the interactions during each component.  The four components of the program are:

• Demonstrate a culture of quality and organizational excellence through an Excellence Appraisal (pre-certification).
• Determine the SaMD’s required review through Review Determination.
• Conduct a Streamlined Review, and
• Verify a SaMD’s continued safety, effectiveness and performance and the organization’s commitment to culture of quality through post-market Real-World Performance.

The Excellence Appraisal would evaluate software manufacturers based on five culture of quality and organizational excellence principles (Excellence Principles), including: product quality; patient safety; clinical responsibility; cybersecurity responsibility; and proactive culture.  Organizations would be certified into one of two levels depending on their experience in developing products.  While FDA envisions the use of accredited third-parties for Excellence Appraisal in the future program, they have clarified in this version of the working model that the Excellence Appraisals will be performed by FDA during the 2019 testing of the program.

To us, the Excellence Principles relate closely to the quality system regulation (21 C.F.R. Part 820), yet there is no connection made in the Working Model.  Once precertified, the Working Model describes several factors that may trigger the need for an additional Excellence Appraisal, including patient or product issues, significant restructuring, mergers or acquisitions, continuous improvements that lead to a change in performance capability, new activities or incorporation of a new clinical domain, or continued recurrence of safety signals.  However, it does not describe how inspectional findings in a quality system inspection might affect the precertification status.  And to avoid inspectional findings, precertified software manufacturers should also be careful to ensure that their processes continue to meet quality system requirements, especially in cases where new formats for providing information may be acceptable in an Excellence Appraisal or Streamlined Review.

For precertified software manufacturers, bringing a SaMD to market would begin with a Review Pathway Determination.  While the Review Pathway Determination was described previously, it has been clarified that it requires submission to FDA of information about the SaMD and its risk.  In 2019, the information proposed for this component of the program would be provided during an optional Pre-Submission meeting or as part of the premarket submission.  In the future, FDA anticipates that the SaMD product-level elements would be submitted when the precertified organization is ready to 1) market their SaMD if review is not required or 2) submit their SaMD for Streamlined Review if review is required based on the manufacturer-determined risk category.  Depending on the organization’s precertification level and the SaMD’s Risk Categorization according to the International Medical Device Regulators Form (IMDRF), FDA will confirm whether the product requires a streamlined premarket submission or no premarket submission.  To stay within existing regulatory authority, the Regulatory Framework limits the Pre-Cert Program pilot to Pre-Cert De Novo applications and Pre-Cert 510(k)s following classification of the device in a Pre-Cert De Novo.   FDA also intends, in the future, to post review pathway determinations (when no additional review is required) or following clearance or approval (when a marketing submission is required), which should further assist companies in properly categorizing their SaMD.

For those products that require a marketing submission (which includes all pilot products), the program’s streamlined review promises a shortened review timeline, but does not give estimates on how much time might be saved.  The Working Model further states that FDA anticipates that the amount and complexity of clinical data will ultimately be the driver for the duration of the review.  In our review of the product content that would be submitted, at least for 510(k) devices, there does not appear to be much that differs from a traditional submission.  There is some elimination of possible redundancy (e.g., the device description serves as the software description), and the software development environment description and lower level software testing reports need not be submitted.  In our experience, these parts are not a large portion of the submission under review and are less likely to result in deficiencies, while review of clinical and non-clinical performance data, which is still submitted in a streamlined review, and response to FDA deficiencies related to performance data and substantial equivalence are more likely to slow down the overall time to market.

The last phase of the Pre-Cert Program’s TPLC approach includes collection and analysis of real-world performance analytics (RWPA), encompassing at least three types of analysis: Real-world Health Analytics (RWHA), User Experience Analytics (UXA) and Product Performance Analytics (PPA).  The RWPA plan would be developed in advance of introducing the SaMD to the market.  FDA envisions engaging in an iterative process with SaMD manufacturers to refine the types of data elements most relevant to the SaMD.  FDA would collect RWPA data to FDA periodically (e.g., quarterly), though the mechanism for this data collection is not described.  FDA would be analyzing the data in addition to the SaMD manufacturer and the data would be used to drive decisions such as the need to make software modifications as well as FDA’s ability to identify potential emerging issues across product classes.  It is not clear how FDA will work with SaMD manufacturers, especially if there are differences in opinion in terms of actions that should be taken based on the data.  Questions on timeliness of FDA’s review are also raised as they will be reviewing data after a SaMD manufacturer has already conducted their review and made plans for any necessary actions.

For the pilot program, FDA intends to test the process, which we applaud.  The Test Plan will include retrospective tests of SaMD submissions that have been previously reviewed and prospective tests.  For prospective tests, Pre-Cert companies volunteering to participate in the testing would not only undergo the Pre-Cert process but would also submit a full submission so that FDA can compare results of the current review process to the streamlined review process.  The Working Model states that FDA will not establish precertified companies during the testing in 2019 which suggests that only those companies that are already part of the program will also be included in the submission testing.  Reviewing the proposed methods, we find ourselves asking questions similar to those FDA typically asks device sponsors regarding their test plans.  For example, FDA states they will “consider the Pre-Cert model to be confirmed when the program framework remains static over multiple premarket submissions” but does not provide justification of this endpoint, provide a specific number of submissions over which the framework remains static that would be needed to validate the program, or state how long FDA would run the test.  Further, the Test Plan does not provide a submission sample size with justification or discuss how many divisions and reviewers will participate to account for typical review variability.  If limited to only De Novos or follow-on 510(k) submissions from the current companies participating in the precertification pilot, there may not be enough submissions to truly validate the program.

Overall, we aren’t convinced that the time and cost for market entry will be reduced given the now multi-step process and questions that these new steps raise.  For example, how much time and effort will be required for an organization to become precertified?  In addition to the precertification step, there are two additional steps needed to bring SaMD to market, including a review to confirm the appropriate market application (or that no marketing application is needed) and postmarket commitments including periodic (e.g., quarterly) review of real-world performance data by FDA.  Will FDA have adequate resources to perform these steps without delaying the overall SaMD development timeline?  It is not clear whether the promised benefit of a shorter submission review will outweigh the additional efforts.  This will likely be dependent on the organization and the number of SaMD products it develops and supports in the market.