Yesterday, the Niskanen Center submitted comments on the Food and Drug Administration’s (FDA) recent update to its ongoing development of the Software Precertification Pilot Program working model: Developing Software Precertification Program: A Working Model (v0.2). Although we commend the FDA on its commitment to continuously updating each new iteration of the Pre-Cert Software Pilot Program, much remains to be done if the United States is to reap the many promising benefits of Software as a Medical Device (SaMD). In particular, the FDA needs to focus greater attention on alleviating uncertainty with regards to the the use of artificial intelligence (AI) and machine learning (ML) in medical device technology.

As we note in our comments: “Investment in [AI] in the health care space has recently exploded, reaching an impressive $794 million in 2016, and is projected to continue growing exponentially. This excitement over the potential of AI in health care is not unfounded: AI can revolutionize the U.S. health care market by offering more accurate, more personalized, and cheaper diagnostic solutions, improving patient care outcomes and alleviating growing financial pressures on the system.” In order to continue promoting the investment and research necessary to actualize those benefits, however, AI/ML and SaMD innovators need greater regulatory clarity from the FDA.  

To help lift the fog of uncertainty, we offer 12 recommendations for how the FDA can develop regulatory pathways and approval standards specific for AI-enabled medical devices:

  1. Ensure grant of Pre-Cert status to a diverse portfolio of organization for the pilot program and beyond;
  2. Create a scoring sheet and minimum thresholds for identifying whether each of the excellence principles principles are met, based on demonstrated elements within each organizational domain.
  3. Include examples of key performance indicators (KPI) for each element used to appraise organizational excellence;
  4. Include an additional element under the Deployment and Maintenance domain of the excellence appraisal chart to evaluate an organization’s ability to collect and analyze post-launch, real-world performance data;
  5. Include a new organizational domain in the excellence appraisal chart called “Artificial Intelligence Development Best Practices” with appropriate elements and KPIs;
  6. Explicitly allow for outsourcing of core organizational activities in the excellence appraisal process;
  7. Develop an excellence appraisal process template and example including estimated timelines for each application stage;
  8. Abolish two levels of precertification and adopt a single precertification standard;
  9. Simplify the SaMD risk categorization framework by adopting a single formula yielding a single number that is categorized into three risk levels: low, moderate, and high;
  10. Require precertified organizations to proceed to Streamlined Review only for high-risk SaMDs.
  11. Clarify the Streamlined Premarket Review process requirements; and
  12. Include a domain that measures dynamic physician reliance on SaMD tools under User Experience Analytics for Real-World Performance Analytics assessments.

As Commissioner Scott Gottlieb has noted, this pilot program aims to develop “a new and pragmatic approach to digital health technology.” By leveraging advancements in AI/ML, innovators are well on their way to improving patient care outcomes, while minimizing the financial stress weighing on the domestic health care market. The innovators are doing their part; now it’s up to the FDA to help clear the path to a better and brighter future of personalized medicine for all Americans.

From the executive summary:

Ongoing developments in Artificial Intelligence (AI) hold the potential to revolutionize the U.S. health care market by offering more accurate, more personalized, and cheaper diagnostic solutions. The results of these improvements will be enhanced patient care outcomes, alleviation of growing financial pressure on the domestic health care system, and more personalized and efficacious treatment options for patients. Unfortunately, there is currently a gap in the regulatory approval process for AI-powered software-based medical devices. Although Food and Drug Administration (FDA) has taken an important first step in plugging this gap with the release of Developing Software Precertification Program: A Working Model (v0.2), there is much that can be done to improve on the agency’s proposal.

To that end, these comments offer feedback and specific recommendations based on the most recent version of the FDA’s proposed Software Precertification Pilot Program (Pre-Cert Program). The FDA’s current framework for addressing the needs of organization and innovators developing AI-based medical and diagnostic tools is a commendable step in the right direction. In order to fully realize the potential future gains of this technology’s application to the medical marketplace, however, there are a number of significant improvements the agency can incorporate into the next version of its Pre-Cert Program. We offer 12 recommendations for the use of AI in medical technologies that can help ensure a strong, flexible, and adaptive regulatory framework that will usher American health care innovation into the 21st century and beyond.

Read the full comments here.