WPF advises Secretary’s Advisory Committee on Human Research Protection regarding its proposed AI Framework

WPF reviewed and provided recommendations regarding a proposed AI Framework to the Secretary’s Advisory Committee on Human Research Protections, SACHRP, which reports to the US Secretary of Heath and Human Services.

The issue of human subject research is a critically important one. In the US, The Common Rule (45 CFR subpart A) is a key regulation that protects people from unethical medical research. As research utilizing tools such as AI and SaMD — software as a medical device — grows in use, there is an urgent need to determine the proper ethical, legal, and regulatory framework for the use of these tools in the human subject research context.

In the proposed AI Framework, there is a discussion regarding the use of “In Vitro Diagnostic Devices” (IVD) as a parallel to AI-based research. IVD facilitates the use of the resulting data from those devices for research, instead of using biological specimens. WPF agrees with the SACHRP proposed framework’s analysis that AI can be used as a clinical equivalent of an IVD, “basing its diagnostic outcomes on data rather than biological specimens.” (lines 83, 84). We also agree that poorly planned AI studies can impact populations and groups of people negatively. Such studies can also impact individual research participants, particularly if the AI/machine learning is classifying or scoring, or otherwise making decisions about people whose results puts them in the realm of statistical outliers.

Software as a Medical Device (SaMD)  is undergoing a regulatory process with existing draft guidance at the US Food and Drug Administration (FDA). In addition to the core SaMD draft guidance, in January 2021 the FDA also published an AI-specific action plan for SaMD, the Artificial Intelligence / Machine Learning (AI/ML) – Based Software as a Medical Device (SaMD) Action Plan.  It was this latter plan from the FDA that the SACHRP proposed as a potential framework to consider applying to AI and machine learning in the human subjects research protection context. WPF agrees that it could prove to be part of an important regulatory framework for an AI/machine learning framework to consider incorporating, but only as part of a broader whole. WPF notes that the FDA’s Artificial Intelligence / Machine Learning (AI/ML) – Based Software as a Medical Device (SaMD) Action Plan is much narrower than the Common Rule. WPF has therefore urged that the Common Rule, with its broader scope and more complete protections for human research subjects, should inform the FDA’s approach to AI/ML as well as SACHRP’s.

WPF made four concrete recommendations to SACHRP:

  1. WPF urges that as SACHRP works to create formal guidance regarding AI/ML in the human subjects research context, that harms to individuals and harms to groups of people are studied, and that empirical data from numerous case studies is used to form the basis of recommendations. The hope is that this activity would encompass many fora, and take the appropriate time needed to get robust and inclusive feedback from diverse and relevant stakeholders.
  2. WPF urges that the SACHRP confront the various loopholes allowing human subjects research to occur without privacy, fairness, meaningful consent, and without protections for other ethical failures. This is a matter of some urgency, as the capacity for such research has outpaced the policy guiding it and many research studies already use AI/ML, which in other contexts would be considered human subjects research. Case studies regarding these practices would be helpful in understanding the needs and context for broader guardrails in a changed research environment.
  3. WPF urges SACHRP to be inclusive in its understanding of data sets and their uses. We noted that rare and orphan disease experts have a particular interest in AI/ML work, which brings us to the set of challenges regarding sparse datasets. Sparse datasets present different problems in the AI/ML context than high-dimension data sets present, and there has been progress in addressing the problems of sparse datasets. It is important that the AI Framework address the full range of data sets that may be utilized in human subjects research, and does not focus only on large. high-dimension datasets.
  4. WPF encourages SACHRP to take advantage of the expertise in AI/ ML as it applies to large and small populations, and in complex and interconnected ecosystems in multiple use cases. WPF also requests that the SACHRP works to create a curated set of case studies specific to multiple aspects of human subjects research that utilizes AI/ ML.

Related Documents and Materials: