Healthcare AI Governance: Balancing Transparency and Risk
Artificial intelligence (AI) governance in the healthcare industry presents a complex challenge. At a recent Newsweek webinar titled “Health Care’s AI Playbook: Building Safe, Smart and Scalable Systems,” health tech leaders discussed the delicate balance between setting ground rules for AI technology and the risks associated with transparency.
The webinar, held on May 20, featured an expert panel including Dr. Brian Anderson, Dr. Danny Tobey, Dr. Andreea Bodnari, and Dr. Michael Pencina. They addressed health care decision-makers on the limitations of universal standards for AI models, as performance can vary significantly between organizations due to differences in leadership priorities, frontline users, and patient data.
One key discussion point was the development of a national AI outcomes registry. Dr. Anderson’s Coalition for Health AI (CHAI) has partnered with Avanade to create a public registry for health AI applied model cards, which serve as “nutrition labels” for AI tools. This registry aims to centralize information on AI applications, including their development and known risks, to create an industry-wide database of lessons learned.
The registry is still in its early stages but hopes to become a “post-market or post-deployment monitoring network” for CHAI’s member organizations. According to Anderson, this will help understand how AI models perform locally and identify variations in performance across different populations or geographies. For instance, the data might reveal a model degrading over time or producing unexpectedly positive outcomes in a specific clinical specialty.
Dr. Bodnari highlighted the potential benefits of such a registry, drawing parallels with how the National Institute of Standards and Technology (NIST) flags vulnerabilities to certified enterprises. She noted that contributing to a public AI registry could provide actionable information, such as notifications about vulnerabilities in specific patient populations for ambient AI tools.
However, privacy concerns were raised regarding the sharing of information. Dr. Pencina acknowledged that vendors and health systems might hesitate to share certain data due to privacy issues. Dr. Tobey, with his medical and legal background, suggested that legislative incentives would be necessary to encourage health systems to share data in the largely unregulated health AI market.
The panel agreed that incentives, such as safe harbor or presumptions of prudence for organizations participating in voluntary disclosures or registries, could be crucial. Anderson mentioned that CHAI is exploring an AI-specific patient safety outcomes registry to enhance protections for its members.
The discussion underscored the need for transparency and collaboration in healthcare AI governance while addressing the associated risks and challenges. As AI continues to evolve in the healthcare sector, finding this balance will be critical to harnessing its potential while ensuring safety and efficacy.
