Health systems have to ‘step up’ on AI regulation

WELLESLEY, Mass. — Amid heated discussions on how artificial intelligence should be regulated and who should be involved in health care AI governance, the U.S. Food and Drug Administration commissioner said that health systems need to take a leading role.

“I think there’s a lot of good reason for health systems to be concerned that if they don’t step up, they’re going to end up holding the bag on liability when these algorithms go wrong,” said Robert Califf on Tuesday to a group of journalists in Wellesley, Mass. 

Developers can’t fully guarantee the algorithm’s performance, explained Califf, because an AI model’s performance evolves over time and performs differently in different hospitals’ patient populations, unlike a drug. And that model drift is “happening in health systems. And so for the health system to say, ‘This is just a manufacturer [problem],’ then the whole thing will break down and won’t work,” he said.