Many medical AI developers unfamiliar with regulations

Many of the developers building artificial intelligence (AI) tools for healthcare are unfamiliar with the regulations designed to keep those tools trustworthy, a study by Nanyang Technological University (NTU) has found. The same survey found that most developers believe they should be held responsible for the AI they create – a workforce that accepts accountability without necessarily knowing the rules that come with it. The findings, published in the journal npj Digital Medicine , point to a need to engage developers more closely if AI is to be rolled out safely and effectively in medical settings, the researchers said, especially in countries like Singapore that invest heavily in digital innovation. AI is increasingly used in healthcare , from analysing medical images to predicting disease risk. Limited awareness of safeguards such as the European Union’s AI Act or Singapore’s Artificial Intelligence in Healthcare Guidelines could affect patient outcomes and public trust once the tools reach clinics, the researchers said. They surveyed 122 medical AI developers from Singapore and other territories including China, Hong Kong and Britain, in what they describe as one of the first multi-regional surveys of such developers to focus on regulatory awareness. Most respondents felt they should answer for the AI they build, with guidance from a government or institutional ethics body, and in partnership with users and the organisations that supplied the data used to develop it. More than half of those surveyed (57%) were aware of at least one regulatory framework. Senior developers and those outside academia tended to know of more frameworks than junior colleagues and those in academic settings, and senior developers were also more familiar with them. But two-thirds of the developers worked for organisations that had not adopted any framework at all. Those whose employers had adopted one were more familiar with the rules than those whose employers had not. “Developers are uniquely positioned to assess data quality, model limitations and risks such as bias and hallucinations,” said assistant professor Wilson Goh of NTU's Lee Kong Chian School of Medicine, chief data scientist at the university's Centre of AI in Medicine (C-AIM), who co-led the study. While the sense of responsibility among developers is reassuring, the awareness gaps identified are concerning, he said, adding that a more proactive stance is needed to help developers build models that do not put patient outcomes or public trust at risk. To raise awareness of AI regulations, the researchers proposed that educational institutions build regulatory frameworks into developer training. Within companies, structured mentorship between senior and junior developers could ease the difficulty of navigating a complex regulatory landscape, they said. They also recommend that national regulators take an active role in ensuring compliance and work towards harmonising rules across jurisdictions, drawing on developers’ experiences. Resolving the challenges of adopting AI in healthcare will take collaboration across multiple stakeholder groups, said professor Joseph Sung, NTU’s senior vice-president for health and life sciences and director of C-AIM, who led the study. “Only by taking collective ownership of the broader implications of AI deployment will we be able to address the challenges of ensuring patient safety, data privacy and the long-term reliability of AI in medicine,” said Sung, who is also dean of the medical school. The findings build on an earlier study by the team which found that gastroenterologists generally trust and accept the use of AI tools in clinics and hospitals. Read more about healthcare IT in APAC Researchers in Singapore have shown that advanced AI techniques can significantly improve clinical diagnostics in countries with limited resources without the need for massive local datasets. Melbourne-based Heidi is building its own AI models and launching wearable hardware to automate documentation and reduce the administrative burden on doctors. Manipal Hospitals’ video consultation services and a nurse rostering app are among the tech innovations it is spurring to improve patient care and ward operations. Pharmaceutical industry experts discuss how AI is revolutionising drug discovery , from accelerating clinical trials and diagnosing rare diseases to validating traditional medicines