A new commentary published in the Journal of the Royal Society of Medicine warns on June 27 that current risk-based regulatory approaches to Artificial Intelligence in healthcare are insufficient to protect patients, potentially resulting in over- and undertreatment as well as discrimination against certain patient groups.
The authors state that while AI and machine learning systems can improve clinical accuracy, concerns persist regarding their inherent inaccuracy, opacity, and potential for bias. They say these issues are not adequately addressed by existing regulatory efforts such as the European Union's AI Act.
The AI Act, passed in 2025, categorizes medical AI as "high risk" and imposes strict controls on providers and deployers. However, the authors argue that this risk-based framework fails to address three key issues: individual patient preferences, systemic and long-term effects of AI implementation, and the disempowerment of patients within regulatory processes.
They call for introducing specific patient rights related to AI-generated diagnosis or treatment planning. The commentary emphasizes the importance of these rights but does not detail them within this summary. The authors warn that without urgent engagement from healthcare stakeholders—including clinicians, regulators, and patient groups—these rights may be neglected amid rapid advancements in healthcare technology.
"AI is transforming healthcare, but it must not do so at the expense of patient autonomy and trust," said Professor Ploug. "It is time to define the rights that will protect and empower patients in an AI-driven health system."
