Physicians continued to follow erroneous artificial intelligence (AI) guidance when classifying fictitious patients, even as treatment outcomes showed the classifications were incorrect, according to a study published in PLOS Digital Health on July 13.
The research examined whether doctors could override incorrect AI-based patient classifications when provided with relevant outcome information. In two experiments involving a combined sample of 223 self-reported physicians recruited via the Prolific survey platform, participants decided whether to administer a fictitious drug to individuals misclassified by an AI system as more or less sensitive to treatment. The second experiment used the drug as a model of pseudomedicine, presented as potentially useful but not proven effective.
Across both experiments, most participants trusted the AI's classification and struggled to use patient outcomes to correct errors. Even when shown that the drug was ineffective in Experiment 2—where seven out of ten patients recovered regardless of receiving treatment—participants generally judged it effective. The study found that physicians had difficulty updating their judgments based on direct evidence from patient recovery outcomes.
Participants had mean ages of 38.6 and 36.6 years in Experiments 1 and 2, respectively, with average professional experience of over ten years. Most commonly reported specialties included General Medicine and Pediatrics or Internal Medicine depending on the experiment. In Experiment 1, participants rated the AI system's reliability at an average score of 3.65 out of five and held generally favorable attitudes toward AI use.
In both experiments, doctors largely relied on the AI classification for their decisions: they administered treatment more frequently to those classified as highly sensitive by the algorithm than those classified as lowly sensitive—even though actual patient recovery rates did not differ between groups within each experiment. This pattern persisted despite immediate feedback about individual patient recoveries after each decision.
The authors caution that these findings come from controlled online tasks using fictitious diseases and treatments rather than real clinical settings; professional status was also self-reported by participants. They recommend further research into potential cognitive biases such as automation bias or confirmation bias that may have contributed to these results, and emphasize that while AI systems can support clinical decision-making, human reasoning remains essential.
