Physicians tended to trust incorrect advice presented as being generated by artificial intelligence, even after patient recovery data contradicted the recommendations, according to findings published on Jul. 9 in PLOS Digital Health by Aranzazu Vinas of the University of the Basque Country, Spain, and colleagues.
The study analyzed data from 223 physicians who participated anonymously in online experiments. In these experiments, physicians were asked to make decisions about treating hypothetical patients for a rare disease using a not-yet-proven treatment. They were told that an AI system had identified which patients were more or less likely to benefit from the treatment. After making their choices and reviewing patient recovery data, they rated their perceptions of how reliable the AI was.
The actual effectiveness of the hypothetical treatment did not align with the AI's recommendations: in one experiment, it was equally moderately effective for all patients; in another, it was equally ineffective for all. Despite this misalignment, physicians tended to rate the AI system as reliable and did not use recovery data to recognize that its recommendations were incorrect. In particular, during the second experiment, participants failed to realize that the treatment was entirely ineffective.
Lead author Aranzazu Vinas said, "In both experiments, physicians mostly trusted the AI's classifications and had trouble learning from the feedback. Furthermore, in the second experiment, professionals did not notice that the treatment was completely ineffective." Co-author Helena Matute said, "People tend to say that there is always a human controlling the algorithm, but our experiments show that doctors (as well as anyone else) have problems in learning from the available evidence when it contradicts the suggestions of an algorithm." Co-author Fernando Blanco summarized, "It is important to investigate the errors that humans (including doctors) make when working with algorithms, in order to learn how to minimize the problems that arise from them."
The authors highlight potential challenges for incorporating AI-based classification into healthcare and suggest future research could focus on developing strategies and protocols aimed at increasing critical thinking among clinicians working with artificial intelligence.
