Artificial intelligence may be able to assess clinical interviewing skills as effectively as experienced clinicians, according to research published on Feb. 17 in the journal JMIR Medical Education. The study was conducted by researchers from Juntendo University Faculty of Medicine in Japan, led by Dr. Hiromizu Takahashi and Professor Toshio Naito.
Evaluating clinical interviewing is a critical part of medical training but often requires significant time and effort from faculty members. As medical education expands, the burden on educators has grown, prompting interest in whether AI could help streamline this process.
The researchers designed a cross-sectional validation study using a virtual patient system where seven participants—including students and physicians—conducted interviews with an AI-simulated patient presenting with bilateral leg weakness. These interviews were transcribed and assessed using the Master Interview Rating Scale by both advanced AI models (GPT-o1 Pro and GPT-5 Pro) for the AI-based assessment (ABA) and five experienced instructors for human-based assessment (HBA).
"Our central message is that AI may help make medical training fairer, faster, and more scalable," explains Prof. Naito. The results showed strong agreement between ABA and HBA scores, with minimal differences noted. Furthermore, AI assessments were more consistent across repeated evaluations and required less than half the time per transcript compared to human reviewers.
Dr. Takahashi said, "Rather than replacing teachers, this research suggests a practical 'AI-first, faculty-verified' model in which AI handles the first pass and educators focus their time on coaching, judgment, and high-stakes decisions." Prof. Naito added that rapid feedback through AI could benefit students by making practice opportunities more accessible: "Students could interview an AI-simulated patient and receive feedback almost immediately instead of waiting days or weeks." However, both researchers cautioned that text-only evaluation cannot capture nonverbal cues or cultural context: "AI should be used with human oversight because text-only scoring can miss nuances such as tone, nonverbal communication, and cultural context," they said.
The findings suggest that combining the efficiency of artificial intelligence with clinician expertise may offer a way to improve medical education while reducing faculty workload.
