AI Produces History of Present Illness Summaries Comparable With Residents

THURSDAY, July 20, 2023 (HealthDay News) -- History of present illness (HPI) writeups are of similar quality whether written by a chatbot or senior internal medicine residents, according to a study published online July 17 in JAMA Internal Medicine.

Ashwin Nayak, M.D., from Stanford University in California, and colleagues evaluated the ability of a chatbot to generate an HPI compared with senior internal medicine residents. The analysis included HPIs generated by ChatGPT and those written by four residents based on three patient interview scripts portraying different types of chest pain.

The researchers found that the acceptance rate for chatbot-generated HPIs improved from 10.0 to 43.3 percent by the final round of prompt engineering. Based on the 15-point composite scale, grades of resident and chatbot-generated HPIs differed by less than 1 point (resident mean, 12.18, versus chatbot mean, 11.23). Resident HPIs scored higher on level of detail scale (resident mean, 4.13, versus chatbot mean, 3.57). HPIs were correctly characterized as written by residents or the chatbot by attending physicians 61 percent of the time.

"These findings underscore the potential of chatbots to aid clinicians with medical documentation," the authors write.

Abstract/Full Text (subscription or payment may be required)

Editorial (subscription or payment may be required)