Efficacy of Generative AI-Assisted Patient Education Materials in Orthopedic Surgery Examined

The role of generative artificial intelligence models, such as ChatGPT and Google Gemini, in creating and evaluating patient education materials in the field of orthopedic surgery has been scrutinized in a new comprehensive scoping review.

In studies covering various orthopedic procedures such as knee replacement, hip surgery, ACL reconstruction, and spine surgery, the performance of AI models was analyzed based on criteria of accuracy, readability, and information quality. Research showed that models like ChatGPT could reach accuracy rates of up to 88 percent on topics such as knee replacement FAQs. In a review conducted on lumbar disc herniation, it was determined that 97 percent of the generated statements were understandable, while 52 percent were entirely accurate.

In readability analyses, it was observed that basic AI models generally produced complex texts at an 11th-17th grade level. However, it was determined that this level could be reduced to a 5.0-6.9 grade level using methods such as restrictive prompts and iterative strategies. In quality assessments conducted with tools like DISCERN, variations were recorded in the results depending on the models and prompting techniques used.