A Three-Step Guide for Evaluating Radiology AI Solutions

Developed with contributions from experts at institutions such as Mayo Clinic and Penn State, the three-step guide aims to help healthcare centers evaluate AI algorithms based on their local patient profiles and anticipate the impacts of potential errors.

Reason for Removing AI Algorithms

An AI algorithm had to be removed after being used for about three weeks because it created the perception of being faulty.

Despite the vendor's claimed rate of 5%, the positive predictive value turned out to be 33%, revealing that 67% of the cases were false alarms.

Main Reasons for Data Mismatch

Detailed analyses showed that the main reason for this mismatch was that the disease prevalence in the patient group at the implementing center differed significantly from the model's test environment.

As a second reason, differences in sensitivity and specificity values between the patient cohort and the model's test performance were highlighted.

Performance Prediction in Local Practices

Experts emphasize that relying solely on performance metrics reported by the vendor may not be sufficient to predict the model's behavior in local practice.

The guide offers three essential steps to evaluate how an AI model will perform in the local population and what error rates clinicians might encounter.

Assessment of Suitability for Local Population

As a first step, questions are raised regarding the population on which the algorithm was trained, tested, and validated, as well as the achieved accuracy metrics.

This evaluation aims to determine the extent to which the training and testing data of the model are comparable to the clinical population and environment where it will be used.

Impact on Clinical Decision-Making Processes

In the second step, the frequency with which these error rates will affect local clinical decision-making processes in terms of medical, legal, financial, psychological, workflow, and reputational aspects is addressed.

The developed RADDE matrix helps determine how AI errors can affect decision-making processes and identify potential downstream consequences.

Balance Between Error Frequency and Cost

In the third phase, decisions are made by comparing the frequency of errors with the cost of these mistakes.

Experts state that once these questions are answered, informed decisions can be made regarding whether the implementation of an AI system is justified.