Critical Warning for Artificial Intelligence in Healthcare: Accurate Results Can Be Misleading

Dr. Damla Övek Baydar from the Department of Computer Engineering at Biruni University Faculty of Engineering and Natural Sciences warned that artificial intelligence systems used in healthcare should not be evaluated solely based on their success rates.

Dr. Damla Övek Baydar from Biruni University stated that artificial intelligence technologies are used intensively in medical imaging, determining disease risks, genomic analyses, and clinical decision support systems. Baydar emphasized that while systems are mostly evaluated based on how accurately they make predictions, in a field like healthcare that directly affects human life, simply reaching the correct result may not be sufficient.

Stating that deep learning models can analyze complex data relationships but it is not always understood what information their decisions are based on, Baydar conveyed that this situation is defined as the "black box" problem. Baydar noted that even though a model can make high-accuracy predictions, it must be known whether this prediction is based on a biologically or clinically meaningful sign, or on a coincidental pattern in the data.