New Method Developed to Detect Hallucinations in Large Language Models
Researchers from Skoltech and Sberbank have developed a cost-effective method that identifies out-of-context responses in AI models without requiring additional model training.
Researchers at Skoltech and Sberbank have introduced a cost-effective new approach called TOHA to detect hallucinations produced by large language models in retrieval-augmented generation systems.
New Approach for Hallucination Detection
Researchers at the Skoltech and Sberbank Applied AI Center have proposed a new method called TOHA to detect hallucinations in large language models operating within retrieval-augmented generation systems.
Identification of Out-of-Context Responses
This developed approach makes it possible to identify responses not supported by the provided context by analyzing the topological structure of the model's attention maps.
System Not Requiring Additional Model Training
The method does not require the training of additional models and finds it sufficient to use only a small amount of labeled data for the configuration process.
Publication of the Scientific Article
The research article was published and shared with the scientific community in the Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics.
Experts Involved in the Project
Aleksey Zaytsev, one of the authors of the study, serves as an associate professor at Skoltech and head of the joint Skoltech–Sberbank Laboratory.
Topological Structure and Graph Analysis
The TOHA method, short for TOpology-based HAllucination detector, analyzes the language model's attention matrices, represents them in the form of graphs, and calculates a topological feature called MTop-Div.
Integration into Open-Source Library
This newly developed approach has been successfully integrated into the open-source SIRIN library so that it can be used by researchers and developers.