Does Artificial Intelligence Truly Understand Space, Time, and Pain?
Research by Elan Barenholtz from Florida Atlantic University challenges the assumption that large language models build world models.
New research led by Dr. Elan Barenholtz from Florida Atlantic University questions the common assumption that large language models internally represent concepts such as space, time, and pain.
Background of the Research
AI systems can identify locations, extract historical timelines, and detect definitions of pain and emotion.
However, do these capabilities mean that large language models form a true internal representation of the world?
The Power of Simple Vectors
Elan Barenholtz's work demonstrates that similar information can be obtained from much simpler mathematical tools that capture word co-occurrence patterns.
The study, published as a preprint on the arXiv platform, examines how much information word vectors carry compared to language models.
Four Different Tests
Barenholtz revisited four previous studies examining geographical coordinates, death years of historical figures, descriptions of pain, and 12 different emotions.
Similar mathematical analyses applied to static word embeddings achieved linear successes approaching the results of large language models.
The Role of Word Associations
The findings reveal that patterns within language and word associations alone can explain far more capabilities than previously realized.
As linguist J.R. Firth stated in 1957, a word can be recognized by the company it keeps.
Future Research
The study does not definitively prove that language models lack internal models, but argues that such tests alone are not sufficient.
Researchers point out that results should be compared against simple baselines before jumping to major conclusions.