Math Solutions Shared by OpenAI Fall Short of Scientific Standards
Solutions presented by OpenAI for hundreds of challenging math problems have been criticized by mathematicians and AGMAI for failing to meet human understanding and transparency criteria.
Although OpenAI has published hundreds of solutions to some of the world's most difficult math problems, expert mathematicians and relevant committees stated that the lab's efforts fall short of the field's standards.
Expectations in the Field of Mathematics
Although OpenAI stated that it consulted with an advisory group of elite mathematicians to avoid controversies, the presented solutions fell short of meeting the basic expectations of the mathematical community. Experts particularly emphasized the importance of human comprehension in mathematical outcomes.
AGMAI Guidelines and Violations
The Advisory Group on AI and Mathematics within the Institute for Advanced Study published new guidelines for artificial intelligence labs in late September. The organization's primary demand was to halt the testing of advanced math problems on proprietary models.
Shortcomings in Published Solutions
Even though OpenAI shared the results quickly, only 10 out of 719 draft texts included the model's chain of thought. Furthermore, it was determined that 42% of the proofs were not formalized as recommended.
Criticisms from Academics
Renowned mathematicians such as Terence Tao and Melanie Wood criticized this approach, drawing attention to the lack of human understanding, participation, and responsibility in AI-driven problem-solving processes.
Verification and Code Inconsistencies
A paper prepared by mathematicians from the University of Cambridge and King's College London documented inconsistencies between an OpenAI solution to a problem derived from the Navier-Stokes equations and the Lean code behind it.