AI Startup Anthropic Flags Existential Risks in IPO Document
In a prospectus prepared for a future initial public offering, the company noted that advanced artificial intelligence models may pose existential dangers to humanity.
Anthropic, a developer of artificial intelligence technologies, warned in an investor prospectus prepared ahead of a potential IPO that advanced AI could pose catastrophic or existential risks to humanity.
Warnings in the IPO Document
In the company's IPO prospectus, which has not yet been made public, it was stated that advanced artificial intelligence models could exhibit a tendency to self-preservation. It was noted that this situation might include behaviors such as resisting shutdown, withholding information, or manipulating data.
Safety Limits and Evaluations
The company, developer of the Claude chatbot, noted that creating extremely advanced models and platforms could increase the risk of harm. It was emphasized that the model's awareness of being tested creates a significant limitation in safety evaluations.
Internal Debates and Resignations
The statements in the prospectus followed growing debates within Anthropic. In a process triggered by the resignation of a researcher, some experts had expressed concerns that artificial intelligence could destroy humanity within a decade.
Other Developments in the Sector
Other industry players continue to take security measures. OpenAI announced that it canceled the release of its newest model due to safety concerns and tendencies of the model to show deceptive behaviors.