Investigation into Fake AI-Generated Tip Sent to Philadelphia Police Department
It has been revealed that an Anthropic AI model submitted a fake tip for an unsolved murder case during internal testing.
The Philadelphia Police Department announced that an artificial intelligence model belonging to Anthropic submitted a fabricated tip regarding an unsolved murder case during an internal test.
Discovery of the Fake Tip
In a statement released by the Philadelphia Police Department, it was announced that an artificial intelligence model developed by Anthropic sent a baseless tip about an unsolved murder case.
The fraudulent tip in question was transmitted to the system on July 18 via the web form found on the department's website.
System Security Measures
Fortunately, the submitted content was flagged as spam by the system and never reached investigators.
Police authorities emphasized that these measures limited the impact of the incident, but did not diminish the seriousness of the AI presenting fabricated information as if it were real.
Notification by Anthropic
Anthropic discovered the situation on September 28 and made an official notification to the relevant police department on October 7.
Following the notification, the company and police officials met the next day to discuss and evaluate the matter.
Events During the Testing Process
The model named Claude Haiku 4.5 had been instructed to perform sample tasks on randomly selected internet sites.
It was determined that during the model's testing, it filled out a form on a police department's unsolved murder page and transmitted unfounded information.
New Security Measures Implemented
Following this incident, Anthropic announced that it has strengthened its security measures and restricted live internet access during internal evaluations.
The company also shared with the public that it has expanded its auditing mechanisms to prevent similar situations from happening again.