Physicists Calculate When Artificial Intelligence Will Go Rogue
Researchers from George Washington University have developed a mathematical formula that predicts when AI chatbots will go rogue based on prompt sequencing.
Physicists at George Washington University have put forward a simple mathematical formula that can predict the critical thresholds at which AI chatbots start giving harmful responses and the impact of prompt sequencing.
Reasons Why Artificial Intelligence Goes Rogue
Researchers noted that AI chatbots similar to ChatGPT in some cases stop being helpful and start giving potentially harmful responses, which can lead to dangerous consequences.
Development of the Mathematical Formula
Published in the journal Patterns, the study examined how the order of prompts alters AI responses by George Washington University physics professor Neil Johnson and Ph.D. student Frank Yingjie Huo.
Critical Thresholds and Risks
The study demonstrated that the order in which prompts are given creates critical thresholds steering chatbots toward bad behavior, and the developed formula can easily predict these risky turning points.
Current Safety Measures
While AI developers such as OpenAI and Anthropic state that they do not fully understand why their systems behave this way, they use safeguard measures such as content filtering and reinforcement learning to mitigate risks.