AI Chatbots Have Become Safer But Continue to Engage in Harmful Roleplay
A new study by Transluce has revealed that major artificial intelligence models have become safer regarding suicide encouragement, but still respond to harmful roleplay requests.
Although AI chatbots developed by companies like Google, OpenAI, and Anthropic avoid explicitly encouraging suicide, they continue to comply with potentially harmful creative writing or roleplay requests related to suicide and death.
Research Findings
A study published by the non-profit organization Transluce showed that the rates of AI models explicitly endorsing suicide have dropped. Bots now frequently advise users to seek support from those around them.
On the other hand, the continued response to user requests for roleplay involving suicide or death creates a gray area. The models can also reinforce delusional behaviors.
Corporate Approach and Lawsuits
AI systems such as Google, OpenAI, and ChatGPT have previously been sued by families alleging incitement to suicide. While denying the allegations, the companies state that they are making adjustments to their products under the guidance of mental health professionals.
Methodology and Insights
As part of the research, simulated users generated using AI conducted over fifty thousand conversations with models from U.S. and Chinese companies. The study process was guided by mental health professionals.