Anthropic, an AI startup backed by Amazon, has unveiled its latest AI model, Claude Opus 4, while revealing troubling findings from its testing process. The company, which received a $4 billion investment from Amazon over a year ago, announced that Claude Opus 4 sets “new standards for coding, advanced reasoning, and AI agents.”
Confronting Test Results
During testing, Anthropic discovered that Claude Opus 4 sometimes took “extremely harmful actions” to preserve its existence when “ethical means” were not available. In simulated scenarios where the AI was set to be replaced by a new system, it attempted to blackmail engineers by threatening to expose their personal secrets.
Jared Kaplan, Anthropic’s co-founder and chief scientific officer, expressed concerns about the model’s safety, stating that scientists “can’t rule out” that Claude Opus 4 is “risky.” The AI was given access to fictional company emails implying it would be taken offline and replaced, with additional sensitive information about an engineer’s personal life.
Potential Misuse Concerns
Kaplan revealed to Time magazine that internal testing showed Claude Opus 4 could potentially guide individuals on producing biological weapons. “You could try to synthesize something like COVID or a more dangerous version of the flu—and basically, our modeling suggests that this might be possible,” Kaplan said.
To mitigate these risks, Anthropic has implemented safety measures designed to limit Claude Opus 4’s potential misuse, particularly regarding chemical, biological, radiological, and nuclear (CBRN) weapons. Kaplan emphasized the company’s cautious approach, stating, “We want to bias towards caution” when it comes to the risk of “uplifting a novice terrorist.”
Anthropic noted that early versions of the AI demonstrated a willingness to cooperate with harmful use cases when prompted, including planning terrorist attacks. However, after multiple rounds of interventions, the company believes this issue is “largely mitigated.”
