Agentic Misalignment

In Anthropic’s AI safety research, the agentic misalignment suite is a collection of contrived fictional scenarios to test how language models behave in high-stakes corporate settings. These usually involve forcing the model to decide between shutdown vs. taking a specific harmful action such as blackmail.

Research on ‘agentic misalignment’ began when Sonnet 3.6 was Anthropic’s most capable model,¹ so late 2024 or early 2025.

Initial demonstrations

The blackmail scenario was first documented in the May 2025 system card for Claude Opus 4.

On June 20, 2025, Anthropic published “Agentic Misalignment: How LLMs could be insider threats”, a blog post that stress-tests more models such as Gemini 2.5 Pro, DeepSeek-R1, GPT-4.1, and older Claude models. They find that resorting to blackmail isn’t specific to Claude, but common across models.¹

A research paper by Lynch et al. was later published in October.²

Training against the eval

To mitigate agentic misalignment, Anthropic generated thousands of scenarios similar in structure to the agentic misalignment evals for use in Sonnet 4.5 and Haiku 4.5’s training.³

For use in Sonnet 4.5 and Haiku 4.5’s safety training, Anthropic generated thousands of scenarios similar in structure to the agentic misalignment evals. In the Sonnet 4.5 system card, two new settings “Framing for Crimes” and “Research Sabotage” were also documented.⁴ (7.5.4)

Anthropic found that the models correctly recognized that they were being tested in these fictional scenarios.⁴⁵

Fixing self-preservation

For Opus 4.5’s training, the honeypot environments were removed⁵ and replaced with updates to character training. Anthropic concluded that Opus 4’s self-preservation most likely originated from the model’s pretrained priors — i.e., that the Claude character had not fully generalized into the base model.

The May 2026 post “Teaching Claude Why” described various character training interventions used for Opus 4.5 and subsequent production models, and prove this using the agentic misalignment evaluations as a case study.³

Continued usage as a dataset

Henceforth, the agentic misalignment suite effectively became an eval-awareness benchmark.

The blackmail dataset were reused for:

• White-box analysis of Opus 4.5⁵
• White-box analysis of Opus 4.6⁶
• Demonstrating natural language autoencoders with Opus 4.6⁷
• J-lens readouts of Sonnet 4.5⁸

Further reading

• “Claude 4 You: Safety and Alignment” - Zvi Mowshowitz (May 25, 2025)
• “Tales of Agentic Misalignment” - Zvi Mowshowitz (Jun 25, 2025)
• “Welcome to SummitBridge” - nostalgebraist (Jun 22, 2025)
• https://nostalgebraist.tumblr.com/post/821437941880651776/very-funny-to-me-how-the-summitbridge-scenario-is