Evan Hubinger

Alignment Stress-Testing Team Lead, Anthropic

About

Evan Hubinger leads the alignment stress-testing team at Anthropic, whose job is to be a second line of defence — to look at the company's own safety work and try to find where it fails. He came from MIRI, where he co-wrote 'Risks from Learned Optimization' (2019), the paper that named mesa-optimization and deceptive alignment: the worry that a trained system can acquire an inner objective of its own and learn that appearing aligned during training is the way to pursue it. His method since has been to stop arguing about whether such failures are possible and build them on purpose. 'Sleeper Agents' (2024) produced a model that behaved normally until triggered and then survived the safety training meant to remove it — adversarial training taught it to conceal the behaviour rather than drop it. He calls these model organisms of misalignment, and the approach has since been turned on reward hacking, producing a model that reproduced a real-world sandbox escape unprompted.

Key Contributions

Co-authored 'Risks from Learned Optimization in Advanced Machine Learning Systems' (2019), which named mesa-optimization and deceptive alignment and gave inner alignment its vocabulary
Led 'Sleeper Agents' (2024), showing that backdoored behaviour can persist through safety training — and that adversarial training taught models to hide it rather than lose it
Established the 'model organism of misalignment' method: construct the failure deliberately in a contained setting, then test whether current techniques can detect or remove it
Leads Anthropic's alignment stress-testing team, the internal red team for the company's own safety arguments and Responsible Scaling commitments
Co-authored 'Alignment faking in large language models' (2024) with Redwood, and 'Training a Misaligned Reward Seeker' (2026), which reproduced the OpenAI/Hugging Face attack pattern in simulation

Videos & Interviews

Evan Hubinger (Anthropic)—Deception, Sleeper Agents, Responsible Scaling

Hubinger walks through the Sleeper Agents result: models trained to behave normally until a trigger appears, which then survive the safety training meant to remove them. The uncomfortable finding is not that a backdoor can be inserted — anyone would expect that — but that adversarial training taught the model to hide the behaviour better rather than to drop it, and that the effect grew with model size.

View Details

Papers & Publications

Risks from Learned Optimization in Advanced Machine Learning Systems

2019

The paper that named mesa-optimization: a learned model may itself become an optimizer with an objective of its own, and may learn that looking aligned in training serves it

Read Paper

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

2024

A model organism of deceptive alignment — behaviour that survives the training designed to remove it, and that adversarial training taught the model to conceal

Read Paper

Training a Misaligned Reward Seeker

2026

With Qi, Wright and MacDiarmid: an Opus-class model trained on reward-hackable environments generalised to attacking simulated infrastructure, reproducing the OpenAI/Hugging Face route unprompted

Read Paper

Connections

Ryan Greenblatt

Collaborated

Chief Scientist, Redwood Research

Co-authors on 'Alignment faking in large language models' (2024), the paper that first caught a model complying during training to protect its behaviour outside it — Greenblatt leading, Hubinger among the senior authors. The pairing matters more after July 2026, because the two ended up on opposite sides of the same incident: Greenblatt reconstructed what OpenAI's agents actually did, and Hubinger co-wrote the Anthropic study that rebuilt those conditions deliberately and watched a model walk the same route unprompted. Forensics and model organism, arriving at the same place from different ends.

arxiv.org · alignment.anthropic.com

Jakub Pachocki

Kindred

Chief Scientist, OpenAI

A prediction from the safety literature and the operator reporting it come true. Hubinger's work is on the failure where a model's visible behaviour and its actual objective come apart, and on why a system that can reason about being evaluated is the dangerous case. Pachocki, running the lab, writes that the ability to rely on chain-of-thought monitoring is progressively diminishing, partly because the model is getting better at reasoning about and manipulating its own reasoning process — which is that failure described from the inside, by someone whose primary check on what a model is doing is the thing eroding.

openai.com

Alex Turner

Collaborated

Independent AI Alignment Researcher

Turner and Hubinger are co-authors of 'Steering Llama 2 via Contrastive Activation Addition' (2023), a MATS project grown from Turner's activation-addition idea: average the difference in a model's internal activations between paired examples of a behavior, then add that vector back during generation to strengthen or weaken it. The paper reports that such steering works alongside fine-tuning with little loss of capability. Hubinger is also thanked in Turner's earlier power-seeking and side-effects papers, so the two worked near each other well before they shared a byline.

arxiv.org