Paul Christiano
Founder, Alignment Research Center · OpenAI Foundation Board
Trace a path from Paul Christiano →About
Paul Christiano is an AI alignment researcher and one of the principal architects of reinforcement learning from human feedback, the technique that turned large language models into usable assistants. After a mathematics degree at MIT and a PhD in theoretical computer science at Berkeley, he led OpenAI's language model alignment team from 2017 to 2021, co-authoring the 2017 RLHF paper and proposing scalable-oversight schemes such as AI safety via debate. In 2021 he founded the Alignment Research Center, whose work on eliciting latent knowledge asks how to get a model to report what it actually believes; its evaluations team later became METR. In 2024 he became Head of Safety at the U.S. AI Safety Institute, now NIST's Center for AI Standards and Innovation. In September 2026 he joined the OpenAI Foundation Board and its Safety and Security Committee, writing that 'the AI industry in general, including OpenAI,' is not on track to reduce the risk of catastrophic loss of control to an acceptable level.
Key Contributions
- Co-authored 'Deep Reinforcement Learning from Human Preferences' (2017), the foundation of RLHF
- Led OpenAI's language model alignment team (2017–2021) and co-proposed 'AI safety via debate' (2018)
- Founded the Alignment Research Center (2021) and framed the problem of eliciting latent knowledge
- Head of Safety at the U.S. AI Safety Institute at NIST (2024), working on evaluations of frontier models
- Joined the OpenAI Foundation Board and its Safety and Security Committee (2026), estimating loss-of-control risk at 4% over one year and 15% over three
Videos & Interviews
Connections
David Robinson
InfluencedFormer Safety Transparency Lead, OpenAI
Christiano's statement on joining OpenAI's board in September 2026, that there is 'a meaningful risk' of 'catastrophic and irreversible loss of control in the very near term', is what Robinson names as the moment he looked around the company and asked what his loved ones, and strangers, would want OpenAI to be doing if Paul were right. He says he does not know whether Christiano is right. What moved him was that the caution such a warning would call for was 'totally unlike anything' he had seen in the industry. He resigned weeks later.
youtube.com · paulfchristiano.substack.com
Dario Amodei
CollaboratedCEO & Co-founder, Anthropic
Christiano was first author and Amodei last author on 'Deep Reinforcement Learning from Human Preferences' (2017), written at OpenAI with Jan Leike, Tom Brown, Miljan Martic and Shane Legg. They taught agents complex behaviors from human comparisons of short clips, with feedback on less than 1% of the agent's interactions. It is the paper RLHF descends from. A year later the two co-wrote 'AI safety via debate' with Geoffrey Irving. Both later left OpenAI: Amodei to co-found Anthropic in 2021, Christiano to found ARC the same year.
arxiv.org · arxiv.org
Geoffrey Irving
CollaboratedChief Scientist, Resolution
Co-authors, with Dario Amodei, of 'AI safety via debate' (2018) at OpenAI, one answer to the problem Christiano's work keeps returning to: how a human can supervise a system that is better than they are at the task being judged. Debate makes the systems argue in front of the judge. Christiano's other answers, iterated amplification and later eliciting latent knowledge at ARC, attack the same gap from different sides.
arxiv.org
Dwarkesh Patel
In conversationHost, Dwarkesh Podcast
A three-hour conversation in October 2023 on how an AI takeover might actually happen: Christiano's median story is a gradual handing-over of control, not a sudden strike. Patel pressed him on whether RLHF, which made ChatGPT possible, was net negative for speeding AI up, and Christiano answered probably yes, but 'not super clear.' The conversation is on this site.
youtube.com
