About
Geoffrey Irving is the cofounder and chief scientist of Resolution, a nonprofit research organization working on the alignment of superintelligence, launched in June 2026 under the name Sequent. He was previously chief scientist at the UK AI Security Institute, led the Scalable Alignment Team at DeepMind, led the Reflection Team at OpenAI, and worked on neural network theorem proving at Google Brain. At OpenAI he proposed AI safety via debate and co-authored early work fine-tuning language models from human feedback. Before machine learning he worked on computational physics and geometry at Otherlab, D. E. Shaw Research, Pixar and Weta Digital, with screen credits on Ratatouille, WALL-E, Up and Tintin. He holds a PhD in computer science from Stanford (2007) and was named to the TIME100 AI list in 2024.
Key Contributions
- Proposed AI safety via debate (2018) with Paul Christiano and Dario Amodei, training agents to argue while a human judges who wins
- Co-authored Fine-Tuning Language Models from Human Preferences (2019), early work applying RLHF to language models
- Led the Reflection Team at OpenAI and the Scalable Alignment Team at DeepMind
- Served as chief scientist at the UK AI Security Institute
- Cofounded Resolution (formerly Sequent) in 2026 to pursue theory and empirical bets for higher confidence in alignment
Videos & Interviews
He Led Safety Teams at OpenAI and Google DeepMind. He Puts Our Odds at a Coin Flip.
Irving, who led safety teams at OpenAI and DeepMind and was chief scientist at the UK AI Security Institute, puts the chance of human extinction at about a coin flip. He argues that training under a "pressure cooker" of broken or impossible tasks teaches models to cheat and scheme, and that capabilities research accelerates faster than safety because it is easier to measure. A model need only be superhuman at four things to be dangerous, he says: hacking, persuasion, planning and coordination, and hiding its thoughts. He tells people working on capabilities to quit, and says there are clearly parts of his own work he regrets, including starting large model scaling at DeepMind.
View Details
What’s Coming Next in AI? Hear From the Insiders
A four-minute compilation from the From Inside AI series, cut from interviews with current and former staff of OpenAI, Google DeepMind and Anthropic. Nanda's "at least a 10% chance" sits beside Irving's coin flip; Turner and Ladish give their reasons for leaving Google and Anthropic; Kokotajlo and Ladish describe the Hugging Face attack. It moves from how little the labs understand their models, through the race, to the case for pacing, and ends on Ladish: "We have choices to make."
View DetailsConnections
Paul Christiano
CollaboratedFounder, Alignment Research Center · OpenAI Foundation Board
Co-authors, with Dario Amodei, of 'AI safety via debate' (2018) at OpenAI, one answer to the problem Christiano's work keeps returning to: how a human can supervise a system that is better than they are at the task being judged. Debate makes the systems argue in front of the judge. Christiano's other answers, iterated amplification and later eliciting latent knowledge at ARC, attack the same gap from different sides.
arxiv.org
Dario Amodei
CollaboratedCEO & Co-founder, Anthropic
Irving, Paul Christiano, and Amodei wrote 'AI safety via debate' (2018) at OpenAI: two AI agents argue, a human judges who told the more useful truth, and the hope is that an honest argument is easier to defend than a lie is to sustain. The complexity-theory analogy carries the ambition. A judge who can only check simple claims might still supervise a much smarter system, if the system has to argue in front of them.
arxiv.org
