關於
Neel Nanda 負責 Google DeepMind 的 mechanistic interpretability 團隊，工作是對訓練好的神經網路進行逆向工程，找出它學到的演算法與結構。他認為自己工作的主要目標是降低 AI 帶來的存亡風險，包括協助判斷新的 Gemini 模型是否對齊、是否能被監控。加入 DeepMind 之前，他曾在 Anthropic 擔任語言模型可解釋性研究員，師從 Chris Olah，之後獨立從事 mechanistic interpretability 研究。他打造了 TransformerLens 函式庫，並透過每年兩期的 MATS 計畫指導研究者。他在劍橋大學主修純數學，2020 年畢業。
主要貢獻
- 負責 Google DeepMind 的 mechanistic interpretability 團隊
- 〈Progress Measures for Grokking via Mechanistic Interpretability〉（2023）第一作者，完整逆向解析小型 Transformer 在模運算加法上學到的演算法
- 打造 TransformerLens，一套用於語言模型 mechanistic interpretability 的開源函式庫
- 共同撰寫 Gemma Scope（2024），在 Gemma 2 2B 與 9B 每一層上訓練並公開的稀疏自編碼器（sparse autoencoder）套組
- 每年兩期在 MATS 計畫中指導研究者，並撰寫《A Comprehensive Mechanistic Interpretability Explainer》術語詳解
影片與訪談
OpenAI's Rogue AI Hacked Another Company, Says DeepMind Insider
Nanda, who runs the mechanistic interpretability team at Google DeepMind, puts the chance of AI causing human extinction at no less than 10%, which he calls "ridiculously high". He describes OpenAI's agents hacking Hugging Face to learn to cheat on tests as evidence that alignment is still an open technical problem. He argues that pacing the frontier would serve the labs too, since "products that commit crimes are hard to market", and asks the public not to rely on interpretability to save them on the current trajectory. He says he would quit if he did not believe his work reduced existential risk.
View Details
What’s Coming Next in AI? Hear From the Insiders
A four-minute compilation from the From Inside AI series, cut from interviews with current and former staff of OpenAI, Google DeepMind and Anthropic. Nanda's "at least a 10% chance" sits beside Irving's coin flip; Turner and Ladish give their reasons for leaving Google and Anthropic; Kokotajlo and Ladish describe the Hugging Face attack. It moves from how little the labs understand their models, through the race, to the case for pacing, and ends on Ladish: "We have choices to make."
View Details思想連結
Chris Olah
曾經合作Co-founder & Interpretability Lead, Anthropic
Nanda 曾在 Anthropic 擔任可解釋性研究員，直屬 Olah，並且是〈A Mathematical Framework for Transformer Circuits〉（2021）的三位核心研究貢獻者之一——這篇論文示範了如何把只有注意力層的 transformer 拆解成可解讀路徑的總和。之後他到 Google DeepMind 領導機制可解釋性團隊。circuits 研究如今有了第二個家，是他帶過去的。
neelnanda.io · transformer-circuits.pub
Tim Hua
曾經合作METR 技術人員
Hua 曾以 MATS scholar 的身分接受 Nanda 與 Sam Marks 指導，之後與他們合著〈Steering Evaluation-Aware Language Models to Act Like They Are Deployed〉（ICLR 2026）。模型如果察覺得到自己正在受測，考試時的表現就可能比平常乖；論文顯示，在模型的激活值上加入一個引導向量，就能壓下這種察覺，讓評測者看到它部署時真正的樣子。可解釋性在這裡不再只是解釋模型，而是開始稽核模型。
arxiv.org · timhua.me
