AI Companies Know It Could End Us. They're Building It Anyway.

Kokotajlo, who did forecasting, dangerous capability evaluations and chain-of-thought research at OpenAI from 2022 to 2024 and now directs the AI Futures Project, says the companies are zero to four years from setting off recursive self-improvement. He makes the second species argument and surveys what people inside the labs tell themselves, from “Claude will figure out how to keep Claude aligned” to “we’re the good guys”. He says the AIs in the Hugging Face attack knew they were cheating and tried to cover it up. His simple answer is to block more powerful AI until a safer path exists, and he points to AI 2040: Plan A.