Nick Bostrom
Philosopher & Founding Director, Future of Humanity Institute (2005–2024)
Trace a path from Nick Bostrom →About
Nick Bostrom (b. 1973) is a Swedish-born philosopher who founded and directed the Future of Humanity Institute at the University of Oxford from 2005 until its closure in 2024. His 2014 book Superintelligence: Paths, Dangers, Strategies made the case that advanced AI could pose an existential risk not through malice but through competence: a sufficiently capable optimizer pursuing almost any goal is dangerous by default. He is the source of the orthogonality thesis (intelligence and final goals are independent) and instrumental convergence, and his 'paperclip maximizer' became the field's canonical illustration of misaligned optimization. He is also known for the simulation argument and foundational work on existential risk and anthropic reasoning.
Key Contributions
- Wrote Superintelligence (2014), which moved AI existential risk from the fringe into mainstream research and policy debate
- Formulated the orthogonality thesis and instrumental convergence, severing 'danger' from 'malice' in AI risk
- Introduced the paperclip maximizer as the canonical scene of misaligned optimization
- Developed the simulation argument (2003) and formalized the concept of existential risk
- Founded and led Oxford's Future of Humanity Institute (2005–2024), a hub for long-term risk research
Questions they sharpened View the streams
Books
Videos & Interviews
What happens when our computers get smarter than we are?
Bostrom's 2015 TED talk on machine superintelligence as 'the last invention humanity will ever need to make.'
View Details
Nick Bostrom: Simulation and Superintelligence | Lex Fridman Podcast #83
A long-form conversation on the simulation argument, existential risk, and the control problem.
View DetailsConnections
Stuart Russell
In contrastProfessor of Computer Science, UC Berkeley
Both concluded that a sufficiently capable optimizer is dangerous without needing to be hostile, and they arrived from opposite ends of the building. Bostrom argued it philosophically — orthogonality and instrumental convergence make the danger a property of optimization itself — while Russell rewrote a premise his own textbook had taught for thirty years: stop handing machines fixed objectives. One diagnosis leaves you a problem to fear; the other leaves you an architecture to change.
Sam Altman
InfluencedCEO, OpenAI
In February 2015, ten months before OpenAI launched, Altman posted 'Machine intelligence, part 1' — the development of superhuman machine intelligence is 'probably the greatest threat to the continued existence of humanity' — and sent readers to Bostrom's Superintelligence as 'the best thing I've seen on this topic.' A philosopher's argument that a capable optimizer is dangerous by default became a founding rationale for deliberately building one, on the theory that it is safer done in the open. Whether that inference actually follows is the unresolved question inside the entire industry.
blog.samaltman.com
Alex Turner
InfluencedIndependent AI Alignment Researcher
Turner's best-known result began as an attempt to make Bostrom's instrumental convergence thesis precise — the claim that agents with almost any final goal would pursue the same intermediate ends, such as survival and resources. Turner recast that intuition as 'optimality probability' and proved that in many environments most reward functions make power-seeking optimal. The theorem does not say a superintelligence will seek power; it shows which features of an environment push a goal-directed agent toward keeping its options open.
arxiv.org · lesswrong.com
Max Tegmark
In contrastPhysicist & AI Safety Researcher
Two institutional answers to the same premise, founded a decade apart and an ocean apart. Bostrom's Future of Humanity Institute treated existential risk as a research problem for philosophers and stayed inside Oxford until the university closed it in 2024; Tegmark's Future of Life Institute treated it as a public one and spent its capital on open letters, media and legislation. Read together they pose a question this atlas keeps meeting: whether careful argument or organized alarm does more for a risk nobody can yet measure.
