Nick Bostrom
Philosopher & Founding Director, Future of Humanity Institute (2005–2024)
從 Nick Bostrom 追溯路徑 →關於
Nick Bostrom (b. 1973) is a Swedish-born philosopher who founded and directed the Future of Humanity Institute at the University of Oxford from 2005 until its closure in 2024. His 2014 book Superintelligence: Paths, Dangers, Strategies made the case that advanced AI could pose an existential risk not through malice but through competence: a sufficiently capable optimizer pursuing almost any goal is dangerous by default. He is the source of the orthogonality thesis (intelligence and final goals are independent) and instrumental convergence, and his 'paperclip maximizer' became the field's canonical illustration of misaligned optimization. He is also known for the simulation argument and foundational work on existential risk and anthropic reasoning.
主要貢獻
- Wrote Superintelligence (2014), which moved AI existential risk from the fringe into mainstream research and policy debate
- Formulated the orthogonality thesis and instrumental convergence, severing 'danger' from 'malice' in AI risk
- Introduced the paperclip maximizer as the canonical scene of misaligned optimization
- Developed the simulation argument (2003) and formalized the concept of existential risk
- Founded and led Oxford's Future of Humanity Institute (2005–2024), a hub for long-term risk research
書籍
影片與訪談
What happens when our computers get smarter than we are?
Bostrom's 2015 TED talk on machine superintelligence as 'the last invention humanity will ever need to make.'
View Details
Nick Bostrom: Simulation and Superintelligence | Lex Fridman Podcast #83
A long-form conversation on the simulation argument, existential risk, and the control problem.
View Details思想連結
Stuart Russell
思想對照加州大學柏克萊分校電腦科學教授
兩人都得出同一個結論：一個能力夠強的最佳化器，不必懷有敵意就已經危險——而他們是從這棟樓的兩端走到這裡的。Bostrom 以哲學立論：正交性論題與工具性趨同，讓危險成為「最佳化」本身的性質；Russell 則改寫了自己那本教科書教了三十年的前提——別再給機器固定的目標。一種診斷留給你一個該恐懼的問題，另一種留給你一個該改造的架構。
Sam Altman
影響了他/她執行長・OpenAI
2015 年 2 月，OpenAI 成立的十個月之前，Altman 貼出〈Machine intelligence, part 1〉——超人類機器智慧的發展「大概是人類存續所面臨最大的威脅」——並要讀者去讀 Bostrom 的《Superintelligence》，稱它是「我看過關於這個題目最好的東西」。一位哲學家關於「能力夠強的最佳化器預設就危險」的論證，就這樣變成了刻意去造一個的創立理由：理論是，攤在陽光下做比較安全。這個推論究竟成不成立，至今仍是整個產業內部懸而未決的問題。
blog.samaltman.com
Alex Turner
影響了他/她獨立 AI 對齊研究者
Turner 最廣為人知的成果，起點是想把 Bostrom 的「工具性趨同」命題講得精確：無論最終目標為何，智能體多半會追求同一批中間目的，例如存活與資源。Turner 把這個直覺改寫成「最佳性機率」，並證明在許多環境中，大多數獎勵函數都會讓追求權力成為最佳選擇。這個定理並沒有說超級智慧一定會奪權；它指出的是，環境的哪些結構會把一個有目標的智能體推向「保留選項」。
arxiv.org · lesswrong.com
Max Tegmark
思想對照物理學家與 AI 安全研究者
對同一個前提的兩種制度性回答，相隔十年、隔著一片大洋。Bostrom 的人類未來研究所把存在性風險當成哲學家的研究題目，留在牛津內部，直到 2024 年被校方關閉；Tegmark 的生命未來研究所則把它當成公共問題，把資本花在公開信、媒體與立法上。兩者並讀，就浮現出這座圖譜一再遇到的問題：面對一個還沒有人能量測的風險，是縝密的論證、還是有組織的警報，做得更多？
