Nick Bostrom

Nick Bostrom is a Swedish philosopher known for his work on existential risk, artificial intelligence and anthropic reasoning. He was the founding director of the Future of Humanity Institute at the University of Oxford from 2005 until it closed in 2024. His simulation argument (2003) claims that at least one of three surprising propositions about advanced civilisations must be true. Superintelligence (2014) argues that machines more intelligent than humans could be extremely dangerous unless the problem of controlling them is solved first, and it helped bring AI safety into public debate.

Key ideas

• The orthogonality thesis: almost any level of intelligence can be combined with almost any final goal.
• Instrumental convergence: very different goals lead an intelligent agent to seek similar means, such as resources and self-preservation.
• A superintelligent AI could be hard to control, so its goals must be aligned with human values before it is built.
• Either civilisations rarely reach a posthuman stage, or they rarely run ancestor simulations, or we are almost certainly living in a simulation.
• Reducing existential risk has very high expected value because of how much future may be lost.

Major works

• Superintelligence: Paths, Dangers, Strategies (2014)
• Anthropic Bias (2002)
• Deep Utopia: Life and Meaning in a Solved World (2024)

In the Book corner

• Superintelligence: Paths, Dangers, Strategies 2014 · Intermediate
  A systematic study of what might happen if machines come to exceed human beings in general intelligence. It opens with a fable…

In the conversation

No topics yet. Start one about this thinker.