Existential risk
On this page
A risk that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential; Nick Bostrom argues that reducing such risks should be a global priority.
Also: existential risks
The argument at a glance
A guide to the entry below, quoting its own words. Skip to the full entry and its sources.
- Question
- Should the reduction of risks to humanity's long-term survival and potential take priority over other ways of doing good, and if so, on what kind of argument: an empirical claim about probabilities, a philosophical claim about the value of future lives, or a decision-theoretic claim about how to act under uncertainty?
- Reasons , COT reconstruction: the numbering is ours, not the philosopher's
-
(COT's reconstruction; the three kinds of premise are kept apart) Empirical premises:
- the future could contain a very large number of worthwhile lives if humanity survives and develops;
- some present actions can reduce, by some non-negligible amount, the probability of a catastrophe that would prevent this. Value premises:
- the lives of future people count, and count no less for occurring later;
- preventing the non-existence of a vast number of worthwhile future lives is a very great good, not merely a matter of indifference. Decision-theoretic premise:
- we should choose so as to maximise expected value, or at least give great weight to expected value, when the probabilities can be estimated. Conclusion:
- even very small reductions in existential risk have greater expected value than most other available goods, so their reduction should be a global priority.
- Intended conclusion
- Reducing existential risk should be treated as a priority for humanity as a whole, because of the size of what an existential catastrophe would destroy rather than because such a catastrophe is likely.
Objections and replies
- Objections , Raised by other thinkers and critics, as reported
-
Each kind of premise is contested. Against (3), defenders of pure time discounting hold that future well-being may count for less simply because it lies in the future; Lukas Meyer's Stanford entry on intergenerational justice treats whether we have reasons to discount future persons' well-being as an open debate. Against (4), person-affecting views, set out in that entry's discussion of the non-identity problem, hold that an outcome can be worse only if it is worse for someone, which threatens the claim that failing to bring future people into existence is a great loss to anyone. Against (1) and (2), the empirical estimates and the effect of present action on them are uncertain, a point the live text above records with Bostrom's own admissions; and against (5), the fanaticism worry noted above questions whether expected value should govern when probabilities are tiny and stakes vast.
- Replies
-
On discounting, Meyer's entry reports that Sidgwick, Ramsey, Rawls and Parfit all reject pure time discounting, the view that future people's claims count for less merely because they are future. On person-affecting views, Bostrom argues that even from a person-affecting perspective the assessment of flawed realisation should turn on whether conditions give a good prospect of realising a large amount of value over the future, so that at least part of his case does not depend on impersonal value. On the empirical premises, defenders reply that the argument requires only that some reduction be possible, not any particular estimate, a reply whose force depends on the size of the reduction actually available.
Coordinate 3.3.4Coordinate 3.2.1Coordinate 6.20Nick BostromToby OrdDerek ParfitSources and primary textsRead the full entry
What it means
An existential risk, in Nick Bostrom's definition in 'Existential Risk Prevention as Global Priority' (Global Policy, 2013), is one that threatens 'the premature extinction of Earth-originating intelligent life' or 'the permanent and drastic destruction of its potential for desirable future development.' The notion is therefore wider than extinction. Bostrom distinguishes four classes: human extinction before technological maturity; permanent stagnation, in which humanity survives but never reaches maturity, for instance after an unrecovered collapse; flawed realisation, in which maturity is reached in a dismally and irremediably flawed form; and subsequent ruination, in which good prospects are later destroyed.
The question. Should the reduction of risks to humanity's long-term survival and potential take priority over other ways of doing good, and if so, on what kind of argument: an empirical claim about probabilities, a philosophical claim about the value of future lives, or a decision-theoretic claim about how to act under uncertainty?
Provenance. The comparative intuition behind the argument is Derek Parfit's. In Reasons and Persons (Oxford: Clarendon Press, 1984), pp. 453–454, as Bostrom quotes it, Parfit compares three outcomes: peace; a nuclear war that kills 99 per cent of the world's existing population; and a nuclear war that kills 100 per cent. Of the second and third he writes: 'I believe that the difference between 2 and 3 is very much greater.' Nick Bostrom reformulated the point as a priority argument in 'Existential Risk Prevention as Global Priority' (2013).
The text's own reasoning. Bostrom uses the comparison to introduce his own claim that the loss in an existential catastrophe includes the value of all the future lives it would prevent, and he puts this in numbers. On his most conservative estimate the expected loss exceeds the value of 10^16 human lives; this, he writes, 'implies that the expected value of reducing existential risk by a mere one millionth of one percentage point is at least a hundred times the value of a million human lives.' He works under the simplifying assumption that, holding quality and duration fixed, the value of a life does not depend on when it occurs.
The case for prioritising such risks is an expected-value argument. Because the potential future contains vast numbers of lives, even very small reductions in the probability of existential catastrophe would, on Bostrom's estimates, outweigh the direct benefit of saving very many people now. He proposes as a rule of thumb the Maxipok principle: 'Maximise the probability of an "OK outcome", where an OK outcome is any outcome that avoids existential catastrophe.' He explains the neglect of these risks by their character as a global public good that benefits future generations who cannot pay for it, by the weak response of moral sentiment to very large numbers, and by their multidisciplinary nature. He also holds that 'The biggest existential risks are anthropogenic and related to potential future technologies.'
Structure. (COT's reconstruction; the three kinds of premise are kept apart) Empirical premises: (1) the future could contain a very large number of worthwhile lives if humanity survives and develops; (2) some present actions can reduce, by some non-negligible amount, the probability of a catastrophe that would prevent this. Value premises: (3) the lives of future people count, and count no less for occurring later; (4) preventing the non-existence of a vast number of worthwhile future lives is a very great good, not merely a matter of indifference. Decision-theoretic premise: (5) we should choose so as to maximise expected value, or at least give great weight to expected value, when the probabilities can be estimated. Conclusion: (6) even very small reductions in existential risk have greater expected value than most other available goods, so their reduction should be a global priority.
Intended conclusion. Reducing existential risk should be treated as a priority for humanity as a whole, because of the size of what an existential catastrophe would destroy rather than because such a catastrophe is likely.
Artificial intelligence is the most discussed case. Vincent Müller's Stanford Encyclopedia entry on the ethics of AI reports arguments that massive negative consequences, 'including existential risk for the human species (XRisk), have significant probability', notes that the concern is sometimes widened to any catastrophic risk to the species, of which AI is only one, as in the work of Toby Ord, and connects it to the control problem.
Objections. Each kind of premise is contested. Against (3), defenders of pure time discounting hold that future well-being may count for less simply because it lies in the future; Lukas Meyer's Stanford entry on intergenerational justice treats whether we have reasons to discount future persons' well-being as an open debate. Against (4), person-affecting views, set out in that entry's discussion of the non-identity problem, hold that an outcome can be worse only if it is worse for someone, which threatens the claim that failing to bring future people into existence is a great loss to anyone. Against (1) and (2), the empirical estimates and the effect of present action on them are uncertain, a point the live text above records with Bostrom's own admissions; and against (5), the fanaticism worry noted above questions whether expected value should govern when probabilities are tiny and stakes vast.
The framework is contested at several points, some acknowledged by Bostrom himself: Maxipok is not an absolute principle, since other moral ends matter; classifying an outcome as flawed realisation depends on contested value judgements and on population ethics; probability estimates are highly subjective; and careless pursuit of safety, for instance through oppressive security measures, could itself raise risk. The entry also reports that some participants question the assumptions of the AI-risk debate and the fanaticism of letting low-probability, high-stakes outcomes dominate decisions.
Replies. On discounting, Meyer's entry reports that Sidgwick, Ramsey, Rawls and Parfit all reject pure time discounting, the view that future people's claims count for less merely because they are future. On person-affecting views, Bostrom argues that even from a person-affecting perspective the assessment of flawed realisation should turn on whether conditions give a good prospect of realising a large amount of value over the future, so that at least part of his case does not depend on impersonal value. On the empirical premises, defenders reply that the argument requires only that some reduction be possible, not any particular estimate, a reply whose force depends on the size of the reduction actually available.
Key thinkers and texts. Derek Parfit, Reasons and Persons (1984), the closing pages of Part Four; Nick Bostrom, 'Existential Risk Prevention as Global Priority', Global Policy (2013); Toby Ord, The Precipice (2020); Henry Sidgwick, Frank Ramsey and John Rawls on time preference, as reported by Meyer.
Current status. The priority claim is widely discussed and widely contested. Its defenders and critics agree that it combines premises of different kinds, and much of the debate concerns which of them carries the weight: the value of merely possible future people, the treatment of very small probabilities, or the empirical estimates.
Related entries on this site. Person-affecting view, Procreation asymmetry, Non-identity problem, Pascal's mugging, Effective altruism and the AI control problem take up the separate premises; Repugnant conclusion and Mere addition paradox bear on premise (4).
On this site it belongs to Coordinates 3.3.4 (Ethics of Technology and AI), 3.2.1 (Consequence-Based Approaches) and 6.20 (Philosophy of Death).
See also the glossary entries Technological singularity, Orthogonality thesis, Value alignment, Expected utility theory and Repugnant conclusion.
The questions it belongs to
- 3.3.4 Ethics of Technology and AIWho is responsible when a machine causes harm and no one intended it?
- 3.2.1 Consequence-Based ApproachesIf an act produces the best outcome, can it be wrong?
- 6.20 Philosophy of DeathCan something be bad for you if you are not there for it?
Thinkers
To read
- On What Matters — Derek Parfit
- Reasons and Persons — Derek Parfit
- Superintelligence: Paths, Dangers, Strategies — Nick Bostrom
Sources and further reading
- Nick Bostrom, 'Existential Risk Prevention as Global Priority', Global Policy 4 (1), 2013 (author's copy)
- Vincent C. Müller, 'Ethics of Artificial Intelligence and Robotics', Stanford Encyclopedia of Philosophy (substantive revision 27 March 2026)
- Lukas Meyer, 'Intergenerational Justice', Stanford Encyclopedia of Philosophy (substantive revision 17 July 2026)
Cite this page
Responsibility: Coordinates of Thought Editors. Each statement was drafted from the sources listed above and checked against them in a separate pass before publishing; this page is not peer-reviewed by outside specialists. Source-checked 15 November 2026. How to cite this site · How we write and check.
APA (7th edition)
Coordinates of Thought Editors. (2026, November 15). Existential risk (Arguments). Coordinates of Thought. Retrieved [date accessed], from https://coordinatesofthought.com/arguments/existential-risk/
MLA (9th edition)
“Existential risk (Arguments).” Coordinates of Thought, 15 Nov. 2026, coordinatesofthought.com/arguments/existential-risk/. Accessed [date accessed].
Chicago (17th edition)
Coordinates of Thought Editors. “Existential risk (Arguments).” Coordinates of Thought. Source-checked November 15, 2026. https://coordinatesofthought.com/arguments/existential-risk/.
BibTeX
@misc{cot-glossary-existential-risk-2026,
author = {{Coordinates of Thought Editors}},
title = {Existential risk (Arguments)},
howpublished = {Coordinates of Thought},
year = {2026},
month = nov,
url = {https://coordinatesofthought.com/arguments/existential-risk/},
note = {Source-checked 15 November 2026}
}
RIS
TY - ELEC AU - Coordinates of Thought Editors TI - Existential risk (Arguments) T2 - Coordinates of Thought PB - Coordinates of Thought PY - 2026 DA - 2026/11/15 UR - https://coordinatesofthought.com/arguments/existential-risk/ LA - en ER -
For reference managers (Zotero, Mendeley, EndNote): BibTeX file · RIS file