Skip to content
Coordinates of ThoughtHome of Philosophy
Join
Debate

Thesis Tiny probabilities of vast payoffs should simply be ignored, however large the expected value

Its coordinates: 2.6 Rationality · 2.8 Formal Epistemology

Where it stands

Our effective altruism group had its usual Thursday argument, and I lost it, so I'm bringing it here.

The setup is Pascal's mugging, a puzzle named after Pascal's wager. A stranger with no weapon asks for your wallet. In return he promises, by powers he can't demonstrate, a thousand quadrillion years of bliss. You think he is almost certainly lying. But "almost certainly" isn't "certainly", and if your credence that he's honest is even one in a trillion trillion, the expected value of handing over the wallet beats keeping it. And he can always raise the number until it does.

My claim: a rational agent should ignore possibilities below some very small probability, whatever payoff is attached to them. That isn't a fudge. It's how every sane person already lives. Nobody insures against a meteorite landing on their breakfast.

The obvious objection, which my friends pressed hard, is that any cut-off is arbitrary, and that you could slice a serious risk into many tiny ones, each below the threshold, and then ignore the lot.

The stakes aren't academic for us. A good deal of longtermist reasoning multiplies very small chances of changing the far future by astronomically large numbers of future people. If I'm right, much of that reasoning goes too. Am I?

LikeReplyFollow1 reply

Join the conversation: members reply here, and joining is free. Reading is always free.

Join free to reply Sign in

1 reply

For the thesis 0

No replies on this side yet.

Against the thesis 1

  1. Wanjiru Njenga

    Against Contributor

    Keep the maths and fix the prior. The mugger's trick works only if your credence in his promise stays roughly fixed as the promised payoff grows. It shouldn't. A claim to deliver a thousand quadrillion years of bliss is a claim to extraordinary powers, and the bigger the claim, the less likely it is to be true. One standard reply is that your prior should shrink at least as fast as the promised value rises, so the product never runs away.

    That handles the mugger without a threshold, and without your arbitrariness problem. It doesn't save every longtermist argument, but it sorts them by the right question: how credible is this particular causal story?