How much should one result change your mind? Thinking in degrees of belief

Last year a friend called me in a panic. A screening test had come back positive for a condition that, she had read, the test detects ninety-nine per cent of the time. She was sure she had it. I asked her two questions she had not thought to ask: how common is the condition in people like her, and how often does the test flag people who do not have it? The answers changed everything, and they show why I think the most useful idea in modern epistemology is that belief comes in degrees.

All or nothing, or by degrees?

Traditional epistemology mostly asks whether you know or believe something: yes or no. Formal epistemology asks a different question: how confident should you be, and how should that confidence change when new evidence arrives? Most of the field starts from Bayesian answers, named after Thomas Bayes, the eighteenth-century minister whose essay on probability was published after his death, in 1763, by his friend Richard Price.

The basic idea is easy to state. Before the evidence, you have some level of confidence in a hypothesis, your prior. Evidence then raises or lowers it according to how much more likely that evidence would be if the hypothesis were true than if it were false.

My friend's test

Here is a version of her case with round numbers. Suppose the condition affects one person in a hundred among people like her. The test catches 99 per cent of real cases, but it also gives a positive result to 5 per cent of people who do not have the condition.

Imagine ten thousand such people. A hundred have the condition, and the test catches 99 of them. Of the 9,900 who do not, 5 per cent, which is 495 people, also test positive. So of 594 positive results, only 99 are real. Her chance of having the condition, given a positive test, is about one in six, not ninety-nine in a hundred.

Her mistake has a name: base-rate neglect. The 99 per cent figure answers the question "how likely is a positive result if I have the condition?" She needed the answer to the reverse question, "how likely is it that I have the condition, given a positive result?" Bayes's rule is the bridge between the two, and the base rate, how common the condition is in the first place, is the pillar it rests on. (She had the follow-up test. She was fine.)

I see the same error at work every week. In credit scoring, a model flags an applicant as "high risk", and someone reads the flag as near-certainty without asking how many flagged applicants actually default. The arithmetic is identical, and so is the human cost of getting it wrong.

Why should confidence obey the rules?

A natural question is why anyone's degrees of belief should obey the rules of probability at all. Frank Ramsey and Bruno de Finetti gave an influential answer in the first half of the twentieth century, tying degrees of belief to the bets a person would accept. The Dutch book argument goes like this. If your confidences break the rules, for instance if you are 60 per cent sure it will rain tomorrow and also 60 per cent sure it will not, then a clever bookmaker can offer you a set of bets, each fair by your own lights, that together guarantee you lose money whatever the weather does.

The argument has critics. Most of us are not betting on our beliefs, and if confidence is not really a disposition to bet, the bookmaker is only hypothetical. But I find it a useful picture of what incoherence costs: you can be exploited by your own standards.

Richard Jeffrey added an important refinement. Real evidence is often uncertain. You glimpse something in poor light and become more confident, not certain. Jeffrey showed how to update on evidence of that kind, which is much closer to how most of life arrives.

An ancient anticipation

Degrees of belief are not a modern invention. In the second century BCE the Academic sceptic Carneades, who argued that nothing can be known with certainty, still had to explain how a sceptic could act. According to Sextus Empiricus, he distinguished grades of impression: those that are merely persuasive, those that are persuasive and not contradicted by other impressions, and those that are persuasive, uncontradicted and thoroughly tested. Sextus reports an example: a man enters a dark room, sees a coil of rope, and takes it for a snake. He does not simply act on his first impression; he examines it, and checks again, until he is confident enough to act.

That is not Bayesianism, and I do not want to force the comparison. Carneades had no numbers and no rule for combining evidence. But the core insight is shared. When certainty is unavailable, the rational response is not to suspend all judgement but to hold beliefs with the confidence the evidence supports, and to test them in proportion to what is at stake.

My argument

I want to make a modest claim. You do not need to do calculations to benefit from thinking in degrees. You need three habits.

Ask for the base rate. Before you are impressed by a test, a witness or a model, ask how common the thing is to begin with.

Ask the reverse question. "How likely is this evidence if I'm right?" is not the same as "how likely am I to be right, given this evidence?"

Update by steps, not leaps. A single piece of evidence rarely takes you from doubt to certainty. If it seems to, check whether you have ignored the base rate.

Critics of Bayesianism raise real problems. Where do priors come from? Can two people with different priors reasonably reach different conclusions from the same evidence? Do numbers lend false precision to judgements that are really vague? These are live debates, and some philosophers prefer imprecise probabilities, ranges rather than single numbers, for exactly that reason.

But the alternative, all-or-nothing belief, is worse at the things that matter most in ordinary life: medical results, risk, news that arrives one piece at a time. Carneades's man in the dark room did not need probability theory. He needed the habit of asking how sure he was, and what it would take to be surer. That habit is the best thing formal epistemology has to give the rest of us.