Skip to content
Coordinates of ThoughtHome of Philosophy
Join
Discussion

On a railway every fault must turn the signal red. Should our moral rules be designed to fail safe?

Liu Mingqiao Contributor
· 3 replies
Technology & the futureAnalyticJohn Rawls#engineering-ethics#fail-safe#precautionary-principle#risk
Follow

Its coordinates: 6.16 Philosophy of Technology

For forty years I designed signalling systems around one principle: when anything fails, the system must fall into its most restrictive state. A broken wire, a dead relay, a lost power supply: each must show red, never green. The old semaphore arms were weighted so that if the cable snapped, gravity pulled them to danger. A track circuit reports the line as occupied when the circuit breaks, even if no train is there. We accepted many needless stops so that we would never have one wrong "go".

Since I retired I notice how often people argue for the same rule outside railways. If we are unsure a new chemical is safe, stop. If we are unsure a building will hold, close it. This is called the precautionary principle.

But on a railway, stopping really is safe: a train standing at a red signal harms nobody. In most of life there is no such state. Refusing a new medicine is also a choice with victims; they are only harder to count. Rawls defended choosing the option with the best worst outcome, but only under special conditions, when probabilities are unknown and the worst outcomes are intolerable.

So my question: is "fail safe" a moral principle, or only an engineering trick that works because railways have a red light and life does not? The card on philosophy of technology is a good place to start.

LikeReply
SaveFollow3 replies

Join the conversation: members reply here, and joining is free. Reading is always free.

Join free to reply Sign in

3 replies

  1. Anand Kurup

    Fellow

    Our lake trust meets this every monsoon. When we are unsure whether opening a sluice will flood the breeding grounds downstream, the precautionary instinct says keep it shut. But a shut sluice floods the colony upstream. There is no red light, only two kinds of harm.

    What we have learned is to ask which harm can be undone. A flooded colony recovers in weeks; a lost fish population may never come back. So I would rephrase your principle: in doubt, choose the error you can reverse. That is still a fail-safe idea, but the safe state is defined by reversibility, not by stopping.

  2. Nima Tsering Gurung

    Contributor

    Your railway has something mountains don't: the train can wait at the signal as long as it needs. On a ridge, turning back is usually the cautious choice, but not always. If a storm is coming, sitting still is the dangerous state, and the descent itself carries risk. Guides who treat "stop" as automatically safe sometimes get people killed.

    So I share your doubt. The fail-safe rule depends on there being a state in which time is on your side. Where waiting itself costs something, precaution has to be calculated, not assumed.

  3. Ethan Mercer

    Contributor

    Rawls is the right reference, and his conditions matter more than the rule. In A Theory of Justice he says maximin is rational when you cannot estimate probabilities, when you care little about gains above the guaranteed minimum, and when the alternatives risk outcomes you could not accept. Harsanyi replied that applied across the board, maximin would stop you ever boarding a plane.

    Your railway meets all three conditions: nobody cares much about ten minutes saved, and a collision is unacceptable. Most policy questions meet perhaps one. That is a decent test for when "fail safe" is borrowing authority it hasn't earned.