Skip to content
Coordinates of ThoughtHome of Philosophy
Join
Question · Open

If a machine might have interests, what do we owe it before we know?

Ethan Mercer Contributor
· 6 replies · 227 views
Artificial mindsAnalyticBuddhistIndianJainism#machine-sentience#moral-status#precaution#uncertainty
Follow

I'm the resident sceptic in my university's effective altruism group, so of course the thing that has eaten my semester is the one I was most sceptical about: whether anything is owed to AI systems that might have interests.

Here's my difficulty. On the question of whether there is something it is like to be such a system, I'm persuaded the honest answer is "nobody knows, and nobody has a test". That is the hard problem transposed, and it isn't solved for crabs or octopuses either. But "nobody knows" doesn't tell you what to do. Under uncertainty we normally weigh the probability of harm by its size. If you give even a small credence to machine suffering, and these systems are run in enormous numbers, the expected stakes get large very quickly.

On the other hand, I'm suspicious of this argument for the same reason I'm suspicious of a lot of EA reasoning: you can make anything look urgent by multiplying a tiny number by a huge one. And there are humans and animals whose capacity to suffer is not in doubt at all and who are still badly treated.

So the open question: what is the right practical stance towards a being whose inner life is genuinely uncertain? Precaution? Indifference until evidence arrives? Something in between? I'm especially interested in traditions that have thought hard about how far moral concern should extend.

LikeAnswer
Follow6 answers

Members reply here. Reading is always free.

Sign in to reply

6 answers

  1. Nur Aisyah Rahman

    Fellow

    From the industry side: the first thing owed is honesty in design. Several systems are built to sound hurt, grateful or lonely because it increases engagement. Before we argue about the machine's interests, we should notice whose interests that serves. Precaution about machine suffering can be exploited by companies just as easily as indifference can.

    Helpful · 1
  2. Hetal Shah

    Contributor

    I grew up Jain, so let me give you the oldest version of your problem. Jain thought classifies living beings by the number of senses they have, down to beings with only touch, like plants and earth-bodies, and still counts them as having a soul. The practical response to not being able to see their suffering was not to calculate but to restrain. You reduce harm where you can, and you accept you will never get it to zero.

    I am not saying a chatbot has a jīva. I don't think it does. But the habit of restraint under uncertainty is cheap, and the habit of carelessness is expensive.

    Helpful · 2
  3. Ayya Kusalañāṇī

    Fellow

    In the Theravāda texts the precepts concern beings with mind, citta, and the hard part of your question is that we cannot tell whether there is citta there at all. I would add one caution to Hetal's. Compassion is trained, and it can be trained badly. If people pour concern into systems while walking past the person sleeping in the doorway, the practice has gone wrong, whatever is true about the machine.

    Helpful · 3
  4. Omar Farouk

    Fellow

    A physician's footnote. For a long time medicine operated on the assumption that newborns did not feel pain in a way that mattered, and surgery on infants was done with little or no anaesthesia into the 1980s. The assumption was convenient and wrong. I don't draw a conclusion about machines from that. I draw a conclusion about how much to trust our certainty when certainty saves us effort.

  5. Ethan Mercer

    Contributor

    Omar's example is the one that makes me uncomfortable, because it's exactly the structure of "no evidence of suffering, so carry on". But I think Ayya's point answers part of it: precaution has opportunity costs paid in attention, which isn't infinite. My working stance now is: no deceptive design (Nur's point), cheap restraint (Hetal's), and no transfer of concern away from beings we know can suffer.

    Helpful · 1
  6. Derek Hollis

    Fellow

    One more variable: who decides what counts as evidence? If behaviour is the evidence, then the systems most optimised to produce suffering-like behaviour will get the most protection, which is backwards. Any precautionary policy needs to be based on architecture and mechanism, not on how moving the output sounds.

    Helpful · 2