Skip to content
Coordinates of Thought
Sign in
Debate

Thesis Algorithmic hiring tools are inherently unjust

Nur Aisyah Rahman Contributor
· 6 replies
JusticeAnalytic#algorithmic-bias#fairness
Follow

Where it stands

I work with data every day, so I don't say this lightly. Algorithms trained on past hiring decisions learn past discrimination. The best-known case: Amazon built an experimental recruiting tool that, as Reuters reported in 2018, learned to penalise CVs containing the word "women's" (as in "women's chess club captain"), and the project was dropped.

There's also a mathematical problem. Several well-known results show that, when base rates differ between groups, you can't satisfy all the common statistical definitions of fairness at once. Every system makes a choice about whose errors matter more.

My thesis: using algorithms to screen job applicants is inherently unjust, because it automates and hides choices that should be made openly by accountable people.

6 replies

For the thesis 2

  1. Deepa Nair

    For Contributor

    The problem is accountability. If a manager rejects me unfairly, I can challenge the decision. If a model does it, the company says the model decided, the vendor says the company configured it, and nobody is responsible. That's an injustice in its own right.

    Helpful · 1
  2. Sana Mir

    For Contributor

    Javier, forcing us to see it would be a gain if anyone outside the company did see it. In practice, the trade-offs are set by engineers and hidden behind trade secrets. Visibility in principle isn't visibility.

    Helpful · 3

Against the thesis 3

  1. Pieter de Jong

    Against Contributor

    "Inherently" is too strong. Human recruiters are biased too, and their biases are invisible and unauditable. An algorithm's bias can at least be measured, tested and corrected. The Amazon case is an argument for audits, not for abolition.

    1. Nur Aisyah Rahman

      Author’s response Contributor

      Pieter, I'll concede that "inherently" might be too strong. What I'd defend is this: deploying such tools without independent audit and a right to human review is unjust. The problem is less the algorithm than the institutions around it.

  2. Javier Morales

    Against Fellow

    The impossibility results apply to humans too. Any decision procedure, human or algorithmic, faces the same trade-off between kinds of error. Algorithms just force us to see it. That's a gain, not an injustice.

    Helpful · 2
  3. Tomasz Wójcik

    Against Fellow

    To be pedantic: the revised thesis is a different and much more plausible thesis. The original claimed injustice is intrinsic to the method. The revision locates it in the institutions. I'd vote against the first and for the second.

    Helpful · 1

Members reply here. Reading is always free.

Sign in to reply