What happened, by date
-
2014
Amazon builds an AI tool to rank applications: 100 résumés in, the top 5 out (as reported by Reuters).
-
The mechanism
the tool learns from ten years of résumés received by the company, most of them from men. It infers what a “good” profile looks like: like yesterday’s.
-
2015
internal finding: the system is not neutral. It penalizes résumés containing the word “women’s” (for example “women’s chess club captain”) and downgrades graduates of two women’s colleges.
-
Early 2017
the team is disbanded, the tool abandoned.
-
October 2018
the story is made public by Reuters.
What HUMICS would have put on the table, beforehand
HUMICS examines an AI project from the human angle — impact on people, explainability of a decision, sustainability. On such a sensitive HR use, the grid would have asked, as early as 2014, the questions that ended up costing years: can a ranking be explained? can it be contested? who monitors the effect on groups of candidates? Asked beforehand, the question costs 45 minutes. Asked afterward, years.