The trolley problem began in Philippa Foot's 1967 paper, with a runaway tram that could be steered toward one man instead of five. MIT's Moral Machine put it to driverless cars and logged about 40 million decisions in 233 countries and territories.

The thought experiment every driverless-car argument eventually runs into began as a side example in a 1967 paper about abortion. The philosopher Philippa Foot put a runaway tram into "The Problem of Abortion and the Doctrine of the Double Effect" (full text) to test a principle Catholics were using against abortion.

Half a century later an MIT-led team rebuilt the dilemma around self-driving cars. The results, published in Nature on 24 October 2018 (the paper), rest on about 40 million decisions from people in 233 countries and territories.

Foot sets the tram up in a single sentence. The man is "the driver of a runaway tram which he can only steer from one narrow track on to another", with five men working on one track and one on the other, and anyone on the track he enters bound to be killed.

Nobody stands at a lever anywhere in the paper.

She didn't call it the trolley problem. That label came in 1976, from the philosopher Judith Jarvis Thomson's paper "Killing, Letting Die, and the Trolley Problem" (record) in The Monist.

When a car can't spare everyone, who writes the rule? The MIT study leaves that open.

Foot's question was narrower. The doctrine of double effect, as she defines it, turns on the gap between what a person foresees will follow from an action and what he strictly intends, as end or as means.

On the doctrine, it can be permissible to bring about, through what she, borrowing Bentham's term, calls oblique intention, a harm you may not directly intend.

She set the tram beside a judge facing a mob. The rioters will kill five hostages unless a culprit is produced, and the judge can stop that only by framing and executing an innocent man. Both cases trade one life for five, yet most people say the driver should steer for the one and recoil from the frame-up.

Double effect offers an answer, and Foot puts it this way: "it is one thing to steer towards someone foreseeing that you will kill him and another to aim at his death as part of your plan." The driver foresees the death. The judge needs it.

Foot once found that conclusive, but by 1967 she thought the real work was done by the difference between a duty to avoid injuring people and a duty to bring them aid.

The driver faces a conflict of negative duties, injure five or injure one, and should do the least injury he can. The judge would be inflicting injury to give aid, a different kind of choice.

Her verdict is that the intention distinction "plays only a quite subsidiary role" in these cases. She adds that rejecting double effect doesn't force anyone to the conclusion that the size of the evil "must always be our guide". Counting heads, as consequentialism does in judging actions by their outcomes, was never her point.

The Moral Machine turned it into counting. Built at MIT by Edmond Awad, the paper's lead author, and colleagues, the site asked visitors to judge driverless-car scenarios and pick which group the car should spare. The scenarios varied nine factors, and the paper counts about 40 million decisions in ten languages from over 2.3 million people.

The strongest preferences, the paper reports, were for sparing humans over animals, sparing more lives and sparing young lives. Clustering the country-level results produced three groups, which the paper labels Western, Eastern and Southern.

Awad, a senior lecturer in the Department of Economics and the Institute for Data Science and AI at the University of Exeter, said in an email that the scale let the team compare the strength of each factor's pull and see how widely it was shared.

"A common misreading is that our results showed only cultural differences," he said. What the team found was "broad agreement in the direction of preferences, with variations in their strength."

His examples: respondents across countries generally preferred saving younger lives over older ones, and women over men, Awad said. The preference for the young was weaker in collectivist countries, he said, and the preference for saving women was stronger in the group of countries that includes France and several of its former colonies.

A comment posted on Nature's site in August 2019 called it "highly problematic" to cluster China, Japan and South Korea under the concept of Confucianism, and argued that an individualist versus collectivist split isn't a useful predictor of ethical preferences. It is a reader comment in the article's public comments section, signed by three people, not a peer-reviewed reply.

"Our sample is self-selected," the paper states, meaning anyone who found the site could play.

The paper's Extended Data captions on the Nature page say most of those who filled in the site's demographic survey were male, most had been to college and most were in their 20s or 30s, and the US sample over-represents men and younger people compared with the American Community Survey. The result is a poll of no country.

A lot of players questioned the premise. Connecting the dilemmas to cars "prompted people to question the setup," Awad said in the email, and "many asked why the car couldn't stop or why can't we save everyone."

Foot had already conceded the artificiality in 1967. Real life, she wrote, almost never delivers the certainty the example assumes, and the man on the track might find a foothold on the side of the tunnel and cling on as the tram passes. She put the certainty in on purpose, to pull the moral question away from the odds.

"Whether these cases are rare, very rare, or extremely rare, we need to agree beforehand on how they should be solved," the paper says.

And the authors don't think the 40 million clicks should necessarily settle it. In their words, "the ethical preferences of the public should not necessarily be the primary arbiter of ethical policy."

That leaves the result in an odd spot. Countries leaned the same way on direction, by Awad's account, and differed on strength, and the people who gathered the votes say votes shouldn't necessarily be the main input.

Foot's driver had one choice, injure five or injure one. A car ranking a young pedestrian against an old one is answering a question her tram never asked.

The Nature abstract says the authors discuss how these preferences can contribute to developing global, socially acceptable principles for machine ethics.