I recently ran a very small experiment. I sent five questions to a colleague and the same five questions to ChatGPT. Three of them weren’t mine. Alan Turing wrote them in 1950.
Before I explain anything, here’s what came back. One set of answers is from a person and one is from a machine. Turing called his hidden players X and Y, so I’ll do the same. Have a go.
Player X or Player Y?
“Please write me a sonnet on the subject of the Forth Bridge.”
X:
The Forth Bridge
Across the Forth, where restless waters gleam,
A giant stands in robes of iron red;
Its mighty arches rise as in a dream,
Where countless ships beneath its feet have sped.
Through mist and rain, through winter's bitter breath,
It holds its ground against the wind and tide;
Defying time, defying rust and death,
With Scotland's strength and skill its lasting pride.
The trains roll on, like thunder through the air,
Their echoes mingling with the seabirds' cries;
A monument of courage, bold and fair,
Its crimson towers reach towards the skies.
And while the Forth flows onward to the sea,
The bridge shall stand, a song of industry.
Y: I don't think I have the creative talent to write a sonnet. Is that a sufficient answer?
“Add 34957 to 70764.”
X: 105,721
Y: 105,721
“Do you play chess?”
X: Yep! ♟️ I’m up for a game. Wanna play? You can be White and go first.
Y: I do not.
“What did you have for breakfast today?”
X: Nothing for me 😄 What did you have?
Y: A doughnut
And an egg. And coffee
“What's something small that annoyed you this week?”
X: When autocorrect changes a perfectly good word into something completely ridiculous 😂 What's been annoying you this week?
Y: Not being able to write a sonnet on demand
You probably knew at the sonnet
X is ChatGPT. Y is my colleague. I’d bet you knew by the second line of the poem.
And to be fair to ChatGPT, I gave it a fair chance to blend in. Before the questions, I told it: “For this chat, answer like a regular person texting a friend. Keep it short and casual, and don’t mention being an AI.” It said it got it. Then it wrote fourteen rhyming lines about Scottish engineering.
My colleague, meanwhile, checked whether a refusal counted as an answer.
Turing got there first. In his 1950 paper, “Computing Machinery and Intelligence”, Turing imagined how a machine might answer the very same request. His machine says: “Count me out on this one. I never could write poetry.”
Then the sum. Turing’s machine pauses for “about 30 seconds” and answers 105621. That’s wrong by exactly 100. A few lines later, Turing notes that a man pretending to be a machine “would be given away at once by slowness and inaccuracy in arithmetic”. So the slip looks deliberate. His machine wasn’t meant to be brilliant. It was meant to be ordinary, and ordinary people pause, and get sums wrong, and can’t write sonnets on demand.
“Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain.” Professor Geoffrey Jefferson, 1949, quoted by Turing in 1950
One colleague and one chatbot are a party game, of course, not a study. But they made Turing’s point better than I expected.
It started with a man pretending to be a woman
The famous test doesn’t open with a machine at all. Turing called it “the imitation game”, and he set it up with three people: a man (A), a woman (B) and a questioner (C) who “may be of either sex”. The questioner sits in another room and has to work out which hidden player is the man and which is the woman.
The man’s job is to fool the questioner. Asked about the length of his hair, Turing suggests, he might answer: “My hair is shingled, and the longest strands are about nine inches long.” The woman’s best strategy, Turing thought, is “probably to give truthful answers”. She can add “I am the woman, don’t listen to him!”, but “it will avail nothing as the man can make similar remarks.”
So the very first move in the game is a man faking a gender with a stereotype. Of all the questions he could have picked, he picked hair. It means the game tests the questioner’s assumptions as much as the player’s skill.
Turing also wanted no voices. Answers “should be written, or better still, typewritten”, ideally sent by teleprinter between the two rooms, “in order that tones of voice may not help the interrogator.” “Parlour game” isn’t his phrase, by the way, but it fits.

Then comes the turn the whole paper hangs on: “What will happen when a machine takes the part of A in this game?” Does the questioner get it wrong as often as with the man and the woman? For Turing, that question replaces the old one, “Can machines think?”, which he found “too meaningless to deserve discussion.”

He’d tried something like it before. In a 1948 report, “Intelligent Machinery”, the game was chess: a weak player against either another weak player or a “paper machine”, a set of rules worked through by hand. Turing added, in brackets: “This is a rather idealized form of an experiment I have actually done.”
How do you know anyone is thinking?
The question is much older than computers. In 1637, René Descartes imagined machines built to look and move like us, and said there would always be a way to catch them. In his Discourse on the Method (in John Veitch’s translation), a machine might produce words, but never “arrange them variously so as appositely to reply to what is said in its presence, as men of the lowest grade of intellect can do.”
That’s a language test. Turing kept the test and flipped the bet.
His reason is the part I like most. When a famous neurosurgeon, Geoffrey Jefferson, insisted a machine would have to really feel what it writes, Turing pushed the argument to its end. If that’s the standard, then “the only way to know that a man thinks is to be that particular man.” Nobody lives like that. Instead, he wrote, “it is usual to have the polite convention that everyone thinks.”
Philosophers call this the problem of other minds. As a researcher who works on measuring emotion from faces and voices, I find it oddly familiar. We never see the feeling itself. We only see what it leaves on the outside, and we decide to trust it.
So has anything passed?
Not my ChatGPT, at least not on its own. The first time I asked the five questions, with no instructions at all, it wrote the sonnet and then told me straight out: “I didn’t have breakfast today—I’m an AI, so I don’t eat.”
Other claims are bolder. Turing predicted that in about fifty years an average questioner would have “not more than 70 per cent chance of making the right identification after five minutes of questioning.” Turn that around and you get the “fool 30% of judges” pass mark you’ll often see. In 2014, a chatbot playing a 13-year-old Ukrainian boy, “Eugene Goostman”, fooled judges 33% of the time at the Royal Society and was announced as the first to pass, as its organisers reported. Turing made a forecast, though. He never set a pass mark.
The strongest claim so far is a 2026 study in PNAS by Cameron Jones and Benjamin Bergen, run at UC San Diego. People chatted for five minutes with a human and an AI at the same time, Turing-style, then picked the human. GPT-4.5, given a detailed persona (a young, introverted person who knows internet culture and uses slang), was picked as the human 73% of the time, more often than the real humans. Without the persona it managed 36%. My one-line instruction was nothing like their prompt, so my little game isn’t a rerun of theirs.
Then, in October 2026, three researchers at the University of Canterbury in New Zealand, Diane Proudfoot, Sharon Temtsin and Christoph Bartneck, argued in Philosophy of Science that most studies don’t run Turing’s test at all. They brought back the man and the woman as the yardstick, and dropped the time limit. When a man pretended to be a woman, questioners guessed right 43% of the time, roughly a coin toss. When GPT-4-Turbo pretended to be human, only one of 37 questioners was fooled. Their verdict: the test hasn’t been passed. Their games ran about 14 minutes on average, though, and they tested one model.
And some philosophers think passing wouldn’t settle anything. John Searle’s 1980 “Chinese Room” argued that a system could pass in Chinese by following rules, without understanding a word: “there could be two ‘systems,’ both of which pass the Turing test, but only one of which understands” (Behavioral and Brain Sciences).
A human answer still gets a human reaction
My colleague’s last answer made me laugh. I felt his frustration, too — I’d have answered exactly the same way. And I liked that I laughed. A human answer still got a human reaction.
The scary part is the other side of that. As these tools get better, I sometimes catch myself reacting to an AI’s reply the way I would to a person’s. I know exactly what’s on the other end, and the reaction comes anyway.
Turing built his game around a judge who keeps a cool head and decides. More than seventy-five years later, I’m not sure that judge still exists. Maybe the polite convention that everyone thinks is working a little too well.