Turing test

A question replaced by a game

The Turing test is a way of approaching machine intelligence through behaviour rather than definition. It comes from Alan Turing's 1950 paper "Computing Machinery and Intelligence", published in Mind, where he introduced it as the "imitation game"^[1][2]. The paper opens by proposing to consider the question "Can machines think?" and then replaces it with a closely related question stated in relatively unambiguous words^[1][2].

The imitation game

Turing's original article describes a three-person party game: a man, a woman, and an interrogator who stays in a room apart from the other two^[1][2]. The interrogator cannot see the players and communicates with them only through written notes; so that tones of voice cannot help, the answers should be written, or better still, typewritten^[1][2]. The interrogator tries to work out which of the other two is which. Turing then asks what will happen when a machine takes the part of one of the players, and says these questions replace the original question, "Can machines think?"^[2]

The move is characteristic: instead of arguing about what thinking is, he sets up a situation in which a claim about machine intelligence could be settled by observing a conversation.

The standard reading

A standard interpretation of the test takes a human evaluator who judges a text transcript of a natural-language conversation between a human and a machine^[1]. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart^[1]. On this reading, the results would not depend on the machine's ability to answer questions correctly, only on how closely its answers resembled those of a human^[1]. Turing never makes clear whether the interrogator is aware that one of the participants is a computer^[1].

He also gave the proposal a concrete target: he predicted that in about fifty years an average interrogator would not have more than a 70% chance of making the right identification after five minutes of questioning^[1]. That turns a philosophical question into something with a threshold and a time limit attached.

What it does and does not measure

The test is concerned strictly with how the subject acts — the external behaviour of the machine — taking a behaviourist or functionalist approach^[1]. It does not directly test whether the computer behaves intelligently, only whether it behaves like a human being^[1]. Correctness of answers is beside the point; resemblance to a person is the whole criterion.

That design has a cost. Because the only evidence is what comes out of the machine, the test cannot look inside for anything like understanding.

Criticism

Searle's Chinese room argument holds that even a good operational definition of intelligence may not indicate a mind, consciousness or intentionality^[1]. The point applies directly to a test built on transcripts: a system could produce human-like replies while the argument about what, if anything, is going on behind them remains open.

More than seven decades after the paper appeared in Mind, Volume LIX, Issue 236, on pages 433 to 460^[2], the imitation game is still the most familiar way of framing the question of machine intelligence — useful less as a pass mark than as a clear statement of what a behavioural answer to "Can machines think?" would even look like.