Amazon launches Mechanical Turk

A name borrowed from a chess machine

In the 18th century, Wolfgang von Kempelen built a chess-playing automaton known as the Mechanical Turk. It appeared to play chess by itself, but was later revealed to conceal a human chess master inside^[1]. The trick is simple: a machine that seems to think is actually a person working out of sight. Amazon borrowed the name for a service that runs the same trick in the open.

Launch

Amazon Mechanical Turk, usually shortened to MTurk, went live in 2005 and is owned by Amazon^[1]. It is a crowdsourcing microtask marketplace operated under Amazon Web Services^[1]. Venky Harinarayan is named as its creator, and the concept was first proposed by Jeff Bezos^[1].

How a task moves through the system

The employers are called requesters. They post jobs called Human Intelligence Tasks, or HITs: identifying specific content in an image or a video, for example. The workers are called Turkers, and they complete those tasks for a fee^[1]. A microtask in this sense is a piece of work small enough that one person can finish it in a short sitting, which is what makes it possible to distribute a large job across many hands.

Amazon coined the term "artificial artificial intelligence" for processes that outsource parts of a computer program to humans^[1]. The label is oddly literal: a program that cannot settle something on its own hands the decision to a person, and takes the answer back as if it had computed it.

Growth

By March 2007 there were more than 100,000 workers on the marketplace, in over 100 countries. By January 2011 there were over 500,000 registered workers from over 190 countries^[1]. The numbers describe a workforce that grew from a small experiment into something spread across much of the world.

Inside a dataset

The 2009 ImageNet paper records one use of the service. Candidate images were collected from the Internet and verified by people through Amazon Mechanical Turk, and across 80 randomly sampled synsets the labelling precision averaged 99.7%^[2]. Verifying candidate images means making a very large number of small judgements — is this picture really of this thing? — and the paper states that this step went through the marketplace.

Why it matters

Two things about the launch turned out to matter. One is organizational: a big block of human judgement could be cut into many small pieces that different people pick up and finish, with the payment attached to each piece. The other is conceptual. "Artificial artificial intelligence" names a pattern in which a system appears automatic while part of the work is done by people — exactly what the 18th-century chess machine was doing, except that here the people are visible, recruited and paid. The ImageNet case shows what that pattern is good for: the workers supply the kind of check that a program could not yet make reliably, and the result is a labelled dataset that others can build on.