Microsoft demonstrates deep-learning speech translation

Background

Before this demonstration, speech recognition had improved slowly for decades. Rick Rashid, Microsoft's chief research officer, described the state of the art bluntly: speech recognisers had until recently understood only 75 to 80 percent of the words a person might say during an ordinary conversation ^[2]. He characterised the field's last big accuracy jump as the arrival of hidden Markov modelling in 1979, a technique that had shaped the following decades of work ^[2].

The demonstration

On 25 October 2012, at a conference in Tianjin, China, Rashid demonstrated a speech recognition and translation system ^[2]. Microsoft Research described what the audience saw as the conversion of Rashid's spoken English words into computer-generated Chinese speech ^[1]. In other words, the pipeline took audio of a person speaking English, recognised the words, translated them, and produced spoken Chinese output rather than text on a screen.

How it was built

Microsoft Research said the breakthrough was patterned after deep neural networks, and that the approach significantly reduced errors in both spoken and written translation ^[1]. The training burden was described in modest, concrete terms: teaching the system to handle Rashid's own voice took about an hour of him speaking English, plus a few hours of a native Chinese speaker ^[2]. That gave the demonstration a personal quality — the voice being recognised and the voice producing the Chinese output were both drawn from a small circle of people rather than a large crowd-sourced corpus.

Results

In his account of the system's accuracy, Rashid said Microsoft's recogniser could correctly identify 86 to 88 percent of the words in arbitrary speech ^[2]. Compared with the 75 to 80 percent figure he cited for conversational speech, the gain was substantial. Rashid framed it as the most dramatic change in accuracy since the introduction of hidden Markov modelling in 1979 ^[2].

Aftermath

NBC News reported that Rashid published a blog post about the demonstration on 8 November 2012, roughly two weeks after the Tianjin talk ^[2]. The same report placed the work in a wider context: Google had used the same deep-learning technique earlier in 2012 to build a computer that taught itself to recognise cat pictures ^[2]. Together, the two examples pointed to a shift in method — instead of hand-engineered rules and statistical models tuned feature by feature, systems were being trained on data with deep neural networks — and the speech demonstration showed that the shift could be applied to language as well as to images.