Google launches GNMT in production for Chinese-to-English translation

Background

Much of machine translation has historically been assembled from separate stages: one component splits a sentence into words or phrases, another picks candidate translations for those pieces, and another decides how to reorder them into a sentence in the target language. Each stage is built and tuned on its own, which means an error made early in the chain tends to survive to the end.

A neural approach replaces that chain with a single network trained end to end on pairs of sentences, so that the mapping from source sentence to target sentence is learned as one model rather than assembled by hand.

What was announced

On 27 September 2016, Google Research published a post titled “A Neural Network for Machine Translation, at Production Scale”, announcing the Google Neural Machine Translation system, known as GNMT, together with a technical report titled “Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”.^[1]

The post also announced that the system had gone live in production on one language pair: Chinese to English.^[1] At that point the Google Translate mobile and web apps used GNMT for 100% of their Chinese-to-English machine translation, which came to about 18 million translations a day.^[1]

How the system works

GNMT is a recurrent neural network (RNN) sequence-to-sequence model with three parts: an encoder, a decoder, and attention over the encoded vectors.^[1] The encoder reads the source sentence and turns it into a sequence of vectors, one per position, rather than into a single fixed summary of the whole sentence. The decoder then generates the target sentence from those vectors.

The attention component is what makes the second step practical: at each point while producing a word, the decoder can look back at the parts of the encoded source that are most relevant to that word, instead of having to carry the whole sentence in one internal state. Because the whole thing is one network, it is trained on sentence pairs as a unit.

What made production deployment possible

Running a model like this inside a consumer product is a compute problem as much as a modelling problem: the system has to answer requests fast enough for interactive use, at the volume of a public translation service. Google said the production deployment was made possible by TensorFlow and its Tensor Processing Units, which supplied enough compute while still meeting Translate's latency requirements.^[1]

Reported results

Google reported that GNMT reduces translation errors by more than 55%–85% on several major language pairs, compared with the system it replaced.^[1] Those figures were measured on sampled sentences from Wikipedia and news websites, and scored by bilingual human raters.^[1]

What it meant for Translate

Chinese to English was, at the time, one of more than 10,000 language pairs supported by Google Translate, and Google said GNMT would be rolled out to more pairs over the following months.^[1] That framing matters: the announcement was not just a research result but a statement that a neural translation model could carry the traffic of a large public service, with the compute, latency and quality constraints that come with it.