Skype Translator

What it is

Skype Translator is a Microsoft feature that translates conversations in two ways at once: spoken words in a call, and text in an instant-message chat. Microsoft's own description calls it automatic, simultaneous translation of speech and instant-message conversations at web scale ^[1]. The point is not to paste a sentence into a separate translation box, but to let people who speak different languages take part in the same conversation.

The first public preview

15 December 2014 was the first time a preview of Skype Translator was made available for public use ^[1]. The first phase covered just two spoken languages, Spanish and English ^[1]. Text had a much wider reach: users who signed up and ran Windows 8.1 on a desktop or device could use more than 40 instant-messaging languages ^[1]. The requirement to sign up, and the tie to one version of Windows, are signs of a controlled trial rather than a finished product shipped to everyone.

The gap between two spoken languages and more than 40 written ones is not an accident. A text message is already clean characters; the machine only has to deal with words and grammar. Speech has to pass through speech recognition first, which turns sound into text, and accents, background noise and people talking over each other all make that step fail. Once recognition is wrong, the translation is wrong too, so early previews usually start with a small number of language pairs.

Where the technology came from

Microsoft said the preview drew on more than a decade of investment in speech recognition, automatic translation and machine learning ^[1]. Those three items map onto three stages of a pipeline. Speech recognition converts sound into words. Automatic translation turns words in one language into words in another. Machine learning is what lets the first two stages be tuned from data instead of being written out as hand-made rules: the system adjusts its internal settings so that its output moves closer to correct answers on examples it has seen.

The demonstration at launch

The preview was announced together with a classroom demonstration linking a school in Mexico City with one in Tacoma, Washington ^[1]. A classroom is a demanding setting on purpose. Students speak spontaneously, at their own speed, with their own vocabulary; nothing is read from a prepared script. A conversation between two classrooms therefore exposes weaknesses that a carefully rehearsed sentence would hide.

Getting better with use

Because the system relies on machine learning, Microsoft said translation quality would improve as more people used it ^[1]. This is the usual logic of machine-learning systems: more use produces more real speech and more real dialogue, and that material can be used to adjust the models so that the next round of recognition and translation is more accurate. The trade-off is that such gains rarely arrive all at once; they accumulate over time, and the early version is by definition the weakest one.

Where it sits

On 19 December 2014, Microsoft's official blog listed the public launch among the week's news in its round-up ^[2]. Seen as a product, the significance of Skype Translator is that it joined speech recognition, machine translation and machine learning together and put the result inside a tool people already use for ordinary chatting, rather than leaving it in a laboratory demo. Real-time speech translation has long been one of the recognised hard problems in artificial intelligence, and any publicly available version is both a technology demonstration and a large-scale test under real conditions.