OpenAI open-sources the Whisper speech-recognition model

Background

Turning speech into text is one of the older goals in machine learning. A speech-recognition system is usually built by collecting audio, transcribing it by hand and training a model on the resulting pairs of sound and text. The bottleneck in that recipe is scale: hand-made transcripts are slow and expensive to produce, and the resulting collections tend to cover a small number of languages and fairly clean recording conditions.

What was released

On 21 September 2022 OpenAI released Whisper, an automatic speech recognition (ASR) system, as open-source software ^[1]. The release came under the MIT License, a permissive licence that allows others to use, modify and redistribute the software ^[1]. Publishing the model itself, rather than only a description of it, meant that outside developers could run it on their own audio instead of going through a hosted service.

How it was built

Whisper was trained on 680,000 hours of labelled audio, much of it multilingual ^[1]. That figure is the centre of the design: instead of assembling a small, carefully curated set of clean recordings, the project leaned on the volume and variety of material already available. The paper that followed describes the method in those terms — models were trained simply to predict transcripts of audio found on the internet ^[3]. The paper's title, "Robust Speech Recognition via Large-Scale Weak Supervision", names the same idea: the supervision is weak in the sense that the transcripts were not produced to a single curated standard, and the scale is meant to compensate for that.

Beyond transcription, the system could translate speech into English, a second task supported by 125,000 hours of translation training data ^[1]. In practice this means audio in one language could be turned directly into English text, rather than only being transcribed in its own language first.

The written account appeared later than the software. The full paper followed on arXiv in December 2022 ^[2], submitted on 6 December 2022 by Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey and Ilya Sutskever ^[3]. So for a few months the model was available to download and use before the detailed description of how it had been trained was public.

Why it mattered

Three properties of the release sit together. First, the licence: MIT terms put few restrictions on reuse, which matters for a component that other products and research projects might want to build on ^[1]. Second, language coverage: the training mixture was largely multilingual, so a single model was intended to handle many languages rather than one ^[1]. Third, the second task: speech translation into English was trained alongside transcription ^[1].

The scale figures are the most concrete part of the story. 680,000 hours of labelled audio is far more than a single team could transcribe by hand, and 125,000 hours of translation data is the same argument applied to the translation task ^[1]. The claim being tested was that a general model fed enough varied, imperfectly labelled audio could handle unfamiliar speakers, background noise and languages without being tuned for each setting — robustness coming from breadth of data rather than from tailoring the system to one benchmark.

Place in the sequence

Whisper is best understood as an open, downloadable artefact rather than a service announcement: a trained speech model and its licence, released first and documented in full some weeks later ^[1][2]. For people outside the lab, that combination — a speech recogniser that transcribes, translates into English, and can be inspected and reused under a permissive licence — is what made the release notable.