Whisper

What it is

Whisper is an automatic speech recognition system that OpenAI released as open-source software on 21 September 2022 under the MIT License^[1]. It takes audio in and produces text: it can transcribe speech in the language being spoken, and it can also translate speech into English.

How it works

The model is an encoder-decoder transformer. Audio is resampled to 16,000 Hz and converted into an 80-channel log-magnitude Mel spectrogram, the image-like representation of sound that the encoder reads; the decoder then generates the text^[1]. The architecture is fairly standard. What is unusual is the scale and mixture of the supervision: training used 680,000 hours of labelled audio-transcript pairs, of which 117,000 hours covered 96 non-English languages and 125,000 hours were translation data into English^[1].

That mixture shapes how the model is used. Whisper was not built as a separate system for each language or each task; it was trained as one multilingual, multitask model that learns transcription, language handling and translation together from a very large body of audio and transcripts.

The paper

The work was described in "Robust Speech Recognition via Large-Scale Weak Supervision", a paper dated December 2022 with Alec Radford as first author^[2][3]. The paper reports scaling training to 680,000 hours of multilingual and multitask supervision^[6]. Its central claim concerns generalisation: the resulting models often matched prior fully supervised results in zero-shot transfer, without any fine-tuning, and approached human accuracy and robustness^[6]. In practice that meant a single trained model could be pointed at new datasets and domains rather than being retrained for each one. OpenAI released the models and inference code as a foundation for further work on robust speech processing^[6].

Releases and updates

The weights were revised several times: large-v2 on 8 December 2022, large-v3 in November 2023 at OpenAI's DevDay, and Whisper Large v3 Turbo on 30 September 2024^[1][5]. Alongside the open weights, OpenAI offered a hosted service — a Whisper API for transcription and translation debuted on 1 March 2023^[1]. In March 2025 OpenAI released transcription models based on GPT-4o and GPT-4o mini with lower error rates than Whisper^[1], and on 26 August 2026 OpenAI announced that the whisper-1 API model would be shut down on 26 February 2027^[4].

Where it sits

Whisper's place in speech recognition comes from two things at once. The models and the inference code were published openly under a permissive licence, so that others could run them locally and build on them^[1][6]. At the same time the same technology was offered as an API, which made transcription and translation available without running the model yourself^[1].

The later history shows the split clearly: the open weights continued through large-v2, large-v3 and v3 Turbo, while the hosted line moved on to GPT-4o-based transcription models and the original whisper-1 endpoint was scheduled for retirement^[1][4][5]. Whisper therefore remains both a widely reusable open model family and the starting point for the speech features that came after it.