Gemini 1.5 Pro

A mid-size model built for long inputs

Gemini 1.5 Pro is a mid-size multimodal model in the Gemini 1.5 generation from Google. It was announced on 15 February 2024 in a post by Demis Hassabis that carried a note from Sundar Pichai ^[1]. Multimodal means the model is built to take in more than one kind of input — text, images, audio, video — instead of text alone, and "mid-size" places it in the middle of the generation rather than at its largest or smallest end.

Mixture of experts

The model used a mixture-of-experts (MoE) architecture ^[1]. In an MoE model the network is divided into many separate sub-networks, often called experts, and a routing step picks which few of them handle each piece of input. Only a fraction of the network runs for any given token, so the model can hold a large total number of parameters while keeping the computation per step smaller than a dense model of the same total size would need.

The context window

The model shipped with a standard context window of 128,000 tokens, with up to 1 million tokens available in private preview ^[1]. A context window is the material a model can keep in view at once, prompt and answer together; a token is the unit of text it reads and writes, roughly a fragment of a word. Context length matters because anything outside the window is invisible to the model when it answers.

Google put the 1 million figure in concrete terms: it said that much room let the model process about one hour of video, 11 hours of audio, codebases of more than 30,000 lines, or more than 700,000 words in a single prompt, and that in research it was tested at up to 10 million tokens ^[1].

A long window is only useful if the model actually uses material from the middle of it. In "Needle In A Haystack" tests, which hide a piece of text inside a much longer block and ask the model to retrieve it, Gemini 1.5 Pro found the embedded text 99% of the time in blocks of up to 1 million tokens ^[1].

Benchmarks and a language with fewer than 200 speakers

Google said the model outperformed Gemini 1.0 Pro on 87% of the benchmarks used to develop its large language models and performed at a broadly similar level to Gemini 1.0 Ultra ^[1].

A second demonstration showed what the context window was for. Given a grammar manual for Kalamang, a language with fewer than 200 speakers, the model learned to translate English into Kalamang at a level similar to a person learning from the same content ^[1]. The manual sat in the prompt, and the model worked from it rather than from anything learned in training.

Longer context and product integration

At Google I/O on 14 May 2024, Google made a 2-million-token context window available by waitlist to API developers and Google Cloud customers, and said Gemini 1.5 Pro was being integrated into Gemini Advanced ^[2]. Gemini 1.5 Pro powered Deep Research in Gemini Advanced when that feature launched on 11 December 2024 ^[3].

Place in the line

Gemini 1.5 Pro was succeeded by Gemini 2.0 Pro and Gemini 2.5 Pro in 2025, and the Gemini 1.5 models have since been discontinued ^[4]. What the 1.5 generation put forward, and what later models kept, was the idea that how much a model can hold at once is a capability in its own right rather than only a technical setting: a model that can read an hour of video or a whole codebase in one pass can be asked questions that a shorter window makes impossible.