Google DeepMind shows Project Astra at I/O

Background

AI assistants have mostly worked as turn-by-turn systems: a user asks, the model answers, and little of the exchange is kept. Cameras and voice input exist, but an assistant does not usually hold on to a running record of what it has seen and heard.

The announcement

At Google I/O on 14 May 2024, Demis Hassabis presented Project Astra, which Google DeepMind described as its vision for the future of AI assistants^[1]. It was set out as a direction rather than as a finished product, and it arrived alongside other model news from the same event^[1].

How it works

According to Google, Astra continuously encodes video frames and speech into a timeline of events and caches that timeline for fast recall^[1]. The operative word is continuous: instead of waiting for a question and then looking at a single image, the system keeps converting what the camera sees and what is said into a record it can go back to. Caching is what makes going back cheap — the encoded timeline is kept close at hand rather than rebuilt every time it is needed.

The models announced alongside

The same event brought model news that bears directly on assistants. Google announced Gemini 1.5 Flash, which it described as the fastest Gemini model served in its API^[1]. Flash was trained by 1.5 Pro through distillation, a procedure in which a smaller model is trained to reproduce the behaviour of a larger one instead of being trained from scratch^[1]. Google also announced a 2-million-token context window for Gemini 1.5 Pro, offered by waitlist to developers and Google Cloud customers^[1]. A context window is the amount of input a model can hold at once; a longer window lets a system keep far more material in view without discarding it.

The December update

In December 2024, an updated Astra built on Gemini 2.0 added mixed-language conversation, the use of Google Search, Lens and Maps, and up to 10 minutes of in-session memory^[2]. Google also said that a small group of testers would soon try Astra on prototype glasses^[2].

Why it matters

Two things stand out in the way Astra was described. First, the assistant is defined by continuity rather than by a single answer: what makes it useful is the accumulating timeline of what it has seen and heard, plus the ability to recall it quickly^[1]. That is a different design target from a chat box that starts fresh with each question, and it is the part of the idea that later work on assistants would be measured against.

Second, the details mark how early the work was. In-session memory reached up to 10 minutes^[2], and the glasses were prototypes given to a small group of testers^[2]. The framing was a vision of what assistants could become, not a description of something already in hand.

Put together with the Gemini 1.5 announcements from the same day — a faster, distilled model^[1] and a much larger context window^[1] — the presentation sketched an assistant that keeps watching, keeps listening, and keeps a record to look back on. Whether such a system works reliably outside a demo is a separate question, and one the announcement did not answer.