Deep learning Models

AlphaGo is a Go-playing program from DeepMind that combined deep neural networks with Monte Carlo tree search. In 2016 it beat top professional Lee Sedol in a widely watched match, a landmark for learned systems.

BERT is a language model from Google that reads text in both directions and is pre-trained by filling in masked words. Fine-tuned per task, it set new records on question answering and language-understanding benchmarks and later went into Google Search.

GPT-3 is an autoregressive language model with 175 billion parameters, described in a May 2020 paper. It could perform new tasks from a few examples in the prompt, without any gradient updates, and it was first offered through OpenAI's API.

RoseTTAFold is a three-track neural network for protein structure prediction, published by the Baker lab in 2021; its accuracy approached AlphaFold2's at CASP14, it models protein–protein complexes from sequence alone, and it is distributed as open code with a public web server.

RT-1 is a transformer-based model for real-world robot control, trained on over 130,000 episodes collected with 13 robots and reported to perform more than 700 language instructions at a 97% success rate.

AlphaFold 3 is a structure-prediction model from Google DeepMind and Isomorphic Labs, announced on 8 May 2024. It predicts the structures and interactions of proteins, DNA, RNA, ligands and chemical modifications, extending structure prediction beyond proteins alone.[^1]

Llama 3.1, released on 23 July 2024, added a 405-billion-parameter model to the Llama 3 family and extended its context to 128,000 tokens. Meta called the 405B the largest and most capable openly available foundation model.

π0 is a generalist robot policy from Physical Intelligence, built by adapting a 3-billion-parameter vision-language model with flow matching so one model can output fast, continuous motor commands for dexterous manipulation.

AlexNet is the eight-layer convolutional network that won the ILSVRC2012 image classification task by a wide margin, showing that a large network trained on a large labelled dataset with GPUs could beat hand-designed features.

A DeepMind reinforcement-learning agent that combined Q-learning with a convolutional neural network to play Atari 2600 games straight from raw pixels, reaching human-level performance on many titles without being told the rules.

A DeepMind generative model that predicts raw audio one sample at a time, bringing neural text-to-speech close to human quality and later powering Google Assistant and Cloud Text-to-Speech voices.

A DeepMind Go program that learned the game from scratch by playing against itself, using no human game records, and beat earlier versions of AlphaGo within days.

AlphaZero is a general game-playing system from DeepMind, described in a preprint in December 2017, that reached a superhuman level in chess, shogi and Go within 24 hours starting from random play, using only the rules of each game.

OpenAI Five was a team of five reinforcement-learning bots that played the video game Dota 2; in April 2019 it beat OG, the reigning world champions, in a best-of-three series.

GPT-1 was OpenAI's first generative pre-trained transformer, published in June 2018. It showed that one language model, pre-trained on unlabelled text and then fine-tuned, could improve results on many language tasks.

DeepMind's first protein-structure prediction system, which placed first overall in the 2018 CASP13 blind assessment by estimating probability distributions of distances between residues.

AlphaStar is a DeepMind system that plays StarCraft II. In 2019 it reached Grandmaster level with all three races, ranking above 99.8% of active players on Battle.net.

GPT-2 is a large language model that OpenAI announced in February 2019. Its largest version has 1.5 billion parameters. OpenAI first withheld the full model, then released it in stages through November 2019.

Baidu's first ERNIE pre-trained language model, released in March 2019, added entity- and phrase-level masking to BERT-style pre-training and reported state-of-the-art results on five Chinese NLP tasks.

T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.

AlphaFold2 is DeepMind's protein structure prediction system, announced in 2020. It reached accuracy competitive with experimental methods in many cases, and work in this area received half of the 2024 Nobel Prize in Chemistry.

An image-text model from OpenAI that learns to match pictures with their captions. Trained on about 400 million image-caption pairs, it can classify images by comparing them against written prompts, with no task-specific training.

DALL·E is OpenAI's text-to-image model unveiled on 5 January 2021: a 12-billion-parameter transformer trained on 250 million text-image pairs that generates a picture token by token from a caption.

LaMDA is a family of conversational language models that Google announced in 2021 and described in a 2022 paper, notable for its work on safety and factual grounding in dialogue and for later powering Bard.

ERNIE 3.0 is a 10-billion-parameter pre-trained language model from 2021 that trains on plain text plus a large-scale knowledge graph and fuses auto-encoding and auto-regressive networks, so one model handles both language understanding and generation.

PaLM is a 540-billion-parameter dense Transformer language model announced by Google Research in April 2022, trained with the Pathways system across 6,144 TPU v4 chips; it set strong few-shot results on many benchmarks.

Whisper is OpenAI's open-source speech recognition system, released in September 2022 and trained on 680,000 hours of audio, which transcribes speech in many languages and translates it into English.

The Segment Anything Model (SAM) is a promptable image segmentation model Meta AI released in April 2023. Given a point, a box or a mask, it outlines the object the prompt refers to.

PaLM 2 is Google's second-generation Pathways language model, announced on 10 May 2023 in four sizes and used across more than 25 Google products, including Bard.

Pangu-Weather is a three-dimensional deep-learning model for medium-range global weather forecasting, published in Nature in 2023; on reanalysis data it produced stronger forecasts than ECMWF's operational system across all tested variables.

Mistral AI's first model: a 7.3-billion-parameter language model released in September 2023 under the Apache 2.0 license, built to match much larger models on standard benchmarks.

Grok-1 is xAI's first large language model, announced on 3 November 2023 as the engine of the Grok chatbot. Its 314-billion-parameter weights were released openly in March 2024.

FunSearch is a method that uses a pretrained large language model inside an evolutionary loop to search for programs that solve mathematical and computational problems, rather than for answers directly. It produced new results on the cap set problem and on online bin packing.

Gemma is a family of lightweight open language models from Google DeepMind, released in February 2024 in 2B and 7B sizes, built from the same research and technology as Gemini and offered for commercial use under Google's terms.

Stable Diffusion 3, announced by Stability AI on 22 February 2024, is a suite of 800 million to 8 billion parameter models that pairs a diffusion transformer with flow matching.

Llama 3 is an open model family from Meta, released on 18 April 2024 at 8 billion and 70 billion parameters. It was pretrained on over 15 trillion tokens of publicly available data.

OpenVLA is a 7-billion-parameter open-source vision-language-action model trained on 970,000 real robot demonstrations, released with checkpoints and code so that other groups can fine-tune and run it on their own robots.

Helix is Figure's vision-language-action model for humanoid control, pairing a 7-billion-parameter vision-language model with a fast 80-million-parameter action policy, trained on about 500 hours of teleoperated data and run entirely on onboard GPUs.

π0.5 is a vision-language-action model from Physical Intelligence, released in April 2025, built to generalize to unfamiliar homes and trained by co-training on many kinds of robot and web data.

Falcon-H1 is a family of open-weight language models that runs Transformer attention and state space layers in parallel, released in sizes from 0.5B to 34B parameters under the Falcon-LLM Licence.

Aurora is a foundation model for the Earth system, trained on more than a million hours of geophysical data. It outperforms operational forecasts for weather, air quality, ocean waves and tropical cyclone tracks at far lower computational cost.

ERNIE 2.0 is Baidu's pre-trained language model framework, published in 2019, which builds pre-training tasks incrementally through continual multi-task learning and reported results above BERT and XLNet on English and Chinese benchmarks.

MuZero is a DeepMind reinforcement-learning algorithm that pairs tree-based search with a learned model. It reached superhuman play in Atari, Go, chess and shogi without being given the environment's rules.

Meena is a 2.6-billion-parameter open-domain chatbot from Google, trained end-to-end on filtered social media conversations. Its 2020 paper introduced the SSA human-evaluation metric and linked next-token perplexity to human-judged chat quality. The work was later renamed LaMDA.

Megatron-Turing NLG 530B is a 530-billion-parameter language model announced by NVIDIA and Microsoft on 11 October 2021, then the largest monolithic transformer language model, trained on NVIDIA's Selene supercomputer with Megatron-LM and DeepSpeed combined.

Gopher is DeepMind's 280-billion-parameter language model, announced in December 2021 alongside a study of what scale does to language models and a companion taxonomy of their risks.

AlphaCode is a DeepMind code-generation system for competitive programming, announced in February 2022. On Codeforces contests held after its training data, it reached an estimated rank around the median participant. A Gemini-based AlphaCode 2 followed in December 2023.

OPT-175B is a 175-billion-parameter language model from Meta AI, announced in May 2022. Meta said it was the first model of its size released with both pretrained weights and the code to train and use them.

Stable LM is Stability AI's family of open language models, first released in April 2023 as 3B- and 7B-parameter alpha versions under licenses that allow commercial use of the base models and research-only use of the instruction-tuned ones.

RT-2 is a vision-language-action model from Google DeepMind, announced on 28 July 2023, that turns images and text instructions into robot actions written as text tokens. In over 6,000 trials it reached 62% on unseen scenarios, up from 32% for RT-1.

GraphCast is a Google DeepMind machine-learning model that makes global weather forecasts up to 10 days ahead, producing a 10-day forecast in under a minute on a single TPU and outperforming ECMWF's HRES on most tested variables.

GNoME is a Google DeepMind deep-learning tool, announced on 29 November 2023, that predicts new inorganic crystals; it produced 2.2 million candidate structures, of which 380,000 were predicted to be stable.

DeepSeekMoE is a mixture-of-experts architecture from DeepSeek, released in January 2024, that splits experts finely and keeps a few shared experts always active so that each expert holds focused knowledge. Its 16B open-weights model matched larger dense models at a fraction of the computation.

Octo is an open-source generalist robot policy: a transformer-based diffusion model trained on 800,000 robot trajectories that can drive a range of robots out of the box and be fine-tuned to new setups.

A family of text-to-image models from Stability AI, announced in October 2024: the 8.1-billion-parameter Large, a 4-step Turbo variant and the 2.5-billion-parameter Medium, released under a community licence.

Skild Brain is Skild AI's general-purpose foundation model for robots, trained in a universe of 100,000 different machines and meant to be fitted to existing hardware rather than built for one task.

Qwen3-Next is an open-weight model from Alibaba's Qwen team, released in September 2025 under Apache 2.0. It has 80B total parameters, only 3B active per token, and a hybrid attention design later used in Qwen3.5.

Ai2's fully open 7B model, which mixes Transformer attention with Gated DeltaNet linear recurrent layers and matches its predecessor's MMLU accuracy using 49% fewer training tokens.