Home›Models
Models
Models
Foundation, generative, multimodal and specialised models.
Total 139 · Clear filters
2021
-
Dec 8, 2021
Gopher
Gopher is DeepMind's 280-billion-parameter language model, announced in December 2021 alongside a study of what scale does to language models and a companion taxonomy of their risks.
-
Oct 11, 2021
Megatron-Turing NLG 530B
Megatron-Turing NLG 530B is a 530-billion-parameter language model announced by NVIDIA and Microsoft on 11 October 2021, then the largest monolithic transformer language model, trained on NVIDIA's Selene supercomputer with Megatron-LM and DeepSpeed combined.
-
Jul 7, 2021
OpenAI Codex (2021 model)
OpenAI Codex is a 2021 GPT language model fine-tuned on public code from GitHub. It translates natural language into code, and was introduced alongside the HumanEval benchmark for testing code correctness.
-
Jul 5, 2021
ERNIE 3.0
ERNIE 3.0 is a 10-billion-parameter pre-trained language model from 2021 that trains on plain text plus a large-scale knowledge graph and fuses auto-encoding and auto-regressive networks, so one model handles both language understanding and generation.
-
May 18, 2021
LaMDA
LaMDA is a family of conversational language models that Google announced in 2021 and described in a 2022 paper, notable for its work on safety and factual grounding in dialogue and for later powering Bard.
-
Jan 5, 2021
CLIP
An image-text model from OpenAI that learns to match pictures with their captions. Trained on about 400 million image-caption pairs, it can classify images by comparing them against written prompts, with no task-specific training.
-
Jan 5, 2021
DALL·E
DALL·E is OpenAI's text-to-image model unveiled on 5 January 2021: a 12-billion-parameter transformer trained on 250 million text-image pairs that generates a picture token by token from a caption.
2020
-
May 28, 2020
GPT-3
GPT-3 is an autoregressive language model with 175 billion parameters, described in a May 2020 paper. It could perform new tasks from a few examples in the prompt, without any gradient updates, and it was first offered through OpenAI's API.
-
Jan 28, 2020
Meena
Meena is a 2.6-billion-parameter open-domain chatbot from Google, trained end-to-end on filtered social media conversations. Its 2020 paper introduced the SSA human-evaluation metric and linked next-token perplexity to human-judged chat quality. The work was later renamed LaMDA.
2019
-
Nov 19, 2019
MuZero
MuZero is a DeepMind reinforcement-learning algorithm that pairs tree-based search with a learned model. It reached superhuman play in Atari, Go, chess and shogi without being given the environment's rules.
-
Oct 23, 2019
T5 (Text-to-Text Transfer Transformer)
T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.
-
Jul 29, 2019
ERNIE 2.0
ERNIE 2.0 is Baidu's pre-trained language model framework, published in 2019, which builds pre-training tasks incrementally through continual multi-task learning and reported results above BERT and XLNet on English and Chinese benchmarks.
-
Mar 2019
ERNIE 1.0
Baidu's first ERNIE pre-trained language model, released in March 2019, added entity- and phrase-level masking to BERT-style pre-training and reported state-of-the-art results on five Chinese NLP tasks.
-
Feb 14, 2019
GPT-2
GPT-2 is a large language model that OpenAI announced in February 2019. Its largest version has 1.5 billion parameters. OpenAI first withheld the full model, then released it in stages through November 2019.
-
Jan 2019
AlphaStar
AlphaStar is a DeepMind system that plays StarCraft II. In 2019 it reached Grandmaster level with all three races, ranking above 99.8% of active players on Battle.net.
2018
-
Oct 11, 2018
BERT
BERT is a language model from Google that reads text in both directions and is pre-trained by filling in masked words. Fine-tuned per task, it set new records on question answering and language-understanding benchmarks and later went into Google Search.
-
Jun 11, 2018
GPT-1
GPT-1 was OpenAI's first generative pre-trained transformer, published in June 2018. It showed that one language model, pre-trained on unlabelled text and then fine-tuned, could improve results on many language tasks.
-
Jun 2018
OpenAI Five
OpenAI Five was a team of five reinforcement-learning bots that played the video game Dota 2; in April 2019 it beat OG, the reigning world champions, in a best-of-three series.
2013
-
Dec 19, 2013
Deep Q-Network (DQN)
A DeepMind reinforcement-learning agent that combined Q-learning with a convolutional neural network to play Atari 2600 games straight from raw pixels, reaching human-level performance on many titles without being told the rules.