Home›Models
Models
Models
Foundation, generative, multimodal and specialised models.
Total 386 · Clear filters
2022
-
Apr 4, 2022
PaLM
PaLM is a 540-billion-parameter dense Transformer language model announced by Google Research in April 2022, trained with the Pathways system across 6,144 TPU v4 chips; it set strong few-shot results on many benchmarks.
-
Mar 15, 2022
GPT-3.5 (text-davinci-002/003)
GPT-3.5 is the family of OpenAI language models between GPT-3 and GPT-4, released through API models such as text-davinci-002 and text-davinci-003 and underlying the first version of ChatGPT.
-
Feb 9, 2022
GPT-NeoX-20B
GPT-NeoX-20B is a 20-billion-parameter open autoregressive language model released by EleutherAI in February 2022, with weights under Apache 2.0 and its training code, described by its makers as the largest publicly accessible general-purpose model of its kind at the time.
-
Feb 2, 2022
AlphaCode
AlphaCode is a DeepMind code-generation system for competitive programming, announced in February 2022. On Codeforces contests held after its training data, it reached an estimated rank around the median participant. A Gemini-based AlphaCode 2 followed in December 2023.
-
Jan 27, 2022
InstructGPT
InstructGPT is a family of GPT-3 models trained with reinforcement learning from human feedback so that they follow user instructions. OpenAI announced them in January 2022 and made them the default models in its API.
2021
-
Dec 23, 2021
ERNIE 3.0 Titan
ERNIE 3.0 Titan is a Chinese pre-trained language model with over 260 billion parameters, described by its authors as the largest dense Chinese model of its time and reported to beat state-of-the-art systems on 68 NLP datasets.
-
Dec 8, 2021
Gopher
Gopher is DeepMind's 280-billion-parameter language model, announced in December 2021 alongside a study of what scale does to language models and a companion taxonomy of their risks.
-
Oct 11, 2021
Megatron-Turing NLG 530B
Megatron-Turing NLG 530B is a 530-billion-parameter language model announced by NVIDIA and Microsoft on 11 October 2021, then the largest monolithic transformer language model, trained on NVIDIA's Selene supercomputer with Megatron-LM and DeepSpeed combined.
-
Aug 2021
Jurassic-1
Jurassic-1 is the first family of large language models from AI21 Labs, announced with the AI21 Studio developer platform in August 2021. Its largest model, Jumbo, has 178 billion parameters.
-
Jul 7, 2021
OpenAI Codex (2021 model)
OpenAI Codex is a 2021 GPT language model fine-tuned on public code from GitHub. It translates natural language into code, and was introduced alongside the HumanEval benchmark for testing code correctness.
-
Jul 5, 2021
ERNIE 3.0
ERNIE 3.0 is a 10-billion-parameter pre-trained language model from 2021 that trains on plain text plus a large-scale knowledge graph and fuses auto-encoding and auto-regressive networks, so one model handles both language understanding and generation.
-
Jun 2021
GPT-J-6B
GPT-J-6B is a six-billion-parameter open language model released by EleutherAI in June 2021 under the Apache 2.0 license. Its published scores sit close to GPT-3 Curie, making it a widely used free baseline for large-model research.
-
May 18, 2021
LaMDA
LaMDA is a family of conversational language models that Google announced in 2021 and described in a 2022 paper, notable for its work on safety and factual grounding in dialogue and for later powering Bard.
-
Mar 21, 2021
GPT-Neo
GPT-Neo is a family of open text-generation models released by EleutherAI in March 2021 at 125M, 1.3B and 2.7B parameters, built to replicate the GPT-3 architecture and distributed with freely downloadable weights under an MIT license.
2020
-
May 28, 2020
GPT-3
GPT-3 is an autoregressive language model with 175 billion parameters, described in a May 2020 paper. It could perform new tasks from a few examples in the prompt, without any gradient updates, and it was first offered through OpenAI's API.
-
Jan 28, 2020
Meena
Meena is a 2.6-billion-parameter open-domain chatbot from Google, trained end-to-end on filtered social media conversations. Its 2020 paper introduced the SSA human-evaluation metric and linked next-token perplexity to human-judged chat quality. The work was later renamed LaMDA.
2019
-
Nov 19, 2019
MuZero
MuZero is a DeepMind reinforcement-learning algorithm that pairs tree-based search with a learned model. It reached superhuman play in Atari, Go, chess and shogi without being given the environment's rules.
-
Oct 23, 2019
T5 (Text-to-Text Transfer Transformer)
T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.
-
Jul 29, 2019
ERNIE 2.0
ERNIE 2.0 is Baidu's pre-trained language model framework, published in 2019, which builds pre-training tasks incrementally through continual multi-task learning and reported results above BERT and XLNet on English and Chinese benchmarks.
-
Mar 2019
ERNIE 1.0
Baidu's first ERNIE pre-trained language model, released in March 2019, added entity- and phrase-level masking to BERT-style pre-training and reported state-of-the-art results on five Chinese NLP tasks.
-
Feb 14, 2019
GPT-2
GPT-2 is a large language model that OpenAI announced in February 2019. Its largest version has 1.5 billion parameters. OpenAI first withheld the full model, then released it in stages through November 2019.
-
Jan 2019
AlphaStar
AlphaStar is a DeepMind system that plays StarCraft II. In 2019 it reached Grandmaster level with all three races, ranking above 99.8% of active players on Battle.net.
2018
-
Oct 11, 2018
BERT
BERT is a language model from Google that reads text in both directions and is pre-trained by filling in masked words. Fine-tuned per task, it set new records on question answering and language-understanding benchmarks and later went into Google Search.
-
Jun 11, 2018
GPT-1
GPT-1 was OpenAI's first generative pre-trained transformer, published in June 2018. It showed that one language model, pre-trained on unlabelled text and then fine-tuned, could improve results on many language tasks.
-
Jun 2018
OpenAI Five
OpenAI Five was a team of five reinforcement-learning bots that played the video game Dota 2; in April 2019 it beat OG, the reigning world champions, in a best-of-three series.
2013
-
Dec 19, 2013
Deep Q-Network (DQN)
A DeepMind reinforcement-learning agent that combined Q-learning with a convolutional neural network to play Atari 2600 games straight from raw pixels, reaching human-level performance on many titles without being told the rules.