Home›Models
Models
Models
Foundation, generative, multimodal and specialised models.
Total 367 · Clear filters
2020
-
Jan 28, 2020
Meena
Meena is a 2.6-billion-parameter open-domain chatbot from Google, trained end-to-end on filtered social media conversations. Its 2020 paper introduced the SSA human-evaluation metric and linked next-token perplexity to human-judged chat quality. The work was later renamed LaMDA.
2019
-
Oct 23, 2019
T5 (Text-to-Text Transfer Transformer)
T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.
-
Jul 29, 2019
ERNIE 2.0
ERNIE 2.0 is Baidu's pre-trained language model framework, published in 2019, which builds pre-training tasks incrementally through continual multi-task learning and reported results above BERT and XLNet on English and Chinese benchmarks.
-
Mar 2019
ERNIE 1.0
Baidu's first ERNIE pre-trained language model, released in March 2019, added entity- and phrase-level masking to BERT-style pre-training and reported state-of-the-art results on five Chinese NLP tasks.
-
Feb 14, 2019
GPT-2
GPT-2 is a large language model that OpenAI announced in February 2019. Its largest version has 1.5 billion parameters. OpenAI first withheld the full model, then released it in stages through November 2019.
2018
-
Oct 11, 2018
BERT
BERT is a language model from Google that reads text in both directions and is pre-trained by filling in masked words. Fine-tuned per task, it set new records on question answering and language-understanding benchmarks and later went into Google Search.
-
Jun 11, 2018
GPT-1
GPT-1 was OpenAI's first generative pre-trained transformer, published in June 2018. It showed that one language model, pre-trained on unlabelled text and then fine-tuned, could improve results on many language tasks.