Models

An open bilingual chat model from Zhipu AI and Tsinghua KEG with about 6.2 billion parameters, small enough to run on one consumer GPU, with weights fully open for research.

Large language modelsOpen sourceChina

Stable Diffusion 2 is an open-source text-to-image model family released by Stability AI on 24 November 2022, built on a new OpenCLIP text encoder and shipped with upscaling, depth-guided and inpainting models; version 2.1 followed weeks later.

Open sourceGenerative modelsCreative tools

Galactica is a large language model for science, unveiled by Meta AI and Papers with Code in November 2022. Its public demo was taken down about three days later, after users found it produced fake papers.

Large language modelsResearch historyOpen source

GLM-130B is a 130-billion-parameter open bilingual Chinese–English dense language model from Tsinghua University's KEG group and Zhipu AI, built to give researchers an open model that matches GPT-3 at the hundred-billion scale.

Large language modelsResearch historyOpen source

OPT-175B is a 175-billion-parameter language model from Meta AI, announced in May 2022. Meta said it was the first model of its size released with both pretrained weights and the code to train and use them.

Large language modelsResearch historyOpen source

GPT-NeoX-20B is a 20-billion-parameter open autoregressive language model released by EleutherAI in February 2022, with weights under Apache 2.0 and its training code, described by its makers as the largest publicly accessible general-purpose model of its kind at the time.

Large language modelsResearch historyOpen source

RoseTTAFold is a three-track neural network for protein structure prediction, published by the Baker lab in 2021; its accuracy approached AlphaFold2's at CASP14, it models protein–protein complexes from sequence alone, and it is distributed as open code with a public web server.

Research historyOpen sourceDeep learning

GPT-J-6B is a six-billion-parameter open language model released by EleutherAI in June 2021 under the Apache 2.0 license. Its published scores sit close to GPT-3 Curie, making it a widely used free baseline for large-model research.

Large language modelsResearch historyOpen source

GPT-Neo is a family of open text-generation models released by EleutherAI in March 2021 at 125M, 1.3B and 2.7B parameters, built to replicate the GPT-3 architecture and distributed with freely downloadable weights under an MIT license.

Large language modelsResearch historyOpen source

An image-text model from OpenAI that learns to match pictures with their captions. Trained on about 400 million image-caption pairs, it can classify images by comparing them against written prompts, with no task-specific training.

Open sourceFoundation modelsMultimodal

T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.

Large language modelsOpen sourceFoundation models

ERNIE 2.0 is Baidu's pre-trained language model framework, published in 2019, which builds pre-training tasks incrementally through continual multi-task learning and reported results above BERT and XLNet on English and Chinese benchmarks.

Large language modelsResearch historyOpen source

Baidu's first ERNIE pre-trained language model, released in March 2019, added entity- and phrase-level masking to BERT-style pre-training and reported state-of-the-art results on five Chinese NLP tasks.

Large language modelsResearch historyOpen source