EleutherAI

What it is

EleutherAI is a grassroots collective of AI researchers and volunteers. It began in July 2020 as a Discord server, which is a chat service organised into topic channels, and it was first named LibreAI before being renamed EleutherAI later that month. Its best-known outputs are datasets and open models that anyone can study and reuse. ^[1]

Origins

The founding members were Connor Leahy, Leo Gao and Sid Black. Their first aim was to replicate OpenAI's GPT-3 in the open, meaning to build a model of similar kind whose details and weights anyone could examine. ^[1]

For compute, EleutherAI first used Google's TPU Research Cloud, which gave researchers access to Google's tensor processing units. From early 2021 it also accepted GPU compute from CoreWeave, a cloud provider that supplied the hardware for its GPT-NeoX-20B model. ^[1][3]

Datasets and models

On 31 December 2020 EleutherAI released The Pile, an English text dataset of 825 GiB assembled from 22 subsets, for training large language models. ^[2]

The group then released a series of open GPT-style models. GPT-Neo came out in March 2021 in 125M, 1.3B and 2.7B parameter sizes. GPT-J-6B followed in June 2021, and GPT-NeoX-20B in February 2022. ^[1] In 2021 members Katherine Crowson and Ryan Murdock also developed the VQGAN-CLIP technique for generating images from text. ^[1]

In April 2023 the group released Pythia, a suite of 16 models from 70M to 12B parameters with 154 checkpoints each, built to study how models learn during training. ^[5]

Institute

In early 2023 EleutherAI incorporated as the EleutherAI Institute, a nonprofit research institute. It is run by Stella Biderman, Curtis Huebner and Shivanshu Purohit. Its donors included Stability AI, Hugging Face, Canva, Lambda Labs and former GitHub CEO Nat Friedman. ^[1][4]

Leahy and Black later left to found the alignment company Conjecture, and Leahy joined the nonprofit's board of advisers. ^[4]

The group's focus moved toward interpretability, alignment and scientific research. By 2025 it no longer trained frontier-style models of its own. It maintained datasets, did research and worked on policy. ^[1][4]

Criticism and later work

The Pile drew criticism. Its Books3 component was taken offline in 2023 after an anti-piracy complaint. A July 2024 investigation found that it included subtitles from over 170,000 YouTube videos. ^[1]

In June 2025 EleutherAI released Common Pile v0.1, an 8 TB dataset of public-domain and openly licensed text, along with two 7B Comma v0.1 models trained on it. ^[6]