Hugging Face Transformers library

What it is Transformers is an open-source Python library from Hugging Face that provides pretrained Transformer models under one unified API. The models it offers are backed by the models shared on the Hugging Face Hub.^[1] A Transformer is a neural-network design that processes sequences of data, such as words, all at once and relates each element to the others. Most modern language models are built on this design. A pretrained model has already been trained on a large body of data, so a developer can use it or adapt it rather than starting from nothing. ## Origins The library began as pytorch-pretrained-bert, a PyTorch port of Google's BERT. It was later renamed pytorch-transformers and then Transformers as it grew.^[1] PyTorch is an open-source deep-learning framework, and a port in this sense means the same model rewritten to run in that framework. The rename to Transformers came with version 2.0 in 2019. That version added TensorFlow 2.0 support alongside PyTorch.^[3] TensorFlow is another framework for building machine-learning systems, so the library could be used with either of the two most common tools. The library is described in a paper titled “HuggingFace's Transformers: State-of-the-art Natural Language Processing”. Thomas Wolf, Lysandre Debut, Victor Sanh and others wrote it, and it was first posted to arXiv on 9 October 2019.^[1] arXiv is an online repository where researchers share papers before or alongside formal publication. ## Growth and the split from the Hub The first v4 release candidate came out on 19 November 2020. At that time the library supported about 40 architectures and around 1,000 Hub checkpoints, and it was installed about 20,000 times a day.^[2] An architecture here means a particular model design, and a checkpoint is a saved set of trained weights for that design. In late 2020 the Hub-related logic was split out of Transformers into the separate huggingface_hub library.^[4] This separation let the file-sharing functions be used on their own. The first v5 release candidate was announced on 1 December 2025. By then the library had more than 400 architectures, over 750,000 compatible Hub checkpoints, more than 3 million pip installs a day and over 1.2 billion installs in total. Pip is the standard package manager that Python developers use to install libraries.^[2] ## Version 5 changes Version 5 goes PyTorch-only. It sunsets TensorFlow and Flax support, so developers who relied on those frameworks need to move to PyTorch. Flax is a neural-network library built on JAX.^[2] Version 5 also adds continuous batching and paged attention, which are techniques for handling many requests efficiently on the same hardware. It includes a “transformers serve” server that is compatible with the OpenAI API, so tools written for that interface can call the locally run models.^[2] ## Impact Transformers became one of the most widely used libraries for working with pretrained language and other models. Its design, which hides the differences between many model architectures behind one interface, is a large part of why. Readers who want to follow the library should note that its version changes can break existing code, which is why each major version is documented with its own release notes.