ChatGLM-6B

What it is

ChatGLM-6B is an open bilingual chat model released in 2023 by Zhipu AI and the Knowledge Engineering Group (KEG) at Tsinghua University, with about 6.2 billion parameters^[1]. The size matters: six billion parameters is small enough to fit on a single consumer graphics card, which is unusual for a model built to hold a conversation. "Bilingual" means one set of weights handles Chinese and English; "chat" means the model was tuned to answer in dialogue rather than merely continue text.

How it was trained

Pre-training used roughly 1 trillion Chinese and English tokens^[1]. A token is the unit of text the model actually reads — roughly a word, or a fragment of one — so the number describes how much text went in. That stage teaches the model to predict what comes next. Three further stages turned it into an assistant: supervised fine-tuning, feedback bootstrapping, and reinforcement learning from human feedback^[1]. Supervised fine-tuning shows the model example exchanges so it learns the shape of an answer; in reinforcement learning from human feedback (RLHF), humans rank competing answers and the model is pushed toward the preferred ones. Together these stages turn a text predictor into something that follows instructions.

Running on a personal machine

The memory budget is the model's most practical feature. Inference needs about 13 GB of GPU memory at FP16, about 8 GB at INT8, and about 6 GB at INT4^[1]. These numbers come from quantization: storing the weights in lower-precision numbers. FP16 keeps each weight in a 16-bit floating-point value; INT8 and INT4 squeeze it into 8-bit and 4-bit integers, cutting memory at some cost in accuracy. Without a GPU, full-precision inference on CPU needs about 32 GB of memory, and the repository suggests quantized versions on machines with less RAM^[1]. On a Mac, loading the model at half precision needs about 13 GB of memory, but the quantized kernels require CUDA, so a Mac falls back to the CPU^[1].

Fine-tuning is light too. P-Tuning v2 — a parameter-efficient method that trains a small set of extra parameters instead of the whole model — needs about 7 GB at INT4, 14 GB at FP16, and 9 GB at INT8^[1]. Adaptation to a specific task is therefore feasible on much the same hardware used for inference.

License

The code is released under Apache-2.0, and the weights are fully open for academic research^[1]. Commercial use is free but requires registering through a questionnaire^[1]. In practice the model is open for research without conditions and open for business after a formality — looser than a research-only release, but not the same as an unconditional open-source license.

Versions

A v1.1 checkpoint on 15 May 2023 added English instruction data to reduce answers that mixed the two languages^[1]. ChatGLM2-6B followed on 25 June 2023, pre-trained on 1.4 trillion tokens, with the base context length extended from 2K to 32K^[1]; context length is how much text the model can keep in view at once, so the model could handle far longer inputs. CodeGeeX2-6B, a code model built on ChatGLM2-6B, was released on 25 July 2023^[1], showing the base model being reused for a specialised domain.

Where it sits

Zhipu's consumer chatbot was later known as ChatGLM before being renamed GLM^[2]. ChatGLM-6B is the point where that line of work became something individuals could download and run: an open checkpoint, a published memory budget, and a quantized path down to about 6 GB of GPU memory^[1].