EXAONE 4.0

A single family in two sizes

EXAONE 4.0 is a family of large language models from LG. Its technical report was posted on 15 July 2025 and covers two sizes: a 32B model and a 1.2B on-device model ^[1]. The two entries are built for different settings. The 32B model is the large one, and most of the published architecture detail belongs to it. The 1.2B model is small enough to be labelled an on-device model — small enough, that is, to be run on a local device rather than only on a remote server.

What the release changes

Earlier EXAONE releases divided the work between two kinds of model. EXAONE 3.5 was the general model, and its strength was usability; EXAONE Deep was the model built for reasoning. EXAONE 4.0 merges the usability of EXAONE 3.5 with the reasoning ability of EXAONE Deep into a single model, so a user does not have to choose between a chat-oriented model and a reasoning-oriented one ^[1].

Two further changes come with the release. The model adds agentic tool use, meaning it can take part in workflows where a model calls tools to get work done; LG framed this as preparation for the agentic AI era, a period in which models are expected to act on a task rather than only answer about it ^[1]. And multilingual support was extended: where earlier support covered English and Korean, EXAONE 4.0 adds Spanish ^[1].

The 32B model in detail

The 32B model holds 30.95 billion parameters excluding embeddings, spread over 64 layers ^[2]. Parameters are the learned weights of the network; embeddings are the lookup tables that turn tokens into vectors, and they are left out of this count. Its context window — the amount of text the model can keep in view at once — is 131,072 tokens, and its vocabulary is 102,400 tokens, the set of units its tokenizer can produce ^[2].

The attention design is hybrid: local sliding-window attention and global full attention are mixed in a 3:1 ratio ^[2]. In a sliding-window layer each token attends only to a bounded neighbourhood, which keeps the cost of long sequences down; in a global full-attention layer every token can see every other token, which keeps information flowing across the whole context. Mixing the two is a way of keeping most of the reach of full attention while paying much less for it, since most layers only ever look at a short window.

Rotary position embeddings are dropped on the global attention layers, which the model card presents as a way of handling long contexts better ^[2]. Rotary embeddings encode position by rotating the query and key vectors; removing them from the global layers changes how position is represented once the context becomes long.

A normalisation scheme called QK-Reorder-Norm applies LayerNorm to the outputs of the attention and MLP blocks and RMS normalisation after the query and key projections ^[2]. Normalisation keeps the scale of activations under control during training and generation, and where it is placed matters for stability — putting it after the Q and K projections is a specific choice about which quantities get rescaled before attention scores are computed.

Attention itself is grouped-query: 40 query heads share 8 key-value heads ^[2]. Grouped-query attention gives several query heads one shared key-value head, which shrinks the cache of keys and values that must be stored while a model generates text, while keeping much of the quality of ordinary multi-head attention.

Licence and where it sits

The licence was updated with this release: commercial use is allowed through licensed serving partners ^[2]. Read together, the pieces describe a consolidation release rather than a single new trick. One family now covers both everyday use and reasoning, two sizes cover a large model and a small on-device one, and tool use plus a third language widen who can use it and for what. LG's own framing places the tool-use capability at the centre of that plan, as groundwork for a period in which models are expected to act through tools rather than only produce answers ^[1].