Tencent Hy4 preview

What it is

Tencent Hy4 preview, also called Hy4-preview or Hunyuan Hy4, is a next-generation large language model that Tencent released and open-sourced on Friday, 28 August 2026. Parameter counts are a rough guide to size: total parameters count everything stored in the network, while active parameters count what is used for each token. Tencent tailored it for productivity tasks.^[2] It is a mixture-of-experts (MoE) model with 770 billion total parameters and 49 billion activated parameters, plus a multi-token-prediction layer.^[1] It supports a context length of 1 million tokens.^[1]

It came less than two months after Hy3's 6 July 2026 launch.^[2]

Origins

Hy4 preview follows Hy3 as the next model in Tencent's line. Its weights, including an FP8-quantized version, were released under the Apache License 2.0.^[1][2] Apache 2.0 allows use, modification and redistribution, so developers can build on the released weights under its terms.

Main features

The model has 78 layers: one dense layer and 77 MoE layers. Each MoE layer contains 256 routed experts and one shared expert, and each token activates the top 8 routed experts.^[1] A shared expert is processed for every token, while routed experts are chosen for each token by a router.

The architecture uses Gated DeepSeek Sparse Attention with IndexCache, and identity Hyper-Connections with four residual streams.^[1] Sparse attention limits the number of earlier tokens each token attends to, which reduces the cost of long inputs. Hyper-Connections are a way of widening the residual pathway through which information passes between layers.

Tencent reported the following scores: 92.3 on GPQA Diamond, 85.4 on Terminal-Bench 2.1, 65.7 on SWE-bench Pro, 64.3 on Deep SWE, 74.1 on Toolathlon Verified and 37.1 on Apex Agents.^[1]

In an internal blind evaluation of 203 engineering tasks by 163 Tencent experts, Hy4 averaged 2.99. For comparison, the same evaluation gave 2.92 for GLM 5.3 and 2.94 for Kimi K3.^[1]

Reasoning defaults to high, with a no_think option. Tencent notes that the model can over-reason and over-verify its own work.^[1]

Place in the field

Caixin placed Hy4 against rivals including Moonshot's Kimi K3, DeepSeek V4 Pro and Z.ai's GLM-5.3.^[1] Hy4 preview is larger than Hy3, which has 295 billion total parameters.^[1][2]