Falcon-H1

What Falcon-H1 is

Falcon-H1 is a family of open-weight language models in the Falcon line. The weights of its 34B instruction-tuned build appeared on Hugging Face on 1 May 2025^[1]. What sets the family apart is its architecture: it uses a parallel hybrid design that runs Transformer attention alongside state space models in the same network^[2].

Why put two architectures side by side

Earlier Falcon models were built on either Transformer or Mamba alone^[2]. The two approaches have different strengths. Transformer attention lets every token look directly at every earlier token, which makes it good at pulling together information scattered across a long text, but the cost of doing so grows with the length of the sequence. A state space model instead carries a compressed summary of what it has read so far and updates that summary step by step, which is cheaper per token but keeps less raw detail. The Falcon-H1 technical report describes a parallel hybrid: rather than choosing one of the two, the model runs attention and state space components alongside each other, so both kinds of processing are available inside a single model^[2].

Sizes and forms

The series ships at 0.5B, 1.5B, 1.5B-deep, 3B, 7B and 34B parameters, each in base and instruct form^[2]. Base models are trained to predict the next piece of text and serve as a starting point for further training; instruct models are tuned to follow directions and answer questions the way a user would pose them. A 1.5B-deep build sits alongside the standard 1.5B entry in the lineup^[2].

Languages, runtimes and licence

The instruction-tuned models support 18 languages, among them Arabic, Chinese, Japanese and Urdu^[1]. The model card lists transformers, vLLM 0.9 and llama.cpp as supported runtimes^[1] — the standard Hugging Face library, a serving engine built for high-throughput inference, and a lightweight implementation aimed at running models locally. Weights are released under the Falcon-LLM Licence^[1].

Release timeline

The 34B instruct weights were the first visible piece, on 1 May 2025^[1]. Quantised instruction-tuned builds followed on 12 and 13 May 2025^[3]. Quantisation stores weights at lower numerical precision, so a model occupies less memory and can run on smaller hardware, at some cost in accuracy. The technical report was posted to arXiv on 30 July 2025^[2]. In January 2026 a 90M Tiny instruct model was added, extending the family below its smallest original size^[3].

Where it sits

The report frames the work as revisiting model design, data strategy and training dynamics rather than only scaling up^[2]. For readers following open-weight releases, Falcon-H1 is a concrete example of that shift: instead of treating the Transformer as the only option, it places attention and state space layers in the same model, covers the range from 0.5B to 34B parameters^[2], and publishes the weights under a named licence so that others can build on them^[1].