Snowflake Arctic

What Snowflake Arctic is

Snowflake Arctic is an open-weight large language model that Snowflake released on 24 April 2024 ^[1][2]. Its defining feature is not raw size but the ratio between the parameters it stores and the parameters it uses: 480 billion total parameters, of which 17 billion are active for any given token ^[1][2]. Snowflake's pitch is aimed at enterprise workloads rather than at general chat alone.

A dense and a sparse model in one

Arctic's architecture is a hybrid. A conventional dense transformer of about 10 billion parameters sits alongside a residual mixture-of-experts (MoE) multilayer perceptron made of 128 experts of 3.66 billion parameters each ^[1][2]. A mixture-of-experts layer splits work among many specialised sub-networks and routes each token to only a few of them, so the compute per token stays far below what the total parameter count would suggest. In Arctic the routing is a top-2 gate: two experts are chosen per token, which is where the 17 billion active parameters come from ^[1][2]. The dense part is always on, the sparse part is added residually, and the result is a model that is stored at 480 billion parameters yet runs at the cost of a much smaller one.

Built to a budget

Snowflake put an unusual amount of emphasis on training cost. The company says the model was trained for under 2 million US dollars, or less than 3,000 GPU-weeks ^[1]. Against that budget it makes two comparisons: it says Arctic matches Llama 3 70B on enterprise metrics while using 17 times less compute ^[1], and that it stays competitive with DBRX on language understanding while using 7 times less compute ^[1].

The enterprise score

To make "enterprise metrics" concrete, Snowflake defines an enterprise intelligence score as the average of four benchmarks: HumanEval+, MBPP+, Spider and IFEval ^[1]. Together these cover code generation, SQL query writing and instruction following — the tasks that come up in data and application work rather than in general knowledge quizzes.

Choosing where to be weaker

One deliberate choice stands out. Snowflake accepted a lower MMLU score on purpose, arguing that world-knowledge scores rise with training compute ^[1]. In other words, the model trades some breadth of factual knowledge for efficiency on the tasks Snowflake cares about. That is a defensible design decision for a company selling into data warehouses, and an unusual thing to state in public.

Context, checkpoints and access

The model was trained with a 4K attention context window, with a 32K window planned ^[1]. It is released under Apache 2.0, with ungated access to weights and code ^[1]. Both base and instruct checkpoints are published, and the LoRA fine-tuning recipe fits on a single node ^[1][2]. Alongside the language model, Snowflake released the Arctic Embed family of retrieval models ^[1] — a pairing that matches how enterprise systems are actually built, with a generator and a retriever working together.

Where it comes from

The team behind Arctic had earlier built ZeRO, DeepSpeed, PagedAttention and vLLM, and LLM360 ^[1]. That lineage shows in the priorities: systems efficiency, serving throughput and open release rather than leaderboard chasing.