Snowflake

A cloud data platform

Snowflake is a cloud data platform: customers keep data with the company and query it there, and third parties list data and applications for sale through its marketplace. The company says it serves 12,062 customers worldwide and lists 3,400 entries in that marketplace^[1]. It also reports an average of 6.3 billion queries a day run by its customers^[1]. Sridhar Ramaswamy is chief executive^[1].

Those numbers describe a business built on other organisations' data work — storing it, querying it, and increasingly running models over it.

Arctic

On 24 April 2024 Snowflake released Arctic, its own foundation language model^[1][2]. The architecture is unusual: rather than a plain transformer, the company calls it a dense-MoE hybrid — a 10-billion-parameter dense transformer combined with a residual mixture-of-experts MLP holding 128 experts of 3.66 billion parameters each^[1][2].

A mixture-of-experts layer splits one large block of the network into many separate sub-networks, called experts, and a small gating network decides which few experts each token passes through. Capacity grows with the number of experts while the arithmetic per token stays small. In Arctic that arrangement yields 480 billion parameters in total, of which 17 billion are active for any given token, chosen by top-2 gating^[2][3].

What it cost

Snowflake said the model cost under 2 million US dollars to train, or less than 3,000 GPU-weeks^[2]. The people behind it had worked on efficiency tooling before: the team had earlier built ZeRO, DeepSpeed, PagedAttention, vLLM and LLM360^[2] — systems for splitting training across many accelerators and for serving models cheaply, which is the background the low training figure rests on.

Measured on enterprise work

Snowflake defined an enterprise intelligence score as the average of four benchmarks: HumanEval+, MBPP+, Spider and IFEval^[2] — two code-generation tests, a text-to-SQL test and an instruction-following test. The company says Arctic matches Llama 3 70B on those metrics using 17 times less compute^[2], and that it accepted a lower MMLU score on purpose^[2]. MMLU is a broad academic-knowledge test, so the trade-off says the model was tuned for the tasks that turn up inside a data platform rather than for general trivia.

Open weights, and where it sits

Arctic shipped under the Apache 2.0 licence with ungated weights and code, released alongside the Arctic Embed retrieval family^[2][3]. Ungated means a download needs no approval step.

For a company that sells a data platform, publishing an openly licensed model with an explicit cost argument is a way of pointing at where it expects AI work to happen — inside the warehouse and the queries running there — and of appealing to developers who care about cost per unit of useful output.