Apple Foundation Models (AFM, 2024)

What it is

Apple Foundation Models, usually shortened to AFM, are the large language models that power Apple Intelligence. A language model is a program trained on large amounts of text so that it can read and write language. A foundation model is a general model that can be adapted to many tasks. Apple described its first generation on 10 June 2024 ^[1]. It has two members: AFM-on-device, with about 3 billion parameters, which runs on the phone or computer itself, and AFM-server, a larger model that runs in Private Cloud Compute, Apple's system of servers for requests that are too heavy for a device ^[1][2].

A parameter is one of the numbers inside a neural network that are adjusted during training. More parameters usually mean more capacity, but also more memory and computing cost, which is why the on-device model is kept small.

Origins

Apple published the design in June 2024 in an overview post on its Machine Learning Research site ^[1]. A fuller technical report, "Apple Intelligence Foundation Language Models", followed on arXiv on 29 July 2024, with Tom Gunter as first author ^[2]. The report, of which there was also a version in HTML, describes the architecture, the training data and the results ^[3].

The models were built with Apple's open-source AXLearn framework, which is built on JAX and XLA, two tools for running numerical computing on accelerator chips ^[1]. Apple said the training used licensed data and public web data crawled by its Applebot crawler, and that users' private data was not used ^[1].

Key features

The on-device model has 26 layers and a model dimension of 3,072. It uses 24 query heads and 8 key/value heads, a design called grouped-query attention, along with RoPE, a method for encoding word positions. Its vocabulary has 49K entries, and its input and output share the same embeddings. It has 2.58 billion non-embedding parameters ^[3][1]. The server model uses a larger vocabulary of 100K entries, but Apple did not publish its parameter count ^[1][3].

Training happened in stages. Pre-training used 6.3 trillion core tokens, followed by 1 trillion further tokens weighted toward code and math at a length of 8,192, and then 100 billion tokens at a context of 32,768 ^[3]. A token is a small piece of text, often a word or part of one, and context length is how much text the model can consider at once. The server model was trained from scratch on 8,192 TPUv4 chips. The on-device model was trained on 2,048 TPUv5p chips and started from a pruned 6.4B model, then trained with distillation, a method where a model learns from another model's outputs ^[3].

After pre-training, post-training used two new algorithms: iTeC (iterative teaching committee) and MDLOO, an online form of reinforcement learning from human feedback (RLHF) ^[3].

Running on a phone demanded compression. On the device the model is reduced to about 3.7 bits per weight on average, using a mix of 2-bit and 4-bit palettization. Apple reported about 30 tokens per second on an iPhone 15 Pro ^[1]. Small task-specific add-ons called LoRA adapters customise the model for each feature ^[1].

Apple reported these results: a HELM MMLU score of 61.4 for the on-device model and 75.4 for the server model. In side-by-side tests with human judges, AFM-on-device won 47.7% of comparisons against Phi-3-mini, and AFM-server won more than 50% against GPT-3.5 ^[3].

Where it sits

This generation was the base for the Apple Intelligence features shipped in late 2024. Its successor, the second generation announced in June 2025, used a mixture-of-experts server model and added image understanding ^[4]. The 2024 models are therefore best read as the first step in a series that Apple has updated each year.