Apple publishes the technical report on its Apple Intelligence foundation models

Background

In June 2024 Apple introduced its first generation of foundation models for Apple Intelligence. Those models, AFM-on-device and AFM-server, were described in an overview on Apple's Machine Learning Research site ^[3]. The overview gave the outline, but the detailed design needed a longer document. That document arrived on 29 July 2024, the same day that developer betas of iOS 18.1 with Apple Intelligence began ^[3].

What happened

On 29 July 2024 Apple posted "Apple Intelligence Foundation Language Models" to arXiv, a public preprint server where researchers share papers before or alongside formal publication ^[1]. Tom Gunter was the first author, and the report had about 210 authors in total ^[1]. Apple also published an HTML version of the report ^[2].

Apple updated its June overview post to reflect the report ^[3]. This meant that readers could follow the same numbers in both places.

Key details

The report covers the two models. AFM-on-device has about 3 billion parameters. AFM-server was built for Private Cloud Compute, Apple's system of servers for requests too heavy for a device ^[1].

The report discloses the training hardware. The server model was trained on 8,192 TPUv4 chips, and the on-device model on 2,048 TPUv5p chips. The core pre-training used 6.3 trillion tokens ^[2]. A token is a small piece of text, often a word or part of one. Disclosing these numbers gave researchers and the public a way to compare Apple's effort with the training of other models, even though the figures do not show the quality of the result by themselves.

Apple also described its responsible-AI principles in the report and said that no private user data was used in training ^[1][3]. Those statements are Apple's own claims, as set out in the report, and the report is the primary source for them.

Impact

The report made Apple's foundation models easier to examine. Before it, the public knew the broad design from the June post. After it, anyone could read the method, the scale and the data policy, and check them against what Apple later did with the models. For students and general readers, the report is also a clear example of how a company documents a large language model: the architecture, the hardware, the data sources, the evaluation and the safety principles are all set out in one place.

Note that the report is long and technical. Readers who are new to the subject can start with the June overview, which is written more simply, and then turn to the report for the details.

Because the report was posted to arXiv, a public server, anyone can read it without a subscription or a request to Apple. That openness is part of what made the release useful to people outside the company.