PaLM 2

What PaLM 2 is

PaLM 2 is Google's second-generation Pathways language model, announced at the company's I/O event on 10 May 2023 ^[1]. It is a large language model: a system trained on a very large amount of text so that it can continue, transform and answer questions about language. The "pre-training" in that description refers to the first, broad stage of training, in which the model learns statistical patterns of text before any narrower tuning.

It came in four sizes, from smallest to largest: Gecko, Otter, Bison and Unicorn ^[1]. Offering a model family in several sizes is a common practice in the field, because a larger model usually handles harder tasks while a smaller one is cheaper to run and can sit closer to the user. In PaLM 2's case the smallest member, Gecko, was light enough to run on mobile devices, even offline ^[1].

Training and capabilities

Google trained the model on text spanning more than 100 languages ^[1]. The training data included scientific papers and web pages with mathematical expressions, and the model was pre-trained on public source code ^[1] — the kind of material that gives a model practice with formulas and with programming syntax, not just ordinary prose.

Google said PaLM 2 demonstrates improved capabilities in logic, common sense reasoning, and mathematics ^[1], and that it passed advanced language-proficiency exams at the "mastery" level ^[1].

Versions and how developers reach it

Specialised versions included Med-PaLM 2 for medicine and Sec-PaLM for security ^[1]. Alongside these, the general model was offered to developers through the PaLM API and on Google Cloud Vertex AI ^[1]. Customers could use PaLM 2 in Vertex AI with enterprise-grade privacy, security and governance ^[1], which matters for organisations that cannot send data through consumer-facing tools.

Developers had already had a look: since March, Google had been previewing the PaLM API with a small group of developers ^[1]. The May announcement widened that from a preview to a product offering.

What was built on it

At the same announcement, Google introduced more than 25 new products and features powered by PaLM 2, including Bard ^[1]. The improved multilingual capabilities were put to work immediately: Google used them to expand Bard to new languages on 10 May 2023 ^[1].

That breadth is the notable part. Rather than releasing a single model behind a single chat interface, Google placed one model family underneath a large set of products at once, spanning consumer applications and a cloud platform, with medical and security variants branching off from the same base.

Where it sits in the line

PaLM 2 was not the end of the sequence. Google said at the time that its next model, Gemini, was still in training ^[1], and the handover came within the year: Bard switched to a fine-tuned Gemini Pro on 6 December 2023 ^[2]. PaLM 2 is therefore best read as the stage at which Google's language models moved from research releases into the ordinary surface of its products, and as the model that held that position until Gemini was ready.