Meta launches Galactica and pulls its demo three days later

Background

Scientists work with large volumes of papers, equations, code and reference material. Meta AI and Papers with Code built Galactica as a language model meant to help with that work.^[1]

What happened

On 15 November 2022 Meta AI and Papers with Code unveiled Galactica, with a public demo. The largest version had 120 billion parameters. It was trained on 48 million examples of scientific articles, textbooks, lecture notes, websites and encyclopedias.^[1][2]

The paper reported 68.2% on LaTeX equations, against 49.0% for GPT-3. On the MATH benchmark it scored 20.4%, against 8.8% for PaLM 540B.^[3]

Key details

Users quickly found failures. They showed the model producing fake papers attributed to real authors, and confident articles on invented topics.^[1]

Meta took the demo offline on 17 November 2022, about three days after launch. The models remained available to researchers.^[1]

Michael Black of the Max Planck Institute said the outputs were “wrong or biased but sounded right and authoritative”.^[1]

Language models generate text by predicting likely next words, so they can write fluent sentences whether or not the content is true. Scientific writing makes errors harder to spot, because citations, equations and technical terms all look authoritative. LaTeX is the typesetting language used for mathematical equations. The paper compared Galactica with GPT-3 and PaLM 540B, both earlier large language models.^[3]

The demo was public, so anyone could type a question and see the output. That made the model's tendency to produce confident errors visible to the general public, not only to the people who built it.^[1]

The paper was posted to arXiv, so its methods and benchmark results could be read and checked by people outside the two organisations.^[3]

The models were built for scientists, and the public demo was the part of the release that reached a general audience.^[1]

Impact

The episode showed the gap between fluent output and reliable output in language models.