Open letter calls for a six-month pause on giant AI experiments

Background

Large language models are built by training a system on very large amounts of text and adjusting its parameters until it can produce and work with language; in general, more computing power and data going into training means a more capable system. A "training run" is the process of doing that work — feeding the data through the model and updating it over and over until training stops. The letter was aimed at that process rather than at a finished product. The Future of Life Institute published "Pause Giant AI Experiments: An Open Letter" on 22 March 2023.^[1]

What the letter asked for

The central demand was a pause: all AI labs should immediately pause, for at least six months, the training of AI systems more powerful than GPT-4.^[1] The capability line named in the letter was GPT-4, so the request covered systems that would be trained beyond that level.

The letter also attached conditions to what would count as a pause. It had to be public and verifiable, and it had to include all key actors; if a pause could not be enacted quickly, governments should step in with a moratorium.^[1] Verification and full coverage were therefore part of the request, not footnotes to it: a pause announced by one lab alone would not meet the description, and a government moratorium was named as the fallback if voluntary action did not arrive quickly.

Using the pause for safety work

The six months were given a job. AI labs and independent experts were asked to use the pause to develop shared safety protocols, audited and overseen by independent outside experts.^[1] That combines two things: common technical practice that different labs would follow, and outside scrutiny of whether they actually follow it.

Who signed

The signatories named by the institute included Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak, Yuval Noah Harari and Emad Mostaque.^[1] The Future of Life Institute said the letter was signed by more than 30,000 experts, researchers, industry figures and other leaders.^[2] The list as described by the institute was not limited to AI researchers; it included people from industry and other public figures.

Governance recommendations

Beyond the pause, the letter recommended a set of governance measures: new regulatory authorities for AI; oversight of highly capable systems and of large pools of computing; provenance and watermarking systems, which mark content so that its origin can be traced; an auditing and certification ecosystem; liability for harm caused by AI; and public funding for technical AI safety research.^[1]

The list mixes several kinds of instrument. Regulators and auditors are institutions; watermarking and provenance are technical tools for tracking where material came from; liability rules decide who pays when an AI system causes harm; and public funding pays for safety research that might otherwise not be done commercially. Oversight of large pools of computing targets the inputs to training rather than the model itself — the idea being that very large training runs are visible at the level of computing capacity.

Six months on

The institute marked six months since the letter's release on 22 September 2023, and published reflections from signatories on what should happen next.^[2]

What the document is

The letter is a short public statement with a specific ask, conditions on how the pause would work, and a list of policy proposals, signed by a mix of researchers, executives and public figures; the signatory count reported by the institute put it in the tens of thousands.^[1][2] Its central proposal was a public and verifiable pause on training above a stated capability line, covering all key actors, with a government moratorium as a fallback — and, alongside it, a set of measures for regulating and auditing advanced AI.