Center for AI Safety

What CAIS is

The Center for AI Safety (CAIS) is an American nonprofit organization based in San Francisco that promotes the safe development and deployment of artificial intelligence.^[1] In its own account, it is a research and field-building nonprofit whose mission is to reduce societal-scale risks associated with AI by conducting safety research, building the field of AI safety researchers, and advocating for safety standards.^[2]

Where it came from

CAIS was founded in 2022 by Dan Hendrycks and Oliver Zhang.^[1] Hendrycks, who serves as the center's director, holds a PhD in Computer Science from UC Berkeley.^[2] The organization's stated aims place it on two tracks at once: producing technical work on safety and enlarging the pool of people who do that work.

Research

CAIS's work encompasses research in technical AI safety and AI ethics, advocacy, and support to grow the AI safety research field.^[1] Its researchers published "An Overview of Catastrophic AI Risks", which details risk scenarios and mitigation strategies, including the use of AI in autonomous warfare, for engineering pandemics, and AI capabilities for deception and hacking.^[1] The document is a survey rather than a single result: it lays out how a technology that is generally useful could be turned to large-scale harm, and what could be done about it.

CAIS has also worked on concrete weaknesses in deployed systems. In collaboration with Carnegie Mellon University researchers, it described an automated way to discover adversarial attacks on large language models that bypass safety measures.^[1] Adversarial attacks here means inputs crafted so that a model behaves in ways its operators intended to prevent; making the search for such inputs automatic matters because manual probing cannot keep up with models that answer freely in natural language.

Building a field

A large part of the center's activity is aimed at people rather than at models. CAIS runs a compute cluster to support AI safety research and an online course titled "Intro to ML Safety".^[1] It offers an "Introduction to AI Safety" course in partnership with Coursera, covering major risks from AI including malicious use, competitive pressures, and loss of control.^[2] Those three categories are a way of organizing the problem: misuse by bad actors, pressure on developers who fear falling behind, and the difficulty of keeping systems aimed at human intentions as they become more capable.

CAIS also runs a fellowship for philosophy professors to address conceptual problems.^[1] The premise behind such a program is that some safety questions — what counts as harm, how to specify an objective, who decides — are not settled by engineering alone. In addition, the center runs an AI and Society Fellowship, a three-month research program that investigates the societal impacts of advanced AI and the institutions and policies that could help societies respond well.^[2]

The extinction-risk statement

CAIS became widely known in May 2023, when it published a one-sentence statement on AI risk of extinction signed by hundreds of AI professors and leaders of major AI companies.^[1] Among the signatories were Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and other AI industry leaders.^[1] The statement's significance lay in its brevity and its signatories: a short claim about extreme risk, endorsed by people who were themselves building the systems in question.

Policy work

The Center for AI Safety Action Fund, an affiliated body, is a sponsor of the California bill SB 1047, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.^[1] Sponsoring legislation is a different kind of activity from research: it puts the organization into a concrete policy fight over how frontier models should be regulated.

Where it sits

CAIS occupies an unusual position in the AI safety landscape. It does technical work of the kind that appears at machine learning conferences, it organizes other researchers through courses, fellowships and compute, and it takes public positions through statements and legislation. Its founding in 2022 placed it at a moment when concern about large models was moving from a niche research topic into public debate, and its one-sentence statement became one of the more visible artifacts of that shift. The center's own description of its mission — research, field-building, advocacy — is a fair summary of how those strands fit together.^[2]