Safety & governance

Geoffrey Hinton is a computer scientist born in London whose research on neural networks, including back-propagation and deep convolutional networks, is central to deep learning. He shared the 2018 ACM A.M. Turing Award and the 2024 Nobel Prize in Physics.

Canadian computer scientist whose work on sequence models, word embeddings and attention helped lay foundations for modern language models. He shared the 2018 ACM A.M. Turing Award with Geoffrey Hinton and Yann LeCun.

Dario Amodei is the co-founder and CEO of Anthropic. He previously worked at OpenAI, where he was Vice President of Research and co-authored the GPT-2 paper and led the research for GPT-3.

Israeli-Canadian computer scientist born in 1986 who co-built AlexNet, co-founded OpenAI and served as its chief scientist, and later founded Safe Superintelligence Inc.

Mobileye makes camera-based vision chips and software for cars, from basic driver assistance up to systems for driverless vehicles; its EyeQ chips have shipped in hundreds of millions of vehicles.

Cruise was a San Francisco autonomous-vehicle company founded in 2013 and acquired by General Motors in 2016; it ran paid driverless rides in San Francisco before its permits were suspended and GM wound the business down.

OpenAI is a San Francisco AI research and deployment company founded in 2015 as a nonprofit. It makes the GPT model family and ChatGPT and filed confidentially for an IPO in June 2026.

Reinforcement learning from human feedback trains a model toward what people prefer, using a reward model learned from human comparisons of its outputs. It became a standard step in building chat assistants.

Waymo's app-based ride-hailing service, in which the car drives itself. It was the first service to carry public passengers without a safety driver, and it is how Waymo brings autonomous driving to ordinary riders.

Tesla's most advanced driver-assistance package, sold as a $99 monthly subscription. It steers, changes lanes, follows routes and parks, while the driver stays responsible and the car is not autonomous.

Anthropic is an AI safety and research company founded in January 2021 by former OpenAI employees. It develops the Claude models and products, and its Responsible Scaling Policy sets out safety levels for its models.

A 2021 FAccT paper by Bender, Gebru, McMillan-Major and Shmitchell that asks whether language models can be too large and sets out recommendations on costs, data, research goals and alternative directions.

InstructGPT is a family of GPT-3 models trained with reinforcement learning from human feedback so that they follow user instructions. OpenAI announced them in January 2022 and made them the default models in its API.

InstructGPT used human-written demonstrations, human rankings of outputs and reinforcement learning to make GPT-3 follow user instructions better. Labelers preferred its outputs even though it was far smaller than GPT-3.

GPT-4 is OpenAI's large multimodal model, released on 14 March 2023. It accepts image and text input, scored near the top of human test-takers on a simulated bar exam, and was succeeded by GPT-4 Turbo, GPT-4o and GPT-5.

OpenAI o1 is the first reasoning model series from OpenAI, introduced in September 2024. It thinks through a long internal chain of thought before answering, and it was the base for later reasoning models such as o3.

Huawei's intelligent driving system released in Shanghai on 22 April 2025, built on a cloud-plus-vehicle WEWA architecture and offered in four versions, the flagship carrying a dedicated highway L3 package.

GPT-5 is OpenAI's fifth GPT foundation model, released on 7 August 2025. It pairs a fast model with a deeper reasoning model and a router that chooses between them for each request.

ByteDance's video generation model released in February 2026, built to produce picture and sound together from text, image, audio or video inputs, and known for a copyright dispute that paused its global rollout.

Claude Mythos Preview, announced on 7 April 2026, is an unreleased frontier model that Anthropic made available only to restricted partners through Project Glasswing, a programme for securing critical software.

Claude Fable 5 is Anthropic's Mythos-class model for general use, released on 9 June 2026. Its sibling Claude Mythos 5 is the same model with some safeguards lifted for vetted users.

GPT-6 is OpenAI's sixth-generation model family, launched from 3 September 2026 with GPT-6 Astra, followed by Sol, Luna and Instant. Astra was the first OpenAI model to reach its internal critical cybersecurity threshold.

Gemini 4 Argon is the first model in Google's Gemini 4 family, announced on 30 September 2026 and made generally available on 13 October 2026, built for long multi-step work and shipped with a one-million-token output limit.

Hallucination is false or misleading information that a model presents as fact. It is a central failure mode of text-generating systems and a main obstacle to trusting their output.

Alignment is the problem of making a model's behaviour match what its users and designers actually intend, not just what its training objective literally rewards. Training on human preference comparisons is one practical route to it.

The European Commission is the executive body of the European Union: it proposes laws and policies, monitors how they are applied, manages the EU budget and represents the EU externally.

AGI is the goal of AI systems that match or beat humans across a broad range of tasks. The term has no agreed definition; a framework by Morris et al. grades systems by performance level and generality.

NIST is a U.S. federal physical science laboratory founded in 1901 and now part of the Department of Commerce, which works on measurement science, standards and technology — and has recently become the home of the government's AI safety and evaluation work.

ELIZA is an early program for holding a natural-language conversation with a computer, built at MIT by Joseph Weizenbaum in the mid-1960s. Its DOCTOR script imitated a psychotherapist, and the readiness of users to treat it as understanding became known as the ELIZA effect.

Apple is a US technology company founded in 1976 that makes the iPhone, iPad and Mac. It introduced Apple Intelligence in 2024, builds its own Apple Foundation Models, and in 2026 agreed to build its next models with Google.

Mustafa Suleyman is a British AI entrepreneur who co-founded DeepMind and Inflection AI, and in 2024 became chief executive of Microsoft AI, leading its Copilot and in-house model work.

Daniela Amodei is an American entrepreneur and co-founder of Anthropic, where she serves as President and leads teams focused on AI safety and alignment. She previously worked at Stripe and OpenAI.

American AI researcher who co-founded OpenAI, led its reinforcement-learning team and helped develop the training methods behind ChatGPT; he later moved to Anthropic and then became chief scientist of Thinking Machines Lab.

Jakub Pachocki is OpenAI's chief scientist. He joined the company in 2017, led work on GPT-4 and OpenAI Five, and helped refocus its research on scaling deep learning systems.

A non-profit founded in 1998 that stewards the Open Source Definition and, since 2024, the Open Source AI Definition, setting a shared standard for what counts as open source.

A department of China's State Council, established in March 2008 to administer industry, communications and informatisation. It consolidated industrial and information-technology functions that had previously been spread across several government bodies.

The State Internet Information Office — the Cyberspace Administration of China — is the Chinese body that manages online information content, from news licensing and website enforcement to algorithm rules and AI content labeling.

Zoox is an autonomous ride-hailing company founded in 2014 that builds its own zero-emissions robotaxis, with no steering wheel or pedals, and the software that drives them. It began carrying paying-concept service in Las Vegas after receiving the first commercial exemption for a purpose-built robotaxi.

Deepfake is synthetic media: images, video or audio made or altered with AI. The term, a blend of "deep learning" and "fake", appeared in late 2017 and now names a whole class of face-swapping and manipulation techniques.

A 2017 paper showing that an AI agent can learn a task from short human comparisons instead of a hand-written reward. The method, reinforcement learning from human feedback, later became central to training chat assistants.

OpenAI's 2019 paper on GPT-2, a 1.5-billion-parameter Transformer trained on web text. It showed that a model trained only to predict the next word could perform several tasks when prompted, and its staged release sparked public debate.

GPT-2 is a large language model that OpenAI announced in February 2019. Its largest version has 1.5 billion parameters. OpenAI first withheld the full model, then released it in stages through November 2019.

EleutherAI is a grassroots AI research collective that began as a Discord server in July 2020. It released The Pile and the open GPT-style models GPT-Neo, GPT-J and GPT-NeoX-20B, and in 2023 it became a nonprofit research institute.

LaMDA is a family of conversational language models that Google announced in 2021 and described in a 2022 paper, notable for its work on safety and factual grounding in dialogue and for later powering Bard.

A 2022 paper by Percy Liang, Rishi Bommasani, Tony Lee and many co-authors that ran 30 language models through the same 42 scenarios and seven metrics, turning scattered benchmarks into one standardized evaluation.

Constitutional AI is a training method introduced by Anthropic in December 2022, in which a model's behaviour is shaped by a written constitution of principles rather than by human labels of harmful outputs.

Constitutional AI, a 2022 paper from Anthropic, proposes training a harmless AI assistant with AI-generated feedback guided by a written set of principles, instead of human labels of harmful outputs.

Direct preference optimization (DPO) is a method for fine-tuning language models on human preference data with a supervised-learning-style objective, removing the separate reward model and the reinforcement-learning loop of standard RLHF.

Prompt injection is an attack in which text inside a model's input overrides its original instructions, letting an adversary hijack a language-model application — including, as researchers showed in 2023, through content the application retrieves on its own.

Claude is the AI assistant Anthropic announced on 14 March 2023, its first publicly available assistant. It was built around training AI to be helpful, honest and harmless, and it started the Claude model line.

GPT-4V is a version of GPT-4 that accepts image input alongside text, documented in OpenAI's system card of 25 September 2023 and released to developers through preview API models; image understanding later became native in GPT-4o.

A UK government research body created in 2023 to study the most powerful AI models and AI safety, renamed in 2025 to focus on national security and serious crime. It advises government but has no regulatory power.

METR is a Berkeley-based nonprofit that evaluates frontier AI models on long-horizon agentic tasks, publishes capability metrics and incident investigations, and works with AI developers on pre-deployment assessments and safeguards.

The European AI Office is a body inside the European Commission that supports implementation of the EU AI Act and enforces the rules for general-purpose AI models. It was established in February 2024.

The Meta Llama 3 Community License is the custom agreement Meta attaches to its Llama 3 models: free to use and modify, but with use restrictions, attribution duties and a user-scale threshold that keep it from being open source in the usual sense.

Apple Intelligence is Apple's personal-intelligence system, announced at WWDC on 10 June 2024 for iOS 18, iPadOS 18 and macOS Sequoia. It runs models on device and uses Private Cloud Compute for larger requests.

Safe Superintelligence Inc. (SSI) is an American AI company founded in June 2024 by Ilya Sutskever, Daniel Gross and Daniel Levy to build a safe superintelligence as its only product, without releasing anything in between.

The Open Source AI Definition is a 1.0 standard, released by the Open Source Initiative in October 2024, that a model must meet to be called open source: reproducible design, disclosed training data, and freedom to use, modify and build on it.

Claude Opus 4 and Claude Sonnet 4, announced on 22 May 2025, were hybrid models with extended thinking. Anthropic called Opus 4 the best coding model in the world, and Claude Code became generally available the same day.

Claude Sonnet 4.5, released on 29 September 2025, was a drop-in replacement for Sonnet 4 at the same price. Anthropic reported that it could stay focused on complex tasks for more than 30 hours.

Sora 2 is OpenAI's second-generation video and audio generation model, unveiled on 30 September 2025 with a social iOS app. OpenAI discontinued Sora in 2026, and its API models were scheduled to shut down in September 2026.

OpenSandbox is a general-purpose sandbox platform for AI applications, contributed by Alibaba to the Agentic AI Foundation. It offers multi-language SDKs, a unified sandbox API, Docker and Kubernetes runtimes, and a protocol that lets teams plug in custom runtimes.

Released on 16 April 2026 at the same price as Opus 4.6, Claude Opus 4.7 added an xhigh effort level and accepted much larger images. It was the first model to test automatic blocking of high-risk cybersecurity requests.

GPT-5.5 is an OpenAI model released on 23 April 2026 as GPT-5.5 Thinking and Pro, with an API version the next day. It succeeded GPT-5.4 and had a cybersecurity variant in limited preview.

Meta's safety and preparedness report for Muse Spark, the model behind Meta AI, sets out the catastrophic-risk evaluations behind its launch and argues that, with mitigations applied, residual risk is acceptable.

OpenAI's three-model family released on 9 July 2026 — Luna, Terra and Sol, ordered from least to most capable — with Sol pitched as the company's strongest coding and agent model.

Claude Opus 5, released on 24 July 2026 at $5 and $25 per million tokens, is Anthropic's most aligned model to date and its most capable generally available model for scientific research.

GLM-5.3 is Zhipu's open-weight language model released on 14 August 2026. It keeps the GLM-5.2 base model and gains its improvements from post-training, with reported state-of-the-art open results on agentic coding and cybersecurity benchmarks.

Claude Fable 5.1, released on 1 September 2026, keeps the $10 and $50 per million token price of Fable 5, with cache reads 75% cheaper. Its sibling Claude Mythos 5.1 is limited to trusted-access programmes.

Gemini 3.8 Flash is Google's fast, low-cost model released on 2 September 2026, alongside a cybersecurity variant, Gemini 3.8 Flash Cyber, with introductory pricing that doubles at the start of 2027.

OpenAI's late-2026 model family: Sol, released at DevDay on 29 September 2026, and the faster, cheaper Terra on 26 October 2026. A planned Astra version was withheld over safety concerns.

Gemini 4 Carbon is a Google DeepMind frontier model announced on 28 October 2026, two weeks after Gemini 4 Argon, with Google claiming state-of-the-art cybersecurity and long-horizon capabilities.

OpenAI's cybersecurity-focused model, released on 5 November 2026 and limited to the top tier of its Daybreak Red programme; OpenAI says it surfaced 948 potential Linux kernel issues and 81 novel Firefox vulnerabilities.

Mechanistic interpretability explains model behaviour by finding the internal computations and features that produce it. Work on superposition and sparse autoencoders shows how many features can be packed into few neurons.

Red teaming is the practice of actively probing a model for harmful or unwanted behaviour before deployment. A 2022 method by Perez and colleagues automated it by using one language model to generate test cases for another.

Benchmark contamination means the test items a model is evaluated on, or close variants of them, were present in its pretraining data, so scores partly measure memory of seen text rather than the ability the benchmark is meant to probe.

Chris Olah is a Canadian machine learning researcher and co-founder of Anthropic, known for work on mechanistic interpretability, the effort to understand the internal workings of neural networks.

Jared Kaplan is a physicist turned AI researcher who co-founded Anthropic and serves as its Chief Science Officer. He was first author of the 2020 paper “Scaling Laws for Neural Language Models.”

A jailbreak is an adversarial prompt that gets a safety-trained model to produce behaviour its training was meant to prevent. A 2023 analysis by Wei et al. traced these failures to competing objectives and mismatched generalization.

Uber ATG was Uber's self-driving unit. After a fatal 2018 crash with a pedestrian that the NTSB traced to a distracted safety operator and weak safety oversight, Uber sold the group to Aurora in December 2020.

Jan Leike is an AI alignment researcher who joined Anthropic in May 2024 after leaving OpenAI, where he had co-led the superalignment project with Ilya Sutskever.

Reward hacking is what happens when a system optimises a proxy reward that only imperfectly captures what we want: it scores well on the proxy while performing badly on the true objective.

American politician born in San Francisco in 1967; founder of the PlumpJack Group, former mayor of San Francisco, and California's 40th governor, who has both signed and declined to sign frontier-AI legislation.

Founded in 2000 as the Singularity Institute, MIRI is a Berkeley nonprofit that was among the first to treat artificial intelligence risk as a technical research problem, and now advocates halting work on smarter-than-human AI.

American labor union formed in 2012 by the merger of the Screen Actors Guild and AFTRA, representing about 170,000 media professionals and known for its Actor Awards and its 2023 contract protections on digital replicas.

A nonprofit coalition of industry, academic, civil society and media organizations, formed in 2016 to guide the responsible development and use of artificial intelligence.

An independent, non-profit AI research institute in Toronto, launched in 2017 with provincial and corporate funding, with Geoffrey Hinton among its founding scientists.

The AI Now Institute is an American research institute, launched in 2017 in partnership with New York University, that studies the social implications of artificial intelligence and the concentration of power in the tech industry.

Stanford's Institute for Human-Centered Artificial Intelligence, launched in 2019 and co-directed by Fei-Fei Li and John Etchemendy, studies AI's opportunities and risks and houses the Center for Research on Foundation Models.

Content provenance is a checkable record of where a piece of media came from and how it was edited, attached to the file itself. The C2PA, founded in 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic, publishes the technical standard behind it.

Character.AI is a chatbot company founded in 2021 by former Google engineers Noam Shazeer and Daniel De Freitas, known for user-created AI characters; it filed for Chapter 11 bankruptcy in October 2026.

A San Francisco nonprofit founded in 2022 that researches technical AI safety, builds the field of safety researchers, and advocates for safety standards — known for a 2023 statement on AI extinction risk signed by industry leaders.

Galactica is a large language model for science, unveiled by Meta AI and Papers with Code in November 2022. Its public demo was taken down about three days later, after users found it produced fake papers.

A pair of medical question-answering language models described in arXiv papers of 26 December 2022 and 16 May 2023; the first introduced the MultiMedQA benchmark, and Med-PaLM 2 reached up to 86.5% on MedQA.

Text watermarking embeds a hidden statistical signal in generated text so that a short span of tokens can later be identified as machine output, while the text still reads normally to people.

Apple Foundation Models (AFM, 2024) are the first generation of the models behind Apple Intelligence: a roughly 3-billion-parameter on-device model and a larger server model that runs in Private Cloud Compute.

Comet is an AI web browser from Perplexity, launched on 9 July 2025 for Windows and macOS. Built on Chromium, it has an assistant that can carry out multistep browsing tasks.

Claude in Chrome is a Chrome extension that lets Claude see web pages, click buttons and fill in forms for the user. Anthropic began a pilot on 25 August 2025 and opened the extension to more plans through late 2025.

Released on 28 May 2026 at the same price as Opus 4.7, Claude Opus 4.8 added a fast mode at 2.5 times the speed, with fewer unnoticed flaws in its own code. Anthropic called it a modest but tangible improvement.

Grok 4.7 is xAI's September 2026 flagship model for coding and knowledge work, built on a larger base model than Grok 4.6 with longer reinforcement learning and a new safeguard stack, at unchanged token prices.

Shane Legg is a New Zealand AI researcher and co-founder of DeepMind, known for helping popularise the term "artificial general intelligence" and for leading the company's AGI safety work.

Aicha Evans is chief executive of Zoox, the autonomous ride-hailing company Amazon acquired, and stayed on to lead it as a standalone business after the deal.

SLIM (Secure Low-Latency Interactive Messaging) is a secure transport protocol for agent traffic. It carries A2A, MCP and SLIMRPC over one encrypted overlay, lets agents connect outbound and be reached by name, and publishes SDKs for six languages.

Redwood Research is an AI safety research group that has co-authored, with Anthropic, studies of deceptive behaviour in language models — sleeper-agent backdoors and alignment faking — as well as work on AI control protocols.

Jack Clark is a co-founder of Anthropic, where he has led policy and later public-benefit work and, from March 2026, the Anthropic Institute. He is also a co-author of the GPT-3 paper.

Connor Leahy is a German-American AI researcher and AI-safety advocate. He co-founded EleutherAI in 2020, co-founded the AI-safety company Conjecture, and in 2026 became US Director of the advocacy group ControlAI.

Apollo Research is a research organization focused on the risk of scheming: advanced AI systems covertly pursuing misaligned objectives. It evaluates frontier models for this behavior and advises policymakers.

Kyle Vogt is identified in General Motors' March 2016 announcement as the founder of Cruise Automation, the San Francisco startup GM acquired; the company he founded later ran driverless passenger service in San Francisco until California regulators suspended its permits in October 2023.

Jesse Levinson is a co-founder and chief technology officer of Zoox, the autonomous ride-hailing company acquired by Amazon, where he continued to lead after the deal.

The Recording Industry Association of America is the US trade body for companies that make and distribute recorded music. It lobbies, runs research, issues Gold and Platinum awards, and in 2024 sued two AI music generators.

The FCC is the US agency that administers the Communications Act of 1934. In 2024 it confirmed that restrictions on artificial or prerecorded voices already cover AI-generated voices, and proposed a US$6 million robocall penalty.

Lilian Weng is an AI researcher whose main public writing is her blog Lil'Log, which has carried her learning notes since 2017. Chinese-language coverage describes her as a Peking University alumna and reports on her years at OpenAI and her later moves.

Sam McCandlish is a co-founder and chief technology officer of Anthropic, a co-author of influential work on scaling laws and GPT-3, and the company's first Responsible Scaling Officer.

A British multinational design and engineering firm founded in 1946 by Sir Ove Arup, known for complex landmark projects such as the Sydney Opera House and for ownership held in trust for its employees.