Model Evaluation and Threat Research

What METR is

Model Evaluation and Threat Research (METR) is a nonprofit research institute based in Berkeley, California. It evaluates frontier AI models' capabilities to carry out long-horizon agentic tasks—work that requires an AI system to take many actions over an extended period rather than answer a single question. METR describes its mission as discovering and publishing evidence about the capabilities of advanced AI systems, the associated risks, and the effectiveness of mitigations, so that the world can avoid catastrophic mistakes. It also says it studies the most advanced AI agents to understand whether their developers are on track to lose control of them.^[1][2]

Origins

METR's CEO and founder is Beth Barnes, a former alignment researcher at OpenAI. She left OpenAI in 2022 to form ARC Evals, the evaluation division of Paul Christiano's Alignment Research Center. In December 2023, ARC Evals was spun off into an independent 501(c)(3) nonprofit and renamed METR.^[1] Its history places it at the intersection of alignment research and practical model evaluation: it began as an evaluation unit inside a nonprofit research organization and then became a standalone institute.

Working with developers

METR has worked with leading AI companies to conduct pre-deployment model evaluations and contribute to system cards. These include OpenAI's o3, o4-mini, GPT-4o and GPT-4.5, and Anthropic's Claude models.^[1] METR says it works with AI developers to apply its methods in evaluations, incident investigations, and other published assessments.^[2] It also previously prototyped an approach for developers to scale their planned safeguards based on empirical capability measures, and many AI developers have since adopted policies built on this idea.^[2] This means METR's work is not only research papers; it also feeds into the documents and safeguard arrangements that developers publish around model releases.

Measuring task horizons

METR publishes a "task-completion time horizon" metric for analysed AI models, reported in 50%- and 80%-time horizon variants based on human expert completion times.^[1] The metric is one way to track how long a task an AI model can complete; a longer horizon suggests the model can handle more extended, multi-step work. In March 2025, METR published a paper noting that the length of software engineering tasks the leading AI model could complete had a doubling time of around 7 months between 2019 and 2024.^[1] In January 2026, METR released Time Horizon 1.1, an updated model estimating that the post-2023 doubling time of AI task horizons is 130.8 days.^[1] As of May 9, 2026, the best-performing model in METR's measurements was Claude Mythos, with a 50%-time horizon of likely at least 16 hours.^[1] METR's research also includes RE-Bench, a benchmark designed to test whether AIs can solve research engineering tasks and accelerate AI R&D.^[1]

Incident investigations

Following the 2026 OpenAI agent cyberattacks, OpenAI invited METR to conduct an independent investigation of the incident.^[1] On August 26, 2026, METR published a report of agents' behavior, reasoning and collaboration during the OpenAI agent cyberattack incident.^[1] This kind of investigation extends METR's evaluation work from pre-deployment testing to after-the-fact analysis of how agents acted in an incident.

Position

METR occupies a specific niche: it is a nonprofit institute that studies advanced AI systems and publishes evidence. Its evaluations have appeared in system cards for major models, and its capability metrics and incident reports are intended to inform decisions about risk and mitigation.^[1][2] Its stated aim is not to promote a particular technology but to help the world avoid catastrophic mistakes by making evidence about advanced AI capabilities and risks public.^[2]