Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 60 · Clear filters
- 2026
Muse Spark Safety & Preparedness Report
arXiv
Meta's safety and preparedness report for Muse Spark, the model behind Meta AI, sets out the catastrophic-risk evaluations behind its launch and argues that, with mitigations applied, residual risk is acceptable.
- 2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
- 2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
- 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
arXiv
Terminal-Bench 2.0 is a curated benchmark of 89 hard tasks carried out in computer terminal environments, built to measure how well AI agents handle realistic command-line work; frontier models and agents score below 65% on it.
- 2025
Kimi K2: Open Agentic Intelligence
arXiv
Kimi K2 is Moonshot AI's trillion-parameter mixture-of-experts model, pre-trained with the MuonClip optimizer and post-trained on large-scale agentic data plus reinforcement learning; the report reports state-of-the-art results among open-source non-thinking models and releases base and post-trained checkpoints.
- 2025
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
arXiv
The Gemini 2.5 technical report, submitted to arXiv on 7 July 2025, documents Google's Gemini 2.5 Pro and 2.5 Flash models: their reasoning, long-context video handling, and cost-versus-capability positioning.
- 2025
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
arXiv
An eight-terabyte collection of public domain and openly licensed text for pretraining language models, released together with two 7B-parameter models trained on it to show the data is enough.
- 2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
- 2024
The Llama 3 Herd of Models
arXiv
Meta's 2024 paper describing Llama 3 as a family of foundation models — 8B, 70B and 405B parameters — trained on about 15 trillion tokens and reported to match leading models such as GPT-4 across a wide range of tasks.
- 2024
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
arXiv
FineWeb is a 15-trillion-token pretraining dataset built from 96 Common Crawl snapshots and released in 2024 with its curation code and ablation models; its educational subset, FineWeb-Edu, improves results on knowledge- and reasoning-heavy benchmarks.
- 2024
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
arXiv
τ-bench is a benchmark, submitted to arXiv in June 2024, that tests language agents in dynamic conversations with a simulated user, with domain-specific API tools and policy rules, and scores them by database state rather than by the text of their replies.
- 2024
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
arXiv
OSWorld is a benchmark and environment for testing multimodal agents on 369 real computer tasks in Ubuntu, Windows and macOS; humans complete over 72.36% of the tasks, while the best model reaches only 12.24%.
- 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
arXiv
A benchmark for code LLMs that keeps collecting fresh contest problems from LeetCode, AtCoder and CodeForces, so that scores reflect ability rather than memorised test data, and that also tests code repair and execution reasoning.
- 2024
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
arXiv
A report from Google's Gemini Team introducing the Gemini 1.5 family, multimodal models built around very long context, with Gemini 1.5 Pro recalling information across up to 10 million tokens.[^1]
- 2024
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
arXiv
Dolma is an openly released English corpus of three trillion tokens, built from web text, papers, code, books, social media and Wikipedia, with a paper documenting both the dataset and the pipeline used to curate it.
- 2023
Scaling deep learning for materials discovery
Nature
A 2023 Nature paper from researchers at Google DeepMind presenting a scaled-up deep-learning approach to predicting stable crystal structures, screening 2.2 million candidates and reporting a large jump in stability-prediction accuracy.
- 2023
GAIA: a benchmark for General AI Assistants
arXiv
GAIA is a benchmark of real-world questions that are easy for humans but hard for AI assistants, testing reasoning, multimodality, web browsing, and tool use. It compares AI progress against average human performance.
- 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv
A benchmark of 2,294 real software engineering problems taken from GitHub issues and the pull requests that resolved them, in which a model must edit a codebase to fix an issue; on release the best model solved under 2% of them.
- 2023
WebArena: A Realistic Web Environment for Building Autonomous Agents
arXiv
WebArena is a reproducible environment of four fully functional websites — e-commerce, a social forum, collaborative software development and content management — built for testing language-guided web agents. Its best GPT-4 agent completed 14.41% of tasks; humans completed 78.24%.[^1]
- 2023
Segment Anything
arXiv 2304.02643
A 2023 paper that introduces the Segment Anything project: a promptable segmentation task, a model, and a dataset of over 1 billion masks on 11 million images, all released to advance foundation models for computer vision.