Papers | TellWorld

Home›Papers

Papers

Papers that changed AI, with a plain-language explainer for each.

Total 43 · Clear filters

2026
Kimi K3: Open Frontier Intelligence
arXiv
Kimi K3 is Moonshot AI's 2.8-trillion-parameter open mixture-of-experts model with native vision and a one-million-token context, released with full weights. Its report describes the architecture, training and serving work behind it, and reports frontier-level results that still trail the strongest proprietary models.
Large language models Open source China
2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
Large language models Open source China
2026
Qwen3.5-Omni Technical Report
arXiv
The Qwen3.5-Omni Technical Report documents Alibaba's Qwen3.5-Omni, a hundreds-of-billions-parameter model that understands long audio and video and generates speech across 10 languages, reporting state-of-the-art results on 215 audio and audio-visual benchmarks.
Large language models Open source China
2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
Large language models China Agents & tool use
2025
Kimi K2: Open Agentic Intelligence
arXiv
Kimi K2 is Moonshot AI's trillion-parameter mixture-of-experts model, pre-trained with the MuonClip optimizer and post-trained on large-scale agentic data plus reinforcement learning; the report reports state-of-the-art results among open-source non-thinking models and releases base and post-trained checkpoints.
Large language models Open source China
2025
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
arXiv
An eight-terabyte collection of public domain and openly licensed text for pretraining language models, released together with two 7B-parameter models trained on it to show the data is enough.
Large language models Open source Data & datasets
2025
Qwen3 Technical Report
arXiv
The Qwen3 Technical Report presents Alibaba's Qwen3 family of dense and mixture-of-experts language models, which unify thinking and non-thinking modes, add a controllable thinking budget, and are released under the Apache 2.0 licence.
Large language models Open source China
2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
arXiv
The DeepSeek-R1 report described how a reasoning model was trained largely with reinforcement learning, rewarding correct answers, and reached performance comparable to OpenAI's o1 on math, code and reasoning benchmarks. Its weights were released openly.
Large language models Open source China
2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
Large language models Open source China
2024
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
arXiv
FineWeb is a 15-trillion-token pretraining dataset built from 96 Common Crawl snapshots and released in 2024 with its curation code and ablation models; its educational subset, FineWeb-Edu, improves results on knowledge- and reasoning-heavy benchmarks.
Large language models Open source Data & datasets
2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
arXiv
A benchmark for code LLMs that keeps collecting fresh contest problems from LeetCode, AtCoder and CodeForces, so that scores reflect ability rather than memorised test data, and that also tests code repair and execution reasoning.
Large language models Research history Data & datasets
2024
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
arXiv
Dolma is an openly released English corpus of three trillion tokens, built from web text, papers, code, books, social media and Wikipedia, with a paper documenting both the dataset and the pipeline used to curate it.
Large language models Open source Data & datasets
2023
Scaling deep learning for materials discovery
Nature
A 2023 Nature paper from researchers at Google DeepMind presenting a scaled-up deep-learning approach to predicting stable crystal structures, screening 2.2 million candidates and reporting a large jump in stability-prediction accuracy.
Deep learning Data & datasets Science
2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv
A benchmark of 2,294 real software engineering problems taken from GitHub issues and the pull requests that resolved them, in which a model must edit a codebase to fix an issue; on release the best model solved under 2% of them.
Research history Agents & tool use Data & datasets
2023
Segment Anything
arXiv 2304.02643
A 2023 paper that introduces the Segment Anything project: a promptable segmentation task, a model, and a dataset of over 1 billion masks on 11 million images, all released to advance foundation models for computer vision.
Research history Data & datasets Computer vision
2023
RedPajama: open reproduction of the LLaMA training dataset
Together AI blog / GitHub
RedPajama is an open-source project that began by reproducing the LLaMA training dataset, releasing a 1.2-trillion-token corpus as a clean-room, fully open implementation.
Large language models Open source Data & datasets
2022
Holistic Evaluation of Language Models (HELM)
arXiv
A 2022 paper by Percy Liang, Rishi Bommasani, Tony Lee and many co-authors that ran 30 language models through the same 42 scenarios and seven metrics, turning scattered benchmarks into one standardized evaluation.
Large language models Research history Safety & governance
2022
LAION-5B: An open large-scale dataset for training next generation image-text models
arXiv
LAION-5B is an openly available dataset of 5.85 billion CLIP-filtered image-text pairs, released so that researchers outside a handful of labs could study and reproduce large vision-language models.
Open source Multimodal Data & datasets
2022
Training Compute-Optimal Large Language Models (Chinchilla)
NeurIPS
A DeepMind study showing that, for a fixed computing budget, language models should be trained on far more data than was then standard; its 70-billion-parameter Chinchilla beat larger contemporaries.
Large language models Foundation models Data & datasets
2021
Program Synthesis with Large Language Models (MBPP)
arXiv
A 2021 arXiv paper by Austin, Odena, Nye and eight other authors that introduced MBPP, a benchmark of 974 short Python programming tasks, and measured how language models from 244M to 137B parameters turn natural-language descriptions into working code.
Large language models Research history Data & datasets