Papers | TellWorld

Home›Papers

Papers

Papers that changed AI, with a plain-language explainer for each.

Total 28 · Clear filters

2026
Muse Spark Safety & Preparedness Report
arXiv
Meta's safety and preparedness report for Muse Spark, the model behind Meta AI, sets out the catastrophic-risk evaluations behind its launch and argues that, with mitigations applied, residual risk is acceptable.
Large language models Foundation models Safety & governance
2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
Large language models Open source China
2025
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
arXiv
The Gemini 2.5 technical report, submitted to arXiv on 7 July 2025, documents Google's Gemini 2.5 Pro and 2.5 Flash models: their reasoning, long-context video handling, and cost-versus-capability positioning.
Large language models Foundation models Multimodal
2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
Large language models Open source China
2024
The Llama 3 Herd of Models
arXiv
Meta's 2024 paper describing Llama 3 as a family of foundation models — 8B, 70B and 405B parameters — trained on about 15 trillion tokens and reported to match leading models such as GPT-4 across a wide range of tasks.
Large language models Research history Open source
2024
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
arXiv
A report from Google's Gemini Team introducing the Gemini 1.5 family, multimodal models built around very long context, with Gemini 1.5 Pro recalling information across up to 10 million tokens.[^1]
Large language models Research history Foundation models
2022
Scalable Diffusion Models with Transformers (DiT)
arXiv
A 2022 paper by William Peebles and Saining Xie that replaces the U-Net backbone of diffusion models with a transformer operating on latent patches, showing that image quality improves predictably as compute grows and reaching a state-of-the-art FID of 2.27 on ImageNet 256×256.
Research history Generative models Image generation
2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (Imagen)
arXiv
Imagen is a text-to-image diffusion model that uses a large pretrained language model to encode prompts; it reached a zero-shot FID of 7.27 on COCO and introduced the DrawBench benchmark.
Research history Multimodal Generative models
2022
Training Compute-Optimal Large Language Models (Chinchilla)
NeurIPS
A DeepMind study showing that, for a fixed computing budget, language models should be trained on far more data than was then standard; its 70-billion-parameter Chinchilla beat larger contemporaries.
Large language models Foundation models Data & datasets
2021
High-Resolution Image Synthesis with Latent Diffusion Models
arXiv 2112.10752 (CVPR 2022)
A 2021 paper by Rombach, Blattmann, Lorenz, Esser and Ommer that runs diffusion models inside the latent space of pretrained autoencoders, cutting the compute needed for high-resolution image synthesis while adding cross-attention for text and bounding-box control.
Research history Generative models Image generation
2021
Zero-Shot Text-to-Image Generation (DALL·E)
arXiv
A 2021 OpenAI paper describing a single 12-billion-parameter autoregressive transformer trained on 250 million image-text pairs, which generates images from text and, evaluated zero-shot, is competitive with domain-specific models.
Research history Multimodal Generative models
2021
Learning Transferable Visual Models From Natural Language Supervision (CLIP)
ICML
CLIP is an OpenAI model trained on 400 million image-text pairs to map pictures and sentences into one shared space, so it can classify images zero-shot from a written description.
Foundation models Multimodal Computer vision
2020
Denoising Diffusion Probabilistic Models
NeurIPS
A 2020 paper by Jonathan Ho, Ajay Jain and Pieter Abbeel that trained a U-Net to reverse a gradual noising process, making diffusion models a practical rival to GANs and paving the way for DALL-E 2, Imagen and Stable Diffusion.
Deep learning Generative models Image generation
2020
Language Models are Few-Shot Learners (GPT-3)
NeurIPS
The GPT-3 paper showed that a language model with 175 billion parameters could perform many tasks from a few examples in its prompt, with no fine-tuning, and that this ability grew with model size.
Large language models Foundation models
2020
Scaling Laws for Neural Language Models
arXiv
A 2020 paper finding that language model loss falls predictably, as a power law, with model size, data and compute. It gave labs a quantitative reason to build much larger models and shaped the scale-up that followed.
Large language models Foundation models Hardware & compute
2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
arXiv 1909.08053
An NVIDIA paper from 2019 presenting an intra-layer model parallelism method, implemented with a few communication operations in native PyTorch, that trained transformer models of up to 8.3 billion parameters and improved several benchmarks.
Research history Foundation models Hardware & compute
2019
Language Models are Unsupervised Multitask Learners (GPT-2)
OpenAI
OpenAI's 2019 paper on GPT-2, a 1.5-billion-parameter Transformer trained on web text. It showed that a model trained only to predict the next word could perform several tasks when prompted, and its staged release sparked public debate.
Large language models Foundation models Safety & governance
2018
A Style-Based Generator Architecture for Generative Adversarial Networks
arXiv
This 2018 paper introduces a style-based GAN generator that separates high-level attributes from stochastic variation and allows scale-specific control, improving distribution quality, interpolation, and disentanglement.
Generative models Computer vision Image generation
2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
NAACL
Google's 2018 BERT paper introduced an encoder-only Transformer that reads text in both directions and is pre-trained by hiding words. It set new results on eleven language tasks and made download-and-fine-tune the default workflow.
Large language models Research history Foundation models
2018
Large Scale GAN Training for High Fidelity Natural Image Synthesis
arXiv
A 2018 paper by Andrew Brock, Jeff Donahue and Karen Simonyan that trained generative adversarial networks at the largest scale attempted at the time, introduced BigGANs and set a new state of the art in class-conditional image synthesis.
Generative models Computer vision Image generation