Models

AlphaProof is a Google DeepMind reinforcement-learning system that proves mathematical statements in the formal language Lean. At the 2024 International Mathematical Olympiad it solved three problems, and together with AlphaGeometry 2 reached silver-medal level.

ReasoningScienceSymbolic AI

Mistral AI's second-generation flagship model, released in July 2024 with 123 billion parameters, a 128K-token context window, wide language and code coverage, and weights published under a research license.

Large language modelsEnterpriseCode

Mistral NeMo is a 12-billion-parameter language model released by Mistral AI with NVIDIA in July 2024, notable for a 128K-token context window, a new multilingual tokenizer called Tekken, and an Apache 2.0 release.

Large language modelsOpen sourceHardware & compute

SmolLM is a family of small language models with 135M, 360M and 1.7B parameters, released openly by Hugging Face in July 2024 for on-device use and trained on a purpose-built corpus of generated textbooks, code and filtered web text.

Large language modelsOpen sourceConsumer apps

A 7-billion-parameter model released by Mistral AI on 16 July 2024 for mathematical and STEM reasoning, built on Mistral 7B, openly licensed, and now listed as deprecated.

Open sourceReasoningScience

Codestral Mamba is a 7B open code model from Mistral AI, released in July 2024, that swaps the Transformer for the Mamba state-space architecture to get linear-time inference.

Open sourceDeveloper toolsCode

iFlytek's large language model released on 27 June 2024 and trained entirely on a domestic platform co-built with Huawei; iFlytek presented it as reaching GPT-4 level while acknowledging weaker coding and multimodal abilities.

Large language modelsChinaFoundation models

Claude 3.5 Sonnet, released on 20 June 2024, was the first model of the Claude 3.5 family. It ran twice as fast as Claude 3 Opus at the mid-tier price and powered the new Artifacts and Projects features.

Large language modelsEnterpriseConsumer apps

An open-source mixture-of-experts code model from DeepSeek, released in June 2024, that widened language coverage and context length over its predecessor and reported benchmark results above several closed models.

Large language modelsOpen sourceChina

Nemotron-4 340B is a family of open models from NVIDIA, released on 14 June 2024 as base, instruct and reward models. It was built to generate synthetic data for training other language models, and it fits on a single eight-GPU DGX H100.

Large language modelsOpen sourceEnterprise

Codestral is Mistral AI's family of code models, first released in May 2024 as an open-weight 22B-parameter model with a 32K context window, followed by faster and longer-context versions aimed at IDE code completion.

Large language modelsDeveloper toolsCode

Snowflake's open-weight model with 480 billion total and 17 billion active parameters, built as a dense-MoE hybrid, released in April 2024 under Apache 2.0 and trained for under 2 million dollars.

Large language modelsOpen sourceEnterprise

OpenELM is a family of small open language models from Apple, published in four sizes from 270M to 3B parameters with its training framework, logs, checkpoints and MLX conversion code.

Large language modelsResearch historyOpen source

Grok-1.5 is xAI's upgrade to Grok-1, with better reasoning and a 128,000-token context window. Its companion Grok-1.5V was xAI's first multimodal model. Neither model's weights were released publicly.

Large language modelsMultimodalReasoning

DBRX is Databricks' openly licensed large language model, a mixture-of-experts transformer with 132 billion total parameters and 36 billion active per token, announced on 27 March 2024.

Large language modelsOpen sourceEnterprise

Claude 3 Opus, announced on 4 March 2024, was the most capable model in the Claude 3 family. It added vision and launched with a 200K-token context window and a Needle In A Haystack recall above 99%.

Large language modelsEnterpriseMultimodal

StarCoder2 is a family of open code models in 3B, 7B and 15B sizes, released in February 2024 by BigCode (Hugging Face and ServiceNow) with NVIDIA and trained on The Stack v2, published together with its dataset and training code.

Large language modelsOpen sourceData & datasets

DeepSeekMath is a 7B-parameter open language model built for mathematical reasoning and released in 2024. It reached 51.7% on the competition-level MATH benchmark without external tools, and introduced GRPO, a reinforcement-learning algorithm that uses less memory than PPO.

Large language modelsOpen sourceChina

Aya 101 is a 13-billion-parameter, instruction-tuned language model from Cohere For AI that follows instructions in 101 languages. It was released under Apache 2.0 in February 2024.

Large language modelsOpen sourceFoundation models

Spark V3.5 is a large language model that iFlytek released on 30 January 2024. It was the first Spark model trained entirely on the domestic FlyingStar One computing platform, and iFlytek compared it with GPT-4 Turbo.

Large language modelsChinaFoundation models

AlphaGeometry is a Google DeepMind system that pairs a neural language model with a symbolic deduction engine to solve olympiad geometry problems, reaching near gold-medal level on a benchmark of 30 problems.

Open sourceReasoningScience

FunSearch is a method that uses a pretrained large language model inside an evolutionary loop to search for programs that solve mathematical and computational problems, rather than for answers directly. It produced new results on the cap set problem and on online bin packing.

Deep learningReasoningScience

Phi-2 is a 2.7-billion-parameter base language model from Microsoft Research, announced on 12 December 2023. Microsoft reported that it matched or beat models up to 25 times larger on complex benchmarks, making it a reference point for small-model training.

Large language modelsOpen sourceReasoning

GNoME is a Google DeepMind deep-learning tool, announced on 29 November 2023, that predicts new inorganic crystals; it produced 2.2 million candidate structures, of which 380,000 were predicted to be stable.

Deep learningData & datasetsScience

DeepSeek's first model release: a family of open-source code models from 1.3B to 33B parameters, trained from scratch on 2 trillion tokens and reported to beat Codex and GPT-3.5 on several benchmarks.

Large language modelsOpen sourceChina

Phi-1.5 is a 1.3-billion-parameter language model from Microsoft Research, trained on curated synthetic "textbook" text rather than web crawls; its report claims natural-language performance comparable to models five times larger.

Large language modelsOpen sourceReasoning

Code Llama is a family of coding models that Meta released in August 2023 by further training Llama 2 on code. It ranges from 7 billion to 70 billion parameters and uses the Llama 2 community license.

Large language modelsOpen sourceDeveloper tools

SparkDesk V2.0 is the second version of iFlytek's large language model, launched on 15 August 2023 with upgraded coding and multimodal abilities, and released alongside the iFlyCode 1.0 programming assistant.

Large language modelsChinaFoundation models

Phi-1 is a 1.3-billion-parameter Transformer language model for Python from Microsoft Research, which showed that filtered and synthetic "textbook quality" training data can partly substitute for model and data scale.

Large language modelsOpen sourceData & datasets

StarCoder is an open-access code language model of about 15.5 billion parameters, released on 4 May 2023 by the BigCode project led by Hugging Face and ServiceNow.

Large language modelsOpen sourceData & datasets

GPT-3.5 is the family of OpenAI language models between GPT-3 and GPT-4, released through API models such as text-davinci-002 and text-davinci-003 and underlying the first version of ChatGPT.

Large language modelsFoundation modelsProduct history

AlphaCode is a DeepMind code-generation system for competitive programming, announced in February 2022. On Codeforces contests held after its training data, it reached an estimated rank around the median participant. A Gemini-based AlphaCode 2 followed in December 2023.

Large language modelsResearch historyCode

Megatron-Turing NLG 530B is a 530-billion-parameter language model announced by NVIDIA and Microsoft on 11 October 2021, then the largest monolithic transformer language model, trained on NVIDIA's Selene supercomputer with Megatron-LM and DeepSpeed combined.

Large language modelsFoundation modelsHardware & compute

MuZero is a DeepMind reinforcement-learning algorithm that pairs tree-based search with a learned model. It reached superhuman play in Atari, Go, chess and shogi without being given the environment's rules.

Research historyDeep learningReasoning

T5 is a family of Google Transformer models that recasts every language task as text in, text out. Its 11-billion-parameter version set state of the art on several benchmarks, and the code, models and C4 dataset were released openly.

Large language modelsOpen sourceFoundation models

AlphaStar is a DeepMind system that plays StarCraft II. In 2019 it reached Grandmaster level with all three races, ranking above 99.8% of active players on Battle.net.

Research historyDeep learningReasoning

AlphaZero is a general game-playing system from DeepMind, described in a preprint in December 2017, that reached a superhuman level in chess, shogi and Go within 24 hours starting from random play, using only the rules of each game.

Research historyDeep learningReasoning

IBM Watson is a question-answering computer system built by an IBM research team led by David Ferrucci; in February 2011 it defeated the two foremost all-time champions of the televised quiz show Jeopardy!

Research historyEnterpriseReasoning

IBM's chess computer that in May 1997 became the first computer system to defeat a reigning world chess champion in a match under standard tournament time controls.

Research historyProduct historyHardware & compute