Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 32 · Clear filters
Information theoryA framework that treats information as a measurable quantity to be encoded and transmitted, founded in Shannon's 1948 paper; it gave the field the bit, entropy and channel capacity.Turing testThe Turing test is Alan Turing's 1950 proposal for replacing the question "Can machines think?" with a game: a machine passes if an interrogator cannot tell its written answers apart from a human's.AI winterAI winter is a period of sharply reduced funding and interest in artificial intelligence research. The field's hype cycles have produced two such downturns, usually dated to about 1974–1980 and 1987–2000.Hopfield networkA recurrent neural network introduced by John Hopfield in 1982 that stores patterns as minima of an energy function and recalls them from partial or distorted input, bringing statistical mechanics into the study of neural networks.Boltzmann machineA Boltzmann machine is a neural network whose configurations are scored by a global energy and trained by a stochastic learning algorithm. Introduced by Geoffrey Hinton and Terry Sejnowski, it brought energy-based models into cognitive science and machine learning.Imitation learningImitation learning trains an agent by supervised learning on expert demonstrations rather than by trial and error. Its simplest form is behaviour cloning, and its best-known weakness is distribution shift.Topic modelTopic models represent documents as mixtures of latent topics, giving a compact statistical description of text collections. The 2003 Latent Dirichlet Allocation paper formalized this idea as a hierarchical Bayesian model.Connectionist Temporal ClassificationConnectionist temporal classification (CTC) is a method for training recurrent neural networks to label unsegmented sequences, such as speech or handwriting, without frame-by-frame alignment. Introduced in 2006, it let CTC-trained LSTMs win the first international contests for recurrent networks.Graph neural networkGraph neural networks are neural networks built for graph-structured data, in which nodes repeatedly exchange messages with their neighbours. Since the model was formulated in 2009, they have become a standard way of learning on graphs.Deep neural network acoustic modelingDeep neural network acoustic modeling uses multilayer networks to map speech to units such as phones. A 2010 NIPS paper used Deep Belief Nets and a mean-covariance RBM to extract speech features, reaching 20.5% phone error on TIMIT.Xavier (Glorot) initializationXavier (Glorot) initialization is a weight-initialization scheme from Glorot and Bengio's 2010 paper that sets each layer's starting weight scale so signals stay stable across layers, giving substantially faster convergence in deep feedforward networks.Knowledge distillationKnowledge distillation trains a small model to reproduce the outputs of a larger model or an ensemble, so that much of the predictive performance is kept at lower compute and deployment cost.Content provenance (C2PA)Content provenance is a checkable record of where a piece of media came from and how it was edited, attached to the file itself. The C2PA, founded in 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic, publishes the technical standard behind it.LoRA (low-rank adaptation)LoRA (low-rank adaptation) is a fine-tuning technique that freezes a pre-trained model's weights and trains small low-rank matrices instead, cutting the number of trainable parameters and the memory needed while keeping quality close to full fine-tuning.Speculative decodingSpeculative decoding speeds up text generation from large autoregressive models: a smaller model proposes several tokens and the large model checks them in parallel, producing identical output faster.Text watermarkingText watermarking embeds a hidden statistical signal in generated text so that a short span of tokens can later be identified as machine output, while the text still reads normally to people.Prompt injectionPrompt injection is an attack in which text inside a model's input overrides its original instructions, letting an adversary hijack a language-model application — including, as researchers showed in 2023, through content the application retrieves on its own.Computer useComputer use is an Anthropic capability, released in public beta in October 2024, that lets an AI model operate a computer the way a person does — reading the screen, moving a cursor, clicking and typing — so it can drive ordinary software built for humans.Open Source AI Definition (OSAID)The Open Source AI Definition is a 1.0 standard, released by the Open Source Initiative in October 2024, that a model must meet to be called open source: reproducible design, disclosed training data, and freedom to use, modify and build on it.Vision-Language-Action model (VLA)A vision-language-action model (VLA) is a robot policy that takes in camera images and a language instruction and outputs actions directly, usually by adapting a pretrained vision-language model to control.