Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 43 · Clear filters
Information theoryA framework that treats information as a measurable quantity to be encoded and transmitted, founded in Shannon's 1948 paper; it gave the field the bit, entropy and channel capacity.k-means clusteringk-means clustering splits n observations into k groups by repeatedly sending each point to its nearest centroid and recomputing each centroid as its cluster's mean. It is simple and scales to large data, but k must be fixed in advance.Support vector machineA supervised maximum-margin model for classification and regression that separates classes by the widest possible margin and uses kernel functions to handle non-linear data.Cross-validationA model validation technique that repeatedly splits data into folds, trains on some parts and validates on others, then averages the results to estimate how well a model or statistical result will generalise to unseen data.Autonomous drivingAutonomous driving is the operation of a road vehicle by a computer system instead of a person. Its six automation levels, its two main technical traditions and its long line of research vehicles define how the field is discussed today.TokenizationTokenization splits text into the tokens a language model actually reads; most modern systems do it with byte pair encoding, a 1994 compression method adapted for translation in 2015.AdaBoostAdaBoost, short for Adaptive Boosting, is a statistical classification meta-algorithm introduced by Yoav Freund and Robert Schapire in 1995. It reweights training examples, trains weak classifiers in sequence and combines their outputs into one weighted classifier.Topic modelTopic models represent documents as mixtures of latent topics, giving a compact statistical description of text collections. The 2003 Latent Dirichlet Allocation paper formalized this idea as a hierarchical Bayesian model.Connectionist Temporal ClassificationConnectionist temporal classification (CTC) is a method for training recurrent neural networks to label unsegmented sequences, such as speech or handwriting, without frame-by-frame alignment. Introduced in 2006, it let CTC-trained LSTMs win the first international contests for recurrent networks.Xavier (Glorot) initializationXavier (Glorot) initialization is a weight-initialization scheme from Glorot and Bengio's 2010 paper that sets each layer's starting weight scale so signals stay stable across layers, giving substantially faster convergence in deep feedforward networks.Attention mechanismAttention scores how important each element of a sequence is relative to the others and turns those scores into weights, letting a model read any part of its input directly. It is the operation at the core of the transformer.Attention and self-attentionAttention lets a neural network weigh the parts of its input by relevance. Self-attention applies this idea inside a single sequence, so each token gathers information from every other token. It is the core operation of the Transformer.Knowledge distillationKnowledge distillation trains a small model to reproduce the outputs of a larger model or an ensemble, so that much of the predictive performance is kept at lower compute and deployment cost.Positional encodingA component of sequence models, especially transformers, that encodes each token's position as a vector, giving order-blind attention a sense of sequence; its forms range from the original transformer's sinusoids to RoPE and ALiBi.TransformerThe Transformer is a neural network architecture introduced in 2017 that relies on self-attention instead of recurrence or convolution. Most of today's large language models are built on it.Retrieval-augmented generationRetrieval-augmented generation (RAG) lets a language model fetch relevant documents from an external store and put them into its prompt before answering, so it can use knowledge beyond its training data and cite where an answer came from.LoRA (low-rank adaptation)LoRA (low-rank adaptation) is a fine-tuning technique that freezes a pre-trained model's weights and trains small low-rank matrices instead, cutting the number of trainable parameters and the memory needed while keeping quality close to full fine-tuning.Speculative decodingSpeculative decoding speeds up text generation from large autoregressive models: a smaller model proposes several tokens and the large model checks them in parallel, producing identical output faster.Test-time computeTest-time compute is the computation a model spends while answering a prompt, beyond what training already fixed into its weights. A 2024 study by Snell et al. showed how to spend it adaptively, and how it can stand in for a bigger model.Computer useComputer use is an Anthropic capability, released in public beta in October 2024, that lets an AI model operate a computer the way a person does — reading the screen, moving a cursor, clicking and typing — so it can drive ordinary software built for humans.