Deep learning Concepts

An artificial neural network is a computing system of many simple connected units whose behaviour is set by connection weights. Its roots lie in 1943 models of neurons, and its learning versions began with Frank Rosenblatt's perceptron in 1958.

A recurrent neural network introduced by John Hopfield in 1982 that stores patterns as minima of an energy function and recalls them from partial or distorted input, bringing statistical mechanics into the study of neural networks.

Autonomous driving is the operation of a road vehicle by a computer system instead of a person. Its six automation levels, its two main technical traditions and its long line of research vehicles define how the field is discussed today.

Backpropagation is the procedure that lets a multilayer neural network learn by sending output errors backward to adjust its weights, using gradient descent. A 1986 Nature paper made it widely known, and it is standard in most neural networks today.

Imitation learning trains an agent by supervised learning on expert demonstrations rather than by trial and error. Its simplest form is behaviour cloning, and its best-known weakness is distribution shift.

A convolutional neural network applies the same small pattern detectors across a whole image, so a feature is found wherever it appears. Trained with backpropagation in 1989 and refined as LeNet-5 in 1998, it became standard in computer vision.

Attention scores how important each element of a sequence is relative to the others and turns those scores into weights, letting a model read any part of its input directly. It is the operation at the core of the transformer.

A generative adversarial network pits a generator that makes fake data against a discriminator that judges it, both trained in a zero-sum game; proposed by Ian Goodfellow and colleagues in 2014, it led to a series of architectures for generating images.

Attention lets a neural network weigh the parts of its input by relevance. Self-attention applies this idea inside a single sequence, so each token gathers information from every other token. It is the core operation of the Transformer.

A residual connection adds a block's input to its output, so the block learns only the change it needs to make. This makes very deep networks trainable, and it is a standard part of today's transformers.

The Transformer is a neural network architecture introduced in 2017 that relies on self-attention instead of recurrence or convolution. Most of today's large language models are built on it.

Pre-training trains a large model on huge amounts of unlabeled text, and fine-tuning then adapts it to a specific task with a small labeled dataset. This two-stage recipe is the basis of the term foundation model.

Quantization stores a language model's weights and activations at lower numerical precision, cutting the memory and bandwidth that inference requires. Methods such as LLM.int8() and GPTQ made very large models runnable on far smaller hardware.

Reinforcement learning is the branch of machine learning and optimal control in which an agent learns, by trial and error, which actions in a dynamic environment maximise a reward signal.

Unsupervised learning trains algorithms on data that carry no labels, letting them find structure such as clusters, low-dimensional representations or generative models. It underlies clustering tools and neural generative models from the Hopfield network and Boltzmann machine to the variational autoencoder.

A multimodal model handles more than one kind of input or output, such as images together with text, inside a single model. Work on CLIP and Flamingo showed how such models are trained and how they can be steered toward new tasks.

Supervised learning trains a model to map inputs to outputs using paired labelled examples, so that it can predict answers for new, unseen data. Its central measure of success is how well the model generalises beyond the training set.

A vision-language-action model (VLA) is a robot policy that takes in camera images and a language instruction and outputs actions directly, usually by adapting a pretrained vision-language model to control.

A supervised maximum-margin model for classification and regression that separates classes by the widest possible margin and uses kernel functions to handle non-linear data.

A Boltzmann machine is a neural network whose configurations are scored by a global energy and trained by a stochastic learning algorithm. Introduced by Geoffrey Hinton and Terry Sejnowski, it brought energy-based models into cognitive science and machine learning.

A recurrent neural network (RNN) is a neural network with feedback connections that carry information from one step of a sequence to the next. Its long short-term memory variant (LSTM) was designed to keep useful memories over longer spans.

Tokenization splits text into the tokens a language model actually reads; most modern systems do it with byte pair encoding, a 1994 compression method adapted for translation in 2015.

AdaBoost, short for Adaptive Boosting, is a statistical classification meta-algorithm introduced by Yoav Freund and Robert Schapire in 1995. It reweights training examples, trains weak classifiers in sequence and combines their outputs into one weighted classifier.

A word embedding represents each word as a list of numbers learned from text, so that words used in similar ways get similar lists. Embeddings are a basic building block of modern language models.

Connectionist temporal classification (CTC) is a method for training recurrent neural networks to label unsegmented sequences, such as speech or handwriting, without frame-by-frame alignment. Introduced in 2006, it let CTC-trained LSTMs win the first international contests for recurrent networks.

Deep neural network acoustic modeling uses multilayer networks to map speech to units such as phones. A 2010 NIPS paper used Deep Belief Nets and a mean-covariance RBM to extract speech features, reaching 20.5% phone error on TIMIT.

Graphics processing units (GPUs) run many arithmetic operations in parallel, which suits neural-network training. Their use from 2010 onward made deep learning practical at much larger scales.

Xavier (Glorot) initialization is a weight-initialization scheme from Glorot and Bengio's 2010 paper that sets each layer's starting weight scale so signals stay stable across layers, giving substantially faster convergence in deep feedforward networks.

A variational autoencoder is a generative model whose encoder maps an input to a distribution over a latent space and whose decoder reconstructs the input from a sample; training maximises a lower bound on the data likelihood.

Knowledge distillation trains a small model to reproduce the outputs of a larger model or an ensemble, so that much of the predictive performance is kept at lower compute and deployment cost.

A technique that re-centres each layer's inputs to zero and rescales them to unit variance over a mini-batch, making deep networks faster and more stable to train.

A component of sequence models, especially transformers, that encodes each token's position as a vector, giving order-blind attention a sense of sequence; its forms range from the original transformer's sinusoids to RoPE and ALiBi.

Tensor Cores are specialised units inside NVIDIA GPUs that perform a small matrix multiply-and-add in one step, making the matrix arithmetic behind neural-network training and inference dramatically faster.

Diffusion models learn to reverse a gradual noising process, so they can turn random noise into a new image by removing noise step by step. Latent diffusion, which runs this in a compressed space, became the basis of Stable Diffusion.

A state space model represents a sequence as a hidden state that evolves with the input. S4 brought the idea into deep learning in 2021, and Mamba made its parameters input-dependent, enabling linear-time modeling of long sequences.

Sim-to-real transfer is the practice of training robot policies in simulation and then adapting them to physical robots, using synthetic data to supplement costly real-world demonstrations.

A loss function turns the cost of a prediction into a single number, and training a model means making that number as small as possible. Its choice decides which mistakes a model is punished for.

Overfitting is when a model fits its training data so closely that it performs poorly on new data. It is one half of the bias–variance trade-off and is usually spotted by comparing training and validation performance.

Regularization is the practice of making a problem's answer simpler, usually by adding a penalty term to the objective, so that ill-posed problems become solvable and models avoid overfitting.

A function applied to each node's weighted sum, giving neural networks their non-linearity. Sigmoid, tanh and ReLU shaped early deep learning; GELU now dominates transformers.

Embodied AI is the study of agents that learn by interacting with an environment, using egocentric perception, rather than only from datasets curated from the internet. Simulators serve as virtual testbeds where such agents are trained and evaluated.

Mechanistic interpretability explains model behaviour by finding the internal computations and features that produce it. Work on superposition and sparse autoencoders shows how many features can be packed into few neurons.

Reward hacking is what happens when a system optimises a proxy reward that only imperfectly captures what we want: it scores well on the proxy while performing badly on the true objective.

A KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.

Long context is a model's capacity to take in and work over very large inputs — documents, images, recordings — in a single request. Systems now reach millions of tokens, though finding information inside that window is a separate skill.

A context window is the full span of text a model can refer to while producing a response, measured in tokens and including the response itself. Its size, and how evenly a model uses it, shape what the model can do.

k-means clustering splits n observations into k groups by repeatedly sending each point to its nearest centroid and recomputing each centroid as its cluster's mean. It is simple and scales to large data, but k must be fixed in advance.

A model validation technique that repeatedly splits data into folds, trains on some parts and validates on others, then averages the results to estimate how well a model or statistical result will generalise to unseen data.

Graph neural networks are neural networks built for graph-structured data, in which nodes repeatedly exchange messages with their neighbours. Since the model was formulated in 2009, they have become a standard way of learning on graphs.

A regularisation technique for neural networks: during training it randomly zeroes the inputs or outputs of some neurons so that units cannot co-adapt, which reduces overfitting.

Speculative decoding speeds up text generation from large autoregressive models: a smaller model proposes several tokens and the large model checks them in parallel, producing identical output faster.

An autoencoder is an unsupervised neural network that compresses its input into a low-dimensional code and reconstructs it back, learning a latent representation useful for dimensionality reduction, denoising and anomaly detection.

Decision tree learning builds a classification or regression tree from data by repeatedly splitting it into more homogeneous groups. It is a widely used supervised method whose rules can be read directly, and whose main weaknesses are instability and a tendency to overfit.

Feature engineering is the preprocessing step that turns raw data into more useful inputs for machine learning and statistical models, shaping, cleaning and trimming the attributes a model learns from.