Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 55 · Clear filters
Artificial neural networksAn artificial neural network is a computing system of many simple connected units whose behaviour is set by connection weights. Its roots lie in 1943 models of neurons, and its learning versions began with Frank Rosenblatt's perceptron in 1958.Information theoryA framework that treats information as a measurable quantity to be encoded and transmitted, founded in Shannon's 1948 paper; it gave the field the bit, entropy and channel capacity.Turing testThe Turing test is Alan Turing's 1950 proposal for replacing the question "Can machines think?" with a game: a machine passes if an interrogator cannot tell its written answers apart from a human's.k-means clusteringk-means clustering splits n observations into k groups by repeatedly sending each point to its nearest centroid and recomputing each centroid as its cluster's mean. It is simple and scales to large data, but k must be fixed in advance.Support vector machineA supervised maximum-margin model for classification and regression that separates classes by the widest possible margin and uses kernel functions to handle non-linear data.Cross-validationA model validation technique that repeatedly splits data into folds, trains on some parts and validates on others, then averages the results to estimate how well a model or statistical result will generalise to unseen data.AI winterAI winter is a period of sharply reduced funding and interest in artificial intelligence research. The field's hype cycles have produced two such downturns, usually dated to about 1974–1980 and 1987–2000.Hopfield networkA recurrent neural network introduced by John Hopfield in 1982 that stores patterns as minima of an energy function and recalls them from partial or distorted input, bringing statistical mechanics into the study of neural networks.Boltzmann machineA Boltzmann machine is a neural network whose configurations are scored by a global energy and trained by a stochastic learning algorithm. Introduced by Geoffrey Hinton and Terry Sejnowski, it brought energy-based models into cognitive science and machine learning.Autonomous drivingAutonomous driving is the operation of a road vehicle by a computer system instead of a person. Its six automation levels, its two main technical traditions and its long line of research vehicles define how the field is discussed today.Recurrent neural networks and LSTMA recurrent neural network (RNN) is a neural network with feedback connections that carry information from one step of a sequence to the next. Its long short-term memory variant (LSTM) was designed to keep useful memories over longer spans.Backpropagation and gradient descentBackpropagation is the procedure that lets a multilayer neural network learn by sending output errors backward to adjust its weights, using gradient descent. A 1986 Nature paper made it widely known, and it is standard in most neural networks today.Imitation learningImitation learning trains an agent by supervised learning on expert demonstrations rather than by trial and error. Its simplest form is behaviour cloning, and its best-known weakness is distribution shift.Convolutional neural networksA convolutional neural network applies the same small pattern detectors across a whole image, so a feature is found wherever it appears. Trained with backpropagation in 1989 and refined as LeNet-5 in 1998, it became standard in computer vision.AdaBoostAdaBoost, short for Adaptive Boosting, is a statistical classification meta-algorithm introduced by Yoav Freund and Robert Schapire in 1995. It reweights training examples, trains weak classifiers in sequence and combines their outputs into one weighted classifier.Topic modelTopic models represent documents as mixtures of latent topics, giving a compact statistical description of text collections. The 2003 Latent Dirichlet Allocation paper formalized this idea as a hierarchical Bayesian model.Word embeddingsA word embedding represents each word as a list of numbers learned from text, so that words used in similar ways get similar lists. Embeddings are a basic building block of modern language models.Connectionist Temporal ClassificationConnectionist temporal classification (CTC) is a method for training recurrent neural networks to label unsegmented sequences, such as speech or handwriting, without frame-by-frame alignment. Introduced in 2006, it let CTC-trained LSTMs win the first international contests for recurrent networks.Graph neural networkGraph neural networks are neural networks built for graph-structured data, in which nodes repeatedly exchange messages with their neighbours. Since the model was formulated in 2009, they have become a standard way of learning on graphs.Deep neural network acoustic modelingDeep neural network acoustic modeling uses multilayer networks to map speech to units such as phones. A 2010 NIPS paper used Deep Belief Nets and a mean-covariance RBM to extract speech features, reaching 20.5% phone error on TIMIT.