Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 25 · Clear filters
Boltzmann machineA Boltzmann machine is a neural network whose configurations are scored by a global energy and trained by a stochastic learning algorithm. Introduced by Geoffrey Hinton and Terry Sejnowski, it brought energy-based models into cognitive science and machine learning.Topic modelTopic models represent documents as mixtures of latent topics, giving a compact statistical description of text collections. The 2003 Latent Dirichlet Allocation paper formalized this idea as a hierarchical Bayesian model.Connectionist Temporal ClassificationConnectionist temporal classification (CTC) is a method for training recurrent neural networks to label unsegmented sequences, such as speech or handwriting, without frame-by-frame alignment. Introduced in 2006, it let CTC-trained LSTMs win the first international contests for recurrent networks.Deep neural network acoustic modelingDeep neural network acoustic modeling uses multilayer networks to map speech to units such as phones. A 2010 NIPS paper used Deep Belief Nets and a mean-covariance RBM to extract speech features, reaching 20.5% phone error on TIMIT.Xavier (Glorot) initializationXavier (Glorot) initialization is a weight-initialization scheme from Glorot and Bengio's 2010 paper that sets each layer's starting weight scale so signals stay stable across layers, giving substantially faster convergence in deep feedforward networks.Variational autoencoderA variational autoencoder is a generative model whose encoder maps an input to a distribution over a latent space and whose decoder reconstructs the input from a sample; training maximises a lower bound on the data likelihood.Generative adversarial networkA generative adversarial network pits a generator that makes fake data against a discriminator that judges it, both trained in a zero-sum game; proposed by Ian Goodfellow and colleagues in 2014, it led to a series of architectures for generating images.Knowledge distillationKnowledge distillation trains a small model to reproduce the outputs of a larger model or an ensemble, so that much of the predictive performance is kept at lower compute and deployment cost.DeepfakeDeepfake is synthetic media: images, video or audio made or altered with AI. The term, a blend of "deep learning" and "fake", appeared in late 2017 and now names a whole class of face-swapping and manipulation techniques.Diffusion modelsDiffusion models learn to reverse a gradual noising process, so they can turn random noise into a new image by removing noise step by step. Latent diffusion, which runs this in a compressed space, became the basis of Stable Diffusion.LoRA (low-rank adaptation)LoRA (low-rank adaptation) is a fine-tuning technique that freezes a pre-trained model's weights and trains small low-rank matrices instead, cutting the number of trainable parameters and the memory needed while keeping quality close to full fine-tuning.Speculative decodingSpeculative decoding speeds up text generation from large autoregressive models: a smaller model proposes several tokens and the large model checks them in parallel, producing identical output faster.Text watermarkingText watermarking embeds a hidden statistical signal in generated text so that a short span of tokens can later be identified as machine output, while the text still reads normally to people.Test-time computeTest-time compute is the computation a model spends while answering a prompt, beyond what training already fixed into its weights. A 2024 study by Snell et al. showed how to spend it adaptively, and how it can stand in for a bigger model.Computer useComputer use is an Anthropic capability, released in public beta in October 2024, that lets an AI model operate a computer the way a person does — reading the screen, moving a cursor, clicking and typing — so it can drive ordinary software built for humans.Vision-Language-Action model (VLA)A vision-language-action model (VLA) is a robot policy that takes in camera images and a language instruction and outputs actions directly, usually by adapting a pretrained vision-language model to control.Unsupervised learningUnsupervised learning trains algorithms on data that carry no labels, letting them find structure such as clusters, low-dimensional representations or generative models. It underlies clustering tools and neural generative models from the Hopfield network and Boltzmann machine to the variational autoencoder.Feature engineeringFeature engineering is the preprocessing step that turns raw data into more useful inputs for machine learning and statistical models, shaping, cleaning and trimming the attributes a model learns from.Context windowA context window is the full span of text a model can refer to while producing a response, measured in tokens and including the response itself. Its size, and how evenly a model uses it, shape what the model can do.KV cacheA KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.