Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 13 · Clear filters
Boltzmann machineA Boltzmann machine is a neural network whose configurations are scored by a global energy and trained by a stochastic learning algorithm. Introduced by Geoffrey Hinton and Terry Sejnowski, it brought energy-based models into cognitive science and machine learning.Topic modelTopic models represent documents as mixtures of latent topics, giving a compact statistical description of text collections. The 2003 Latent Dirichlet Allocation paper formalized this idea as a hierarchical Bayesian model.Deep neural network acoustic modelingDeep neural network acoustic modeling uses multilayer networks to map speech to units such as phones. A 2010 NIPS paper used Deep Belief Nets and a mean-covariance RBM to extract speech features, reaching 20.5% phone error on TIMIT.Variational autoencoderA variational autoencoder is a generative model whose encoder maps an input to a distribution over a latent space and whose decoder reconstructs the input from a sample; training maximises a lower bound on the data likelihood.Generative adversarial networkA generative adversarial network pits a generator that makes fake data against a discriminator that judges it, both trained in a zero-sum game; proposed by Ian Goodfellow and colleagues in 2014, it led to a series of architectures for generating images.DeepfakeDeepfake is synthetic media: images, video or audio made or altered with AI. The term, a blend of "deep learning" and "fake", appeared in late 2017 and now names a whole class of face-swapping and manipulation techniques.Diffusion modelsDiffusion models learn to reverse a gradual noising process, so they can turn random noise into a new image by removing noise step by step. Latent diffusion, which runs this in a compressed space, became the basis of Stable Diffusion.Content provenance (C2PA)Content provenance is a checkable record of where a piece of media came from and how it was edited, attached to the file itself. The C2PA, founded in 2021 by Adobe, Arm, BBC, Intel, Microsoft and Truepic, publishes the technical standard behind it.Text watermarkingText watermarking embeds a hidden statistical signal in generated text so that a short span of tokens can later be identified as machine output, while the text still reads normally to people.Test-time computeTest-time compute is the computation a model spends while answering a prompt, beyond what training already fixed into its weights. A 2024 study by Snell et al. showed how to spend it adaptively, and how it can stand in for a bigger model.Unsupervised learningUnsupervised learning trains algorithms on data that carry no labels, letting them find structure such as clusters, low-dimensional representations or generative models. It underlies clustering tools and neural generative models from the Hopfield network and Boltzmann machine to the variational autoencoder.KV cacheA KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.Synthetic dataSynthetic data is training material written by a model rather than collected from people or the web. In the phi-1 experiments a small code model trained partly on model-generated "textbooks" reached competitive benchmark scores, suggesting data quality itself shapes capability.