Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 35 · Clear filters
RegularizationRegularization is the practice of making a problem's answer simpler, usually by adding a penalty term to the objective, so that ill-posed problems become solvable and models avoid overfitting.Decision tree learningDecision tree learning builds a classification or regression tree from data by repeatedly splitting it into more homogeneous groups. It is a widely used supervised method whose rules can be read directly, and whose main weaknesses are instability and a tendency to overfit.Feature engineeringFeature engineering is the preprocessing step that turns raw data into more useful inputs for machine learning and statistical models, shaping, cleaning and trimming the attributes a model learns from.AutoencoderAn autoencoder is an unsupervised neural network that compresses its input into a low-dimensional code and reconstructs it back, learning a latent representation useful for dimensionality reduction, denoising and anomaly detection.Multi-agent systemA multi-agent system is a computing system made of several interacting intelligent agents that coordinate to solve problems too hard for a single agent. Recent frameworks built on large language models let such agents talk to each other.Context windowA context window is the full span of text a model can refer to while producing a response, measured in tokens and including the response itself. Its size, and how evenly a model uses it, shape what the model can do.Long contextLong context is a model's capacity to take in and work over very large inputs — documents, images, recordings — in a single request. Systems now reach millions of tokens, though finding information inside that window is a separate skill.KV cacheA KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.Synthetic dataSynthetic data is training material written by a model rather than collected from people or the web. In the phi-1 experiments a small code model trained partly on model-generated "textbooks" reached competitive benchmark scores, suggesting data quality itself shapes capability.Multimodal modelsA multimodal model handles more than one kind of input or output, such as images together with text, inside a single model. Work on CLIP and Flamingo showed how such models are trained and how they can be steered toward new tasks.Benchmark contaminationBenchmark contamination means the test items a model is evaluated on, or close variants of them, were present in its pretraining data, so scores partly measure memory of seen text rather than the ability the benchmark is meant to probe.Red teamingRed teaming is the practice of actively probing a model for harmful or unwanted behaviour before deployment. A 2022 method by Perez and colleagues automated it by using one language model to generate test cases for another.Mechanistic interpretabilityMechanistic interpretability explains model behaviour by finding the internal computations and features that produce it. Work on superposition and sparse autoencoders shows how many features can be packed into few neurons.JailbreakA jailbreak is an adversarial prompt that gets a safety-trained model to produce behaviour its training was meant to prevent. A 2023 analysis by Wei et al. traced these failures to competing objectives and mismatched generalization.Semantic WebThe Semantic Web is a W3C standards effort that uses RDF and OWL to describe data so different applications, enterprises and communities can share and reuse information, moving from research toward practical tools.