Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 16 · Clear filters
Deep learning on GPUsGraphics processing units (GPUs) run many arithmetic operations in parallel, which suits neural-network training. Their use from 2010 onward made deep learning practical at much larger scales.Mixture of experts (MoE)A mixture of experts model contains many sub-networks but routes each token to only a few of them. This lets total model size grow while the compute spent on each token stays much smaller than the total.Tensor CoresTensor Cores are specialised units inside NVIDIA GPUs that perform a small matrix multiply-and-add in one step, making the matrix arithmetic behind neural-network training and inference dramatically faster.Model parallelismModel parallelism splits one model's parameters and computation across multiple accelerators, so networks too large for a single device can still be trained. NVIDIA's 2019 Megatron-LM paper showed an intra-layer version built in native PyTorch.Scaling lawsScaling laws are empirical trends showing that a language model's test loss falls smoothly, as a power law, when parameters, data and training compute grow. They help plan large training runs and explain the race for compute.Chain of thought and reasoning modelsChain of thought is the technique of having a language model write out its intermediate reasoning before answering. Reasoning models extend it by training the model with reinforcement learning to think at length before responding.Speculative decodingSpeculative decoding speeds up text generation from large autoregressive models: a smaller model proposes several tokens and the large model checks them in parallel, producing identical output faster.Test-time computeTest-time compute is the computation a model spends while answering a prompt, beyond what training already fixed into its weights. A 2024 study by Snell et al. showed how to spend it adaptively, and how it can stand in for a bigger model.Sim-to-real transferSim-to-real transfer is the practice of training robot policies in simulation and then adapting them to physical robots, using synthetic data to supplement costly real-world demonstrations.QuantizationQuantization stores a language model's weights and activations at lower numerical precision, cutting the memory and bandwidth that inference requires. Methods such as LLM.int8() and GPTQ made very large models runnable on far smaller hardware.KV cacheA KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.Prompt engineeringPrompt engineering is the practice of designing the text input that steers a generative AI system toward a desired output. Shared vocabulary and techniques such as chain-of-thought prompting make results easier to describe, compare and reproduce.Tool callingTool calling lets a language model hand a sub-task, such as arithmetic or search, to an external program and use the result in its answer. Toolformer showed in 2023 that a model could learn when and how to call tools on its own.Planning in AI agentsPlanning in AI agents is the process of breaking a goal into intermediate steps, choosing actions, and revising the plan as results arrive, rather than producing an answer in one pass.[^1][^2]Reward hackingReward hacking is what happens when a system optimises a proxy reward that only imperfectly captures what we want: it scores well on the proxy while performing badly on the true objective.Artificial general intelligence (AGI)AGI is the goal of AI systems that match or beat humans across a broad range of tasks. The term has no agreed definition; a framework by Morris et al. grades systems by performance level and generality.