Home›Concepts
Concepts
Concepts
How AI works, in plain language: from the artificial neuron to the Transformer, pre-training and reasoning models.
Total 26 · Clear filters
AutoencoderAn autoencoder is an unsupervised neural network that compresses its input into a low-dimensional code and reconstructs it back, learning a latent representation useful for dimensionality reduction, denoising and anomaly detection.QuantizationQuantization stores a language model's weights and activations at lower numerical precision, cutting the memory and bandwidth that inference requires. Methods such as LLM.int8() and GPTQ made very large models runnable on far smaller hardware.KV cacheA KV cache keeps the key and value vectors a model has already computed for earlier tokens, so each new token reuses them instead of recomputing. Its memory footprint is what makes high-throughput serving hard to scale.Synthetic dataSynthetic data is training material written by a model rather than collected from people or the web. In the phi-1 experiments a small code model trained partly on model-generated "textbooks" reached competitive benchmark scores, suggesting data quality itself shapes capability.Benchmark contaminationBenchmark contamination means the test items a model is evaluated on, or close variants of them, were present in its pretraining data, so scores partly measure memory of seen text rather than the ability the benchmark is meant to probe.Semantic WebThe Semantic Web is a W3C standards effort that uses RDF and OWL to describe data so different applications, enterprises and communities can share and reuse information, moving from research toward practical tools.