NVIDIA and Microsoft unveil the 530-billion-parameter Megatron-Turing NLG
On 11 October 2021 NVIDIA and Microsoft announced Megatron-Turing NLG 530B, a language model with 530 billion parameters that they called the largest and most powerful of its kind. The full technical report followed in January 2022.