NVIDIA releases Nemotron-4 340B for synthetic data generation

Background

Training a language model needs large amounts of text, and high-quality examples of good answers are hard to collect at scale. One common approach is to use an existing model to generate examples, a method known as synthetic data. Synthetic data is produced by a model rather than gathered from people. The release aimed to make that practice open and repeatable.

What happened

On 14 June 2024 NVIDIA released Nemotron-4 340B, a family of base, instruct and reward models ^[1]. The company positioned it as an open pipeline for generating synthetic data to train other LLMs. LLM stands for large language model ^[1]. The models were published under the NVIDIA Open Model License on NGC and Hugging Face ^[1][2]. NGC is NVIDIA's catalogue of software and models, and Hugging Face is a platform where open models are hosted.

The base model was trained on 9 trillion tokens ^[1]. A token is a small piece of text, often a word or part of one.

Key details

The three members have different roles. The base model is the general starting point. The instruct model has been further trained to follow requests. The reward model scores responses, and it is the part of the family that can help select or guide the output of other models ^[1].

The reward model ranked first on the RewardBench leaderboard ^[1]. RewardBench is a benchmark that tests how well reward models judge responses, and a leaderboard orders the results.

NVIDIA said that over 98% of its own alignment data was synthetic ^[2]. Alignment is the process that adjusts a model toward the behaviour its developers want, and the figure shows how much of that adjustment was done with material generated by the models themselves.

The license allows distribution, modification and use of the models and their outputs ^[2]. That permission applies to the outputs as well as the models, which matters for anyone who wants to use generated data to train another model.

Impact

The release gave developers an open set of tools for building training data. Instead of collecting every example from people, a team could use these models to generate material, and then train or refine a smaller model on the result. Because the company presented the models as an open pipeline, others could look at the method and reuse it.

For readers, the release is a clear example of a shift in how models are built. Large models are now used not only as products but also as sources of training material for other models. This release made that practice visible and open, and it set out the terms under which the outputs could be used.

For a team with limited data, this approach offers a way to create training material without collecting every example from people. The release made that option available with published terms of use.