The term 'deep learning' gains currency after 2006

A name with more than one start

The words "deep learning" now label a large research field and a good part of the computing industry. The label itself, though, was not introduced once and for all by a single paper. It was used in more than one research community, for more than one thing, before it settled into its present meaning.

Hinton's 2015 account

In a March 2015 post to the Connectionists mailing list, Geoffrey Hinton gave a short history of the phrase. He wrote that the current popularity of the term "deep learning" started with the 2006 paper "A fast learning algorithm for deep belief nets", by Hinton, Osindero and Teh ^[1]. By his account, that paper set off a great deal of discussion about deep belief nets, a family of multilayer neural networks ^[1].

From "deep belief net" to "deep learning"

Around 2007, Hinton wrote, the term "deep belief net" began to be applied loosely to deep neural networks that had been pre-trained as deep belief nets ^[1]. The wording drifted because the networks were useful: such nets were used to build good acoustic models, the part of a speech recogniser that maps a short stretch of audio onto candidate sounds, and that work triggered the re-introduction of neural networks into mainstream speech recognisers ^[1]. A research community takes up a name more readily when the thing being named is producing results it can use.

The distinction, and the pre-training that stopped being needed

Later, Hinton notes, people made a clear distinction between deep belief nets proper and deep neural nets that had merely been initialized as deep belief nets. Eventually they found that, with large datasets and sensible initial weight scales, the pre-training was not needed at all ^[1]. Pre-training here means giving a network a starting set of weights from an earlier training stage before the main training run; once the data and the initialization were good enough, that head start could be dropped. The name, in other words, outlived the technique that had made it popular.

An older use of the same two words

The 2015 thread also reached further back. It points to Rina Dechter's 1986 AAAI paper as an early provenance for the phrase, noting that "deep learning as compared to shallow learning" was terminology used in the study of constraint satisfaction ^[1]. A 2025 peer-reviewed review goes further, stating that Dechter introduced the term "deep learning" to the machine learning community in 1986 ^[2], and that Igor Aizenberg and colleagues applied the term to artificial neural networks in 2000 ^[2].

What the history shows

Two things follow from this. First, "deep learning" was not coined at a single place and moment: the same two words were in use in constraint satisfaction long before they were attached to neural networks, and the 2006 paper is where Hinton dates the start of their present-day popularity ^[1][2]. Second, the term's rise tracked usefulness rather than argument. Speech recognition gave it a foothold, and by Hinton's own account the pre-training that first drew attention later became unnecessary, while the label stayed ^[1]. What began as a description of one training procedure became the name of a field.