Jürgen Schmidhuber

Overview

Jürgen Schmidhuber is a German computer scientist, born on 17 January 1963 in Munich.^[2] He is one of the researchers most closely associated with the long short-term memory (LSTM) network, a recurrent neural network design for learning from sequences. He has directed IDSIA, the Swiss AI lab in Lugano, since 1995.^[2]

Education and early career

Schmidhuber studied computer science at the Technical University of Munich. He earned a diploma in 1987, a doctorate in 1991 and a habilitation in 1993.^[2] A habilitation is a further academic qualification that, in German-speaking countries, is often needed before a professorship.

In 1990 and 1991 his work included adversarial 'artificial curiosity' networks and recurrent network hierarchies pre-trained by self-supervised learning. Self-supervised learning means training a model on the structure of the data itself, without human-provided labels.^[1]

Major contributions

With Sepp Hochreiter he published Long Short-Term Memory in Neural Computation in 1997. The paper was written at IDSIA.^[3] Its design uses memory cells controlled by gates, which let information be kept over long sequences without its training signal fading away.

In 2000, with Felix Gers and Fred Cummins, he added the forget gate that standard LSTMs use. A forget gate lets the memory cell clear out information it no longer needs, so the cell can be reused as a sequence continues.^[4]

With Alex Graves he developed connectionist temporal classification (CTC) in 2006. CTC lets a network learn to map a long input, such as audio or handwriting, onto a shorter sequence of labels without each label having to be aligned by hand with a fixed position in the input. His team's CTC-trained LSTM was the first recurrent network to win international pattern recognition competitions, in 2009.^[1]

In May 2015, with Rupesh Kumar Srivastava and Klaus Greff, he published Highway Networks. These use gating units so that networks hundreds of layers deep can be trained with stochastic gradient descent, the same family of methods used to train LSTMs.^[5]

Career

Schmidhuber taught at the Technical University of Munich from 2004 to 2009, where he headed the Cog Bot Lab. He is professor of AI at the Università della Svizzera italiana (USI) in Lugano.^[2] Since 2021 he has been director of the AI Initiative at King Abdullah University of Science and Technology (KAUST) in Saudi Arabia. In 2014 he co-founded the company NNAISENSE.^[2]

Recognition and disputes

His awards include the Helmholtz Award of the International Neural Network Society (2013) and the IEEE Computational Intelligence Society Neural Networks Pioneer Award (2016).^[2]

Schmidhuber has argued publicly that his own contributions, and those of other researchers, have been under-credited compared with the 2018 Turing Award laureates. He has also published accounts of many priority disputes, which concern who first proposed particular ideas. These claims belong to a wider debate about credit in the field; this article reports them without judging them.^[2]

Influence

Schmidhuber's best-known contributions are LSTM, gated networks and CTC. These remain widely used ideas in sequence learning, including speech and handwriting recognition.