Tomáš Mikolov

Overview

Tomáš Mikolov is a Czech computer scientist, born on 8 October 1982 in Šumperk.^[1] He is best known as the lead author of word2vec, a family of methods that learn vector representations of words from large amounts of text. A word vector is a list of numbers for each word, arranged so that words appearing in similar contexts end up with similar vectors. The training task is simple to state: the model learns to predict a word from the words around it, or the surrounding words from a single word. Because words that occur in similar surroundings are pushed toward similar vectors, the learned numbers come to reflect patterns of usage. Once words are vectors, a program can compare them with ordinary arithmetic.

Education and early research

Mikolov earned his PhD at Brno University of Technology in 2012. His thesis, Statistical Language Models Based on Neural Networks, worked on recurrent neural network language models. A language model estimates how probable a sequence of words is, which is the basis for predicting the next word.^[1]

During his doctoral studies he spent time at Johns Hopkins University, in Yoshua Bengio's lab at the Université de Montréal, and at Microsoft Research in Redmond.^[1]

Major contributions

After his PhD, Mikolov joined Google Brain in 2012. In 2013 he led the group that developed word2vec.^[2]

In January 2013 he was first author of the arXiv paper Efficient Estimation of Word Representations in Vector Space, written with Kai Chen, Greg Corrado and Jeffrey Dean. It introduced two architectures, CBOW and Skip-gram. CBOW predicts a word from the words around it, while Skip-gram does the reverse, predicting the surrounding words from a single word.^[3]

In 2013 he was also first author of the NIPS paper Distributed Representations of Words and Phrases and their Compositionality, written with Ilya Sutskever, Kai Chen, Greg Corrado and Jeff Dean. The paper later received the NeurIPS 2023 Test of Time Award.^[4]

Later career

In 2014 Mikolov moved to Facebook AI Research. There he worked on fastText, an open-source library released in 2016 for text classification and word vectors, which provides word representations for 157 languages.^[1]

In 2020 he joined the Czech Institute of Informatics, Robotics and Cybernetics (CIIRC) at the Czech Technical University in Prague, to start a research group on mathematical models that evolve and grow in complexity.^[1]

Influence

Word vectors let a program judge how related two words are by comparing their vectors, a basic capability for search and translation systems. Word vectors became a common building block in language technology, and the ideas in word2vec were widely adopted in research and applications. The NeurIPS Test of Time Award, given to work whose impact has lasted, reflects that continuing influence.