Ashish Vaswani

Overview

Ashish Vaswani is a computer scientist best known as the first-listed author of “Attention Is All You Need,” the 2017 paper that introduced the Transformer architecture. Many modern large language models build on that design, and the paper is among the most cited in machine learning.^[2] Its title is a claim that attention alone, without the step-by-step recurrence used before, is enough to build a strong model.

Education and early research

Vaswani earned a B.Tech. in computer science from the Birla Institute of Technology, Mesra, in 2002.^[1] He began graduate studies at the University of Southern California in 2004 and completed a PhD in computer science there. His advisers included David Chiang and Liang Huang. His 2014 thesis, “Smaller, Faster, and Accurate Models for Statistical Machine Translation,” dealt with machine translation, the task of having a computer convert text from one language into another. He was also affiliated with the university’s Information Sciences Institute.^[1]

Key contributions

In June 2017, Vaswani and seven co-authors submitted “Attention Is All You Need.” The author list, in order, was Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin.^[2]

Earlier language models mostly read text in order, one word after another. The Transformer instead relies on attention, a mechanism in which each position in a sentence weighs how much it should draw on every other position. In the sentence “The animal did not cross the street because it was tired,” attention lets the model link “it” to “animal,” even though the two words are several positions apart. Because the computation does not have to proceed strictly step by step, the design can run in parallel on modern chips, which makes large training runs practical.

The idea is general rather than tied to one task. Once the architecture was available, the same building blocks could be applied to text generation, translation, speech and images, Because the paper introduced a general design rather than a product, its influence is measured mostly through the research and systems that build on it.

Positions

Vaswani worked at Google Brain from 2016 to 2021.^[1] Google Brain was the company’s internal AI research team. He also co-founded the AI startup Adept, which he later left. With Niki Parmar, another co-author of the Transformer paper, he co-founded Essential AI and served as its CEO. The company emerged from stealth in December 2023 with $56.5 million in funding.^[3]

Impact

The Transformer became a foundation for much of the research that followed, and its influence is visible in how often the paper is cited.^[2] Vaswani’s own career after Google has centred on startups that build AI systems, while the paper that carries his name remains the most direct link between him and the wider history of the field.^[2]