Niki Parmar

Overview

Niki Parmar is one of the eight authors of "Attention Is All You Need", the paper submitted to arXiv on 12 June 2017 ^[1]. That paper is where the Transformer, described by Google as a neural network architecture for language understanding, was presented ^[2]. Parmar also appears in the company's own public account of the work: a Google Research Blog post dated 31 August 2017 names Parmar among the researchers who conducted the Transformer work ^[2].

Those two mentions — a place on the paper's author list and a place in the blog post describing it — form the core of the public record about Parmar. The rest of that record consists of a handful of co-authored papers from the same period and a short institutional profile.

Education

A news article from the University of Southern California's Information Sciences Institute describes Parmar as holding a master's degree in computer science from the University of Southern California ^[4]. The same article describes Parmar as a Google Brain colleague of Ashish Vaswani ^[4], who is one of the other authors of the Transformer paper.

The Transformer paper

"Attention Is All You Need" carries eight authors ^[1]. The architecture the paper describes is built around attention. Rather than reading a sequence strictly one position after another, the model lets each position weigh information coming from other positions and combine it according to relevance, so that dependencies between distant parts of a sentence do not have to travel through a long chain of intermediate steps. Because it is an architecture paper, its contribution is a design rather than a finished product, and it is the design that later work picks up.

Google's blog post is useful for a second reason: it dates the public description of the work to 31 August 2017, a few months after the arXiv submission, and it treats the Transformer as the output of a group of researchers rather than of a single person ^[2].

Work that followed

The Transformer paper was not Parmar's only co-authored publication in this line of work. "Image Transformer" was submitted to arXiv on 15 February 2018, with Parmar as a co-author alongside Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser, Noam Shazeer, Alexander Ku and Dustin Tran ^[3]. Roughly a month later, on 16 March 2018, "Tensor2Tensor for Neural Machine Translation" was submitted, with Parmar one of its thirteen authors ^[5].

Taken together, these three papers sit inside a short window: a June 2017 submission, a February 2018 one and a March 2018 one. They also show the direction the work moved in. The first paper set out a general architecture; the later two connect it to images and to neural machine translation, the task of having a system produce translations directly from data rather than through hand-written rules.

What the record shows

The documented record of Parmar's output is narrow but specific: a position among the eight authors of the Transformer paper ^[1], a mention in Google's blog post about it ^[2], two further co-authored arXiv submissions in 2018 ^[3][5], a master's degree from the University of Southern California, and a working relationship with Vaswani at Google Brain ^[4].

For a reader trying to understand how the Transformer came about, the value of details like these is that they are datable and checkable. Machine-learning papers are usually credited collectively, and an author list is the form in which an individual's part is recorded and later looked up. Parmar's name sits on the author list of a paper that much subsequent work in natural language processing treats as a starting point — which is why the name is worth looking up at all, and why an archive that keeps the submission date, the author list and the institutional notes is more useful than a summary that keeps only the headline.