The Anatomy of a Large-Scale Hypertextual Web Search Engine

What the paper set out to do

A web search engine has to answer a question that sounds simple: given a few words typed by a user, which pages on the web should be shown first? The title announces both the setting and the subject — a large-scale hypertextual web search engine, where "hypertextual" points to the links that connect pages to one another. The paper was written by Sergey Brin and Lawrence Page and appeared in the Proceedings of the 7th International World Wide Web Conference in April 1998 ^[1].

The two authors were PhD students in Stanford's Computer Science Department when they did this work ^[2]. Their starting point was the link structure of the web itself: they developed an algorithm that counted the number and the quality of links to a webpage, and they used that algorithm to build Google's search engine ^[2].

How the ranking worked

A link from one page to another carries information. A page that many other pages point to is, in a rough sense, one that others found worth pointing at. Counting such links is the first half of the algorithm; the second half is the quality of those links. Not every link should weigh the same — a link from a page that is itself heavily linked carries more weight than a link from a page nobody points to. That is the recursive part of the idea: the importance of a page depends on the importance of the pages that link to it. The resulting ranking signal is what became known as PageRank ^[2].

This is a structural signal computed from the graph of pages and links, not from the words printed on a page. It answers a different question from "does this page contain the words the user typed": it asks how the web itself is organised, and lets that organisation influence the order in which results are returned.

Publication and recognition

The paper was published at the Seventh International World Wide Web Conference in 1998, and the IEEE Computer Society cites it, alongside another heavily cited paper, as a description of Brin and Page's early work ^[2]. The IEEE Milestones archive holds both a short version and a long version of the paper ^[1]. It later received a Seoul test-of-time award, the kind of recognition given to work that has held up over the years ^[1].

Why it matters

The algorithm described in the paper did not stay a research result. Brin and Page used it to build Google's search engine ^[2]. The engine's name comes from "googol", the mathematical term for a very large number, and reflects a mission to organise the immense amount of information on the Internet ^[2].

For readers today the paper is also a record of a particular moment: two PhD students in a computer science department, working from the link structure of the web, setting out in a conference paper the ranking idea behind a search engine. Because the IEEE archive keeps both the short and the long version ^[1], it is possible to see how the same argument was presented for a conference audience and at greater length.