Support-vector networks

The problem

The paper addresses two-group classification: given an input vector, a learning machine must assign it to one of two groups. Corinna Cortes and Vladimir Vapnik published *Support-vector networks* in the journal *Machine Learning* in September 1995, while both were at AT&T Bell Labs in Holmdel, New Jersey ^[1]. The abstract presents the support-vector network as a new learning machine for such two-group classification problems ^[1]. The journal records the article as received on 15 May 1993 ^[1]. It appeared in volume 20, pages 273-297, with DOI 10.1007/BF00994018 ^[1].

How it works

The central move is to map input vectors non-linearly into a very high-dimensional feature space and then construct a linear decision surface in that space ^[1]. In plain terms, the classifier does not have to draw a simple straight boundary in the original input space; it first sends each input through a nonlinear transformation into a richer representation, where a linear boundary can separate the two groups. The paper states that special properties of that decision surface ensure high generalization ability of the learning machine ^[1]. Generalization here means performance on examples beyond the training set, not just fitting the training data.

The paper also extends an earlier implementation. That earlier version was restricted to the case of perfectly separable training data; the article moves to the case of non-separable training data ^[1]. This matters because the restricted case assumes the two groups can be separated without error by the chosen surface, while the extended case covers data that do not meet that ideal condition.

Results

The paper demonstrates high generalization ability using polynomial input transformations ^[1]. It compares the support-vector network with classical algorithms that all took part in an Optical Character Recognition benchmark study ^[1]. The benchmark places the new learning machine alongside established methods on a common recognition task. The paper's demonstration therefore combines a high-dimensional mapping, a linear decision surface with special properties, an extension to non-separable data, and a comparison on an OCR benchmark ^[1].

Why it matters

The paper's contribution can be stated as a framework for two-group classification. It combines a nonlinear mapping into a very high-dimensional feature space with a linear decision surface in that space ^[1]. It treats generalization ability as a property tied to special properties of that surface ^[1]. It moves from the restricted, perfectly separable case to the non-separable case ^[1]. And it tests the resulting learning machine with polynomial input transformations against classical algorithms in an OCR benchmark ^[1]. These elements make the paper a compact statement of a particular way to build a classifier: transform the representation, solve a linear separation problem there, and judge the result by generalization and benchmark comparison.

Publication record

*Support-vector networks* appeared in *Machine Learning* in September 1995, volume 20, pages 273-297, DOI 10.1007/BF00994018 ^[1]. The journal records the article as received on 15 May 1993 ^[1]. At the time of writing, both authors were at AT&T Bell Labs in Holmdel, New Jersey ^[1].