ALVINN

A network that steers

ALVINN (An Autonomous Land Vehicle in a Neural Network) was developed in 1988 within the Navlab programme at Carnegie Mellon University's Robotics Institute ^[1]. It was a three-layer, fully connected feedforward network trained by backpropagation, with layers of 1217, 29 and 46 neurons and 36,627 weights in total ^[1]. Its task was road following: it took images from a camera and a laser range finder as input and produced as output the direction the vehicle should travel in order to follow the road ^[2]. The work was published as a NIPS 1988 paper, *ALVINN: An Autonomous Land Vehicle In a Neural Network*, by Dean A. Pomerleau ^[2]; a detailed account appears in Pomerleau's 1992 PhD thesis ^[1].

Where it came from

Research on computer-controlled vehicles at Carnegie Mellon began in 1984 as part of the DARPA Strategic Computing Initiative, and production of Navlab 1 began in 1986 ^[1]. Navlab was a modified Chevy van. The project's stated aim was to train artificial neural networks in real time to perform difficult perception tasks, and ALVINN was the backpropagation network designed to drive that van ^[3].

Inputs and outputs

The input layer had 1217 units, fully connected to a hidden layer of 29 units, which was in turn fully connected to the output layer ^[2]. Two "retinas" supplied the input: a 30x32 grid of grayscale camera values, and an 8x32 grid of laser rangefinder depth readings with a 30 by 80 degree field of view; a single road intensity feedback unit completed the input ^{[1] [2]}. The output layer held 45 units spanning a linear range of steering angles plus one feedback unit, and the most activated steering unit determined the direction chosen ^[1].

Training and driving

Training was supervised, on 1200 simulated road images paired with rangefinder data, for 40 epochs on a Warp machine, which took 45 minutes ^[1]. On unseen simulated road images the trained network predicted the correct steering angle within two units of the true value 90% of the time ^[1].

Real driving raised a problem the simulated images did not. A human driver never strays far from the path, so a network trained only on centred images has seen too few examples of what to do when the vehicle sits off to one side. The team responded by shifting each real image left and right by five amounts, turning each example into 11 ^[1].

On Navlab 1 the system drove at 0.5 m/s along a 400-meter wooded path in snowy, rainy, sunny and cloudy weather ^[1]. The first ALVINN was trained in February 1989 in an eight-hour run on the Warp machine, and on 16 March 1989 a new Navlab record of 1.3 m/s was set ^[1]. The Robotics Institute's own description stresses a different training route: techniques that allow ALVINN to learn in under 5 minutes to control the Navlab autonomously by watching the reactions of a human driver, at speeds of up to 20 miles per hour ^[3].

Why it mattered

Wikipedia calls ALVINN an early demonstration of representation learning, sensor fusion and data augmentation ^[1]. The network was not given a hand-built model of roads, lanes or edges; it learned to map raw sensor grids to steering commands, with camera and rangefinder readings entering the same network ^[1].