ALVINN: An Autonomous Land Vehicle in a Neural Network

The problem

ALVINN stands for Autonomous Land Vehicle In a Neural Network. The task it addresses is road following: the vehicle looks at what its sensors report and decides which direction to travel in order to stay on the road. Underneath that simple description sits an awkward engineering question — how to turn a picture of a road into a steering decision. If that mapping has to be written out by hand, someone must decide in advance how every kind of road appearance should be handled; a network that learns the mapping from examples sidesteps that job.

How ALVINN works

ALVINN is a 3-layer back-propagation network designed for the task of road following.^[1] It takes images from a camera and a laser range finder as input, and produces as output the direction the vehicle should travel in order to follow the road.^[1] Both the visual channel and the range channel feed the same network, and what comes out is not a label attached to a scene but an instruction for movement.

Training was conducted using simulated road images.^[1] The practical reason for simulation is that learning needs many examples, and each example needs a known correct answer. In simulation one can generate road scenes together with the direction the vehicle ought to take in each of them, which gives the network something to be corrected against.

The learning procedure itself is back-propagation. The network produces an output for an input image; that output is compared with the direction it should have produced; the difference becomes an error that is propagated back through the layers, where the connection weights are adjusted slightly in the direction that reduces the error. Repeat this over many images and the network's output gradually moves toward the correct direction. Nothing in this procedure tells the network what a road is; whatever internal representation it ends up using is a by-product of reducing the error on the training images.

Results

The abstract reports successful tests on the Carnegie Mellon autonomous navigation test vehicle, indicating that the network can effectively follow real roads under certain field conditions.^[1] Two details in that sentence deserve attention. The training data were simulated images, while the tests were on a real vehicle, so the reported result concerns carrying a network trained on simulated scenes onto real sensor input. And the claim is deliberately qualified: the network followed real roads under certain field conditions, not under all of them.

The representation depends on the training conditions

The paper also notes something less obvious. The representation developed to perform the task differs dramatically when the network is trained under various conditions.^[1] The same architecture, the same task and the same learning rule can settle into quite different internal organisations depending on the conditions under which training happened. The paper presents that difference as suggesting the possibility of a novel adaptive autonomous navigation system capable of tailoring its processing to the conditions at hand.^[1] If training conditions shape what the network learns, then a system that adapts its processing to the conditions it currently faces could in principle be built along these lines. The paper offers this as a possibility rather than as a system it had built.

Where it appeared, and why it matters

The paper is by Dean A. Pomerleau and appeared in Advances in Neural Information Processing Systems 1, the NIPS 1988 volume edited by D. Touretzky.^[1] Its importance lies in the shape of the demonstration rather than in any single number: a small layered network, trained on examples, taking raw sensor readings in and putting an action out, tested on a real vehicle. Driving is treated as a learning problem instead of a hand-coded pipeline, and the careful qualification about field conditions marks the boundary of what had actually been shown.