Neural Turing Machines

The gap the paper addresses

A conventional neural network has no separate place to keep things. Whatever the model knows at a given moment lives in its weights and in the activations passing through it; there is no distinct store that it can address, write into, and read back from later. "Neural Turing Machines", first submitted to arXiv on 20 October 2014 as 1410.5401, is an attempt to supply that missing piece ^[1]. It was written by Alex Graves, Greg Wayne and Ivo Danihelka ^[1].

Coupling a network to memory

The proposal is to couple a neural network to external memory resources, which the network interacts with through attentional processes ^[1]. The authors present the combined system as analogous to a Turing machine or a von Neumann architecture, with the important qualification that it is differentiable end to end ^[1].

The reference points are worth spelling out. A Turing machine, in the classical sense, is an abstract device with a tape, a head that reads and writes symbols on the tape, and a table of rules telling the head what to do next. A von Neumann machine makes a similar separation between a processor and a store of data. In both, the contact between the computing part and the memory is discrete: the head sits on one cell, writes one symbol, moves one step. Operations of that kind are all-or-nothing, and an all-or-nothing choice has no useful derivative. That is the tension the paper tries to resolve, because the usual way of training a neural network is to compute the derivative of an error with respect to every parameter and step downhill.

Attention is what bridges the two. Rather than committing to a single memory location, the network can produce a weighting over locations and read a blend of what is stored there, or write into several locations in proportion to that weighting. Because the weighting is a continuous quantity, the result of a memory access changes smoothly as the network's output changes, and a derivative can be taken through it. The same reasoning applies to the whole machine: nothing in the pipeline involves an operation that breaks the chain of derivatives from the output back to the parameters.

That property is the point of the design. Because the system is differentiable throughout, it can be trained efficiently with gradient descent ^[1], the standard optimiser used for neural networks, which adjusts parameters in the direction that reduces the error. The memory does not have to be wired up by hand or programmed with explicit read and write instructions; it is learned along with everything else.

Preliminary results

The paper reports preliminary results showing that the model can infer simple algorithms such as copying, sorting and associative recall from input and output examples ^[1]. In each case the network is shown pairs of inputs and the corresponding outputs rather than a description of the procedure, and it has to pick up the underlying rule. Copying means reproducing an input sequence; sorting means emitting the same items in order; associative recall means returning an item that was stored alongside a cue. These are small tasks, but they are the kind of thing that requires somewhere to put intermediate material while the computation is under way.

Why it matters

The lasting contribution is the framing rather than any single score. The paper offers a way to describe a neural network and a memory as one trainable object, and it borrows a vocabulary — the Turing machine, the von Neumann architecture — that makes clear what the network is supposed to be able to do ^[1]. It also sets a pattern: keep the memory external and explicit, keep the addressing soft, and let gradient descent find the addressing scheme. The results themselves are presented as preliminary ^[1], and the tasks demonstrated are simple algorithms rather than open-ended problems. What the work establishes is that the boundary between a network and its memory can be made differentiable, and that a model built this way can learn procedures from examples alone.