TensorFlow

What TensorFlow is

TensorFlow is Google's second-generation machine learning system. Google Research announced that it was released as open source ^[1]. The release was more than a code drop: Google described the system as a standalone library supplied together with tools, tutorials and examples, all under the Apache 2.0 license ^[1]. Google's own summary called it general, flexible, portable, easy to use and completely open source ^[1].

Where it came from

TensorFlow had a predecessor inside Google. DistBelief was developed in 2011 and was used to train the Inception model that won the 2014 ImageNet challenge ^[1]. Before TensorFlow existed, more than 50 teams across Google and Alphabet had already deployed deep neural networks using DistBelief ^[2]. TensorFlow was therefore the second generation of a system that had already become internal infrastructure.

The dates of the release are recorded in several places. The white paper describing the system is dated 9 November 2015; it states that the API and a reference implementation were released as an open-source package under Apache 2.0 that month ^[2]. The TensorFlow Blog dates the open sourcing to 9 November 2015 ^[3]. The GitHub repository for the project was created two days earlier, on 7 November 2015, and is released under the Apache-2.0 license ^[5].

How it works

According to the white paper, TensorFlow expresses computations as stateful dataflow graphs ^[2]. A dataflow graph writes a computation as nodes joined by edges: the nodes stand for operations, and the edges carry the values that pass between them. Calling such a graph stateful means that parts of it can hold values that persist between runs instead of being rebuilt each time. Training a neural network, in this view, is a long sequence of such operations, and writing it as a graph lets the system decide how to place and schedule the work.

The same description sets out the range of hardware the system was built for: a computation can run on systems ranging from phones and tablets up to hundreds of machines with thousands of GPU cards ^[2]. Portability was among the properties Google listed for the system ^[1].

What Google said about speed

Google stated that on some benchmarks TensorFlow was twice as fast as its predecessor DistBelief ^[1]. The official Google Blog put the improvement higher, saying that TensorFlow could build and train neural nets up to five times faster than the first-generation system ^[4].

What the open release meant

Publishing the library under Apache 2.0, with tools, tutorials and examples alongside it, put the system in the hands of people who did not work at Google ^[1]. A team that wanted to build and train a neural network could start from the same code base and the same examples rather than reconstructing the methods from papers.

The lineage is what gives the release its place. TensorFlow is the successor to DistBelief, a system that had been developed in 2011, that had trained a model which won the 2014 ImageNet challenge, and that had been deployed by more than 50 teams inside Google and Alphabet ^[1][2]. TensorFlow carried that work forward and, by being open source, made it available well beyond the company that built it ^[1].