Total 4 · Clear filters
On 11 October 2018 Google researchers posted the BERT paper to arXiv. BERT uses context from both directions in every layer, and a single added output layer is enough to adapt it to tasks such as question answering.
On 11 June 2018 OpenAI published a paper on generative pre-training: a language model first learns from unlabeled text and is then adapted to specific tasks. The approach improved the state of the art on 9 of the 12 tasks studied.
On 12 June 2017 a team of researchers, most of them at Google, posted the paper Attention Is All You Need to arXiv. It introduced the Transformer, a neural network built only on attention, which became the basis for most later large language models.
On 8 August 2014 Science published the paper describing TrueNorth, an IBM chip holding 1 million spiking neurons and 256 million synapses on 5.4 billion transistors, showing a brain-inspired design running on milliwatts.
