Total 39 · Clear filters
On 31 December 2020 EleutherAI released The Pile, an openly available 825 GiB English training corpus assembled from 22 smaller datasets, with a paper showing models trained on it beat both comparison corpora on every component.
In May 2020 OpenAI published the GPT-3 paper, describing a 175-billion-parameter language model that performs new tasks from a few examples given in its text input, without additional training.
On 27 January 2020, Google researchers posted a paper describing Meena, a 2.6-billion-parameter open-domain chatbot that scored 79% on a new chat-quality metric called the Sensibleness and Specificity Average.
On 19 November 2019, DeepMind posted a MuZero preprint to arXiv: it plans with a learned model rather than rules, matched AlphaZero in Go, chess and shogi, and set a state of the art on 57 Atari games.
On 30 October 2019, DeepMind announced that AlphaStar reached Grandmaster level in StarCraft II for all three races, ranking above 99.8% of active players and playing under human-like interface limits.
On 11 October 2018 Google researchers posted the BERT paper to arXiv. BERT uses context from both directions in every layer, and a single added output layer is enough to adapt it to tasks such as question answering.
On 11 June 2018 OpenAI published a paper on generative pre-training: a language model first learns from unlabeled text and is then adapted to specific tasks. The approach improved the state of the art on 9 of the 12 tasks studied.
A preprint posted to arXiv by DeepMind on 5 December 2017 described one general reinforcement-learning algorithm that, given only the rules, learned chess, shogi and Go by self-play and reached superhuman level in all three within 24 hours.
In 2017, DeepMind announced AlphaGo Zero, a Go system that started from random play, learned only by self-play, and outperformed earlier AlphaGo versions with one neural network and far less hardware.
On 12 June 2017 a team of researchers, most of them at Google, posted the paper Attention Is All You Need to arXiv. It introduced the Transformer, a neural network built only on attention, which became the basis for most later large language models.
On 8 September 2016, DeepMind announced WaveNet, a deep neural network that generates raw audio one sample at a time. In blind listening tests it beat Google's existing text-to-speech systems and narrowed the gap to human speech.
On 27 January 2016 DeepMind announced AlphaGo alongside a paper in Nature reporting that the program had beaten European Go champion Fan Hui 5–0 in October 2015 — the first time a computer program had beaten a professional Go player.
On 8 August 2014 Science published the paper describing TrueNorth, an IBM chip holding 1 million spiking neurons and 256 million synapses on 5.4 billion transistors, showing a brain-inspired design running on milliwatts.
Goodfellow, Bengio, and co-authors proposed a generative-model framework with adversarially trained generator and discriminator.
On 19 December 2013, DeepMind researchers posted a paper showing that one convolutional network, trained with a variant of Q-learning, could learn Atari 2600 control policies directly from raw pixels and outperform prior methods on six of seven games.
A 2010 NIPS workshop on deep learning and unsupervised feature learning. Its cited papers examine pre-training and fine-tuning for speech recognition and connect earlier and later work in the field.
In 1969 Marvin Minsky and Seymour Papert published Perceptrons, a mathematical analysis of what perceptrons cannot compute. The book is often credited with contributing to a decline in neural-network research.
Turing introduced the imitation game as a way to discuss machine intelligence through observable behavior.
