Computer vision Papers

In 1958 Frank Rosenblatt described the perceptron, a model whose connection strengths change with training so that it learns to classify stimuli by trial and error. It became the starting point for trainable neural networks.

The 2009 paper introduced ImageNet, a large, cleanly labelled image database organised by the WordNet hierarchy. It later became the basis of an annual benchmark, the ImageNet Large Scale Visual Recognition Challenge.

A CVPR 2011 Best Paper that recovers 3D body joint positions from a single depth image without temporal information, by classifying pixels into body parts, and runs at 200 frames per second on consumer hardware.

The 2012 NeurIPS paper that introduced AlexNet, a large convolutional network trained on GPUs. It won the ILSVRC-2012 image classification competition by a wide margin and is widely seen as a turning point for deep learning in computer vision.

A 2014 study showing that stacking many small 3×3 convolution filters, up to 16–19 weight layers, improves large-scale image recognition; its models took first place in localisation and second in classification at ImageNet Challenge 2014 and were released publicly.

The 2014 paper that introduced the Inception architecture and its 22-layer GoogLeNet network, which took first place in the ImageNet 2014 classification and detection challenge with a 6.67% top-5 error.

Mask R-CNN is a 2017 framework for object instance segmentation. It detects objects and produces a high-quality mask for each instance by adding a parallel mask branch to Faster R-CNN.

This 2018 paper introduces a style-based GAN generator that separates high-level attributes from stochastic variation and allows scale-specific control, improving distribution quality, interpolation, and disentanglement.

A 2020 paper by Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi and Ng that represents a scene as a neural radiance field: a network mapping 5D coordinates to density and colour, optimized from a sparse set of images with known camera poses.

A 2020 paper showing that a plain Transformer applied directly to sequences of image patches, with no convolution at all, can match state-of-the-art convolutional networks on image classification.

A 2023 paper that introduces the Segment Anything project: a promptable segmentation task, a model, and a dataset of over 1 billion masks on 11 million images, all released to advance foundation models for computer vision.

A 3-layer back-propagation network that takes camera and laser range finder images and outputs the direction to steer, trained on simulated road images and tested on Carnegie Mellon's autonomous navigation test vehicle.

In 1989 LeCun and colleagues at AT&T Bell Laboratories trained a convolutional network with backpropagation to read handwritten zip-code digits from U.S. mail, reaching 5.0% test error. The work showed that the convolutional design could be learned end to end on a real task.

MNIST is a database of 60,000 training and 10,000 test images of handwritten digits, normalized to 28×28 pixels and described in the 1998 paper by LeCun and colleagues as a shared benchmark for document recognition.

LeCun, Bottou, Bengio and Haffner’s 1998 review argued that learning systems should replace hand-designed features. It showed that convolutional networks such as LeNet-5 outperformed other methods on handwritten digits, and it introduced the MNIST dataset.

CIFAR-10 and CIFAR-100 are labeled image datasets drawn from the 80 million tiny images collection, documented in a 2009 technical report by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton. CIFAR-10 holds 60,000 32×32 colour images in 10 classes.

A 2009 technical report by Alex Krizhevsky that introduces CIFAR-10 and CIFAR-100 and studies multi-layer generative models for learning features from tiny colour images.

In 2012, Quoc Le, Andrew Ng and colleagues trained a large unsupervised neural network on 10 million internet images and showed it could detect faces without a single labelled face.

R-CNN, posted to arXiv by Girshick, Donahue, Darrell and Malik in November 2013, applied convolutional neural networks to region proposals for object detection and lifted mean average precision on PASCAL VOC 2012 to 53.3%, more than 30% above the previous best[^1].

Zeiler and Fergus ask why large convolutional networks perform so well on ImageNet, introduce a visualization technique for intermediate layers, and use ablation to find an architecture that outperforms Krizhevsky et al. The model also transfers to Caltech-101 and Caltech-256.

DeepFace is a 2014 face verification pipeline that rebuilt the alignment and representation stages of the standard face pipeline, reaching 97.35% accuracy on Labeled Faces in the Wild with a nine-layer network of over 120 million parameters.

A 2014 paper introducing Microsoft COCO, a large dataset of everyday scenes in which common objects are labeled with per-instance segmentations, built to study object recognition as part of scene understanding.

A 2014 paper from Google researchers that introduced the Inception architecture, whose GoogLeNet model set a new state of the art in ILSVRC 2014 by widening and deepening a network within a fixed computational budget.

A 2014 paper by Vinyals and colleagues that trained a single deep network to turn a photograph into a sentence, reporting a BLEU-1 score of 59 on Pascal against 25 for the previous state of the art.

A convolutional network for biomedical image segmentation, posted to arXiv on May 18, 2015, that trains end to end from very few annotated images and won the ISBI cell tracking challenge 2015 in its categories.

A 2015 paper that introduced a Region Proposal Network sharing full-image convolutional features with the detection network, making proposals nearly cost-free and enabling end-to-end training, 5 fps GPU detection, and state-of-the-art accuracy.

The 2015 paper introduced YOLO, which treats object detection as a single regression from image pixels to bounding boxes and class probabilities. One neural network makes predictions in one evaluation, reaching real-time speed.

The 2015 YOLO paper recast object detection as a single neural network's regression over bounding boxes and class probabilities, achieving real-time detection while reducing false positives on empty background.

The 2015 paper introduced residual networks, which add each layer's input back to its output through a shortcut connection. This made networks of over a hundred layers trainable and won first place in several ILSVRC and COCO tasks in 2015.

CycleGAN learns image-to-image translation between two domains without aligned pairs, combining an adversarial loss with a cycle-consistency constraint to constrain the mapping.

A 2017 paper by Lin, Goyal, Girshick, He and Dollar that argues dense one-stage detectors are held back by extreme foreground-background imbalance and proposes focal loss, a reshaped cross-entropy that down-weights well-classified examples.

A 2017 paper by Karras, Aila, Laine and Lehtinen that trains generative adversarial networks by growing the generator and discriminator from low to high resolution, improving stability, quality and variation.

A 2018 paper by Andrew Brock, Jeff Donahue and Karen Simonyan that trained generative adversarial networks at the largest scale attempted at the time, introduced BigGANs and set a new state of the art in class-conditional image synthesis.

A 2020 paper by Alexey Dosovitskiy and colleagues showing that a pure Transformer applied directly to sequences of image patches — the Vision Transformer, or ViT — matches state-of-the-art convolutional networks on image classification when pre-trained on large amounts of data.

CLIP is an OpenAI model trained on 400 million image-text pairs to map pictures and sentences into one shared space, so it can classify images zero-shot from a written description.

SAM 2 is a Meta FAIR foundation model that segments objects in images and videos from simple prompts such as points, boxes or masks, released together with a large video segmentation dataset and training code.

A 2015 arXiv paper by Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho and co-authors that introduced an image-captioning model with visual attention, able to show where it is looking while writing each word.