Skip to content

TellWorldAI World Archive

Ask 中文

Home›Topics›Deep learning›Transformers & attention

Techniques

Transformers & attention Companies

Google Research Google's research division, based in Mountain View and usually dated to 2000, produced the Transformer, BERT and T5 — the work underlying modern large language models.
Google Brain Google Brain was a deep-learning research team started inside Google's X in 2011. Its cat-recognition experiment, TensorFlow, sequence-to-sequence models and the Transformer made it central to modern AI; it merged into Google DeepMind in 2023.

Events in this topic

Dec 1, 2025Update

Hugging Face announces Transformers v5

Hugging Face Hugging Face Transformers library

Feb 22, 2024Release

Stability AI previews Stable Diffusion 3

Transformer Stability AI Stable Diffusion 3

Dec 13, 2022Release

Google publishes RT-1

RT-1 (Robotics Transformer)Google

Mar 22, 2022Release

NVIDIA announces the Hopper architecture and H100

Transformer NVIDIA H100 NVIDIA

Oct 11, 2018Paper

Google researchers publish BERT on arXiv

Transformer BERT BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Pre-training and fine-tuning Google Research Jacob Devlin

Jun 11, 2018Paper

OpenAI publishes the first GPT paper on generative pre-training

Improving Language Understanding by Generative Pre-Training (GPT)OpenAI Pre-training and fine-tuning Ilya Sutskever Alec Radford

Jun 12, 2017Paper

“Attention Is All You Need” is posted to arXiv, introducing the Transformer

Attention Is All You Need Transformer Google Brain Ashish Vaswani Noam Shazeer