Cloud Products

vLLM is an open-source serving system for large language models, built on PagedAttention, which manages the KV cache the way an operating system manages virtual memory. It reports 2-4x higher throughput than earlier serving systems at the same latency.

Amazon SageMaker is AWS's fully managed service for building, training and hosting machine learning models, announced in 2017. It combines hosted notebooks, distributed training and scalable inference endpoints, and each part can be used on its own.

The Hugging Face Hub is the platform where people share machine-learning models, datasets and demo apps. By September 2026 it had more than 3 million models and 18 million users.

The OpenAI API is OpenAI's commercial developer platform, announced on 11 June 2020 with GPT-3. Over the following years it grew into a platform for building AI agents, adding the Responses API, Agents SDK, AgentKit and other tools.

OpenRouter is a marketplace and gateway for large language models, started in early 2023, that lets developers reach hundreds of models from many providers through one interface. It reports more than 10 million users and a $113 million Series B.

Amazon Bedrock is AWS's managed, serverless service for accessing foundation models from several companies, announced on 13 April 2023. It became generally available in September 2023 and added Amazon's own Nova models and the AgentCore agent platform later.

TensorRT-LLM is NVIDIA's open-source library for speeding up and optimizing large language model inference on NVIDIA GPUs, publicly released on 19 October 2023 and usable free from GitHub or as part of NVIDIA NeMo.

SGLang is an open-source system for running complex language model programs: a frontend language plus a runtime that reuses cached attention state and speeds up structured output, reporting up to 6.4x higher throughput than prior inference systems.

Kiro is Amazon Web Services' agentic IDE, announced in July 2025, which turns a prompt into user stories, a design document and a task list before writing code — an approach its makers call spec-driven development.

Apache MXNet is an open-source deep learning library that offers both symbolic and imperative programming, runs from mobile devices to GPU clusters, and ships bindings for eight languages. Its paper was posted in December 2015 by Tianqi Chen, Mu Li and colleagues.

Amazon Q is a generative AI assistant for work from AWS. It was announced in preview at re:Invent on 28 November 2023, respects each user's permissions and became generally available as Amazon Q Business on 30 April 2024.