Deploying MiniMax-M3 with vLLM

Who this is for

This guide is for developers who have four or eight GPUs, want to run MiniMax-M3 on their own machine or server, and want to call it through an OpenAI-compatible API. By the end you should be able to start a vLLM server and send chat and image requests from Python or curl.

What MiniMax-M3 and vLLM are

MiniMax-M3 is an open-source large language model from MiniMax. It has a context window of 512K tokens, can output up to 128K tokens in one response, and accepts image input^[1]. Image input works through both OpenAI-compatible and Anthropic-compatible interfaces; video, audio and documents are not supported yet^[1]. The official code is on GitHub at MiniMax-AI/MiniMax-M3, and the model page is on Hugging Face at MiniMaxAI/MiniMax-M3^[2].

vLLM is a high-performance inference framework for large language models. It works out of the box and exposes an OpenAI-compatible interface, and it supports long-context inference, streaming output, tool calling and parsing of thinking content^[1][3]. Across several GPUs it can use tensor parallelism, which splits the model's computation over the cards and is what the four-GPU example uses, or expert parallelism, which the eight-GPU example turns on as well^[1].

Preparation

Memory decides whether the model runs at all. The figures are about 220 GB of GPU memory for the weights and about 240 GB for every 1M tokens of context^[1]. With four 96 GB GPUs the total context capacity is about 400K tokens, and with eight 144 GB GPUs about 3M tokens^[1]. These are the maximum total cache capacities the hardware supports, not limits on a single request. For a real deployment, estimate from the number of concurrent requests and the context each one uses, then choose the number of cards.

On the software side, check that nvidia-smi lists every card, then check that PyTorch can see CUDA^[1]. Use venv, conda or uv to create a separate environment so vLLM's dependencies do not collide with other projects^[1].

Steps

1. Check the GPUs and CUDA. nvidia-smi should list four or eight cards, and PyTorch should report that CUDA is available.
2. Create the environment and install vLLM with uv^[1]:

uv venv
source .venv/bin/activate
uv pip install vllm --torch-backend=auto

The first two lines create and activate the virtual environment. The last line installs vLLM; keep --torch-backend=auto as written^[1].

1. Download the model (optional). vLLM downloads and caches the model from Hugging Face on first start, so a manual download is not required. If you want to fetch it first, or Hugging Face is slow for you, use ModelScope^[1]:

pip install modelscope
python model_download.py

The example script calls snapshot_download from the modelscope package, with the model ID MiniMaxAI/MiniMax-M3 and revision master^[1]. Downloads through ModelScope do not need the Hugging Face mirror. If vLLM pulls from Hugging Face directly and the network is restricted, run export HF_ENDPOINT=https://hf-mirror.com first^[1].

1. Start the server. The four-GPU command is^[1]:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M3 --trust-remote-code \
    --tensor-parallel-size 4 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

For eight GPUs, add --enable_expert_parallel and set tensor parallelism to 8^[1]:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M3 --trust-remote-code \
    --enable_expert_parallel --tensor-parallel-size 8 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

The first load is slow and can take more than half an hour^[1]. Once the server is up it listens at http://127.0.0.1:8000/v1^[1]; every client below uses that address as its base URL.

1. Test with curl. A reply from the chat endpoint confirms the server works^[1]:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3",
    "messages": [
      {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
      {"role": "user", "content": [{"type": "text", "text": "请简要介绍 MiniMax-M3 模型的特点。"}]}
    ]
  }'

1. Call it from Python. Point the official OpenAI Python SDK at the local server; the API key can be any placeholder string^[1]:

# test_chat.py
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1",
)

response = client.chat.completions.create(
    model="MiniMaxAI/MiniMax-M3",
    messages=[
        {"role": "user", "content": "请介绍 MiniMax-M3 相比 M2.5 有哪些提升？"}
    ],
    max_tokens=8192,
    top_p=0.95,
    temperature=1.0,
)

msg = response.choices[0].message
print("MiniMax-M3:", msg.content)

In the code, response.choices[0].message is the model's reply and msg.content is its text^[1]. The recommended sampling settings are temperature 1.0, top_p 0.95 and top_k 20^[1]. max_tokens sets how long the reply can be. M3 can output up to 128K tokens, but a very large value raises memory use and latency^[1].

Image input

Image input goes in the content array of a user message, alongside a text item, as an image_url entry^[1]. Only images are supported for now; video, audio and documents are not^[1]. To have the model work with a document, convert it to text or images yourself before sending it.

Tool calling and thinking content

Tool calling is turned on at the server. --enable-auto-tool-choice lets the model start tool calls on its own when it needs them, --tool-call-parser minimax_m2 selects MiniMax's tool-call parser, and --reasoning-parser minimax_m2_append_think parses thinking content^[1]. All three are already in the startup commands above.

Common problems

• Hugging Face downloads are slow or fail: use ModelScope, or set the HF_ENDPOINT mirror^[1].
• "MiniMax-M3 model is not currently supported": the vLLM version is too old; run pip install -U vllm^[1].
• CUDA memory errors (torch.AcceleratorError): add --compilation-config "{\"cudagraph_mode\": \"PIECEWISE\"}" to the startup arguments^[1].
• Garbled output: upgrade to the latest vLLM release that matches MiniMax-M3^[1].

Next steps

Check flags and deployment options against the official vLLM documentation at docs.vllm.ai^[3]. The model weights and model card are on the Hugging Face page for MiniMaxAI/MiniMax-M3^[2].

Adapted from the tutorial "MiniMax-M3 vLLM 部署调用" in Datawhale's open-source project self-llm^[1], released under the Apache-2.0 License.