Serving MiniMax-M2 with vLLM

MiniMax-M2 is an open model built for agents and code: planning long tool-call chains and coordinating Shell, browser, Python execution, and MCP tools is its home turf ^[1]. Deploying it on vLLM, the first budget to compute is not parameters but VRAM — the weights need about 220GB, and every 1M context tokens adds roughly 240GB more ^[1]. This page walks the full deployment across two tiers — 96GB x 4 GPUs and 144GB x 8 GPUs — and closes with a tool-calling test.

The VRAM budget and choosing a GPU count

Once the memory math is clear, the card count follows ^[1]: four 96GB cards support about 400K total context tokens, while eight 144GB cards reach about 3M. The two tiers map to two parallel strategies: pure tensor parallelism on 4 GPUs, expert parallelism added on 8. The reference environment is 8 x RTX PRO 6000 with 96GB each ^[1]. When memory runs tight, lowering --mem-fraction-static or trimming --max-model-len buys headroom ^[1].

Environment and installation

Dependencies belong in a virtual environment to avoid conflicts ^[1]. Installation pins versions with uv and points vLLM at the nightly wheel index ^[1]:

pip install --upgrade pip
pip install uv==0.9.7
[ -d ".venv" ] || uv venv
source .venv/bin/activate
uv pip install modelscope==1.31.0
uv pip install openai==2.6.1
uv pip install 'triton-kernels @ git+https://github.com/triton-lang/triton.git@v3.5.0#subdirectory=python/triton_kernels'  vllm --extra-index-url https://wheels.vllm.ai/nightly --prerelease=allow

triton-kernels installs from source on GitHub — the step most likely to fail on a constrained network, usually during the repository clone.

Model download

vLLM fetches and caches the model from Hugging Face on first start; on restricted networks, pre-download with modelscope using the name MiniMaxAI/MiniMax-M2 ^[1]:

# model_download.py
from modelscope import snapshot_download

model_dir = snapshot_download('MiniMaxAI/MiniMax-M2', cache_dir='/root/autodl-tmp', revision='master')
print(f"模型下载成功，保存到: {model_dir}")

The weights are large; when bandwidth is scarce, download on a high-bandwidth machine first and copy to the target server ^[1].

Starting the service: one command per GPU count

Two launch styles exist, and the script variant pins parameters and logs nicely. The script inspects the detected GPU count and picks the command — anything other than 4 or 8 GPUs raises immediately ^[1]:

import torch
from vLLM.utils import launch_server_cmd, wait_for_server

gpu_count = torch.cuda.device_count() if torch.cuda.is_available() else 0
if gpu_count == 4:
    cmd = (
        "SAFETENSORS_FAST_GPU=1 vllm serve "
        "MiniMaxAI/MiniMax-M2 --trust-remote-code "
        "--tensor-parallel-size 4 "
        "--enable-auto-tool-choice --tool-call-parser minimax_m2 "
        "--reasoning-parser minimax_m2_append_think"
    )
elif gpu_count == 8:
    cmd = (
        "SAFETENSORS_FAST_GPU=1 vllm serve "
        "MiniMaxAI/MiniMax-M2 --trust-remote-code "
        "--enable_expert_parallel --tensor-parallel-size 8 "
        "--enable-auto-tool-choice --tool-call-parser minimax_m2 "
        "--reasoning-parser minimax_m2_append_think"
    )
else:
    raise RuntimeError(f"建议使用 4 或 8 张 GPU，当前检测到: {gpu_count}")

server_process, port = launch_server_cmd(cmd, port=8000)
wait_for_server(f"http://127.0.0.1:{port}")
print(f"vLLM Server started: http://127.0.0.1:{port}")

The equivalent command-line form for 4 GPUs ^[1]:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M2 --trust-remote-code \
    --tensor-parallel-size 4 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

The 8-GPU version adds --enable_expert_parallel ^[1]:

SAFETENSORS_FAST_GPU=1 vllm serve \
    MiniMaxAI/MiniMax-M2 --trust-remote-code \
    --enable_expert_parallel --tensor-parallel-size 8 \
    --enable-auto-tool-choice --tool-call-parser minimax_m2 \
    --reasoning-parser minimax_m2_append_think

The two parsers are a fixed pair in the launch line: --tool-call-parser minimax_m2 handles tool-call formatting, and --reasoning-parser minimax_m2_append_think handles reasoning content. First load can take over half an hour at this model size; the service then listens at http://127.0.0.1:8000/v1 ^[1]. A quick curl confirms the path end to end ^[1]:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M2",
    "messages": [
      {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
      {"role": "user", "content": [{"type": "text", "text": "Which team won the worldcup in 2022?"}]}
    ]
  }'

A temperature of 1.0 looks aggressive but is normal for reasoning models: the thinking phase needs enough randomness to explore solution paths, and pushing it down invites repetitive reasoning loops. Determinism matters for argument extraction and classification instead — lower the temperature just for those. Using tiered sampling parameters, official values for agent orchestration calls and low temperature for extraction calls, is a detail worth writing into any calling convention.

Three calling styles

Completions, chat, and streaming, all through the OpenAI SDK. The completions call uses the officially recommended sampling parameters ^[1]:

# test_completion.py
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1",
)

response = client.completions.create(
    model="MiniMaxAI/MiniMax-M2",
    prompt="简要介绍一下 MiniMax M2 模型的特点。",
    max_tokens=8192,
    top_p=0.95,
    temperature=1.0,
)
print(response)

In chat replies the thinking sits inside <think> tags with the final answer after them ^[1]:

# test_chat.py
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1",
)

response = client.chat.completions.create(
    model="MiniMaxAI/MiniMax-M2",
    messages=[
        {"role": "user", "content": "MiniMAX公司的愿景是什么？你觉得他们会成功吗，请作出详细分析。"}
    ],
    max_tokens=8192,
    top_p=0.95,
    temperature=1.0,
)

msg = response.choices[0].message
print("MiniMax-M2:", msg.content)

Streaming suits long generations, printing deltas chunk by chunk ^[1]:

# test_streaming.py
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://127.0.0.1:8000/v1",
)

stream = client.chat.completions.create(
    model="MiniMaxAI/MiniMax-M2",
    messages=[{"role": "user", "content": "请写一篇题为Agent时代大模型应用落地要点的调研报告。"}],
    stream=True,
    max_tokens=32768,
    top_p=0.95,
    temperature=1.0,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta and delta.content:
        print(delta.content, end="", flush=True)

For long-report style tasks, budget max_tokens generously — the 32768-token allowance splits between thinking and the answer, and a smaller cap truncates before the final text completes.

The tool-calling test

Tool calling is enabled by default in the vLLM deployment: the model decides when an external tool is needed and emits structured arguments ^[1]. The demo defines a weather function with a typed tool description ^[1]:

# test_tool_calling.py
from openai import OpenAI
import json

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def get_weather(location: str, unit: str):
    return f"Getting the weather for {location} in {unit}..."

tool_functions = {"get_weather": get_weather}

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location", "unit"]
        }
    }
}]

The request reads the model name dynamically from the model-list endpoint, sidestepping hard-coded paths ^[1]:

response = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=[{"role": "user", "content": "What's the weather like in San Francisco? use celsius."}],
    tools=tools,
    tool_choice="auto"
)

print(response)

tool_call = response.choices[0].message.tool_calls[0].function
print(f"Function called: {tool_call.name}")
print(f"Arguments: {tool_call.arguments}")
print(f"Result: {get_weather(**json.loads(tool_call.arguments))}")

The run is clean: the model picks get_weather with {"location": "San Francisco, CA", "unit": "celsius"} — the prompt named only the city, the model completed the state abbreviation following the example inside the tool description, and the unit follows the user's explicit Celsius request ^[1].

Parameter quick reference

To close ^[1]: the request's model must match the serve-time name; --tensor-parallel-size is usually the GPU count; the 8-GPU example turns on --enable_expert_parallel; --enable-auto-tool-choice lets the model initiate tool calls; oversized max_tokens buys only VRAM and latency; and deterministic output comes from lower temperature and top_p. For sampling, the official recommendation is temperature=1.0, top_p=0.95, top_k=20 ^[1].

Adapted from the MiniMax-M2 vLLM deployment section of Datawhale's open guide *self-llm*^[1], released under the Apache-2.0 License.