Using Ollama's OpenAI-compatible API

Plugging local models into an existing OpenAI stack has an easy path: Ollama's OpenAI compatibility layer. Official SDKs stay untouched — only the base URL changes. Ollama implements a subset of the OpenAI API, and this page walks the endpoints one by one to separate what works from what does not ^[1].

Two entry points

The layer opens two ways. The cloud entry connects to Ollama's hosted service — no Ollama installation needed at all: put the API key in OLLAMA_API_KEY, pip install the openai client, and point the base URL at https://ollama.com/v1 ^[1]:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://ollama.com/v1",
    api_key=os.environ["OLLAMA_API_KEY"],
)

response = client.chat.completions.create(
    model="gemma4:31b",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)

The same client also issues stateless Responses requests ^[1]:

response = client.responses.create(
    model="gemma4:31b",
    input="Say hello in one sentence.",
)
print(response.output_text)

The cloud surface has edges: no stateful Responses, no built-in web search through /v1/responses, and no custom or freeform tool-call replay ^[1].

The local entry is the familiar localhost:11434. One counterintuitive detail: the client must pass an api_key argument, but Ollama ignores the value entirely — any string works; using cloud models through a local server requires signing in to Ollama first ^[1].

One protocol, three languages

Python, JavaScript, and curl differ only in client construction ^[1]:

from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1/',
    api_key='ollama',  # required but ignored
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            'role': 'user',
            'content': 'Say this is a test',
        }
    ],
    model='gpt-oss:20b',
)
print(chat_completion.choices[0].message.content)

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:11434/v1/",
  apiKey: "ollama", // required but ignored
});

const chatCompletion = await openai.chat.completions.create({
  messages: [{ role: "user", content: "Say this is a test" }],
  model: "gpt-oss:20b",
});

console.log(chatCompletion.choices[0].message.content);

curl -X POST http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
  "model": "gpt-oss:20b",
  "messages": [{ "role": "user", "content": "Say this is a test" }]
}'

The local Responses call is even simpler — input is a plain string ^[1]:

from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1/',
    api_key='ollama',  # required but ignored
)

responses_result = client.responses.create(
  model='qwen3:8b',
  input='Write a short poem about the color blue',
)
print(responses_result.output_text)

Base64-only images are a pragmatic local-first choice: no external reachability required, local files encode straight into the request; the cost is payload size, where a few high-resolution images push requests into the megabytes. A sturdier conversation pattern sends the image once and refers to conclusions textually afterward, rather than re-attaching the same image in history.

The endpoint inventory

/v1/chat/completions supports chat, streaming, JSON mode, reproducible outputs, vision, tools, and reasoning control for thinking models; logprobs are absent ^[1]. Three field-level notes: images in messages accept only base64 encoding, not image URLs; tool_choice, logit_bias, user, and n are all unsupported ^[1].

Thinking control comes through two fields — reasoning_effort and reasoning.effort — with model-specific values discoverable via /api/show. Resolution is layered: metadata-driven models match supported names exactly and fall back to the default otherwise; boolean-only models map recognized compatibility efforts to true and "none" to false; models without metadata keep the aliases "minimal" to "low" and "xhigh"/"ultra" to "max"; GPT-OSS is special-cased — "minimal" to "low", "xhigh"/"ultra" to "high" — because its thinking cannot be disabled ^[1].

/v1/completions adds a suffix field among its supported set, accepts only string prompts, and skips best_of and echo along with logprobs ^[1]. On /v1/models, created reflects the last modification time and owned_by is the Ollama username, defaulting to "library" ^[1]. /v1/embeddings takes strings and string arrays — no token arrays — plus an optional dimensions field ^[1].

Audio endpoints are the recent additions: /v1/audio/speech (v0.40.8) streams audio from speech models, caps input at 4096 characters, offers only wav (default) and pcm output, and returns a 400 for unsupported formats rather than ignoring them — the audio is 16-bit mono at 24kHz ^[1]. /v1/audio/transcriptions accepts WAV up to 384 kHz, 512 MB, and 2 hours, and audio beyond the model's single-pass limit (30 seconds for current models) is split at pauses and transcribed piece by piece ^[1]. /v1/audio/translations (v0.40.12) translates speech across languages, with Ollama extensions to_language (ISO 639-1, default en) and language ^[1].

The Responses API (since v0.13.3) implements only the stateless flavor — previous_response_id and conversation are unsupported — and carries the Ollama think extension accepting a boolean, a model-defined string, or null ^[1].

Two adaptation points

Local inference requires pulling first, while cloud requests skip it — identifiers come from ollama.com/api/tags, and a signed-in local server selects cloud models like gemma4:cloud ^[1]:

ollama pull llama3.2

Tooling that hard-codes default OpenAI names such as gpt-3.5-turbo gets a zero-code fix through ollama cp ^[1]:

ollama cp llama3.2 gpt-3.5-turbo

curl http://localhost:11434/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "gpt-3.5-turbo",
        "messages": [
            {
                "role": "user",
                "content": "Hello!"
            }
        ]
    }'

The other unavoidable point is context length: the OpenAI protocol has no field for it, so Ollama's answer is a Modelfile with num_ctx followed by ollama create under a new name ^[1]:

FROM <some model>
PARAMETER num_ctx <context size>

curl http://localhost:11434/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "mymodel",
        "messages": [
            {
                "role": "user",
                "content": "Hello!"
            }
        ]
    }'

Those two tricks cover the most common adaptation needs. The remaining gaps — image URLs, tool_choice, multi-candidate n — call for either caller-side changes or a fallback to Ollama's native API; checking the endpoint inventory before committing saves most of the debugging time later.

Adapted from the OpenAI compatibility section of the Ollama documentation^[1], released under the MIT License.