Ollama

What Ollama is

Ollama is a program for running and managing large language models (LLMs) on a user's own computer rather than through a remote service. It runs on macOS, Linux and Windows, and can be driven either from a command line interface or from a local REST API that other software can call. ^[3] It is written in the Go programming language. ^[3]

"Local" here means the model's weights sit on the user's own disk and the computation happens on the user's own hardware. Instead of sending a prompt to a company's servers and waiting for a reply, the user starts a model on their machine and gets answers from it directly.

Where it came from

Ollama was created by Jeffrey Morgan and Michael Chiang in 2023. ^[3] The project's repository, ollama/ollama, was created on GitHub on 26 June 2023 ^[1], and the first release, v0.0.1, was published on 8 July 2023. ^[2] The code is released under the MIT license, a permissive open-source license that allows others to reuse and modify it, and the repository has roughly 186,000 stars on GitHub. ^[1]

How it works

Two design choices shape how the tool behaves.

The first is the inference backend. Ollama uses llama.cpp to run models locally. ^[3] llama.cpp is an open-source engine for running language models efficiently on ordinary computers, including machines without data-centre accelerators, so a laptop or a desktop workstation can serve as the place where inference happens.

The second is packaging. Ollama uses a model-library format for pulling and running model variants by name. ^[3] In practice this means a user refers to a model by its name, and the tool fetches the matching variant and runs it, rather than requiring the user to hunt down weight files, convert them and wire up a serving stack by hand. Keeping the model names and the runtime in one place is what makes the command line experience short: name a model, get a working model.

Why it mattered

Ollama became associated with the growth of local large-language-model software, letting users run models such as Llama, Gemma, Mistral, Qwen, gpt-oss, GLM and DeepSeek on their own machines. ^[3]

That matters for a few practical reasons. Running a model locally means prompts and documents do not have to leave the machine, which suits work with private or sensitive material. It means the tool still works without a network connection once the model has been downloaded. It also gives people a cheap way to experiment: instead of comparing hosted services through their APIs, a user can switch between model families on the same computer and see how they behave on their own tasks. Because the same interface covers many different model families, the choice of model becomes a name to type rather than a new piece of software to install.

The project sits at a point in the stack where several trends meet: open model weights published by many labs, efficient open-source inference engines, and developers who want to build applications on top of models without depending entirely on a hosted API. Ollama's contribution is mostly plumbing — a consistent way to fetch, run and manage models — rather than new modelling research.

Limits

Local inference is bounded by the machine it runs on. A model that needs more memory than the computer has will not run well, or at all, and larger models generally answer more slowly on consumer hardware than smaller ones. Models that fit comfortably on a laptop are usually reduced in size compared with the largest hosted systems, so quality can differ. Managing downloads, disk space and versions also remains the user's responsibility. These are constraints of running models locally rather than faults specific to any one tool, and they define what this kind of software is good for: private, offline and experimental use, more than replacing large hosted services outright.