Gemma 4 with Ollama and Open WebUI

The fastest path to running a new model locally with a usable chat interface is a two-tool stack: Ollama runs the model, Open WebUI provides the interface. This page deploys Gemma 4 E4B-it with that pairing — the gemma4:e4b tag on the Ollama Library — on the reference stack of Ubuntu 22.04, Python 3.12, PyTorch 2.5.1, and CUDA 12.4 ^[1].

Division of labor

First, the boundaries. Ollama is an open-source LLM serving tool whose goal is letting users run models locally quickly — a single run command handles download, loading, and interaction together ^[1]. Open WebUI is an extensible, feature-rich, user-friendly self-hosted WebUI designed to operate fully offline, supporting multiple runners including Ollama and OpenAI-compatible APIs ^[1]. The combination buys a clean split: the model layer and the serving layer decouple, so swapping a model never touches the interface and vice versa.

If you only need to validate the model itself, llama.cpp offers a one-liner alternative: llama-server -hf ggml-org/gemma-4-E4B-it-GGUF brings up the GGUF build as a service ^[1].

Installing Ollama

Three platforms, three routes. macOS and Windows both ship official installer packages — download and double-click ^[1]. Linux has two paths ^[1]:

Automatic installation in one line:

curl -fsSL https://ollama.com/install.sh | sh

Manual installation downloads the archive and extracts it into /usr ^[1]:

curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

The manual route buys control: the download URL can be swapped for a mirror, which keeps constrained networks moving when the install script fails ^[1]:

curl -L https://git.886.be/https://github.com/ollama/ollama/releases/download/v0.6.0/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

Bringing up the model

Service start and model run are two commands ^[1]:

ollama serve

ollama run gemma4:e4b

The run command doubles as the downloader: a missing tag is fetched before the interactive session opens. To confirm the model is actually loaded, the ps subcommand shows runtime state ^[1]:

ollama ps

Installing and running Open WebUI

Open WebUI installs through pip, pinned at 0.5.20 with a Tsinghua mirror for speed ^[1]:

# 升级 pip
python -m pip install --upgrade pip
# 更换 pypi 源加速库的安装
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

pip install open-webui==0.5.20

The service takes port 8080 by default; --port moves it, say to the common 6006 ^[1]:

open-webui serve

open-webui serve --port 6006

If startup hits Connection to huggingface.co timed out, initialization is trying to reach Hugging Face; setting the mirror endpoint in the environment and restarting fixes it ^[1]:

export HF_ENDPOINT=https://hf-mirror.com

Open WebUI is more than a chat box: multi-user accounts, session history, prompt libraries, and retrieval over uploaded documents all work in the self-hosted form. Placed on a LAN, it lets a small team share one model host while keeping sessions isolated per account. The fully-offline design goal means the whole chain depends on no external service — usable even with the network down.

The handshake in the browser

Opening http://localhost:6006 brings up the interface. The key integration step is model discovery: with the Ollama service running and gemma4:e4b loaded, Open WebUI automatically detects local Ollama and selects the model — no manual connection strings or API endpoints to configure ^[1].

The auto-detection deserves a closer look: it assumes Ollama listens on its default port. If the service moved ports or another host, the connection address must be set by hand in Open WebUI's settings. Same-machine deployment on the default port is the lowest-friction shape for this pairing. Once the interface is up, chat away to verify Gemma 4 E4B-it end to end ^[1].

The deployment checklist

The whole path compresses into a checklist: install Ollama (script or manual archive), start ollama serve, run gemma4:e4b, confirm with ps, pip-install Open WebUI with a pinned version, serve the interface (adjusting port and mirror as needed), and verify the auto-detected model in the browser. None of the seven steps requires writing code — all command-line work, and a solid standard flow for standing up a local chat environment on a fresh machine ^[1].

Adapted from the Gemma 4 Ollama and Open WebUI deployment section of Datawhale's open guide *self-llm*^[1], released under the Apache-2.0 License.