LLM Chatbot

Private chat UI (Open WebUI) for local models served by Ollama

Category
chatbot
Version
2.0.0
Deployment formats
Docker Compose, Helm (Kubernetes), Terraform (local Docker), Terraform (AWS)
Maintainer
AI Solutions Camp
Tags
Chatbot, LLM, Ollama, Open WebUI, Self-hosted, Local Models

Source code

Overview

Self-hosted chat interface: Open WebUI 0.12.4 on top of Ollama 0.40.11, which runs open models such as Llama 3.2 on your own hardware. Chats, accounts and models stay on your machine.

On the first start a one-shot job pulls a default model (llama3.2, about 2 GB), so the chat works right away. Open WebUI provides user accounts with roles, chat history, model switching and streaming answers. The first account becomes the admin; new sign-ups need approval and can be turned off.

Runs on CPU; an NVIDIA GPU is optional and speeds up answers a lot. Both services listen on 127.0.0.1 only and the session secret has no default. Deploy with Docker Compose, Helm (official Open WebUI chart with bundled Ollama), Terraform with the Docker provider, or Terraform on one AWS EC2 instance.

Requirements

  • Docker Engine with the Compose plugin (docker compose)
  • About 8 GB RAM for 1B-3B models such as llama3.2, about 16 GB for 7B-8B models
  • About 15 GB disk for the two images, plus the models (llama3.2: about 2 GB)
  • Optional: NVIDIA GPU with the NVIDIA Container Toolkit for faster answers

Quick start

  1. Get the stack and create your settings:
   git clone https://github.com/aisolutionscamp/stacks.git
   cd stacks/stacks/llm-chatbot
   cp .env.example .env
  1. Set the secret key in .env:
   sed -i "s/^WEBUI_SECRET_KEY=.*/WEBUI_SECRET_KEY=$(openssl rand -hex 32)/" .env

Optional: change DEFAULT_MODEL (default llama3.2, about 2 GB).

  1. Start the stack (the first start downloads the images and the model):
   docker compose up -d --wait

With an NVIDIA GPU: docker compose -f compose.yaml -f compose.gpu.yaml up -d --wait

  1. Check that it works:
   ./smoke-test.sh
  1. Open http://localhost:3000 and create an account. The first account becomes the admin.
  2. Turn off new sign-ups: Admin Panel Settings General "Enable New Sign Ups".
  3. Pull more models in Admin Panel Settings Models, or:
   docker compose exec ollama ollama pull llama3.1:8b

Environment variables

  • WEBUI_SECRET_KEY (required) – Signs Open WebUI login sessions. Generate with openssl rand -hex 32. Changing it logs out all users.
  • DEFAULT_MODEL – Ollama model pulled on start by the model-pull job and preselected in Open WebUI. Empty skips the download.
  • WEBUI_PORT – Host port for Open WebUI, bound to 127.0.0.1.
  • WEBUI_URL – Public URL of the UI, used in links and OAuth callbacks.
  • WEBUI_NAME – Name shown in the UI.
  • ENABLE_SIGNUP – Allow new accounts. The first account becomes the admin; turn sign-up off afterwards. Only read on the first start, then managed in the admin panel.
  • DEFAULT_USER_ROLE – Role for new accounts: pending (needs admin approval), user or admin.
  • ENABLE_OPENAI_API – Also allow OpenAI-compatible APIs (keys are added in the admin panel).
  • OLLAMA_PORT – Host port for the Ollama API, bound to 127.0.0.1. The API has no authentication.
  • OLLAMA_KEEP_ALIVE – How long a model stays loaded after the last request.
  • OLLAMA_NUM_PARALLEL – Requests processed in parallel per model.
  • OLLAMA_MAX_LOADED_MODELS – Models kept in memory at the same time.
  • OLLAMA_VERSION – ollama/ollama image tag.
  • OPEN_WEBUI_VERSION – ghcr.io/open-webui/open-webui image tag.

Documentation

Architecture

Three containers on one Docker network:

| Service | Image | Port (host) | Purpose | |---|---|---|---| | open-webui | ghcr.io/open-webui/open-webui:0.12.4 | 127.0.0.1:3000 - 8080 | Chat UI, user accounts, chat history (SQLite in the open-webui-data volume) | | ollama | ollama/ollama:0.40.11 | 127.0.0.1:11434 - 11434 | Runs the models; stores them in the ollama-data volume | | model-pull | ollama/ollama:0.40.11 | none | One-shot job: pulls DEFAULT_MODEL into Ollama, then exits |

Open WebUI talks to Ollama over the internal network (http://ollama:11434). Both ports are bound to 127.0.0.1, so nothing is reachable from other machines until you put a reverse proxy with TLS in front of Open WebUI. Ollama's API has no authentication; do not publish it.

The default model is llama3.2 (Meta Llama 3.2 3B, about 2 GB download). Other models can be pulled from the Open WebUI admin panel (Admin Panel Settings Models) or with docker compose exec ollama ollama pull . Model names are listed at https://ollama.com/library.

Requirements

  • Docker Engine with the Compose plugin (docker compose)
  • CPU only: 8 GB RAM is enough for 1B-3B models such as llama3.2; 7B-8B models need about 16 GB. Answers on CPU take seconds to tens of seconds.
  • Disk: about 15 GB for the two images (Ollama 8.4 GB, Open WebUI 6.4 GB unpacked) plus the models (llama3.2 about 2 GB, llama3.1:8b about 4.9 GB)
  • Optional: NVIDIA GPU with the NVIDIA Container Toolkit (compose.gpu.yaml)

Configuration

All settings are in .env (copy from .env.example). Required:

  • WEBUISECRETKEY: signs login sessions. Generate with openssl rand -hex 32. Compose refuses to start without it. Keep it stable; changing it logs everyone out.

Common optional settings: DEFAULTMODEL (empty skips the download), WEBUIPORT, OLLAMAPORT, WEBUIURL (set to the public URL when you use a domain), ENABLEOPENAIAPI (set to true to add OpenAI-compatible APIs in the admin panel), OLLAMAKEEPALIVE, OLLAMANUMPARALLEL, OLLAMAMAXLOADED_MODELS.

GPU (NVIDIA):

docker compose -f compose.yaml -f compose.gpu.yaml up -d --wait

Accounts and sign-up

  • The first account you create on the sign-up page becomes the admin. Create it right after the first start, before anyone else can reach the page.
  • Later sign-ups get the role pending (DEFAULTUSERROLE) and can only use the UI after an admin approves them.
  • After creating your admin account, turn sign-up off in Admin Panel Settings General ("Enable New Sign Ups"), and add users yourself in Admin Panel Users.
  • Open WebUI saves settings such as ENABLESIGNUP, DEFAULTUSERROLE and DEFAULTMODELS in its database on the first start. Changing these variables later has no effect; change them in the admin panel instead.

Security notes

  • No default passwords or keys. WEBUISECRETKEY must be set.
  • Ports are bound to 127.0.0.1. For access from other machines, use a reverse proxy with TLS (Caddy, Traefik, nginx) in front of port 3000 and set WEBUI_URL.
  • Do not expose port 11434. Anyone who reaches Ollama can run, pull and delete models.
  • The OpenAI API connection is off by default (ENABLEOPENAIAPI=false), so no request leaves the machine except model and image downloads.
  • Back up the open-webui-data volume: it holds accounts and chats.

Upgrading

  1. Read the release notes of Open WebUI and Ollama.
  2. Back up the data volume, for example:

docker run --rm -v llm-chatbot_open-webui-data:/data -v "$PWD":/backup alpine tar czf /backup/open-webui-data.tgz -C /data .

  1. Change OPENWEBUIVERSION / OLLAMA_VERSION in .env.
  2. docker compose pull && docker compose up -d --wait. Open WebUI runs its database migrations on start.

Testing

smoke-test.sh checks a running stack: Ollama API, model pull, a real completion from Ollama, and Open WebUI /health, the UI and /api/config. With SMOKEWEBUISIGNUP=true (only on an empty test instance, as in CI) it also creates an account and gets a chat completion through Open WebUI.

./smoke-test.sh

Troubleshooting

  • required variable WEBUISECRETKEY is missing: set it in .env (openssl rand -hex 32).
  • open-webui is "starting" for a few minutes on the first run: it downloads the embedding model used for document chat and runs migrations. docker compose logs -f open-webui shows progress.
  • No models in the model list: check docker compose logs model-pull, or pull one with docker compose exec ollama ollama pull llama3.2.
  • Very slow answers: the model runs on CPU. Use a smaller model, or the GPU override.
  • Out of memory / Ollama restarts: choose a smaller model, or lower OLLAMANUMPARALLEL and OLLAMAMAXLOADED_MODELS.
  • Lost the admin password: see the Open WebUI docs on resetting the admin password (it is stored in webui.db in the open-webui-data volume).

Other deployment options

  • Kubernetes (Helm): helm/install.sh installs the official Open WebUI chart (17.0.0) with its bundled Ollama subchart; settings in helm/values.yaml.
  • Terraform (Docker): terraform/local/main.tf runs the same containers on one machine with the kreuzwerker/docker provider.
  • Terraform (AWS): terraform/aws/main.tf starts one EC2 instance (optional GPU) that runs this compose stack; access is limited to allowed_cidr, administration through SSM Session Manager.

docker-compose.yml

# LLM Chatbot: Open WebUI (chat interface) on top of Ollama (local model server).
#
#   cp .env.example .env    # then set WEBUI_SECRET_KEY
#   docker compose up -d --wait
#
# NVIDIA GPU: docker compose -f compose.yaml -f compose.gpu.yaml up -d --wait

name: llm-chatbot

services:
  ollama:
    image: ollama/ollama:${OLLAMA_VERSION:-0.40.11}
    restart: unless-stopped
    environment:
      OLLAMA_KEEP_ALIVE: ${OLLAMA_KEEP_ALIVE:-5m}
      OLLAMA_NUM_PARALLEL: ${OLLAMA_NUM_PARALLEL:-1}
      OLLAMA_MAX_LOADED_MODELS: ${OLLAMA_MAX_LOADED_MODELS:-1}
    volumes:
      - ollama-data:/root/.ollama
    ports:
      # Ollama API has no authentication. Keep it on 127.0.0.1.
      - "127.0.0.1:${OLLAMA_PORT:-11434}:11434"
    healthcheck:
      # "ollama list" talks to the local server, so it only succeeds once the API is up.
      test: ["CMD", "ollama", "list"]
      interval: 10s
      timeout: 5s
      retries: 12
      start_period: 10s

  # One-shot job: pulls DEFAULT_MODEL into the ollama volume, then exits.
  # Leave DEFAULT_MODEL empty to skip it. Already-present models are only checked, not downloaded again.
  model-pull:
    image: ollama/ollama:${OLLAMA_VERSION:-0.40.11}
    restart: "no"
    depends_on:
      ollama:
        condition: service_healthy
    environment:
      OLLAMA_HOST: http://ollama:11434
      DEFAULT_MODEL: ${DEFAULT_MODEL:-llama3.2}
    entrypoint: ["/bin/sh", "-c"]
    command:
      - |
        if [ -z "$$DEFAULT_MODEL" ]; then echo "DEFAULT_MODEL is empty, nothing to pull"; exit 0; fi
        echo "Pulling $$DEFAULT_MODEL"
        ollama pull "$$DEFAULT_MODEL"

  open-webui:
    image: ghcr.io/open-webui/open-webui:${OPEN_WEBUI_VERSION:-0.12.4}
    restart: unless-stopped
    depends_on:
      ollama:
        condition: service_healthy
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
      WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY:?set WEBUI_SECRET_KEY in .env (openssl rand -hex 32)}
      WEBUI_NAME: ${WEBUI_NAME:-LLM Chatbot}
      WEBUI_URL: ${WEBUI_URL:-http://localhost:${WEBUI_PORT:-3000}}
      # The first account created becomes the admin. Set to false after that.
      ENABLE_SIGNUP: ${ENABLE_SIGNUP:-true}
      DEFAULT_USER_ROLE: ${DEFAULT_USER_ROLE:-pending}
      DEFAULT_MODELS: ${DEFAULT_MODEL:-llama3.2}
      # Only local models through Ollama. Set to true to add OpenAI-compatible APIs in the admin panel.
      ENABLE_OPENAI_API: ${ENABLE_OPENAI_API:-false}
    volumes:
      - open-webui-data:/app/backend/data
    ports:
      - "127.0.0.1:${WEBUI_PORT:-3000}:8080"
    healthcheck:
      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 20
      # First start downloads the default embedding model for document chat and runs DB migrations.
      start_period: 120s

volumes:
  ollama-data:
  open-webui-data:

All AI stacks