Deploy Open WebUI and Ollama on Kubernetes with Helm

Updated 2026-10-26 · 12 min read

A production-shaped Helm setup for Open WebUI and Ollama on Kubernetes, from GPU node preparation to TLS, sign-up control, upgrades and the failures you will most likely hit.

To deploy Open WebUI on Kubernetes, install the official open-webui/open-webui Helm chart from https://helm.openwebui.com/. It bundles the community Ollama Helm chart (otwld) as a subchart, so one helm upgrade --install gives you the chat UI, an Ollama server with a persistent model store, and the wiring between them. For GPU inference you also need the NVIDIA GPU Operator or device plugin on the cluster, a GPU limit on the Ollama pod, and a node selector or toleration that lands it on a GPU node.

This guide walks through a production-shaped setup: GPU nodes, a pinned values file, preloaded models, persistence, ingress with TLS, sign-up and SSO settings, upgrades, and the failures you are most likely to hit. If you only want a single machine with Docker Compose, the Ollama chatbot stack is the faster route.

What you are deploying

The Open WebUI chart can install several components. Know which ones you need before you write values.

| Component | Chart key | Default | What it does | |---|---|---|---| | Open WebUI | (top level) | on | Web UI, user accounts, chat history, RAG over uploaded files | | Ollama | ollama.* | on | Subchart from otwld, serves models on port 11434 | | Pipelines | pipelines.* | on | Optional plugin server for filters and custom pipelines | | Redis | websocket.redis.* | on | WebSocket manager, required once you run more than one replica | | Apache Tika | tika.* | off | Optional document text extraction |

Two things to decide up front:

1. Bundled Ollama or separate release. The subchart is convenient, but it ties Ollama's lifecycle to the UI. If several apps will share one Ollama, or you want to upgrade them independently, install the otwld/ollama chart as its own release and point Open WebUI at it with ollamaUrls. Both approaches are shown below.
2. Single replica or HA. One Open WebUI replica with SQLite on a ReadWriteOnce volume is fine for a team. More than one replica needs PostgreSQL (databaseUrl), ReadWriteMany storage or an object storage provider, Redis, and a shared WEBUISECRETKEY. Start with one replica unless you have a reason not to.

Pipelines is enabled by default. If you are not going to write pipelines, turn it off and save a pod.

Prerequisites: GPU nodes in Kubernetes

Kubernetes does not see GPUs on its own. A node advertises nvidia.com/gpu as an allocatable resource only after the NVIDIA driver, the container toolkit and the device plugin are in place.

GPU Operator or device plugin

You have two options:

• NVIDIA GPU Operator. Installs and manages the driver, NVIDIA Container Toolkit, device plugin, GPU Feature Discovery and DCGM monitoring as pods. This is the right choice on self-managed clusters and on nodes whose OS image has no driver.
• Device plugin only. If your nodes already ship the driver and toolkit (managed GPU node pools on EKS, GKE and AKS usually do), the nvidia/k8s-device-plugin chart is enough. Some managed offerings install it for you. Check before installing a second copy.

Installing the GPU Operator:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator --create-namespace \
  --version  --wait

If the nodes already have a driver, add --set driver.enabled=false so the operator does not try to install a second one.

Verify that the GPU is schedulable before touching Open WebUI:

kubectl get nodes -o custom-columns='NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu'

A GPU node shows 1 or more in the GPU column. If it shows , fix that first. Nothing in the Ollama chart can work around a node that does not advertise the resource.

The GPU Operator also creates a RuntimeClass named nvidia. On clusters where the NVIDIA runtime is not the containerd default, pods must request it with runtimeClassName: nvidia, otherwise the container starts without access to the GPU even though the scheduler placed it on a GPU node.

Taints, labels and node selection

GPU nodes are expensive, so most teams taint them to keep ordinary workloads off. A common convention is nvidia.com/gpu=present:NoSchedule; managed providers apply their own taints on GPU pools, so check with kubectl describe node . The Ollama pod then needs a matching toleration and, ideally, a node selector. GPU Feature Discovery labels GPU nodes with nvidia.com/gpu.present=true, which makes a convenient selector.

Open WebUI itself does not need a GPU. Keep it on normal nodes. Do not give it the GPU toleration, so it never occupies a slot on a GPU machine.

Deploy Open WebUI on Kubernetes with the Helm chart

Add both repositories. The Ollama chart is moving from https://otwld.github.io/ollama-helm/ to https://helm.otwld.com/, so use the new URL for standalone installs.

helm repo add open-webui https://helm.openwebui.com/
helm repo add otwld https://helm.otwld.com/
helm repo update
helm search repo open-webui/open-webui --versions | head

Pin the chart version with --version and the image tags in your values. The chart's image.tag defaults to empty, which means "use the chart's appVersion". That is predictable as long as you pin the chart, but setting the tag explicitly makes the running version visible in your values file and in code review. Never run latest here: Open WebUI migrates its database on startup, so an unplanned image change is an unplanned schema migration.

A values file for the bundled setup

Values for the subchart go under the ollama: key, and the Ollama chart's own settings sit under another ollama: inside it. That is why you see ollama.ollama.gpu below.

# values-openwebui.yaml
image:
  tag: "v0.12.4"            # pin; match the chart's appVersion or a tested release

replicaCount: 1

persistence:
  enabled: true
  size: 10Gi                 # chats, uploads, vector store for document RAG
  storageClass: ""           # empty = cluster default

pipelines:
  enabled: false

resources:
  requests:
    cpu: 250m
    memory: 1Gi
  limits:
    memory: 2Gi

extraEnvVars:
  - name: WEBUI_SECRET_KEY
    valueFrom:
      secretKeyRef:
        name: open-webui-secrets
        key: webui-secret-key
  - name: WEBUI_URL
    value: "https://chat.example.com"
  - name: DEFAULT_USER_ROLE
    value: "pending"

ollama:
  enabled: true
  fullnameOverride: open-webui-ollama
  image:
    tag: "0.40.11"           # pin; Ollama tags have no "v" prefix
  runtimeClassName: nvidia   # only if the NVIDIA runtime is not the default
  ollama:
    gpu:
      enabled: true
      type: nvidia
      number: 1
    models:
      pull:
        - llama3.1:8b
        - nomic-embed-text
  persistentVolume:
    enabled: true
    size: 100Gi
    storageClass: ""
  extraEnv:
    - name: OLLAMA_KEEP_ALIVE
      value: "30m"
  nodeSelector:
    nvidia.com/gpu.present: "true"
  tolerations:
    - key: nvidia.com/gpu
      operator: Exists
      effect: NoSchedule
  resources:
    requests:
      cpu: "2"
      memory: 16Gi
    limits:
      memory: 32Gi

Create the secret first, then install:

kubectl create namespace open-webui
kubectl -n open-webui create secret generic open-webui-secrets \
  --from-literal=webui-secret-key="$(openssl rand -hex 32)"

helm upgrade --install open-webui open-webui/open-webui \
  --namespace open-webui \
  --version  \
  -f values-openwebui.yaml

With ollama.ollama.gpu.enabled: true the Ollama chart sets the nvidia.com/gpu limit for you (the resource name is configurable via nvidiaResource, which matters if you use MIG or time-slicing with renamed resources). Do not also add it by hand in resources.

Before the ingress exists, check it with a port-forward:

kubectl -n open-webui port-forward svc/open-webui 8080:80

The service listens on port 80 and forwards to the container's port 8080.

Using a separate Ollama release

If you prefer independent lifecycles, set ollama.enabled: false and install Ollama on its own:

helm upgrade --install ollama otwld/ollama \
  --namespace ollama --create-namespace \
  --version  \
  -f values-ollama.yaml

values-ollama.yaml contains the same keys as the ollama: block above, one level up (ollama.gpu, ollama.models, persistentVolume, tolerations). In the Open WebUI values, point at the service:

ollama:
  enabled: false
ollamaUrls:
  - "http://ollama.ollama.svc.cluster.local:11434"

The Ollama chart ships an ingress, but do not expose Ollama publicly. Its API has no authentication, so anyone who can reach it can pull models onto your disks and use your GPUs. Keep it as a ClusterIP service and add a NetworkPolicy that only admits traffic from the Open WebUI namespace.

Preloading models

ollama.models.pull downloads models when the container starts, so a fresh pod comes up with the models already on disk. ollama.models.run additionally loads them into GPU memory at startup, which removes the first-request delay for the model you use most. ollama.models.create builds a derived model from a Modelfile template, which is the cleanest way to bake in a larger context window:

ollama:
  models:
    pull:
      - llama3.1:8b
    create:
      - name: llama3.1-8b-ctx16k
        template: |
          FROM llama3.1:8b
          PARAMETER num_ctx 16384
    run:
      - llama3.1-8b-ctx16k

Things to know:

• Pin model tags. llama3.1:8b is better than llama3.1, which can resolve to a different variant later. For a hard pin, use the digest tag shown by ollama list.
• Pulls happen on every start, but only download what is missing. With persistence on, a restart is quick. Without persistence, every reschedule re-downloads tens of gigabytes.
• ollama.models.clean (default false) deletes models on disk that are not listed in values. Turn it on if you want the values file to be the single source of truth. Leave it off if people pull models through the UI.
• Size the volume from the models, not the default. A quantized model's disk size is roughly parameters times bits per weight divided by eight, so an 8B model at 4-bit is around 4 to 5 GB and a 70B model at 4-bit is around 40 GB. Add up everything in pull, add room for models users pull from the UI, and leave headroom. The VRAM guide has the same arithmetic for GPU memory.

Persistence

There are two volumes and they hold very different data.

| Volume | Chart key | Holds | Default | Loss means | |---|---|---|---|---| | Open WebUI data | persistence.* | SQLite DB, uploads, vector index, generated secret key | on, 2Gi | Lost users, chats and settings | | Ollama models | ollama.persistentVolume.* | Model blobs under /root/.ollama | off, 30Gi | Re-download on next start |

The Open WebUI volume is the one to back up. Use your cluster's snapshot mechanism (VolumeSnapshot, Velero, or the cloud provider's disk snapshots) on a schedule. If you move to PostgreSQL via databaseUrl, back up the database instead, and consider persistence.provider: s3 (or gcs, azure) for uploaded files so the pod no longer depends on a single disk.

The Ollama volume is a cache. Losing it costs download time, not data. Turn it on anyway. Note that the subchart default is off.

Both volumes default to ReadWriteOnce, so the pods cannot move to another node while the volume is attached elsewhere. For Ollama this is normally fine because it is pinned to GPU nodes. Use existingClaim if you want to pre-create the PVC with a specific storage class or reuse a volume across reinstalls. A volume created by the chart can be deleted with the release, depending on its reclaim policy.

Ingress and TLS

Enable the chart's ingress and let cert-manager issue the certificate:

ingress:
  enabled: true
  class: "nginx"                 # or traefik, gce, your controller's class
  host: chat.example.com
  tls: true
  existingSecret: chat-example-com-tls
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"

The nginx.ingress.kubernetes.io/* annotations apply to the ingress-nginx controller only; translate them for other controllers. They matter for two reasons. Open WebUI streams tokens and uses WebSockets, and long generations exceed default proxy timeouts, which shows up as responses that stop mid-sentence. Document uploads for RAG also hit the default body size limit quickly.

On GKE you can use ingress.class: gce with the chart's managedCertificate.* values instead of cert-manager.

Set WEBUI_URL to the public URL. Open WebUI uses it to build OAuth redirect URLs and links, and SSO fails with a redirect mismatch if it is wrong.

Authentication and sign-up settings

Out of the box, the first account created becomes the admin. Create it immediately after the first deploy, before the URL is shared. Do it over the port-forward if you like, before the ingress goes live.

After that, new sign-ups get DEFAULTUSERROLE, which defaults to pending: the account exists but cannot do anything until an admin approves it. To close sign-ups entirely, set ENABLE_SIGNUP=false.

One behavior trips up almost everyone. Many Open WebUI settings are PersistentConfig: the environment variable only sets the initial value, which is then stored in the database and edited through the admin panel. Changing ENABLESIGNUP or DEFAULTUSERROLE in values after the first start appears to do nothing. Either change it in the admin UI, or set ENABLEPERSISTENT_CONFIG=false to make environment variables always win (admin UI edits are then not saved).

WEBUISECRETKEY signs session tokens. If unset, Open WebUI generates one and stores it in the data volume. That works for one replica, but a fresh volume means everyone is logged out, and multiple replicas must share the same key. Provide it from a Secret, as above.

For SSO, the chart has first-class values under sso.*: sso.github, sso.google, sso.microsoft and a generic sso.oidc (Keycloak, Authentik, Entra ID, Okta). A Keycloak example:

sso:
  enabled: true
  enableSignup: true
  oidc:
    enabled: true
    providerName: "Company SSO"
    providerUrl: "https://sso.example.com/realms/main/.well-known/openid-configuration"
    clientId: "open-webui"
    clientExistingSecret: "open-webui-oidc"
    clientExistingSecretKey: "client-secret"
    scopes: "openid email profile"
  enableRoleManagement: true
  roleManagement:
    rolesClaim: "roles"
    adminRoles: "openwebui-admin"
    allowedRoles: "openwebui-user,openwebui-admin"

Leave sso.mergeAccountsByEmail off unless you trust every identity provider to verify email addresses. The chart itself flags it as insecure. If your ingress already sits behind an authenticating proxy (oauth2-proxy and similar), sso.trustedHeader lets Open WebUI take the user from a header, but then the proxy must be the only path to the pod.

Upgrades

A safe upgrade routine:

1. Read the Open WebUI release notes and the chart's CHANGELOG between your version and the target. Chart major versions do break values.
2. Snapshot the Open WebUI data volume or database. Open WebUI runs migrations on startup and there is no supported downgrade path for a migrated database.
3. Render the change first: helm diff upgrade (the helm-diff plugin) or helm template with old and new versions, and read the diff.
4. Bump the chart version and the image tags in your values in the same commit, then run helm upgrade with --version.
5. Find the Open WebUI pod with kubectl -n open-webui get pods and follow its logs with kubectl -n open-webui logs -f until the migrations finish and the app is serving.

Upgrade Ollama separately and test that your models still load. New Ollama versions sometimes change defaults such as context length or scheduling. The otwld chart's 1.x release moved ollama.models from a plain list to the pull/run/create structure, so old values files need updating before upgrading across that boundary.

Troubleshooting

Ollama pod stuck in Pending with "Insufficient nvidia.com/gpu". The node does not advertise GPUs, or all are taken. Check allocatable resources with the command from the prerequisites, and kubectl -n gpu-operator get pods to see whether the device plugin is running on that node. Also check that the toleration matches the node's taint exactly.

Ollama runs on CPU on a GPU node. Look for the GPU discovery lines in kubectl logs of the Ollama pod. If Ollama reports no compatible GPU, the container started without the NVIDIA runtime. Set runtimeClassName: nvidia, or check that gpu.enabled is true so the GPU limit is set. kubectl exec into the pod and run nvidia-smi to confirm.

Pod restarts while pulling models. Large pulls on startup can outlast the liveness probe. Pull fewer models at startup, pre-populate the volume once, or relax the Ollama chart's probe timings.

Model loads but is slow or partly on CPU. The model plus its KV cache does not fit in VRAM, so layers spill to system RAM. Use a smaller quantization or a shorter context, or check that OLLAMAMAXLOADED_MODELS is not keeping several models resident. ollama ps inside the pod shows the GPU/CPU split per model.

UI loads but shows no models. Open WebUI cannot reach Ollama. Check the connection URL under Admin Settings, Connections; with a separate release it must be the full cluster DNS name. Remember that the connection list is PersistentConfig too.

Responses cut off after a minute or uploads fail. Ingress timeouts and body-size limits, see the ingress section.

Users logged out after every redeploy. No persistent WEBUISECRETKEY and the data volume was recreated, or replicas have different keys.

Next steps

• If a single GPU server is enough, start with the Ollama-based chatbot stack on Docker Compose and move to Kubernetes later.
• Compare runtimes before scaling up: Ollama vs vLLM vs llama.cpp covers when a throughput-oriented server like vLLM is worth the extra operations work.
• Add retrieval over your own documents with the RAG pipeline stack, or browse all chatbot stacks.