LLM Chatbot

Conversational AI chatbot with memory and streaming

Category
chatbot
Version
2.1.0
Deployment formats
Docker Compose, Helm (Kubernetes), Terraform (local Docker), Terraform (AWS)
Maintainer
AI Solutions Camp
Tags
Chatbot, LLM, OpenAI, Streaming, WebSocket, React

Source code

Overview

A production-ready conversational AI chatbot with memory, context management, and multi-model support. Includes a React frontend and FastAPI backend with WebSocket streaming.

Requirements

  • Docker and Docker Compose installed
  • 8GB RAM minimum (16GB for larger models)
  • GPU optional but recommended for speed
  • First model download: 2-5GB depending on model

Quick start

Quick Start (Docker)

  1. Start the services:
   docker-compose up -d
  1. Download a model (required, first time):
   docker exec -it ollama ollama pull llama3.2

Or for a smaller/faster model:

   docker exec -it ollama ollama pull phi3
  1. Open the chat interface:
  • Go to http://localhost:3000
  • Create an account (stored locally)
  • Start chatting!

Available Models:

  • llama3.2 - Meta Llama 3.2 (4.7GB)
  • phi3 - Microsoft Phi-3 (2.2GB)
  • mistral - Mistral 7B (4.1GB)

Environment variables

  • OLLAMA_BASE_URL – URL to Ollama API
  • WEBUI_SECRET_KEY (required) – Secret key for session encryption

docker-compose.yml

version: "3.8"

services:
  # Local LLM Server
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped
    # GPU support (optional, uncomment for NVIDIA):
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: 1
    #           capabilities: [gpu]

  # Chat Web Interface
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - openwebui_data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=your-secret-key-change-me
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  openwebui_data:

All AI stacks