Self-hosted AI guides

Practical guides to running AI on your own infrastructure: self-hosted RAG, LLM inference servers, GPU sizing, vector databases, embeddings, agents and GDPR.

• How to Self-Host a RAG Pipeline with Docker Compose – A complete, working self-hosted RAG stack in one docker-compose.yml: Ollama for chat and embeddings, Qdrant for vectors, and a small Python service for ingestion and queries.
• Ollama vs vLLM vs llama.cpp: Choosing a Self-Hosted Inference Server – How the three most common self-hosted LLM inference servers differ in architecture, hardware support, concurrency and operations, and which one fits your workload.
• How Much VRAM Do You Need for an LLM? Formulas and Tables – Calculate GPU memory for LLM inference from the model's parameter count and config: weights, KV cache and overhead, with estimate tables for 8B, 14B, 32B and 70B models.
• pgvector vs Qdrant vs Weaviate vs Milvus vs Chroma for RAG – A practical comparison of the five vector databases people actually self-host for RAG, with a decision guide by scenario instead of benchmark charts.
• How to Choose the Best Embedding Model for Self-Hosted RAG – A practical way to choose a self-hosted embedding model: what the leaderboard can and cannot tell you, what each dimension costs in storage, and how to serve, swap and rerank.
• How to Evaluate a RAG Pipeline: Metrics, Test Sets and CI – Measure retrieval and generation separately, build a small golden test set, use a self-hosted LLM judge carefully, and run RAG evaluations on every change.
• Deploy Open WebUI and Ollama on Kubernetes with Helm – A production-shaped Helm setup for Open WebUI and Ollama on Kubernetes, from GPU node preparation to TLS, sign-up control, upgrades and the failures you will most likely hit.
• Terraform GPU Instance on AWS for LLM Inference (EC2 + vLLM) – A minimal, pinned Terraform config for one GPU EC2 instance that serves an LLM with vLLM or Ollama, reachable through SSM instead of open ports, plus the quota and cost traps to avoid.
• Self-Hosted AI Agents: Local LLM Tool Calling That Works – How tool calling works on Ollama and vLLM, which open-weight models handle it, and how to wrap it in an agent loop that is safe to point at real systems.
• Self-Hosting LLMs for GDPR: A Practical Checklist – Running the model on your own servers removes one big data flow, but you are still the controller for every prompt, log and embedding. This checklist covers the rest.