Self-hosted MLOps stacks
The infrastructure around the model: serving, monitoring, GPUs and reproducible deployments.
MLOps stacks
No stacks in this category yet.
About MLOps stacks
MLOps stacks cover what it takes to run models reliably: provisioning GPU machines, serving models behind stable APIs, tracking versions and watching latency, errors and cost. Infrastructure as code (Terraform, Helm) makes these setups reproducible across environments.
A minimal production baseline:
- Infrastructure defined in code, so environments can be rebuilt
- Pinned model and image versions
- Metrics and logs for every model endpoint
- A plan for GPU capacity, quotas and cost control
Frequently asked questions
- What does MLOps mean for self-hosted AI?
- It is the practice of deploying, monitoring and maintaining models in production: infrastructure as code, versioned models, observability and controlled rollouts on infrastructure you run yourself.
- Should I use Kubernetes for LLM serving?
- Kubernetes helps once you run several models or need autoscaling and GPU scheduling. For a single model on one GPU machine, Docker Compose is simpler.
- How do I control GPU costs?
- Right-size the model and quantization to the GPU, stop or scale down idle instances, consider spot capacity for interruptible workloads, and tear down test environments with Terraform when you are done.
Guides
- How Much VRAM Do You Need for an LLM? Formulas and Tables – Calculate GPU memory for LLM inference from the model's parameter count and config: weights, KV cache and overhead, with estimate tables for 8B, 14B, 32B and 70B models.
- Terraform GPU Instance on AWS for LLM Inference (EC2 + vLLM) – A minimal, pinned Terraform config for one GPU EC2 instance that serves an LLM with vLLM or Ollama, reachable through SSM instead of open ports, plus the quota and cost traps to avoid.