pgvector vs Qdrant vs Weaviate vs Milvus vs Chroma for RAG

Updated 2026-10-26 · 12 min read

A practical comparison of the five vector databases people actually self-host for RAG, with a decision guide by scenario instead of benchmark charts.

The short answer to pgvector vs Qdrant, and to the wider question of the best vector database for RAG when you self-host: if your application already runs on PostgreSQL and you have fewer than roughly tens of millions of vectors, start with pgvector. If vector search is the core workload and you need heavy metadata filtering, hybrid search and horizontal scaling, use Qdrant. Pick Milvus when you are planning for hundreds of millions to billions of vectors, Weaviate when you want built-in multi-tenancy and hybrid search with a schema-driven API, and Chroma for prototypes and notebooks.

The rest of this guide explains where those recommendations come from, compares the five databases feature by feature, and ends with a decision guide by scenario. There are no benchmark numbers here on purpose: published vector database benchmarks are usually run by a vendor, on one dataset, with one set of index parameters, and they rarely predict how your data and filters will behave. Run your own on a sample of your corpus.

What actually matters in a RAG vector store

For RAG the vector database does four jobs: store embeddings next to the chunk text and metadata, return the top-k nearest chunks quickly, apply filters (tenant, document type, date, permissions) without wrecking recall, and stay up. Raw query speed on an unfiltered benchmark is the least important of these. The questions that decide the choice are:

• Filtering. RAG queries almost always carry a filter, at least a tenant or permission scope. How the engine combines filters with approximate search determines whether you get 10 results or 2.
• Hybrid search. Pure dense retrieval misses exact terms like product codes, error messages and names. You want keyword (BM25) or sparse vectors fused with dense results.
• Operations. Backups, upgrades, monitoring and how many containers you need to run.
• Scale path. What happens when the corpus grows 10x.

Feature comparison at a glance

Versions checked in October 2026: pgvector 0.8.7, Qdrant 1.20, Weaviate 1.40, Milvus 3.0, Chroma 1.5.

| | pgvector | Qdrant | Weaviate | Milvus | Chroma | |---|---|---|---|---|---| | License | PostgreSQL License | Apache 2.0 | BSD-3-Clause core, some enterprise-licensed features | Apache 2.0 | Apache 2.0 | | Written in | C (Postgres extension) | Rust | Go | Go and C++ | Rust core, Python and JS clients | | ANN indexes | HNSW, IVFFlat | HNSW (plus quantization, on-disk storage) | HNSW, flat, dynamic, HFresh | HNSW, IVF family, DiskANN, SCANN, GPU indexes | HNSW (single node) | | Filtering | Full SQL, iterative index scans | Payload filters applied during graph search | Pre-filtering with allow lists | Scalar filters, partition keys | Metadata and document filters | | Hybrid / sparse | Postgres full-text plus sparsevec, fused in SQL | Sparse vectors, Query API with RRF/DBSF fusion | Built-in BM25 + vector hybrid | Sparse vectors, built-in BM25 function, multi-vector hybrid search | Documented for Chroma Cloud, check self-hosted support | | Multi-tenancy | Row-level security, partitions, schemas | Tenant payload index, custom shard keys | Native per-tenant shards | Databases, collections, partitions, partition key | Tenants and databases | | Backups | pg_dump, pgBackRest, PITR | Snapshot API | Backup modules (filesystem, S3, GCS, Azure) | milvus-backup tool | Copy the data directory | | Horizontal scaling | Read replicas; sharding via Citus | Built-in sharding and replication | Built-in sharding and replication | Separate, independently scaled components | Distributed mode exists, aimed at Kubernetes | | Minimum footprint | One Postgres container | One container | One container | Three containers (Milvus, etcd, MinIO) | One container |

A few rows need more nuance, so here they are database by database.

pgvector: vectors inside Postgres

pgvector adds vector, halfvec (16-bit floats), bit and sparsevec column types and two index types to PostgreSQL. HNSW is the one to use for RAG: better recall/latency trade-off than IVFFlat and no training step, at the cost of slower builds and more memory. IVFFlat builds faster and uses less memory but needs data present before you create the index and recall degrades as data drifts.

Documented index limits matter when you pick an embedding model: HNSW and IVFFlat index vector up to 2,000 dimensions and halfvec up to 4,000. A 4,096-dimension model therefore needs halfvec, binary quantization or a truncated (Matryoshka) embedding to be indexed.

Filtering used to be pgvector's weak point. Approximate indexes return a candidate set first and the WHERE clause is applied afterwards, so with the default hnsw.ef_search of 40 and a filter that matches 10% of rows you get about 4 results. Since 0.8.0, iterative index scans fix most of this:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id         bigserial PRIMARY KEY,
  tenant_id  bigint NOT NULL,
  doc_id     bigint NOT NULL,
  content    text NOT NULL,
  embedding  halfvec(1024) NOT NULL
);

CREATE INDEX ON chunks USING hnsw (embedding halfvec_cosine_ops);
CREATE INDEX ON chunks (tenant_id);

-- per session or per transaction
SET hnsw.ef_search = 100;
SET hnsw.iterative_scan = relaxed_order;

SELECT id, content
FROM chunks
WHERE tenant_id = 42
ORDER BY embedding <=> $1
LIMIT 10;

For very selective filters (one small tenant in a large table), a plain B-tree index on tenant_id plus an exact scan of that tenant's rows can beat the ANN index. Postgres lets you choose, or you can partition by tenant.

Hybrid search in pgvector means combining Postgres full-text search (tsvector with ts_rank, which is not true BM25) or a sparsevec column with the dense query, then fusing the two ranked lists with reciprocal rank fusion in SQL. It works, but you write it yourself.

What you get for free is everything Postgres already does: transactions, joins with your application tables, row-level security, point-in-time recovery, mature backup tooling, and managed offerings everywhere. Scaling is mostly vertical plus read replicas. Index builds on large tables need generous maintenanceworkmem and benefit from maxparallelmaintenance_workers. If you outgrow plain HNSW, Timescale's pgvectorscale extension (PostgreSQL License) adds a disk-friendly StreamingDiskANN index.

Docker image: pgvector/pgvector:0.8.7-pg17 (tags exist for Postgres 13 through 18).

Qdrant: purpose-built and operationally simple

Qdrant is a single Rust binary with a REST API on port 6333 and gRPC on 6334. Its main strength for RAG is filtering: payload indexes are taken into account during the HNSW graph traversal, so filtered queries keep their recall without the post-filter problem. Mark the tenant field with is_tenant: true when you create its payload index and Qdrant organizes storage around it, which is the recommended way to host many small tenants in one collection. For a few large tenants, custom shard keys put each one on its own shards.

Hybrid search is native: a point can carry named dense and sparse vectors, and the Query API runs several prefetch queries and fuses them with RRF or distribution-based score fusion (DBSF). You compute the sparse vectors (BM25 weights, SPLADE) on your side or with Qdrant's FastEmbed library.

Memory control is good: scalar, product and binary quantization, keeping original vectors on disk while quantized ones stay in RAM, and on-disk HNSW. Its capacity-planning docs size dense vectors as points x replication factor x dimensions x bytes per dimension, then add about 20% headroom. Distributed mode (sharding plus replication, coordinated with Raft) is part of the open-source build. Backups use the snapshot API per collection or per node, and snapshots can be restored on another cluster.

Docker image: qdrant/qdrant:v1.20.0. Data lives in /qdrant/storage, snapshots in /qdrant/snapshots. Set an API key with QDRANTSERVICEAPI_KEY before exposing it anywhere.

Weaviate: schema, modules and native multi-tenancy

Weaviate is a Go database with a schema-first collection model, an HTTP API on port 8080 and gRPC on 50051. It offers four vector index types: HNSW (the default and general recommendation), flat (brute force, good for many small tenants), dynamic (starts flat and switches to HNSW past a threshold, still marked experimental) and HFresh, a cluster-based index that trades some speed for memory and requires rotational quantization. PQ, BQ, SQ and RQ compression are available.

Two features stand out for RAG. First, hybrid search is built in: BM25 and vector search run in one query with an alpha weight and a fusion method, no extra pipeline. Second, multi-tenancy is native: each tenant gets its own shard, tenants can be deactivated or offloaded to cold storage, and a query is scoped to a tenant by design, which is a clean fit for SaaS products that must isolate customers. Weaviate can also call embedding models itself through vectorizer modules, though for a self-hosted stack you usually keep embedding in your own service and set DEFAULTVECTORIZERMODULE to none.

Backups go through modules (backup-filesystem, backup-s3, backup-gcs, backup-azure). Sharding and replication are built in.

License check: the Weaviate repository is now licensed in two parts. Code outside the wl directory is BSD-3-Clause. Code inside it (at the time of writing: backup deduplication, database users, namespaces and self-recovery) is under a separate enterprise license and is enabled with a license key. The core database you would self-host for RAG is BSD, but read the LICENSE file of the exact version you deploy if your legal team cares.

Docker image: cr.weaviate.io/semitechnologies/weaviate:1.40.2 (also on Docker Hub as semitechnologies/weaviate).

Milvus: built for very large collections

Milvus separates storage and compute. Even the standalone deployment runs three containers: Milvus itself, etcd for metadata and MinIO (S3-compatible object storage) for segments. In cluster mode the query, data, streaming and coordinator components scale independently, and Milvus 3.0 can run its Woodpecker write-ahead log as a separate service. That architecture is overhead at a million vectors and a real advantage at a billion.

Milvus has the widest index menu of the five: FLAT, the IVF family (IVFFLAT, IVFSQ8, IVF_PQ and newer quantized variants), HNSW, DiskANN for collections that do not fit in RAM, SCANN, and GPU indexes such as GPU_CAGRA. Hybrid search can combine several vector fields per entity, for example dense plus sparse, with RRF or weighted rankers, and a built-in BM25 function turns a text field into sparse vectors inside Milvus. Milvus 3.0 also rebuilt the sparse index and added online schema changes (add, backfill and drop fields while serving), which helps when you migrate to a new embedding model.

Multi-tenancy can be modeled at four levels: database, collection, partition or partition key. Partition key is the usual choice for many tenants. Backups use the separate milvus-backup tool, and object storage can be your existing S3. For prototypes, Milvus Lite runs embedded in the Python client (pymilvus) with the same API.

Docker image: milvusdb/milvus:v3.0.4. Use the official milvus-standalone-docker-compose.yaml attached to each GitHub release rather than writing your own; it wires etcd and MinIO correctly. Client port is 19530, health and metrics on 9091. The 2.6 line still receives patch releases if you are not ready for 3.0.

Chroma: the fastest way to get started

Chroma is the default vector store in many tutorials because it runs in-process in Python with no server, or as a single container. Since 1.0 the core is written in Rust. It supports metadata filters (where) and document filters (where_document with $contains and $regex), and organizes data into tenants, databases and collections.

The limits show up in operations. A single-node Chroma server has no replication and no backup API; you back up by copying the persistent directory, ideally while the server is stopped. Chroma's distributed mode is open source but aimed at Kubernetes, and the hybrid Search API with sparse vectors is documented under Chroma Cloud, so verify it works in your self-hosted version before you design around it.

Docker image: chromadb/chroma:1.5.9, port 8000, data in /data. Avoid the .dev tags, which are pre-release builds.

A pinned Docker Compose for local testing

This file starts pgvector and Qdrant side by side so you can load the same chunks into both and compare recall on your own queries. Pin image tags: latest changes under you, and a vector database upgrade can change on-disk formats, so you want upgrades to be a deliberate change you can roll back.

services:
  postgres:
    image: pgvector/pgvector:0.8.7-pg17
    environment:
      POSTGRES_USER: rag
      POSTGRES_PASSWORD: change-me
      POSTGRES_DB: rag
    ports:
      - "127.0.0.1:5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U rag -d rag"]
      interval: 10s
      retries: 5

  qdrant:
    image: qdrant/qdrant:v1.20.0
    environment:
      QDRANT__SERVICE__API_KEY: change-me
    ports:
      - "127.0.0.1:6333:6333"
      - "127.0.0.1:6334:6334"
    volumes:
      - qdrant_storage:/qdrant/storage
      - qdrant_snapshots:/qdrant/snapshots

volumes:
  pgdata:
  qdrant_storage:
  qdrant_snapshots:

Ports are bound to 127.0.0.1 so neither database is reachable from the network while you experiment. For Weaviate and Chroma a single docker run with a volume is enough; for Milvus use the release compose file mentioned above.

Estimating memory before you choose

Memory is what decides when a single node stops being enough, so estimate it up front. Raw vector size is vectors x dimensions x bytes per dimension:

• 1 million chunks, 1,024 dimensions, float32 (4 bytes): 1,000,000 x 1,024 x 4 = about 4.1 GB.
• The same in halfvec / float16: about 2.0 GB.
• 50 million chunks at 1,024 float32 dimensions: about 205 GB before any index overhead.

HNSW adds graph links on top: each vector keeps up to 2 x M neighbor references on the bottom layer (M is 16 by default in pgvector and Qdrant), which is small next to a 4 KB vector but not zero. Multiply by the replication factor if you replicate, add headroom for index builds and the database's normal working memory, and treat the result as a floor rather than a precise figure. When the result no longer fits comfortably in one machine's RAM, your options are quantization, a disk-based index (DiskANN in Milvus, StreamingDiskANN via pgvectorscale, on-disk storage in Qdrant), smaller embeddings, or sharding. The embedding model you choose drives this number directly; see choosing an embedding model for self-hosted RAG.

Decision guide: the best vector database for RAG by scenario

You already run Postgres. Use pgvector. One database to back up, monitor and secure, and you can join chunks with users, documents and permissions in one query. Turn on iterative scans, store embeddings as halfvec, and move vectors to a dedicated database only when you have a concrete reason: index builds affect your transactional workload, memory no longer fits, or you need hybrid features you do not want to build in SQL.

Vector search is the product, up to a few hundred million vectors. Qdrant. It is one binary, filtering keeps recall under realistic tenant and permission filters, hybrid search and quantization are built in, and distributed mode is there when you need it.

SaaS with strict per-customer isolation. Weaviate's native multi-tenancy maps directly onto that requirement. Qdrant with a tenant payload index or shard keys, and pgvector with row-level security or per-tenant partitions, also work; Weaviate just makes it the default model.

Hundreds of millions to billions of vectors. Milvus. Disk-based indexes, GPU indexes, object storage and independently scaled components are designed for this range. Budget for the operational cost: etcd, object storage and several services to monitor.

Prototype, demo or notebook. Chroma, or Milvus Lite if you expect to end up on Milvus. Plan the migration path early. Keep the raw chunks and metadata in a format you can reload, so moving to another store means re-indexing rather than re-chunking.

Qdrant vs Weaviate vs Milvus, if you must pick one of the three. Qdrant for the simplest operations and strong filtering, Weaviate for built-in BM25 hybrid search and native multi-tenancy, Milvus for the largest scale and the widest choice of index types.

Whatever you choose, test it with your data: load a realistic sample, run your real queries with your real filters, and measure recall@k against exact search and p95 latency. The RAG evaluation guide shows how to set that up.

Next steps

• Start from a working stack: the RAG pipeline stack runs ChromaDB, an embedding service and an LLM with Docker Compose, and you can swap the vector store once you know your requirements.
• Run embeddings yourself with the embedding service stack, or browse all RAG stacks.
• Follow the full setup in self-hosting a RAG pipeline with Docker Compose.