Vector databases explained: how to choose one for RAG
How vector search works, how retrieval-augmented generation uses it, and the options for where vectors live, from the engines' own documentation.
- ✓Built on public docs
- ✓Checked 30 Oct 2026
A vector database stores embeddings, which are lists of numbers that represent the meaning of text or other data, and finds the ones nearest to a query vector. In retrieval-augmented generation (RAG), it fetches the passages a language model reads before answering. Vectors can live in PostgreSQL, OpenSearch or a dedicated service.
What a vector database does
A vector database stores embeddings and returns the nearest ones to a query vector, by a distance measure such as cosine distance. Exact search checks every vector and gives perfect recall; approximate indexes trade some recall for speed (pgvector docs, checked 30 Oct 2026).
An embedding model turns text, images or other data into a vector, so that similar meaning produces nearby vectors. The database stores those vectors with the original record and answers questions of the form: give me the ten items closest to this one. Production systems add filters, such as tenant or date, and often combine the result with keyword search.
The RAG pipeline: how RAG uses a vector database
Retrieval-augmented generation combines a language model with a searchable store of documents. The 2020 paper that named it describes models that merge pre-trained parametric memory with non-parametric memory, a dense vector index queried by a retriever (Lewis et al.).
Chunk
Split source documents into passages small enough to embed and to fit in a prompt.
Embed
Turn each passage into a vector with an embedding model.
Index
Store vectors with their text and metadata, with an index for fast search.
Retrieve
Embed the question, fetch the nearest passages, apply filters and optionally combine with keyword results.
Generate
Put the retrieved passages into the model's prompt so the answer rests on them.
RAG differs from fine-tuning: fine-tuning changes the model's weights, while RAG leaves the model unchanged and supplies fresh data at query time. That is why RAG suits content that changes, and why the freshness of the index matters. See embeddings freshness with Kafka and CDC.
Where vectors can live: pgvector, OpenSearch and dedicated services
There are three common homes for vectors: an extension in PostgreSQL, a search engine such as OpenSearch, and a dedicated vector service. Each documents different index types, filtering and operating models.
| Option | Index types | Filtering and hybrid | Operating model |
|---|---|---|---|
| PostgreSQL with pgvector | Exact by default; HNSW and IVFFlat approximate indexes (pgvector) | Approximate indexes apply WHERE filters after the scan; hybrid by combining full-text search and vectors with Reciprocal Rank Fusion | A database you run, or have run for you. See pgvector. |
| OpenSearch | k-NN with Lucene and Faiss engines; HNSW and IVF (OpenSearch docs) | A hybrid query combines keyword and vector scores (OpenSearch docs) | A search cluster you run, or have run for you. See managed OpenSearch. |
| Valkey Search | HNSW approximate search and exact KNN (Valkey) | In-memory; suited to caching and fast recall | In-memory store. See semantic cache. |
| Dedicated vector service | Managed by the service | Varies by product; check the vendor's documentation | Run by the vendor. See pgvector vs Pinecone and the Pinecone alternative. |
Sources: the pgvector README, OpenSearch documentation and Valkey documentation linked in the cells. Checked 30 Oct 2026. Documentation changes; confirm on the linked pages.
How the options were compared
This guide compares what each engine's own documentation states, read on 30 Oct 2026. It does not rank products and quotes no benchmark, because recall, latency and cost depend on the data, the index settings and the hardware.
OSStend runs PostgreSQL and OpenSearch, so it has an interest in those two options. The guide names where a dedicated service is the better tool, and recommends testing candidates on your own vectors, filters and recall target.
Vector database vs relational database
A vector database and a relational database are no longer separate categories. PostgreSQL with pgvector stores vectors beside relational rows, so one database can hold the business record, its permissions and its embedding.
That has practical benefits: the vector and the record change in one transaction, backups cover both, and filters and joins work on the same data. A dedicated vector database may still fit when the collection is very large and vector-only, or when no team wants to operate a database. pgvector can index up to 2,000 dimensions for the vector type, with larger limits for halfvec and bit (pgvector, checked 30 Oct 2026), so check your embedding model's dimension count.
Index types, recall and latency
Approximate indexes make search faster by giving up some recall. In pgvector, HNSW has better speed and recall trade-offs than IVFFlat but builds more slowly and uses more memory; IVFFlat builds faster and uses less memory (pgvector, checked 30 Oct 2026).
| Index | Key settings | Build and memory | Use when |
|---|---|---|---|
| Exact (no index) | None | No build | Small collections, or when perfect recall is required |
| HNSW | m (default 16), ef_construction (64), query-time hnsw.ef_search (40) | Slower build, more memory; can be created on an empty table | The default choice for most production search |
| IVFFlat | lists, and query-time ivfflat.probes (default 1) | Faster build, less memory; create after loading data | Large collections where build time or memory is tight |
Source: pgvector README, checked 30 Oct 2026. Defaults can change between versions.
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);
Hybrid search and filtering
Most production retrieval combines meaning with exact terms. pgvector documents combining Postgres full-text search with vector search and merging the lists with Reciprocal Rank Fusion or a cross-encoder (pgvector); OpenSearch documents a hybrid query with normalisation of scores (OpenSearch docs).
Filtering needs care. In pgvector, approximate indexes apply filters after scanning the index, so a selective filter can return fewer rows than requested; the README lists iterative index scans, partial indexes and partitioning as remedies (checked 30 Oct 2026). Test filter-heavy queries with realistic filters, not just unfiltered ones.
How to choose a vector database for RAG
Choose by where the source data lives, how the retrieval must rank, how large the collection will grow and who will operate it. The table maps common situations to the option that usually fits.
| Situation | Usually fits | Why |
|---|---|---|
| Source data already in PostgreSQL; vectors must stay in your environment | PostgreSQL with pgvector | One database, one backup, same access control |
| Keyword search is already in OpenSearch; you want hybrid ranking | OpenSearch | Keyword and vector scores in one query |
| Repeat questions and fast recall | Valkey or Redis | In-memory reads |
| Very large, vector-only collection and no database operations capacity | A dedicated vector service | No database to run |
A guide, not a rule. Test on your own data. The vector database for RAG page goes deeper on sizing and tuning.
What changes at production scale
At production scale the work moves from the first demo to index freshness, access control, recall monitoring and what happens when an embedding model changes. These are operations questions as much as search questions.
Public documentation covers each engine's settings; it does not cover your data. A pilot with real queries and a recall target is the dependable test. OSStend can run PostgreSQL and OpenSearch for that pilot; see the AI data layer.
Vector database and RAG questions
What is a vector database?
A vector database stores embeddings, which are numeric representations of text, images or other data, and finds the vectors nearest to a query vector. It is used for semantic search, recommendations and retrieval-augmented generation.
Do I need a dedicated vector database for RAG?
Not always. Vectors can live in PostgreSQL with pgvector or in OpenSearch, which suits data that already lives there. A dedicated service fits very large, vector-only collections or teams with no database operations capacity.
What does a RAG architecture look like?
A RAG architecture has an ingestion path and a query path. Ingestion chunks documents, embeds them and indexes the vectors with their text and metadata. The query path embeds the question, retrieves the nearest passages and passes them to the language model with the question.
What is the difference between RAG and fine-tuning?
Fine-tuning changes a model's weights by training it on more data. RAG leaves the model unchanged and retrieves relevant passages at query time, so the answer can rest on current data. The two can be used together.
Is pgvector good enough for production?
pgvector offers exact search and HNSW and IVFFlat indexes, and many workloads fit one PostgreSQL primary with replicas. Whether it fits yours depends on collection size, filters and recall target, so test with your own data.
Related pages
- AI data · Fresh contextEmbeddings freshness with Kafka and Postgres CDCPostgres CDC with Debezium and Kafka Connect keeps embeddings current: re-embed on change, handle deletes, and stop agents answering from stale data.
- AI data · PostgreSQLpgvector in production: indexes and sizingpgvector in production: HNSW against IVFFlat, memory sizing, recall against latency, filtering, replicas and pgvectorscale, with links to the pgvector docs.
- OpenSearchElasticsearch as a service on managed OpenSearchElasticsearch as a service on OpenSearch: dedicated master and data nodes, hybrid keyword and vector search and snapshots, sized for your index and run 24/7.
- AI data · Fast recallSemantic cache for LLM apps with Redis or ValkeySemantic cache with Redis or Valkey: how a similarity-matched cache cuts repeat model calls, how to set thresholds and eviction, and the cost maths.
- Comparisonpgvector vs Pinecone for enterprise workloadspgvector vs Pinecone for enterprise workloads: where each wins on operations, data location, scale, hybrid search and cost model. Sourced, 30 Oct 2026.
- AlternativePinecone pricing and alternatives in 2026Pinecone pricing and alternatives: vector search with pgvector inside PostgreSQL you already run, tailored to the workload, in your cloud or data center.
Talk to an engineer about retrieval for your data
Bring the workload and where the data must live. An engineer will give a straight view on where the vectors should go.
- ✓A design sketch for your workload, not a plan menu
- ✓A zero-downtime cutover path for what you run today
- ✓A price for that design, after the review