Guide

Vector databases explained: how to choose one for RAG

How vector search works, how retrieval-augmented generation uses it, and the options for where vectors live, from the engines' own documentation.

  • Built on public docs
  • Checked 30 Oct 2026
The RAG pipeline: how RAG uses a vector database

A vector database stores embeddings, which are lists of numbers that represent the meaning of text or other data, and finds the ones nearest to a query vector. In retrieval-augmented generation (RAG), it fetches the passages a language model reads before answering. Vectors can live in PostgreSQL, OpenSearch or a dedicated service.

What a vector database does

A vector database stores embeddings and returns the nearest ones to a query vector, by a distance measure such as cosine distance. Exact search checks every vector and gives perfect recall; approximate indexes trade some recall for speed (pgvector docs, checked 30 Oct 2026).

Illustrative: similar documents sit close together, so retrieval is a nearest-neighbour search.

An embedding model turns text, images or other data into a vector, so that similar meaning produces nearby vectors. The database stores those vectors with the original record and answers questions of the form: give me the ten items closest to this one. Production systems add filters, such as tenant or date, and often combine the result with keyword search.

The RAG pipeline: how RAG uses a vector database

Retrieval-augmented generation combines a language model with a searchable store of documents. The 2020 paper that named it describes models that merge pre-trained parametric memory with non-parametric memory, a dense vector index queried by a retriever (Lewis et al.).

  1. Chunk

    Split source documents into passages small enough to embed and to fit in a prompt.

  2. Embed

    Turn each passage into a vector with an embedding model.

  3. Index

    Store vectors with their text and metadata, with an index for fast search.

  4. Retrieve

    Embed the question, fetch the nearest passages, apply filters and optionally combine with keyword results.

  5. Generate

    Put the retrieved passages into the model's prompt so the answer rests on them.

RAG differs from fine-tuning: fine-tuning changes the model's weights, while RAG leaves the model unchanged and supplies fresh data at query time. That is why RAG suits content that changes, and why the freshness of the index matters. See embeddings freshness with Kafka and CDC.

Where vectors can live: pgvector, OpenSearch and dedicated services

There are three common homes for vectors: an extension in PostgreSQL, a search engine such as OpenSearch, and a dedicated vector service. Each documents different index types, filtering and operating models.

OptionIndex typesFiltering and hybridOperating model
PostgreSQL with pgvectorExact by default; HNSW and IVFFlat approximate indexes (pgvector)Approximate indexes apply WHERE filters after the scan; hybrid by combining full-text search and vectors with Reciprocal Rank FusionA database you run, or have run for you. See pgvector.
OpenSearchk-NN with Lucene and Faiss engines; HNSW and IVF (OpenSearch docs)A hybrid query combines keyword and vector scores (OpenSearch docs)A search cluster you run, or have run for you. See managed OpenSearch.
Valkey SearchHNSW approximate search and exact KNN (Valkey)In-memory; suited to caching and fast recallIn-memory store. See semantic cache.
Dedicated vector serviceManaged by the serviceVaries by product; check the vendor's documentationRun by the vendor. See pgvector vs Pinecone and the Pinecone alternative.

Sources: the pgvector README, OpenSearch documentation and Valkey documentation linked in the cells. Checked 30 Oct 2026. Documentation changes; confirm on the linked pages.

How the options were compared

This guide compares what each engine's own documentation states, read on 30 Oct 2026. It does not rank products and quotes no benchmark, because recall, latency and cost depend on the data, the index settings and the hardware.

OSStend runs PostgreSQL and OpenSearch, so it has an interest in those two options. The guide names where a dedicated service is the better tool, and recommends testing candidates on your own vectors, filters and recall target.

Vector database vs relational database

A vector database and a relational database are no longer separate categories. PostgreSQL with pgvector stores vectors beside relational rows, so one database can hold the business record, its permissions and its embedding.

That has practical benefits: the vector and the record change in one transaction, backups cover both, and filters and joins work on the same data. A dedicated vector database may still fit when the collection is very large and vector-only, or when no team wants to operate a database. pgvector can index up to 2,000 dimensions for the vector type, with larger limits for halfvec and bit (pgvector, checked 30 Oct 2026), so check your embedding model's dimension count.

Index types, recall and latency

Approximate indexes make search faster by giving up some recall. In pgvector, HNSW has better speed and recall trade-offs than IVFFlat but builds more slowly and uses more memory; IVFFlat builds faster and uses less memory (pgvector, checked 30 Oct 2026).

IndexKey settingsBuild and memoryUse when
Exact (no index)NoneNo buildSmall collections, or when perfect recall is required
HNSWm (default 16), ef_construction (64), query-time hnsw.ef_search (40)Slower build, more memory; can be created on an empty tableThe default choice for most production search
IVFFlatlists, and query-time ivfflat.probes (default 1)Faster build, less memory; create after loading dataLarge collections where build time or memory is tight

Source: pgvector README, checked 30 Oct 2026. Defaults can change between versions.

pgvector: create an HNSW index (from the pgvector README)
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

Hybrid search and filtering

Most production retrieval combines meaning with exact terms. pgvector documents combining Postgres full-text search with vector search and merging the lists with Reciprocal Rank Fusion or a cross-encoder (pgvector); OpenSearch documents a hybrid query with normalisation of scores (OpenSearch docs).

Filtering needs care. In pgvector, approximate indexes apply filters after scanning the index, so a selective filter can return fewer rows than requested; the README lists iterative index scans, partial indexes and partitioning as remedies (checked 30 Oct 2026). Test filter-heavy queries with realistic filters, not just unfiltered ones.

How to choose a vector database for RAG

Choose by where the source data lives, how the retrieval must rank, how large the collection will grow and who will operate it. The table maps common situations to the option that usually fits.

SituationUsually fitsWhy
Source data already in PostgreSQL; vectors must stay in your environmentPostgreSQL with pgvectorOne database, one backup, same access control
Keyword search is already in OpenSearch; you want hybrid rankingKeyword and vector scores in one query
Repeat questions and fast recallIn-memory reads
Very large, vector-only collection and no database operations capacityA dedicated vector serviceNo database to run

A guide, not a rule. Test on your own data. The vector database for RAG page goes deeper on sizing and tuning.

What changes at production scale

At production scale the work moves from the first demo to index freshness, access control, recall monitoring and what happens when an embedding model changes. These are operations questions as much as search questions.

Public documentation covers each engine's settings; it does not cover your data. A pilot with real queries and a recall target is the dependable test. OSStend can run PostgreSQL and OpenSearch for that pilot; see the AI data layer.

Vector database and RAG questions

What is a vector database?

A vector database stores embeddings, which are numeric representations of text, images or other data, and finds the vectors nearest to a query vector. It is used for semantic search, recommendations and retrieval-augmented generation.

Do I need a dedicated vector database for RAG?

Not always. Vectors can live in PostgreSQL with pgvector or in OpenSearch, which suits data that already lives there. A dedicated service fits very large, vector-only collections or teams with no database operations capacity.

What does a RAG architecture look like?

A RAG architecture has an ingestion path and a query path. Ingestion chunks documents, embeds them and indexes the vectors with their text and metadata. The query path embeds the question, retrieves the nearest passages and passes them to the language model with the question.

What is the difference between RAG and fine-tuning?

Fine-tuning changes a model's weights by training it on more data. RAG leaves the model unchanged and retrieves relevant passages at query time, so the answer can rest on current data. The two can be used together.

Is pgvector good enough for production?

pgvector offers exact search and HNSW and IVFFlat indexes, and many workloads fit one PostgreSQL primary with replicas. Whether it fits yours depends on collection size, filters and recall target, so test with your own data.

Talk to an engineer

Talk to an engineer about retrieval for your data

Bring the workload and where the data must live. An engineer will give a straight view on where the vectors should go.

  • A design sketch for your workload, not a plan menu
  • A zero-downtime cutover path for what you run today
  • A price for that design, after the review
See pricing →

Your request goes to the OSStend team. No newsletter sign-up.

Get a quoteTalk to an engineer
2Engines
3Where should it run?
4What do you need?

Your request goes to the OSStend team. No newsletter sign-up.