Vector database for RAG on your own infrastructure
Retrieval for RAG on PostgreSQL with pgvector or OpenSearch, run in OSStend's cloud, your cloud account or your data center.
45 minutes with an engineer: we review your workload and sketch the design.
A vector database for RAG stores document embeddings and returns the passages closest to a question, so a language model can answer from your data. PostgreSQL with pgvector suits data that sits beside relational tables, and OpenSearch suits combined keyword and vector search. Dedicated vector engines suit very large, vector-only workloads.
- PostgreSQL
- MySQL
- MongoDB
- Redis
- Valkey
- OpenSearch
- Kafka
- RabbitMQ
- AWS
- Google Cloud
- Azure
- Kubernetes
- OpenShift
- Prometheus
- Grafana
- Datadog
- Dynatrace
- Splunk
Why retrieval gets harder in production
A RAG demo retrieves from a handful of documents. A production system retrieves from a changing corpus under access rules, and a wrong or stale passage becomes a wrong answer.
- Stale content.
- The corpus changes, but vectors do not update on their own. Answers come from the old version until the embedding is recomputed.
- Wrong passages, not no passages.
- Approximate search trades recall for speed, so a relevant chunk can be missed without any error.
- Exact terms matter.
- Part numbers, error codes and names are poorly served by vector similarity alone, which is why keyword plus vector search exists.
- Access rules.
- Retrieval has to respect who is allowed to see which document, so filters must work with the index.
- Model changes.
- Vectors from different embedding models are not comparable, so changing the model means re-embedding the corpus.
- Data location.
- The corpus is often internal or customer data, and sending it to a separate hosted service raises review and residency questions.
The RAG retrieval pattern
RAG splits documents into chunks, turns each chunk into a vector, stores them, and at question time retrieves the nearest chunks to pass to the model. The retrieval store is one component, and it needs a pipeline that keeps it current.
Ingest and chunk
Split source documents into passages sized for the embedding model and the context window.
Embed
Convert each passage into a vector with an embedding model, and store the text, vector and metadata together.
Retrieve
Embed the question, run a nearest-neighbour search with any access filters, and optionally combine it with keyword search.
Rerank and generate
Optionally rerank the candidates, then pass the best passages to the model with the question.
Refresh
When a source document changes, re-embed it. A change stream into Kafka is a common way to trigger this; see keeping embeddings fresh with Kafka and CDC.
| Option | Search style | Fits best when | Documentation |
|---|---|---|---|
| PostgreSQL with pgvector | Exact or approximate vector search; keyword via Postgres full-text search | The corpus and its metadata already live in Postgres, and you want one database, transactions and joins | pgvector |
| OpenSearch | Keyword (BM25) and vector search combined in a hybrid query | Exact terms and semantic similarity both matter, or OpenSearch already holds the corpus | OpenSearch hybrid query |
| Dedicated vector engine | Vector search, often with sparse-dense hybrid | Vector-only workloads at a scale that justifies a separate system, or a team that wants no database operations | See pgvector vs Pinecone and the Pinecone alternative |
Sources: each engine's own documentation, checked 30 Oct 2026.
When PostgreSQL with pgvector is enough
pgvector is enough when the corpus fits a single well-sized Postgres server plus read replicas, and when the retrieval metadata and access rules already live in the same database. It stops being enough when the vector count outgrows what one instance and its replicas can hold in memory.
pgvector supports exact and approximate search, HNSW and IVFFlat indexes, and half-precision and binary storage to shrink the working set. It scales vertically and through replicas, and the pgvector docs name tools such as Citus for sharding beyond that. See the full index and sizing detail on the pgvector page. For how retrieval fits with memory and the rest of an agent's context, see context engineering.
When OpenSearch hybrid search fits
OpenSearch fits when keyword relevance matters as much as semantic similarity. Its hybrid query runs several sub-queries and combines their scores into one, with a search pipeline that normalises the scores first.
OpenSearch's documentation describes a hybrid query of up to five sub-queries, a normalization processor that puts BM25 and vector scores on the same scale, and reciprocal rank fusion as another way to merge ranked lists. Its k-NN engines include Lucene and Faiss, with HNSW and, in Faiss, IVF methods (OpenSearch docs, checked 30 Oct 2026). See managed OpenSearch.
Sizing and tuning retrieval
Retrieval cost is driven by chunk count, embedding dimensions, index type and recall target. The same four decisions apply on every engine, even though the settings are named differently.
| Driver | Effect | PostgreSQL with pgvector | OpenSearch |
|---|---|---|---|
| Chunk count × dimensions | Memory and storage | 4 × dimensions + 8 bytes per `vector`; `halfvec` halves the per-dimension size | Vector memory grows with count and dimensions; quantization options reduce it |
| Index type | Speed, recall, build time | HNSW or IVFFlat | HNSW or IVF methods, depending on engine |
| Recall against latency | Answer quality and response time | `hnsw.ef_search`, `ivfflat.probes` | `ef_search` and related method parameters |
| Filters | Result counts and access control | Filter applies after approximate scan; iterative scans, partial indexes, partitioning | Filters on hybrid queries; the docs cover pre-filtering and post-filtering |
| Keyword relevance | Exact-term recall | Postgres full-text search combined with vector results | Native BM25 in the same hybrid query |
Sources: pgvector README and OpenSearch documentation, checked 30 Oct 2026.
Pitfalls documented by the engines
The engines' own documentation covers the behaviour that most often surprises teams moving RAG to production.
- Filtered queries return fewer rows.
- In pgvector, approximate indexes filter after the scan. Iterative index scans exist for this case.
- Score scales differ.
- In OpenSearch, BM25 and vector scores use different scales, so they are normalised before they are combined.
- Tenants share an index.
- In pgvector, vectors from one tenant can affect recall for another when an approximate index is shared.
- Re-embedding after a model change.
- Plan for a full rebuild, and keep the old index until the new one is verified.
What OSStend runs, where it runs and who it is not for
OSStend designs, moves and runs PostgreSQL and OpenSearch in its own cloud, in your cloud account or co-managed in your data center, so the corpus never leaves your environment.
Contact us if
- ✓The corpus is internal or customer data and must stay in your own environment.
- ✓You want to choose between pgvector and OpenSearch on measured results rather than on a vendor's default.
- ✓You want the retrieval layer run by the same team as your other databases.
Not a fit if
- ✕You want a fully hosted vector API with no infrastructure decisions.
- ✕Your workload is vector-only at a scale that a dedicated vector service is built for.
- ✕You need a managed embedding or generation service. OSStend runs the data layer, not the models.
FAQ
What is the best vector database for RAG?
No single engine is best for every RAG system. PostgreSQL with pgvector suits corpora that live beside relational data, OpenSearch suits combined keyword and vector search, and dedicated vector engines suit large vector-only workloads. The right choice depends on corpus size, filters, hybrid needs and where the data may run.
Can PostgreSQL be a vector database for RAG?
Yes. With the pgvector extension, PostgreSQL stores embeddings and runs exact or approximate nearest-neighbour search, with HNSW and IVFFlat indexes. It adds transactions, joins and point-in-time recovery, and scales vertically and through replicas.
Is OpenSearch a vector database?
OpenSearch includes a k-NN vector field type with HNSW and IVF methods across its Lucene and Faiss engines. Its distinguishing feature for RAG is the hybrid query, which combines keyword and vector scores in one search.
What is a RAG database?
A RAG database is the store that a retrieval-augmented generation system queries for context. It holds document chunks with their embeddings and metadata, and returns the passages nearest to a question. PostgreSQL with pgvector, OpenSearch and dedicated vector engines are all used as one.
Which open-source vector database should run on my own infrastructure?
pgvector and OpenSearch are both open source and run in your own cloud account or data center. Dedicated open-source engines such as Qdrant and Milvus also self-host; the best open source vector databases guide compares six. Choose by data model, scale and operations skill.
How do embeddings stay up to date in RAG?
A pipeline re-embeds a record when its source changes. A common design reads the source database's change stream, publishes it to Kafka and runs an embedding job on each change, so the vector store never lags far behind the source. The Kafka and CDC guide covers the pipeline in detail.
Related pages
- KafkaManaged Kafka in your cloud or data centerManaged Kafka with the broker count, partition plan, KRaft, Connect and schema registry designed for your estate, run 24/7 in your cloud or your data center.
- AI data · Fresh contextEmbeddings freshness with Kafka and Postgres CDCPostgres CDC with Debezium and Kafka Connect keeps embeddings current: re-embed on change, handle deletes, and stop agents answering from stale data.
- Comparisonpgvector vs Pinecone for enterprise workloadspgvector vs Pinecone for enterprise workloads: where each wins on operations, data location, scale, hybrid search and cost model. Sourced, 30 Oct 2026.
- AlternativePinecone pricing and alternatives in 2026Pinecone pricing and alternatives: vector search with pgvector inside PostgreSQL you already run, tailored to the workload, in your cloud or data center.
- AI data · PostgreSQLpgvector in production: indexes and sizingpgvector in production: HNSW against IVFFlat, memory sizing, recall against latency, filtering, replicas and pgvectorscale, with links to the pgvector docs.
- GuideContext engineering for AI agents: the data layerContext engineering for AI agents: what it is, how it differs from prompt engineering, and which data stores hold memory, retrieval and fresh context. Dated.
Talk to an engineer about retrieval for RAG
Bring the corpus size, filters and where the data has to stay. An engineer will help you choose between pgvector and OpenSearch.
- ✓A design sketch for your workload, not a plan menu
- ✓A zero-downtime cutover path for what you run today
- ✓A price for that design, after the review