AI data · Retrieval

Vector database for RAG on your own infrastructure

Retrieval for RAG on PostgreSQL with pgvector or OpenSearch, run in OSStend's cloud, your cloud account or your data center.

45 minutes with an engineer: we review your workload and sketch the design.

A vector database for RAG stores document embeddings and returns the passages closest to a question, so a language model can answer from your data. PostgreSQL with pgvector suits data that sits beside relational tables, and OpenSearch suits combined keyword and vector search. Dedicated vector engines suit very large, vector-only workloads.

160+AceMQ customers
10+Years running these systems
24/7Engineers on call
3Places it runs: our cloud, yours, your data center
Engines we run, on the platforms you already use
  • PostgreSQL
  • MySQL
  • MongoDB
  • Redis
  • Valkey
  • OpenSearch
  • Kafka
  • RabbitMQ
  • AWS
  • Google Cloud
  • Azure
  • Kubernetes
  • OpenShift
  • Prometheus
  • Grafana
  • Datadog
  • Dynatrace
  • Splunk

Why retrieval gets harder in production

A RAG demo retrieves from a handful of documents. A production system retrieves from a changing corpus under access rules, and a wrong or stale passage becomes a wrong answer.

Illustrative: similar documents sit close together, so retrieval is a nearest-neighbour search.
Stale content.
The corpus changes, but vectors do not update on their own. Answers come from the old version until the embedding is recomputed.
Wrong passages, not no passages.
Approximate search trades recall for speed, so a relevant chunk can be missed without any error.
Exact terms matter.
Part numbers, error codes and names are poorly served by vector similarity alone, which is why keyword plus vector search exists.
Access rules.
Retrieval has to respect who is allowed to see which document, so filters must work with the index.
Model changes.
Vectors from different embedding models are not comparable, so changing the model means re-embedding the corpus.
Data location.
The corpus is often internal or customer data, and sending it to a separate hosted service raises review and residency questions.
How it works

The RAG retrieval pattern

RAG splits documents into chunks, turns each chunk into a vector, stores them, and at question time retrieves the nearest chunks to pass to the model. The retrieval store is one component, and it needs a pipeline that keeps it current.

  1. Ingest and chunk

    Split source documents into passages sized for the embedding model and the context window.

  2. Embed

    Convert each passage into a vector with an embedding model, and store the text, vector and metadata together.

  3. Retrieve

    Embed the question, run a nearest-neighbour search with any access filters, and optionally combine it with keyword search.

  4. Rerank and generate

    Optionally rerank the candidates, then pass the best passages to the model with the question.

  5. Refresh

    When a source document changes, re-embed it. A change stream into Kafka is a common way to trigger this; see keeping embeddings fresh with Kafka and CDC.

OptionSearch styleFits best whenDocumentation
PostgreSQL with pgvectorExact or approximate vector search; keyword via Postgres full-text searchThe corpus and its metadata already live in Postgres, and you want one database, transactions and joinspgvector
OpenSearchKeyword (BM25) and vector search combined in a hybrid queryExact terms and semantic similarity both matter, or OpenSearch already holds the corpusOpenSearch hybrid query
Dedicated vector engineVector search, often with sparse-dense hybridVector-only workloads at a scale that justifies a separate system, or a team that wants no database operationsSee pgvector vs Pinecone and the Pinecone alternative

Sources: each engine's own documentation, checked 30 Oct 2026.

When PostgreSQL with pgvector is enough

pgvector is enough when the corpus fits a single well-sized Postgres server plus read replicas, and when the retrieval metadata and access rules already live in the same database. It stops being enough when the vector count outgrows what one instance and its replicas can hold in memory.

pgvector supports exact and approximate search, HNSW and IVFFlat indexes, and half-precision and binary storage to shrink the working set. It scales vertically and through replicas, and the pgvector docs name tools such as Citus for sharding beyond that. See the full index and sizing detail on the pgvector page. For how retrieval fits with memory and the rest of an agent's context, see context engineering.

When OpenSearch hybrid search fits

OpenSearch fits when keyword relevance matters as much as semantic similarity. Its hybrid query runs several sub-queries and combines their scores into one, with a search pipeline that normalises the scores first.

OpenSearch's documentation describes a hybrid query of up to five sub-queries, a normalization processor that puts BM25 and vector scores on the same scale, and reciprocal rank fusion as another way to merge ranked lists. Its k-NN engines include Lucene and Faiss, with HNSW and, in Faiss, IVF methods (OpenSearch docs, checked 30 Oct 2026). See managed OpenSearch.

Sizing and tuning retrieval

Retrieval cost is driven by chunk count, embedding dimensions, index type and recall target. The same four decisions apply on every engine, even though the settings are named differently.

Illustrative shape: real designs start from your metrics.
DriverEffect
Chunk count × dimensionsMemory and storage4 × dimensions + 8 bytes per `vector`; `halfvec` halves the per-dimension sizeVector memory grows with count and dimensions; quantization options reduce it
Index typeSpeed, recall, build timeHNSW or IVFFlatHNSW or IVF methods, depending on engine
Recall against latencyAnswer quality and response time`hnsw.ef_search`, `ivfflat.probes``ef_search` and related method parameters
FiltersResult counts and access controlFilter applies after approximate scan; iterative scans, partial indexes, partitioningFilters on hybrid queries; the docs cover pre-filtering and post-filtering
Keyword relevanceExact-term recallPostgres full-text search combined with vector resultsNative BM25 in the same hybrid query

Sources: pgvector README and OpenSearch documentation, checked 30 Oct 2026.

Pitfalls documented by the engines

The engines' own documentation covers the behaviour that most often surprises teams moving RAG to production.

Filtered queries return fewer rows.
In pgvector, approximate indexes filter after the scan. Iterative index scans exist for this case.
Score scales differ.
In OpenSearch, BM25 and vector scores use different scales, so they are normalised before they are combined.
Tenants share an index.
In pgvector, vectors from one tenant can affect recall for another when an approximate index is shared.
Re-embedding after a model change.
Plan for a full rebuild, and keep the old index until the new one is verified.
Enterprise only

What OSStend runs, where it runs and who it is not for

OSStend designs, moves and runs PostgreSQL and OpenSearch in its own cloud, in your cloud account or co-managed in your data center, so the corpus never leaves your environment.

Contact us if

  • The corpus is internal or customer data and must stay in your own environment.
  • You want to choose between pgvector and OpenSearch on measured results rather than on a vendor's default.
  • You want the retrieval layer run by the same team as your other databases.
Talk to an engineer

Not a fit if

  • You want a fully hosted vector API with no infrastructure decisions.
  • Your workload is vector-only at a scale that a dedicated vector service is built for.
  • You need a managed embedding or generation service. OSStend runs the data layer, not the models.

What is the best vector database for RAG?

No single engine is best for every RAG system. PostgreSQL with pgvector suits corpora that live beside relational data, OpenSearch suits combined keyword and vector search, and dedicated vector engines suit large vector-only workloads. The right choice depends on corpus size, filters, hybrid needs and where the data may run.

Can PostgreSQL be a vector database for RAG?

Yes. With the pgvector extension, PostgreSQL stores embeddings and runs exact or approximate nearest-neighbour search, with HNSW and IVFFlat indexes. It adds transactions, joins and point-in-time recovery, and scales vertically and through replicas.

Is OpenSearch a vector database?

OpenSearch includes a k-NN vector field type with HNSW and IVF methods across its Lucene and Faiss engines. Its distinguishing feature for RAG is the hybrid query, which combines keyword and vector scores in one search.

What is a RAG database?

A RAG database is the store that a retrieval-augmented generation system queries for context. It holds document chunks with their embeddings and metadata, and returns the passages nearest to a question. PostgreSQL with pgvector, OpenSearch and dedicated vector engines are all used as one.

Which open-source vector database should run on my own infrastructure?

pgvector and OpenSearch are both open source and run in your own cloud account or data center. Dedicated open-source engines such as Qdrant and Milvus also self-host; the best open source vector databases guide compares six. Choose by data model, scale and operations skill.

How do embeddings stay up to date in RAG?

A pipeline re-embeds a record when its source changes. A common design reads the source database's change stream, publishes it to Kafka and runs an embedding job on each change, so the vector store never lags far behind the source. The Kafka and CDC guide covers the pipeline in detail.

Talk to an engineer

Talk to an engineer about retrieval for RAG

Bring the corpus size, filters and where the data has to stay. An engineer will help you choose between pgvector and OpenSearch.

  • A design sketch for your workload, not a plan menu
  • A zero-downtime cutover path for what you run today
  • A price for that design, after the review
See pricing →

Your request goes to the OSStend team. No newsletter sign-up.

Get a quoteTalk to an engineer
2Engines
3Where should it run?
4What do you need?

Your request goes to the OSStend team. No newsletter sign-up.