Context engineering for AI agents: the data layer
What context engineering is, and which databases and queues sit behind an agent's context window, from public documentation.
- ✓Built on public docs
- ✓Checked 30 Oct 2026
Context engineering is the practice of curating everything a model sees when it runs: the prompt, tools, retrieved data and message history. For agents, much of that context comes from a data layer: a database for memory, retrieval, fast recall, fresh data and task queues.
What context engineering is
Context engineering is the practice of curating and maintaining the full set of tokens a model sees at inference time, including the prompt, tools, external data and message history. Anthropic's engineering team describes it as broader than writing a prompt, and as iterative, because the curation happens each time you decide what to pass to the model (Anthropic, checked 30 Oct 2026).
The reason it matters is that context is finite. The same article describes context rot: as the window fills, the model's ability to recall information from it declines, so more tokens do not simply mean better answers. Choosing what goes in, and what stays out, is the job.
Context engineering vs prompt engineering
Prompt engineering is about writing and organising instructions, especially the system prompt. Context engineering includes that and also covers tools, retrieved data, memory and message history across a long-running task.
| Prompt engineering | Context engineering | |
|---|---|---|
| Scope | The instructions, often one prompt | Everything in the window: instructions, tools, data, history |
| When it happens | Written once, then revised | Every time the agent decides what to pass to the model |
| Needs a data layer | Rarely | Yes, for memory, retrieval and fresh data |
| Typical failure | Vague or conflicting instructions | Irrelevant, stale or oversized context |
Source: Anthropic engineering, checked 30 Oct 2026; the last row is a summary of the failure modes it discusses.
Techniques: compaction, notes, sub-agents and just-in-time retrieval
Anthropic names four techniques for long-running tasks: compaction, structured note-taking, sub-agents and just-in-time retrieval (Anthropic, checked 30 Oct 2026). Most of them need somewhere to keep state outside the context window.
- Compaction
- summarise a conversation near the limit and start a new window with the summary.
- Structured note-taking
- the agent writes notes to memory outside the window and reads them back later.
- Sub-agents
- specialist agents work in clean windows and return a condensed summary to a lead agent.
- Just-in-time retrieval
- the agent keeps lightweight references and loads data at runtime with tools, rather than loading everything up front.
Agentic search and agentic RAG
Agentic search is retrieval where the agent decides what to look up and when, instead of a fixed pipeline retrieving once. Agentic RAG applies the same idea to retrieval-augmented generation.
Both depend on fast, filterable retrieval; see the vector database guide.
The data layer behind the context window
Each kind of context maps to a type of store: short-term state to an in-memory store, durable memory and checkpoints to a database, retrieval to a vector or search index, fresh data to a change stream and tasks to a queue. The table shows common pairings, with each engine's own documentation.
| Context need | Typical store | What the documentation says | Page |
|---|---|---|---|
| Short-term state, repeat answers | Redis or Valkey | In-memory storage with expiry, plus a vector search module in Valkey (Valkey Search) | Semantic cache |
| Checkpoints and long-term memory | PostgreSQL with pgvector | LangGraph provides a Postgres checkpoint saver (LangGraph); Mem0 and Letta document Postgres-backed storage | Agent memory |
| Retrieval | pgvector or OpenSearch | Vector search in Postgres, or a hybrid keyword-plus-vector query in OpenSearch (OpenSearch docs) | Vector database for RAG |
| Fresh context | Kafka with change data capture | A change stream from the source database that a pipeline can re-embed (Debezium) | Managed Kafka |
| Agent task queues | RabbitMQ or Kafka | Consumer acknowledgements and dead-letter exchanges in RabbitMQ (RabbitMQ docs) | RabbitMQ hosting |
Sources are each project's own documentation, checked 30 Oct 2026. Documentation changes; confirm on the linked pages.
How this guide was put together
The guide draws the definition and techniques from Anthropic's engineering article and the store pairings from each project's documentation, all read on 30 Oct 2026. It names no benchmark and ranks no product.
OSStend runs the databases and queues named in the table, so it has an interest in this section. Other stores exist, and the pairings are common patterns, not requirements. Test any design against your own agent's behaviour.
AI agent infrastructure checklist
Agent infrastructure is ordinary data infrastructure with stricter freshness and access needs. A short checklist covers the points that decide whether an agent's context stays correct.
- ✓Memory lives in a durable store with backups, not only in process memory.
- ✓Retrieval has filters for tenant and permission, tested with realistic queries.
- ✓Embeddings are refreshed when source data changes, by a change stream or a schedule.
- ✓Short-term state expires on its own and cannot grow without bound.
- ✓Failed tasks go to a queue with retries and a dead-letter path.
- ✓The data stays where your policy requires: your account, your data center or an air-gapped site.
Where OSStend fits
OSStend runs the data layer for agents: PostgreSQL with pgvector, Qdrant, Redis or Valkey, OpenSearch, Kafka and RabbitMQ, with Langfuse, ClickHouse and Temporal beside them, in your own environment. It does not build or host the agents themselves.
See the AI data layer overview and agent memory for the patterns, and managed PostgreSQL for the engine most memory designs start from. If agents need to read production tables, the Postgres MCP server guide covers the role and policy design.
Context engineering questions
What is context engineering?
Context engineering is the practice of curating and maintaining everything a model sees at inference time: the prompt, tools, retrieved data and message history. It decides what goes into the context window and what stays out.
How is context engineering different from prompt engineering?
Prompt engineering is about writing and organising instructions. Context engineering is broader: it covers tools, retrieved data, memory and message history across a long task, and the curation happens every time the agent decides what to pass to the model.
What data infrastructure does an AI agent need?
Agents commonly need a durable database for memory and checkpoints, a vector or search index for retrieval, an in-memory store for short-term state, a change stream for fresh data and a queue for tasks. The exact mix depends on the agent.
What is agentic RAG?
Agentic RAG is retrieval-augmented generation where the agent decides what to look up and when, instead of a fixed pipeline retrieving once per question. It depends on fast retrieval that can be filtered and repeated.
Related pages
- GuideWhat is a vector database? How to choose oneWhat a vector database is, how RAG uses one, and how to choose between pgvector, OpenSearch and a dedicated vector service. Built on public docs, dated.
- AI data · Fast recallSemantic cache for LLM apps with Redis or ValkeySemantic cache with Redis or Valkey: how a similarity-matched cache cuts repeat model calls, how to set thresholds and eviction, and the cost maths.
- AI data · MemoryAgent memory on PostgreSQL and Redis in productionAgent memory on PostgreSQL and Redis: long-term memory and checkpoints in Postgres with pgvector, short-term state in Redis or Valkey, and retention.
- AI data · RetrievalVector database for RAG on your own infrastructureA vector database for RAG on your own infrastructure: when PostgreSQL with pgvector is enough, when OpenSearch hybrid search fits, and how to size retrieval.
- KafkaManaged Kafka in your cloud or data centerManaged Kafka with the broker count, partition plan, KRaft, Connect and schema registry designed for your estate, run 24/7 in your cloud or your data center.
- RabbitMQRabbitMQ hosting in your cloud or on premiseRabbitMQ hosting for clusters a plan cannot hold: federation and shovel, the plugins you need and upgrades on your change calendar, run 24/7 by engineers.
Talk to an engineer about your agents' data layer
Bring the agent design and where its data must live. An engineer will give a straight view on the stores behind it.
- ✓A design sketch for your workload, not a plan menu
- ✓A zero-downtime cutover path for what you run today
- ✓A price for that design, after the review