Guide

Context engineering for AI agents: the data layer

What context engineering is, and which databases and queues sit behind an agent's context window, from public documentation.

  • Built on public docs
  • Checked 30 Oct 2026

Context engineering is the practice of curating everything a model sees when it runs: the prompt, tools, retrieved data and message history. For agents, much of that context comes from a data layer: a database for memory, retrieval, fast recall, fresh data and task queues.

What context engineering is

Context engineering is the practice of curating and maintaining the full set of tokens a model sees at inference time, including the prompt, tools, external data and message history. Anthropic's engineering team describes it as broader than writing a prompt, and as iterative, because the curation happens each time you decide what to pass to the model (Anthropic, checked 30 Oct 2026).

The reason it matters is that context is finite. The same article describes context rot: as the window fills, the model's ability to recall information from it declines, so more tokens do not simply mean better answers. Choosing what goes in, and what stays out, is the job.

Context engineering vs prompt engineering

Prompt engineering is about writing and organising instructions, especially the system prompt. Context engineering includes that and also covers tools, retrieved data, memory and message history across a long-running task.

Prompt engineeringContext engineering
ScopeThe instructions, often one promptEverything in the window: instructions, tools, data, history
When it happensWritten once, then revisedEvery time the agent decides what to pass to the model
Needs a data layerRarelyYes, for memory, retrieval and fresh data
Typical failureVague or conflicting instructionsIrrelevant, stale or oversized context

Source: Anthropic engineering, checked 30 Oct 2026; the last row is a summary of the failure modes it discusses.

Techniques: compaction, notes, sub-agents and just-in-time retrieval

Anthropic names four techniques for long-running tasks: compaction, structured note-taking, sub-agents and just-in-time retrieval (Anthropic, checked 30 Oct 2026). Most of them need somewhere to keep state outside the context window.

Illustrative: similar documents sit close together, so retrieval is a nearest-neighbour search.
Compaction
summarise a conversation near the limit and start a new window with the summary.
Structured note-taking
the agent writes notes to memory outside the window and reads them back later.
Sub-agents
specialist agents work in clean windows and return a condensed summary to a lead agent.
Just-in-time retrieval
the agent keeps lightweight references and loads data at runtime with tools, rather than loading everything up front.

The data layer behind the context window

Each kind of context maps to a type of store: short-term state to an in-memory store, durable memory and checkpoints to a database, retrieval to a vector or search index, fresh data to a change stream and tasks to a queue. The table shows common pairings, with each engine's own documentation.

Context needTypical storeWhat the documentation saysPage
Short-term state, repeat answersIn-memory storage with expiry, plus a vector search module in Valkey (Valkey Search)Semantic cache
Checkpoints and long-term memoryPostgreSQL with pgvectorLangGraph provides a Postgres checkpoint saver (LangGraph); Mem0 and Letta document Postgres-backed storageAgent memory
RetrievalVector search in Postgres, or a hybrid keyword-plus-vector query in OpenSearch (OpenSearch docs)Vector database for RAG
Fresh contextKafka with change data captureA change stream from the source database that a pipeline can re-embed (Debezium)Managed Kafka
Agent task queuesConsumer acknowledgements and dead-letter exchanges in RabbitMQ (RabbitMQ docs)RabbitMQ hosting

Sources are each project's own documentation, checked 30 Oct 2026. Documentation changes; confirm on the linked pages.

How this guide was put together

The guide draws the definition and techniques from Anthropic's engineering article and the store pairings from each project's documentation, all read on 30 Oct 2026. It names no benchmark and ranks no product.

OSStend runs the databases and queues named in the table, so it has an interest in this section. Other stores exist, and the pairings are common patterns, not requirements. Test any design against your own agent's behaviour.

AI agent infrastructure checklist

Agent infrastructure is ordinary data infrastructure with stricter freshness and access needs. A short checklist covers the points that decide whether an agent's context stays correct.

  • Memory lives in a durable store with backups, not only in process memory.
  • Retrieval has filters for tenant and permission, tested with realistic queries.
  • Embeddings are refreshed when source data changes, by a change stream or a schedule.
  • Short-term state expires on its own and cannot grow without bound.
  • Failed tasks go to a queue with retries and a dead-letter path.
  • The data stays where your policy requires: your account, your data center or an air-gapped site.

Where OSStend fits

OSStend runs the data layer for agents: PostgreSQL with pgvector, Qdrant, Redis or Valkey, OpenSearch, Kafka and RabbitMQ, with Langfuse, ClickHouse and Temporal beside them, in your own environment. It does not build or host the agents themselves.

See the AI data layer overview and agent memory for the patterns, and managed PostgreSQL for the engine most memory designs start from. If agents need to read production tables, the Postgres MCP server guide covers the role and policy design.

Context engineering questions

What is context engineering?

Context engineering is the practice of curating and maintaining everything a model sees at inference time: the prompt, tools, retrieved data and message history. It decides what goes into the context window and what stays out.

How is context engineering different from prompt engineering?

Prompt engineering is about writing and organising instructions. Context engineering is broader: it covers tools, retrieved data, memory and message history across a long task, and the curation happens every time the agent decides what to pass to the model.

What data infrastructure does an AI agent need?

Agents commonly need a durable database for memory and checkpoints, a vector or search index for retrieval, an in-memory store for short-term state, a change stream for fresh data and a queue for tasks. The exact mix depends on the agent.

What is agentic RAG?

Agentic RAG is retrieval-augmented generation where the agent decides what to look up and when, instead of a fixed pipeline retrieving once per question. It depends on fast retrieval that can be filtered and repeated.

Talk to an engineer

Talk to an engineer about your agents' data layer

Bring the agent design and where its data must live. An engineer will give a straight view on the stores behind it.

  • A design sketch for your workload, not a plan menu
  • A zero-downtime cutover path for what you run today
  • A price for that design, after the review
See pricing →

Your request goes to the OSStend team. No newsletter sign-up.

Get a quoteTalk to an engineer
2Engines
3Where should it run?
4What do you need?

Your request goes to the OSStend team. No newsletter sign-up.