An AI assistant remembers your preferences during a conversation. Start a fresh session, and you may have to explain everything again: how you work, which decisions have already been made, and what went wrong last time.

Applications can address this by saving conversation history or retrieving relevant passages. The harder problem is turning that history into useful, evolving knowledge.

Hindsight, an open-source agent memory system from Vectorize, takes that problem on. Released under the MIT license, its GitHub repository displayed approximately 53,500 stars when checked on October 25, 2026. It combines memory storage, retrieval, and reasoning in a system designed to carry knowledge across sessions. GitHub repository

What Is Hindsight?

Hindsight is a memory layer that sits alongside an AI application. It processes information from conversations and documents, organizes it into a memory bank, and makes that knowledge available to future interactions.

Its central idea is to distinguish recorded information from conclusions drawn from it. What a user said, what an agent did, and what the system inferred should remain distinguishable.

Here, “learning” means updating external memories and synthesized knowledge. It does not mean retraining the underlying language model after each conversation.

The accompanying paper, Hindsight is 20/20, reports a striking result: an open-source 20B model achieved 83.6% accuracy with Hindsight, compared with 39.0% for a full-context baseline using the same underlying model. The paper also reports 91.4% on LongMemEval with a stronger model configuration. These are results from the reported experiments, rather than guarantees for other applications. Research paper

How Its Memory Works

Facts, Experiences, Observations, and Mental Models

Hindsight describes four complementary forms of memory:

  • World facts: Information about people, organizations, events, and the outside world.

  • Experiences: Actions and interactions involving the agent itself.

  • Observations: Conclusions consolidated from multiple memories, with supporting evidence.

  • Mental models: Maintained summaries addressing specific questions about a user, project, or topic.

For example, a user’s stated preference for short reports is a fact. The agent’s experience preparing a report is a separate record. A broader conclusion about the user’s preferred communication style can then be built from multiple interactions.

This structure helps keep the evidence behind a conclusion visible. Project documentation

Retain: Turn Conversations Into Structured Memories

The retain operation processes incoming information.

In its default extraction workflow, Hindsight uses a language model to identify facts, entities, relationships, and time references. It connects related information and distinguishes when an event happened from when the system learned about it.

That distinction matters. A customer mentioning an issue today may be describing something that happened three months ago.

Retain also separates facts about the outside world from the agent’s own experiences. Background consolidation can then combine related facts into observations. The current implementation offers alternative extraction modes, including storing chunks without an LLM extraction step. Retain documentation

Recall: Search Through Multiple Paths

The recall operation combines four retrieval approaches:

  • Semantic search finds information with related meaning.

  • Keyword search matches names, identifiers, and specific terms.

  • Graph traversal follows relationships between memories.

  • Temporal retrieval finds information relevant to a particular period.

Results are combined using reciprocal rank fusion, reranked, and trimmed to fit the available token budget.

This lets the system approach “What happened during the June launch?” differently from “What does this customer prefer?” The first depends heavily on timing; the second may require connecting information across several conversations. Recall documentation

Reflect: Reason Over Accumulated Knowledge

The reflect operation uses stored knowledge to answer questions that require synthesis.

A project assistant might examine recurring delays and identify unresolved risks. A support assistant might compare customer questions with existing documentation to find gaps.

Reflect gathers evidence through an iterative process, consulting mental models, observations, and underlying facts. Its responses can reference the memories used to reach an answer.

Banks also support disposition settings, including skepticism, literalism, and empathy, which influence how reflection interprets and presents information. These settings shape the response; they do not establish whether a conclusion is correct. Reflect documentation

Memory Banks and Sensitive Information

Hindsight organizes memories into banks, which can represent individual users, agents, or projects. It also supports preserving the language and native script of retained information. GitHub documentation

Its Memory Defense feature scans incoming content for recognized secrets and personally identifiable information using 45 detection patterns. Depending on the policy, matching content can be redacted or blocked.

One detail matters for deployment: Memory Defense is disabled by default and must be enabled for each bank. Enabling it affects future writes; it does not automatically clean previously stored memories. Memory Defense documentation

Mental Models: Prepare Answers Before They Are Needed

Some questions come up repeatedly:

“What are this user’s preferences?”

“What decisions have been made about this project?”

A mental model stores a maintained answer to a question like these. Hindsight can refresh it in the background as relevant memories change.

Reading the stored answer requires a database read rather than a fresh retrieval-and-generation cycle. The work still happens, but it moves away from the moment the application needs the answer.

Hindsight also tracks freshness, allowing reflection to check an outdated summary against more recent evidence. Mental models documentation

Knowledge pages extend this approach into documents organized like a wiki, with Markdown export available for use outside the system. Project overview

Connecting Hindsight to an Existing Agent

Hindsight offers integrations for frameworks and tools including LangGraph, CrewAI, n8n, and Dify. MCP support exposes memory operations to compatible clients.

For coding workflows, its coding-agent integration can build repository-specific memory from Git history and past sessions. Supported tools include Claude Code, Codex CLI, and Cursor CLI.

Teams can choose automatic memory handling through an integration or call the APIs explicitly when they need more control over what gets retained and retrieved. Integration directory

Getting Started

The official Docker quick start launches the API and management interface:

export OPENAI_API_KEY="your-api-key"

docker run -it --pull always \
  --name hindsight \
  --restart unless-stopped \
  -p 8888:8888 \
  -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY="$OPENAI_API_KEY" \
  -v hindsight-data:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest

The API runs on port 8888, and the management interface runs on port 9999. The project also provides deployment options for external PostgreSQL, Kubernetes, and managed hosting. Installation guide

Install the Python client:

pip install -U hindsight-client

The basic workflow uses three methods:

from hindsight_client import Hindsight

client = Hindsight(base_url="http://localhost:8888")

client.retain(
    bank_id="demo",
    content="Maya prefers concise weekly project updates."
)

memories = client.recall(
    bank_id="demo",
    query="How does Maya prefer to receive project updates?"
)

response = client.reflect(
    bank_id="demo",
    query="How should I prepare a weekly update for Maya?"
)

These calls demonstrate storing information, retrieving it, and reasoning over it. API quick start

Hindsight supports hosted and local model providers. Using another provider requires matching its provider, model, and endpoint settings; changing the API key alone is not necessarily sufficient. Model configuration

What to Evaluate Before Adopting It

Hindsight brings several useful capabilities together: structured extraction, multiple retrieval methods, evidence-backed synthesis, and maintained summaries.

The practical question is whether those capabilities improve an agent’s performance enough to justify their cost and complexity.

A useful evaluation should include changing preferences, corrections to earlier information, conflicting statements, and questions spanning multiple sessions. Measure answer quality alongside ingestion cost, retrieval latency, and reflection cost.

The reported benchmarks make Hindsight a credible candidate for testing. Its value in a particular application depends on whether the agent can reliably use past information to make its next interaction better.

Explore the project: Hindsight on GitHub

Sources & references