CLAUDE.md · vectorize-io/hindsight

Sections

1. Project Overview
2. Development Commands
3. Architecture
4. Key Conventions
5. Environment Setup

First lines (shown under the repo's MIT license)

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as:
- **World facts**: General knowledge ("The sky is blue")
- **Experience facts**: Personal experiences ("I visited Paris in 2023")
- **Mental models**: Consolidated knowledge synthesized from facts ("User prefers functional programming patterns")

## Development Commands

### Local Development (API + UI)
```bash
# Start both API server and control plane UI
./scripts/dev/start.sh
```

### API Server (Python/FastAPI)
```bash
# Start API server only (loads .env automatically)
./scripts/dev/start-api.sh

# Run all tests (parallelized with pytest-xdist)
cd hindsight-api-slim && uv run pytest tests/

# Run specific test file
cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v

# Run single test function
cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_with_chunks -v

# Lint and format
cd hindsight-api-slim && uv run ruff check .
cd hindsight-api-slim && uv run ruff format .

# Type checking (uses ty - extremely fast type checker from Astral)
cd hindsight-api-slim && uv run ty check hindsight_api/
```

### Control Plane (Next.js)
```bash
./scripts/dev/start-control-plane.sh
# Or manually:
cd hindsight-control-plane && npm run dev

# Run frontend unit tests
cd hindsight-control-plane && npm test

# Run frontend linter
cd hindsight-control-plane && npm run lint
```

### CLI Tool (Rust)
```bash
# Build CLI binary (hindsight-cli/target/release/hindsight)
cd hindsight-cli && cargo build --release

# Run CLI tests
cd hindsight-cli && cargo test
```

### System Tests (Blackbox End-to-End)
```bash
# Prerequisite: server needs pg0 installed in api-slim
(cd hindsight-api-slim && uv sync --frozen --extra embedded-db)

# Run blackbox integration tests against local embedded Postgres & stub provider
# (The stub handles LLM, embeddings, and reranking without downloading models)
cd hindsight-system-tests && uv run pytest tests/ -v
```

### Documentation Site (Docusaurus)
```bash
./scripts/dev/start-docs.sh          # pages compile on first visit (~1 GB)
./scripts/dev/start-docs.sh --full   # compile every page up front, like the build (~3.5 GB)
```


### Generating Clients/OpenAPI
```bash
# Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints)
./scripts/generate-openapi.sh

# Regenerate all client SDKs (Python, TypeScript, Rust, Go)
./scripts/generate-clients.sh
```

### Benchmarks
```bash
# Accuracy benchmarks: LoComo, LongMemEval, BEAM and the coding-agent suite come
# from AMB (vectorize-io/agent-memory-benchmark), which owns their datasets,
# judge and scoring. Run them against a local server, or any deployment with
# --api-url. See hindsight-system-evals/README.md.
cd hindsight-system-evals
uv run run-amb --dataset locomo --split locomo10
uv run run-amb --dataset longmemeval --split s -- --category single-session-user

# Performance benchmarks
./scripts/benchmarks/run-perf-test.sh                      # System perf (mock LLM + pg0)
./scripts/benchmarks/run-perf-test.sh --scale tiny          # Quick smoke test
./scripts/benchmarks/run-consolidation.sh
```

## Architecture

### Monorepo Structure
- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv)
- **hindsight-api/**: Published distribution package providing `hindsight-api`, `hindsight-worker`, `hindsight-local-mcp`, and `hindsight-admin` CLI entrypoints
- **hindsight-all/** & **hindsight-all-slim/**: Full and slim standalone distribution bundle packages
- **hindsight-all-npm/**: Node.js lifecycle manager embedding local daemon (published as `@vectorize-io/hindsight-all`)
- **hindsight-control-plane/**: Admin UI (Next.js, npm)
- **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client)
- **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust, Go)
- **hindsight-docs/**: Docusaurus documentation site
- **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, coding-agents, etc.)
- **hindsight-embed/**: Embedded Python CLI and local daemon manager (connects to local API or falls back to uvx hindsight-api)
- **hindsight-system-tests/**: Blackbox end-to-end integration test suite driven through SDK against local stubs
- **hindsight-system-evals/**: End-to-end quality evaluation suites scored by independent LLM judges against real providers
- **hindsight-integration-tests/**: E2E and integration test suites requiring a running server (live API or docker-compose)
- **hindsight-extensions/**: Server extension slots (tenancy/auth, HTTP endpoints, MCP tools, operation validators, memory defense)
- **hindsight-dev/**: Development tools and benchmarks

### Core Engine (hindsight-api-slim/hindsight_api/engine/)
- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code, GitHub Copilot, DeepSeek, Fireworks, Codex (openai-codex), xAI (xai-oauth), Llama.cpp, Nous, etc. (See `hindsight-docs/docs/developer/configuration.mdx` for full provider lists)
- `embeddings.py`: Embedding generation supporting local (SentenceTransformers) and standalone/remote providers (in-process: onnx; remote: tei, openai, cohere, zeroentropy, litellm, google, etc.)
- `cross_encoder.py`: Reranking supporting local (CrossEncoder) and standalone/remote providers (in-process: flashrank, onnx, jina-mlx; remote: tei, cohere, siliconflow, zeroentropy, litellm, google, alibaba, etc.)
- `query_analyzer.py`: Query intent analysis

**retain/**: Memory ingestion pipeline
- `orchestrator.py`: Coordinates the retain flow
- `fact_extraction.py`: LLM-based fact extraction from content

**search/**: Multi-strategy retrieval
- `retrieval.py`: Main retrieval orchestrator
- `graph_retrieval.py`: Graph retrieval abstract base class
- `fusion.py`: Reciprocal rank fusion for combining results
- `reranking.py`: Cross-encoder reranking

**memories/**: The memories store, which owns every table a memory touches (`base.py` is the interface, `postgres.py` + `pg/` the default Postgres store — the only code allowed to name those tables)
- `pg/entity_resolver.py`: Entity extraction and normalization
- `pg/links.py`: Entity link creation and management
- `pg/link_expansion.py`: Link expansion graph retrieval

### API Layer (hindsight-api-slim/hindsight_api/api/)
- `http.py`: FastAPI HTTP routers for all REST endpoints
- `mcp.py`: Model Context Protocol server implementation

Main operations:
- **Retain**: Store memories, extracts facts/entities/relationships, supports document chunking
- **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking
- **Reflect**: Disposition-aware reasoning using memories and mental models
- **Knowledge Base**: Tree of auto-refreshing synthetic documents (mental models) synthesized by LLM from bank observations; searchable, exportable as markdown, and mountable read-only to the filesystem via hindsight-cli (`hindsight fs mount`)
- **Directives & Mental Models**: Manage behavioral guidelines (Directives) and consolidated facts (Mental Models)

### Database
PostgreSQL with pgvector (also supports Oracle 23ai). Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup.