CLAUDE.md · garrytan/gbrain

Sections

1. North Star
2. Two organizational axes (read this first)
3. Architecture
4. Reference map (load on demand)
5. Maintaining CLAUDE.md and the reference docs
6. Search modes and evals
7. Skills
8. Memory operating protocol
9. Bulk-action progress reporting
10. PR acceptance: evidence, not trust
11. Fix waves: ONE PR
12. Capturing test output (NEVER pipe through tail / head)
13. Sync and backfill pacing
14. Build
15. Version locations (single source of truth: VERSION file)
16. Conductor branch-name = workspace-name (IRON RULE)
17. Releasing
18. Post-ship requirements (MANDATORY)
19. "Say to your agent" rule: every feature doc addresses the END USER (IRON RULE)
20. Privacy rule: scrub real names from public docs

First lines (shown under the repo's MIT license)

# CLAUDE.md

GBrain is a personal knowledge brain and GStack mod for agent platforms. Pluggable
engines: PGLite (embedded Postgres via WASM, zero-config default) or Postgres + pgvector
+ hybrid search in a managed Supabase instance. `gbrain init` defaults to PGLite;
suggests Supabase for 1000+ files. GStack teaches agents how to code. GBrain teaches
agents everything else: brain ops, signal detection, content ingestion, enrichment,
cron scheduling, reports, identity, and access control.

## North Star

gbrain aims to be the **next Postgres for memory**: the most well-tested, widest-coverage,
best-for-the-most-at-the-least retrieval + agent memory system for company brains and
personal AI, built to serve a billion people. Every feature and every eval is judged
against this bar. "gbrain is best" is a WHOLE-SYSTEM claim — proven across the full
BrainBench suite (retrieval, longmemeval, calibration, …) — not by any single feature.
When scoping an eval, prove the FEATURE delivers value to gbrain users; do not waste it
proving that gbrain's particular algorithm beats some other algorithm (a research
bake-off, off-mission).

## Two organizational axes (read this first)

GBrain knowledge is organized along two orthogonal axes. Users AND agents must
understand both, or queries misroute silently.

- **Brain** — WHICH DATABASE. Your personal brain is `host`. You can mount
  additional brains (team-published, each with their own DB and access policy)
  via `gbrain mounts add` (v0.19+). Routing: `--brain`, `GBRAIN_BRAIN_ID`,
  `.gbrain-mount` dotfile.
- **Source** — WHICH REPO INSIDE THE DATABASE. A brain can hold many sources
  (wiki, gstack, openclaw, essays). Slugs scope per source. Routing:
  `--source`, `GBRAIN_SOURCE`, `.gbrain-source` dotfile.

Both axes follow the same 6-tier resolution pattern. Read
`docs/architecture/brains-and-sources.md` for topology diagrams (personal, team
mount, CEO-class with multiple team brains) and
`skills/conventions/brain-routing.md` for the agent-facing decision table.

## Architecture

Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md` — and the seven frozen MEMORY_VERBS `recall`/`remember`/`entity`/`synthesize`/`forget`/`context_pack`/`delta` — the last two are v0.45.7 ambient-recall boundary verbs (budget-packed pack + "what changed since"), all seven stamp `protocol_version: 1`, servable alone via `gbrain serve --surface verbs`, see `docs/protocol/MEMORY_VERBS_v1.md` + `docs/guides/ambient-recall.md`). CLI and MCP
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
markdown files (tool-agnostic, work with both CLI and plugin contexts).

**Trust boundary:** `OperationContext.remote` distinguishes trusted local CLI callers
(`remote: false` set by `src/cli.ts`) from untrusted agent-facing callers
(`remote: true` set by `src/mcp/server.ts`). Security-sensitive operations like
`file_upload` tighten filesystem confinement when `remote=true` and default to
strict behavior when unset.

**Cross-cutting invariants (must-never-violate, regardless of which file you touch).**
Per-file detail is in `docs/architecture/KEY_FILES.md`.

- **Trust is fail-closed.** `OperationContext.remote` is REQUIRED on the type. Anything not
  strictly `false` is treated as remote/untrusted (`ctx.remote === false` for trusted-only
  sites; `ctx.remote !== false` for untrust-unless-explicit-false). Don't default it falsy.
- **Source isolation.** Every read-side op routes through `sourceScopeOpts(ctx)`; precedence
  is federated array (`ctx.auth.allowedSources`) > scalar (`ctx.sourceId`) > nothing. Don't
  hand-roll source filtering — a missed thread is a cross-source data leak. Corollary
  (unscoped-check/scoped-write): `engine.getPage` with no opts matches ANY source while
  `putPage` defaults to `'default'` — an existence check + write pair must scope the read
  to the write's source (`getPage(slug, { sourceId: x ?? 'default' })`). Guarded by
  `scripts/check-getpage-scoped-write.mjs` (opt-out marker
  `gbrain-allow-unscoped-getpage` for read-only first-match sites).
- **JSONB: never `JSON.stringify` into a `::jsonb` cast.** postgres.js double-encodes it (a jsonb
  string scalar); PGLite hides the bug. This bites BOTH spellings — the template form
  (`${JSON.stringify(x)}::jsonb`) AND the positional form (`executeRaw(\`…$N::jsonb\`, [JSON.stringify(x)])`,
  the #2339 class that aborted every sync). Fix: pass a raw object to `engine.executeRaw` / use
  `executeRawJsonb` / `sql.json()`; or for the positional path bind through `$N::text::jsonb` (binds as
  text, the cast parses it). Guarded by `scripts/check-jsonb-pattern.sh` (template grep) +
  `scripts/check-jsonb-params.mjs` (positional AST scanner); the real backstop is the DATABASE_URL-gated
  e2e parity tests, since PGLite can't surface the bug. Full rule in `docs/ENGINES.md`.
- **Engine-live paths avoid runtime dynamic `import()` for helper dependencies.** In both
  engines, `src/core/migrate.ts`, `src/core/engine-sql/` and `src/core/schema-migrations/`,
  helpers use static top-level imports. Exceptions: the snapshot loader's lazy `require()`
  cluster in `pglite-engine.ts:tryLoadSnapshot` (lazy so production builds skip the
  test-fixture path; the snapshot hash reads the FILE BYTES of the schema import closure,
  `src/core/snapshot-schema-inputs.ts`, never loaded modules, so coverage can't skew it) and
  the four `ai/gateway.ts` lookups in both engines' `initSchema()` and `_upsertChunksOnce()`,
  each lazy inside a local `try/catch` because eagerly evaluating the gateway's large
  provider/config closure could turn a recoverable default/config-row fallback into a
  module-load failure. Each exception carries `engine-dynamic-import-ok` on the import line;
  `scripts/check-engine-dynamic-import.sh` enforces the rule (it matches `require()` too).
  For history use `git log -G'await[[:space:]]+import\\('`, not `-S`: a dynamic-to-static
  rewrite can keep the token while changing its context.
- **Engine parity.** Both engines implement every `BrainEngine` method. A migrated storage
  domain's SQL lives once in `src/core/engine-sql/<domain>.ts` and both engines delegate to it
  (domains: the `migrated` rows of `scripts/engine-sql-baseline.tsv`; `check:engine-sql-ratchet`
  fails on new SQL in an engine; mark truly dialect-specific code `// engine-sql-ok: <reason>`).
  Executor contract, RLS read brands, capabilities: `docs/ENGINES.md` ("Storage domains and
  engine-sql"). Pinned by `test/e2e/engine-parity.test.ts`. Forward-referenced columns/indexes
  go in `src/core/engine-sql/bootstrap.ts` (guarded by `test/schema-bootstrap-coverage.test.ts`).
- **Contract-first.** `src/core/operations.ts` is the single source; CLI + MCP are generated
  from it. Every op carries `scope: 'read'|'write'|'admin'` + optional `localOnly`. HTTP
  dispatch enforces scope/localOnly before the handler runs.
- **Migrations + schema text.** One file per schema migration in `src/core/schema-migrations/`
  (`bun run new:migration <name>`; `migrate.ts` is the runner; regenerate `registry.generated.ts`,
  never hand-merge it). Fresh-install DDL has one hand-edited copy, `src/schema.sql` + its TS
  fragments; `bun run build:schema` generates both engine blobs. `CREATE INDEX CONCURRENTLY`
  needs `transaction: false` (pre-drop invalid remnants on Postgres; plain `CREATE INDEX` on
  PGLite via `sqlFor.pglite`).
- **Multi-source.** Slug uniqueness is `(source_id, slug)`, not slug. Key batch ops and
  reverse-writes on the composite key; `validateSourceId` before any `source_id` path join.
- **One canonical chat-pricing table.** All paid-cloud chat/completion prices live ONCE in
  `src/core/model-pricing.ts` (`CANONICAL_PRICING` + `canonicalLookup`). Every other table
  (`anthropic-pricing.ts`'s `ANTHROPIC_PRICING`, the contradictions/cross-modal/skillopt cost
  views) is a DERIVED view, never a hand-copied duplicate — so cross-table price drift is
  structurally impossible (takes-quality reads `pricing.overrides`, then canonical). Update a
  price in `model-pricing.ts` only; each consumer keeps its own miss policy, not its own numbers. Pinned by `test/model-pricing.test.ts`
  (drift guard asserts each view equals canonical). Embeddings price separately in
  `embedding-pricing.ts` (different unit).
- **Module-size ratchet.** `scripts/module-size-limits.tsv` pins per-file line ceilings
  (`check:module-size` in verify): growth over a ceiling, >50 lines of stale slack after a
  shrink, a row for a deleted file, and any UNLISTED src file over 1,500 lines all fail.
  Raise a ceiling only via a reviewer-visible TSV edit in the same commit; lower it in the
  same commit as any peel. `check:function-size` ratchets functions the same way
  (`scripts/function-size-baseline.tsv`): no new function over 300 lines.
- **Peeled façades keep their surface.** operations.ts (`src/core/ops/*`), both engines
  (`src/core/engine-sql/*`, `src/core/{postgres,pglite}-engine/*`), migrate.ts
  (`src/core/schema-migrations/*`), doctor.ts (`src/commands/doctor/registry.ts` +
  `checks/*`), sync.ts (`src/commands/sync/*`; reusable clusters in `src/core/sync-*`),
  serve-http.ts (`serve-http-<area>.ts` `mount<Area>(app, ctx)` modules), cli.ts
  (`src/cli/command-table.ts` + `src/cli/commands/*`), jobs.ts (`src/commands/jobs/*`,
  handlers in `src/core/minions/handlers/*`), hybrid.ts (`src/core/search/hybrid/*` stages),
  autopilot.ts and skillpack.ts (`src/commands/skillpack/*`) re-export everything they always
  exported — import sites and published package exports never chase the peel. New code goes
  in the module dirs, not back into the façades (CONTRIBUTING.md "Where does my change go?").
  Engine modules take narrow explicit deps (never an engine-shaped bag); source-text guards
  read `test/helpers/source-surface.ts`, and the flag-registry generator's `facadeExpansion`
  keeps peeled flag text in each command's scan surface.
- **Coverage is measured, honestly.** CI merges per-lane lcov (`scripts/merge-lcov.ts`) into
  a PR-corpus report on every run (advisory until the diff gate graduates via
  `COVERAGE_GATE_ENFORCE`) and a nightly fullCorpus number incl. the full e2e glob. bun
  facts: unique `--coverage-dir` per process (reuse overwrites lcov.info), line records only
  (JSC omits function names), no subprocess coverage (cli.ts is exempt as a documented
  undercount), never-loaded files are a count+list, never fake all-files math.


## Reference map (load on demand)

CLAUDE.md is the always-loaded orientation + dispatcher. Detailed reference loads
on demand — read the linked doc before working in that area.

| When you're working on... | Read first |
|---|---|
| any file in `src/` (what it does + its invariants) | `docs/architecture/KEY_FILES.md` — choose the subsystem, then find the file's entry |
| search / ranking / hybrid / retrieval | `docs/architecture/RETRIEVAL.md` + the `search/*` entries in `KEY_FILES.md` |
| search modes / cost knobs | `docs/guides/search-modes.md` |
| engine detection / Postgres adoption / DB-access repair / degraded serve (`engine status`, `db-repair`, `init --prefer-postgres`, `GBRAIN_DB_ACCESS`) | `docs/ENGINES.md` ("Engine detection and access repair" + "Local Postgres") |
| embedding spend gates / cost gate / `spend.posture` / off switches | `docs/operations/spend-controls.md` |
| the monthly backup-coverage check (`gbrain backup`, render channels, nag budget) | `docs/operations/backup-check.md` + the `backup/*` entries in `KEY_FILES.md` |
| push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` |
| checkpoint compaction / compiled context files (`gbrain compile-context`) | `docs/guides/checkpoint-compaction.md` + `docs/guides/ambient-recall.md` |
| ambient memory writeback (opt-in unprompted fact capture — `memory.auto_writeback`, harness instruction blocks, Stop-hook backstop, read-time TTL) | `docs/guides/ambient-writeback.md` + the ambient-writeback cluster in `KEY_FILES.md` |
| Memorable integration / session receipts / relay consent (`integrations.memorable.*`) | `docs/memorable-agents.md` + the hook-heartbeat/capture-spec/codex-hooks entries in `KEY_FILES.md` |
| chat connectors (live ChatGPT/Claude history sync — `gbrain connectors`) | `docs/guides/chat-connectors.md` + the `src/core/connectors/*` entries in `KEY_FILES.md` |
| schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` |
| thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` |
| publishing the brain's MCP server to other devices and agents (`gbrain mcp expose`, Tailscale default, Grok Bot / Muse hosted path) | `docs/guides/remote-mcp.md` + `docs/mcp/DEPLOY.md` + the `remote-mcp` skill |