AGENTS.md · LMCache/LMCache

Sections

1. AGENTS.md Ownership
2. Project Overview
3. Repository
4. Python Environment
5. Build & Install
6. Testing
7. Linting & Code Quality
8. Coding Conventions
9. AI Coding Discipline
10. Code Review Rules

First lines (shown under the repo's Apache-2.0 license)

# AGENTS.md

Guidelines for AI coding agents (Copilot, Cursor, Claude Code, etc.) working in this repository.

## AGENTS.md Ownership

`AGENTS.md` is a human-owned policy file.
Agents must not modify this file unless the user explicitly requests changes to `AGENTS.md`.
If an agent discovers recurring guidance or environment notes, it should report them in chat instead of editing this file.

## Project Overview

LMCache is a KV cache management engine for LLM serving that reduces Time To First Token (TTFT) and increases throughput. It stores KV caches across multiple tiers (GPU, CPU, disk, S3) and integrates with vLLM and SGLang.

## Repository

The default branch is `dev`. Base all new branches and pull requests against `dev`.

All commits in pull requests must include a DCO `Signed-off-by` trailer. Use
`git commit -s` for new commits; if an existing commit is missing the trailer,
amend or rebase to add it before pushing.

Agent-created pull requests must be opened as drafts. An agent must not mark a
pull request ready for review: the submitting human must first review every
changed line, run the relevant validation, confirm that the PR description and
test evidence are current, and explicitly decide that the change is ready.
Pure agent submissions without this human review are not allowed.

Every time AI-assisted code is added to an existing pull request, move the PR
back to draft before requesting review. Refresh the title and description for
the new head, regenerate the reviewer HTML explainer described below, and rerun
the relevant validation. The submitting human must then review the new diff and
explicitly mark the PR ready again. An earlier human review does not cover later
AI-generated changes.

AI-assisted PRs must include a self-contained reviewer HTML that stays
synchronized with the current head. Before creating or updating that artifact,
read and follow `.claude/skills/create-pr-explainer/SKILL.md`; it is the
authoritative workflow for content depth, bilingual presentation, diagrams,
optional motion, validation, and publication in the PR description.

## Python Environment

We recommend using [uv](https://docs.astral.sh/uv/) to manage Python environments and dependencies:

```bash
# Create and activate a virtual environment
uv venv --python 3.12
source .venv/bin/activate

# Install dependencies
uv pip install torch               # pre-requisite for CUDA extensions
uv pip install -e . --no-build-isolation
```

For tests in an already prepared Python 3.12 `.venv`, run:

```bash
uv run --no-sync --python 3.12 python -m pytest -q tests/v1/platform/test_stream.py
```

This avoids an implicit editable rebuild. If version detection selects an old
tag such as `v1.2`,
`setuptools-scm` can reject it because this repository requires `vX.Y.Z` tags.
For local source-only validation, the build-time override
`SETUPTOOLS_SCM_PRETEND_VERSION_FOR_LMCACHE=0.0.0` bypasses that lookup; do not
use the override for release artifacts.

## Build & Install

```bash
# Standard install with CUDA extensions (requires torch pre-installed)
pip install -e . --no-build-isolation

# Source-only (no native extensions)
NO_NATIVE_EXT=1 pip install -e .

# CPU-only (common C++ extensions, no GPU backend)
NO_GPU_EXT=1 pip install -e . --no-build-isolation

# HIP/ROCm build
BUILD_WITH_HIP=1 pip install -e .
```

## Testing

### Running Tests

```bash
# Run standard test suite (mirrors CI)
pytest -xvs --ignore=tests/disagg \
 --ignore=tests/v1/multiprocess/ \
 --ignore=tests/v1/distributed/ \
 --ignore=tests/skipped \
 --ignore=tests/v1/storage_backend/test_eic.py

# Run a single test file
pytest -xvs tests/v1/test_cache_engine.py

# Run a single test
pytest -xvs tests/v1/test_cache_engine.py::test_function_name
```

Test dependencies: `uv pip install -r requirements/test.txt`

Pytest marker: `@pytest.mark.no_shared_allocator` disables the shared-allocator monkeypatch for a test.

### CPU-only connector import checks

Use Python 3.12 with PyTorch and the common native extensions installed.
A fresh worktree does not include ignored native build artifacts; build them
with `NO_GPU_EXT=1 python setup.py build_ext --inplace` before running layout
and connector import tests. For these focused checks, use
`python -m pytest --confcutdir=tests/v1 tests/v1/gpu_connector/` so the root
`tests/conftest.py` GPU allocator fixtures are not loaded. This validates CPU
imports and shared layout/GDS behavior, not accelerator kernel roundtrips.
When combining these tests with `tests/inprocess/`, use `--noconftest` for
self-contained CPU checks spanning both trees; normal accelerator CI must
retain the root fixtures. Mirrored test directories with identical module
names need package `__init__.py` files to avoid pytest import mismatches.

### Testing Practices

- Write tests against the **public interface and docstring contract**, not the implementation. Test as if you don't know the internals — verify that behavior matches what the docstring describes.
- Avoid accessing private members in tests unless strongly needed.
- All new features and bug fixes should include corresponding tests.
- Ensure existing tests still pass before submitting changes.

## Linting & Code Quality

```bash
# Run all checks (mirrors CI exactly)
pre-commit run --all-files

# Individual tools
ruff check .              # Lint (E, F, B, SLF rules)
ruff format .             # Format (line-length 88)
isort .                   # Import sorting (black profile, from_first=true)
mypy --config-file=pyproject.toml   # Type checking
codespell --toml pyproject.toml     # Spell checking
```

The pre-commit config includes Rust hooks, so the run fails when `cargo`,
`rustfmt`, or `clippy` are missing (`rust-clippy` also fails on macOS while
building `io-uring`). For non-Rust changes, run
`SKIP=rust-fmt,rust-clippy pre-commit run --all-files`.

C++/CUDA files use clang-format (Google style, 80-col). Rust code in `rust/` uses `cargo fmt` and `cargo clippy`.

All Python files require an `# SPDX-License-Identifier: Apache-2.0` header as the first line.

### Import Ordering

`isort` enforces the section-heading convention (`# Standard` / `# Third Party` /
`# First Party` / `# Local`); see `docs/coding_standards.md` Section 7.2.

### SLF (Private Member Access)

SLF lint rules are currently enforced by CI only in `lmcache/v1/multiprocess/` and `lmcache/v1/distributed/`. However, **all new code should follow SLF discipline regardless of location** — never access private members (prefixed with `_`) of other classes. Treat this as a project-wide coding standard for any new or modified code.