CLAUDE.md · idank/explainshell

Sections

1. Tech Stack
2. Workflow Requirements
3. Code Style
4. Environment
5. Common Commands
6. Project Structure
7. Architecture

First lines (shown under the repo's GPL-3.0 license)

# Project Instructions — explainshell

A web tool that parses man pages and explains command-line arguments by matching each argument to its help text.

## Tech Stack

- Python 3.12, Flask, SQLite, bashlex, OpenAI SDK, Google Gemini SDK, LiteLLM (fallback)
- Linting: ruff (Python), biome (JS)
- Testing: pytest (unit + doctests), JS Playwright Test (e2e)
- Dependencies: `requirements.txt` (web serving), `requirements-extraction.txt` (LLM SDKs, offline tooling only), `requirements-dev.txt` (everything + test/lint), `package.json` (Playwright e2e)

## Workflow Requirements

**Before finishing any task**, always:

1. Run `make format`
1. Run tests — choose the right suite based on what changed:
   - `make tests-quick` (lint + unit) — use when changes clearly cannot affect what the web app serves (e.g., extraction pipeline, CLI tooling, tests themselves)
   - `make tests-all` (lint + unit + e2e) — use when changes might affect the web serving path (rendering, matching, storage, templates, static assets, config)
   - When in doubt, run `make tests-all`
   - If e2e tests fail due to snapshot diffs, assess whether the diff is expected, and get user confirmation before running `make e2e-update`
1. Update README.md if the change adds/removes/renames CLI commands, env vars, or user-facing features
1. Update CLAUDE.md if the change affects structure, convention, workflow, etc.

### LLM Evaluation

Use the LLM eval (`tests/evals/llm/llm_eval.py`) to compare before/after metrics when making changes to the LLM extractor. It runs extraction on the corpus listed in `tests/evals/llm/corpus.txt` (paths into the `manpages/` submodule) and writes `summary.json` plus per-page artifacts under `markdown/`, `prompts/`, and `responses/` to a timestamped directory under `tests/evals/llm/runs/`. Summaries include git metadata, model, label, description, aggregate metrics (extracted/failed files, total options, zero-option pages, multi-chunk pages, token usage), and per-page metrics keyed by repo-relative path.

`run` requires `--label <tag>` (folded into the run dir name) and accepts `-d "..."` for a longer description. Always pass a meaningful label and description inferred from context — e.g. the task you're working on, or "baseline" for a pre-change run — so `list` and `compare` output stay self-explanatory.

**Workflow for code changes (API, prompt, chunking, post-processing):**

```bash
# 1. Stash your changes to get a clean baseline
git stash push -- explainshell/extraction/llm/

# 2. Run on the old code
python tests/evals/llm/llm_eval.py run --label baseline --model codex/gpt-5.6-sol/medium --jobs 10 -d "baseline before <short summary of change>"

# 3. Restore your changes
git stash pop

# 4. Run on the new code
python tests/evals/llm/llm_eval.py run --label change --model codex/gpt-5.6-sol/medium --jobs 10 -d "<short summary of change>"

# 5. Compare the two run directories (oldest first)
python tests/evals/llm/llm_eval.py compare tests/evals/llm/runs/<baseline-run> tests/evals/llm/runs/<change-run>
```

**Usage:**

```bash
# Run on the default corpus, parallelizing realtime calls
python tests/evals/llm/llm_eval.py run --label smoke --model codex/gpt-5.6-sol/medium --jobs 10

# Run on specific files (overrides --corpus)
python tests/evals/llm/llm_eval.py run --label probe --model codex/gpt-5.6-sol/medium --jobs 10 path/to/file.1.gz

# Use --batch <size> instead of --jobs to route through the provider's batch API
# (cheaper, but minutes-to-hours of queue latency; pays off only on much larger corpora).

# Compare two run directories
python tests/evals/llm/llm_eval.py compare tests/evals/llm/runs/<baseline-run> tests/evals/llm/runs/<current-run>

# List all saved runs
python tests/evals/llm/llm_eval.py list
```

## Code Style

- Use Python type annotations on all new code (function signatures, return types, and non-obvious variables). Do not retroactively annotate existing code unless you are already modifying it.

## Environment

- Python virtualenv: repo-local `.venv`
- **CRITICAL**: Every Bash tool call runs in a fresh shell with NO venv active. You MUST prefix every Python/pip/pytest/ruff/make command with `source .venv/bin/activate &&`. Example: `source .venv/bin/activate && make tests`. Never run bare `python`, `pytest`, `ruff`, `pip`, or `make` without activating first.

## Common Commands

```bash
# Run unit tests + doctests (excludes e2e)
make tests

# Run a single test file
pytest tests/test_matcher.py -v

# Run a single test method
pytest tests/test_matcher.py::test_matcher::test_no_options -v

# Lint
make lint

# Format
make format

# Run e2e tests (requires playwright)
make e2e

# Update e2e snapshots
make e2e-update

# Run LLM integration test (requires API key in .env)
make test-llm

# Run quick tests (lint + unit, no e2e)
make tests-quick

# Run all tests (lint + unit + e2e)
make tests-all

# Run DB integrity checks
make db-check

# Run web server locally
make serve

# Generate Ubuntu manpage archive (requires Go)
make ubuntu-archive UBUNTU_RELEASE=resolute

# Generate Arch Linux manpage archive (requires manned.org dump)
make arch-archive

# Process a man page into the database
python -m explainshell.manager extract --mode llm:codex/gpt-5.6-sol/medium /path/to/manpage.1.gz
```

## Project Structure

- `explainshell/` - Main package
  - `manager.py` - CLI entry point for man page processing (`python -m explainshell.manager <command>`)
  - `db_check.py` - Database integrity checks (used by `manager.py db-check`)
  - `matcher.py` - Core logic: walks bash AST and matches tokens to help text
  - `models.py` - Core domain types (Option, ParsedManpage, RawManpage) as Pydantic/dataclass models
  - `store.py` - SQLite storage layer
  - `caching_store.py` - Read-only size-aware cached Store variant for production web serving; when `DEBUG=false`, the Flask app stores one per worker process in `app.extensions`
  - `errors.py` - Exception hierarchy (ProgramDoesNotExist, DuplicateManpage, InvalidSourcePath, ExtractionError, SkippedExtraction, FatalExtractionError)
  - `diff.py` - Man page comparison and diff formatting
  - `roff_parser.py` - Roff macro parser (used by `roff_utils.py` for detection helpers)
  - `roff_utils.py` - Roff source detection (dashless opts, nested cmd)
  - `manpage.py` - Man page reading and HTML conversion
  - `help_constants.py` - Shell constant definitions for help text
  - `logger/` - JSON stderr logging for prod under gunicorn (wired in `create_app`); dev uses plain `logging.basicConfig` via runserver.py
  - `util.py` - Shared utilities (group_continuous, Peekable, name_section)
  - `config.py` - Configuration (DB_PATH, HOST_IP, DEBUG, MANPAGE_URLS)
  - `extraction/` - Man page option extraction pipeline
    - `__init__.py` - Public API: `make_extractor(mode)` factory
    - `types.py` - Shared types (ExtractionResult, ExtractionStats, BatchResult, ExtractorConfig, Extractor protocol)
    - `runner.py` - Execution orchestration (sequential, parallel, batch)
    - `common.py` - Shared metadata assembly for all extractors
    - `prefilter.py` - Pre-extraction classification of input .gz files (size, symlink, --filter-db, already-stored, content-dup)
    - `postprocess.py` - Extractor-agnostic option post-processing
    - `llm/` - LLM-based extraction subpackage
      - `extractor.py` - LLM extractor orchestration
      - `prompt.py` - Prompt construction
      - `response.py` - LLM response parsing
      - `text.py` - Man page text preparation and chunking
      - `providers/` - LLM provider implementations (OpenAI, Gemini, LiteLLM fallback)
  - `web/views.py` - Flask routes with URL-based distro/release routing
- `tools/` - Standalone scripts
  - `fetch_manned.py` - Fetch man pages from manned.org weekly dump