the-controller-e2e-eval

Files (1)

• SKILL.md

SKILL.md (first lines)

# E2E Eval — User Story Validation

## When to Use

You've made a UI change and need to prove it works from the user's perspective — not "it compiles" or "the DOM element exists," but "the user can do the thing and see the result."

## Hard Rules

These are non-negotiable. Every test must follow all of them.

### 1. User story first, always

Before writing a single line of test code, state the user story:

- **Actor**: Who is performing the action (e.g., "a user in Architecture mode")
- **Action**: What they physically do (e.g., "presses `r`")
- **Outcome**: What they expect to see (e.g., "an architecture diagram renders in the diagram pane")

The test's **final assertion** must verify the **outcome**. If the outcome doesn't happen, the test fails. No exceptions.

### 2. No silent passes

**BANNED** — the `if (visible)` escape hatch:

```typescript
// NEVER DO THIS — silently passes when the feature is broken
const visible = await foo.isVisible().catch(() => false);
if (visible) {
  await expect(foo).toHaveText("expected");
}
```

**REQUIRED** — unconditional assertion with a timeout:

```typescript
// DO THIS — fails loudly when the feature is broken
await expect(foo).toBeVisible({ timeout: 10_000 });
await expect(foo).toHaveText("expected");
```

If the element should be there, assert it. If it might legitimately not be there, that's a different test or a precondition you need to set up.

### 3. No mocking the interface

- **No `page.evaluate()` to set store state** when a real interaction exists. If the user presses Space to open the workspace picker, the test presses Space.
- **No synthetic `dispatchEvent(new KeyboardEvent(...))`** — use `page.keyboard.press()`.
- **No stubbing API responses** — the test runs against real servers via `eval.sh`.

The only acceptable use of `page.evaluate()` is to **read** state for assertions (e.g., checking a computed style), never to **write** state that bypasses the UI.

### 4. Assert outcomes, not implementation

**WRONG** — asserting CSS values and class names:

```typescript
// This proves the DOM exists, not that the feature works
await expect(page.locator(".diagram-pane")).toBeVisible();
await expect(page.locator(".generate-action")).toBeVisible();
```

**RIGHT** — asserting what the user sees after the action:

```typescript
// This proves the feature works end-to-end
await page.keyboard.press("r");
await expect(
  page.locator(".diagram-pane").locator("svg, canvas, .mermaid, img")
).toBeVisible({ timeout: 60_000 });
```

Structure checks (element exists, layout is correct) are fine as **preconditions** or **smoke tests**, but they are not user story validation. The final assertion must be about the **result of the user's action**.

### 5. One story per test

Each `test()` block validates exactly one user story. Don't bundle "switch to architecture mode AND generate a diagram AND switch back" into one test. The switch-back is cleanup, not a story.

### 6. Timeouts reflect reality

If an action triggers an async operation (e.g., AI-generated diagram, API call), the timeout must be long enough for it to complete in CI. Use 60s for AI/generation operations, 10s for navigation, 30s for server-dependent renders.

## Steps

### 1. Commit your changes

`eval.sh` requires a clean worktree. Commit all your work before proceeding.

### 2. Determine the worktree path

```bash
WORKTREE_PATH=$(pwd)
```

Verify it's a git worktree (not the main repo):

```bash
if [[ -f "$WORKTREE_PATH/.git" ]]; then
  echo "Worktree: $WORKTREE_PATH"
else
  echo "WARNING: Not a worktree — you may be in the main repo. Proceeding anyway."
fi
```

**Do NOT** look up `staged_session` in project.json or ask the user to press `v` — that is unrelated to e2e eval.

### 3. Define the core user story

State the user story explicitly before writing test code:

```
Actor:   A user in Architecture mode with a project loaded
Action:  Presses 'r' to generate the architecture diagram
Outcome: An architecture diagram (SVG/canvas/image) appears in the diagram pane within 60 seconds
```

This is the contract. The test must not pass unless this outcome is achieved.

### 4. Write the test

Create a spec file in `e2e/specs/` that follows the hard rules above:

From kwannoel/the-controller, MIT.

More from kwannoel/the-controller