feature-verify

Files (5)

• SKILL.md
• blackbox-testing.md · references/
• environments.md · references/
• output-template.md · references/
• safety-rules.md · references/

SKILL.md (first lines)

# Feature Verify — Runtime-First API Verification

## Trigger

- Keywords: verify, investigate, diagnose, check if working, post-deploy, smoke test, validate
- User wants to confirm deployed feature behavior
- User provides environment access (API URL, log system, credentials)

## When NOT to Use

| Need | Use Instead |
| ---- | ----------- |
| Modify data or state | `/feature-dev` |
| Code quality review | `/codex-review-fast` |
| Generate unit tests | `/codex-test-gen` |
| Security audit | `/codex-security` |
| Run local tests | `/verify` |
| Review test coverage | `/codex-test-review` |

## Core Principle

```
⚠️ ALL OPERATIONS MUST BE READ-ONLY ⚠️

Claude independent analysis → Codex third-perspective confirmation → Integrated verdict
```

> **Tool safety note**: `allowed-tools` includes `Bash` for curl/log queries. Read-only enforcement is behavioral — all commands MUST be reviewed against `references/safety-rules.md` before execution. Codex independently verifies compliance at P5.

## Degradation Matrix

Auto-detect from `references/environments.md` configuration:

| Level | Available Resources | P3 API | P4 Observation | Confidence Cap |
| ----- | ------------------- | ------ | -------------- | -------------- |
| **L4** | API + Log + Metrics | Full | Log + Metrics | High |
| **L3** | API + Log | Full | Log only | High |
| **L2-API** | API only | Full | Response-only | Medium |
| **L2-OBS** | Log only (API unreachable) | Skip | Time-window scan | Medium |
| **L1** | No runtime access | Skip P3/P4 | Code review only | Low |

**Auto-detection logic** (see `references/environments.md` § Degradation Detection):

| API Status | Log System | Metrics | Level |
|------------|------------|---------|-------|
| Reachable | Yes | Yes | L4 |
| Reachable | Yes | No | L3 |
| Reachable | No | — | L2-API |
| **Unreachable** | **Yes** | — | **L2-OBS** |
| Unreachable | No | — | L1 |

**Fail-closed**: If Endpoint Allowlist section is missing, skip P3 (cannot call unverified endpoints). At L1, skip P3 and P4. Provide code-review-based analysis only with Low confidence. At L2-OBS, skip P3 (API unreachable); execute P4 time-window scan and background service observation only.

## Workflow

```mermaid
sequenceDiagram
    participant C as Claude
    participant U as User
    participant API as Target API
    participant Log as Log System
    participant Cx as Codex

    C->>C: P0: Scope & Safety
    C->>C: P1: Diff-Lite Scoping
    C->>U: P2: Test Charter (approve?)
    U->>C: Approved
    C->>API: P3: API Execute (read-only)
    C->>Log: P4: Observation Correlate
    C->>Cx: P5: Codex independent review
    Cx-->>C: Codex verdict
    C->>U: P5: Integrated Verdict Report
```

## P0: Scope & Safety

Read [safety-rules.md](references/safety-rules.md) and [environments.md](references/environments.md).

| Check | Method | Fail Action |
| ----- | ------ | ----------- |
| Environment select | `--env` flag or ask user; load from `references/environments.md` | Default to test |
| Read-only confirmed | Review `references/safety-rules.md` and load the endpoint allowlist. **This row runs before any request is made** — see below | — |
| API reachable | Deterministic health-check (3x, 2s timeout — see `references/environments.md`) | Unreachable + Log config → L2-OBS; Unreachable + no Log → L1 |
| Deployment aligned | Compare local HEAD with deployed version | Mismatch → warn, lower confidence |
| Degradation level | Check `references/environments.md` for log/metrics config | Set level (L1-L4) |

**The health check is a request, so the allowlist gates it too.** A reviewer found this skill
calling the configured health endpoint *before* enforcing its own deny-all policy — which is the one
request the policy could never have approved, because nothing had loaded the allowlist yet. Validate
the health endpoint and its method against the allowlist first; if the allowlist is missing, or the
health endpoint is not on it, **make no request** and degrade on that basis (unreachable-equivalent),
recording why. "It is only a health check" is exactly the reasoning the deny-all policy exists to
refuse.

## P1: Diff-Lite Scoping

Read [blackbox-testing.md § P1](references/blackbox-testing.md#p1-diff-lite-scoping).

**Scope only — no code quality judgment.**

1. Get diff: `git diff main...HEAD --name-only` (or user-provided scope)
2. Map changed files → affected endpoints → dependency chains
3. Identify L1 regression endpoints, L2 trigger cases, L3 passive targets

**Fallback**: If no git diff available, ask user for feature description and build scope manually.

**`--level` override**: If user passes `--level L2-API`, skip log/metrics cases even if configured. `--level L2-OBS` forces observation-only mode. `--level L2` defaults to `L2-API` for backward compatibility.

## P2: Test Charter

Read [blackbox-testing.md § P2](references/blackbox-testing.md#p2-test-charter-design).

Generate test cases dynamically from P1 results:

| Type | Goal | When |
| ---- | ---- | ---- |
| **L1 Regression** | Affected API returns expected results | L2-API+ (N/A for L2-OBS) |
| **L2 Active Trigger** | New code path exercised, verify response | L2-API+ (N/A for L2-OBS) |
| **L3 Passive Observe** | Background service running, check logs | L3+ only |

From sd0xdev/sd0x-harness, MIT.

More from sd0xdev/sd0x-harness