grafel-feedback

Files (1)

• SKILL.md

SKILL.md (first lines)

# grafel-feedback

Generate an anonymized quality report that you can share with grafel maintainers
to help improve extractor coverage, resolver accuracy, and framework support — without
revealing any source code, file paths, or identifier names.

## Privacy promise

The report contains:
- **Entity name hashes**: per-report ephemeral salt (from `crypto/rand`), 4-hex output
  (e.g. `ent_a3f7`, `op_92c1`). Salt is never persisted and never logged.
- **Path templates**: `<go>/<seg-1>/<seg-2>.go` — depth preserved, all segments replaced.
- **Count ranges**: exact entity counts are bucketed (1-5, 6-20, 21-100, 100+).
- **Structural labels only**: kind names (`function`, `class`), language names, and
  framework annotation names (`@GetMapping`, `@Inject`) are not hashed — they are
  public framework vocabulary and essential for maintainers to diagnose issues.

The report does **not** contain:
- Source code (zero lines of code).
- Real file paths (depth + extension only).
- Real identifier names (hashed to 4 hex).
- Any network requests (fully offline).
- Any automatic issue creation (you decide whether to share).

## How to run

```
grafel feedback [--group <name>] [--out <path>] [--yes]
```

**Flags:**
- `--group <name>` — which group to analyse (default: inferred from your current directory).
- `--out <path>` — where to write the report (default: `~/.grafel/feedback/<group>-<timestamp>.md`).
- `--yes` — skip the confirmation prompt (useful for CI or scripting).

**Example:**
```
grafel feedback --group my-service
```

The CLI will show you what will (and will not) be collected, then ask for confirmation
before generating the report.

## Phase: synthesize the interpretation (do this after `grafel feedback` completes)

The deterministic collector emits statistics but **no interpretation** — historically every
genuinely useful finding in a feedback report was prose a human wrote by staring at the tables.
This phase makes *you*, the agent, write that prose automatically. After the `.md` report is
generated, **open it, read every section, and append a new `## Findings & Interpretation`
section to the end of the same file.**

### Privacy constraint (read first — non-negotiable)

The synthesis operates **only** on the already-anonymized report the collector produced: kind
names, counts, percentages, orphan rates, disposition vectors, framework vocabulary, and phase
timings. You must **never**:

- re-read the source tree, re-run indexing, or open any file other than the report `.md`;
- introduce a real identifier, real file path, repo name, or any string not already present in
  the anonymized report;
- infer or reconstruct a real name from a hash.

Everything you write must be derivable from the report's public framework vocabulary and its
numbers. If a hypothesis would require looking at source to confirm, phrase it as a hypothesis
for maintainers — do not go look. State this constraint explicitly at the top of the section you
write (one sentence), so a reader knows the interpretation is source-blind.

### What to write in `## Findings & Interpretation`

Produce these four subsections, in order:

#### 1. Ranked extractor / resolver gaps

Read the **Orphan Rate** table (section 2), the **Resolution Disposition** vector (section 3), and
the kind distribution (section 1). Rank the kinds with the highest orphan rate **and** a
meaningful N (ignore kinds with N < 10 — small samples are noise). For each of the top gaps, write:
a one-line **observation** (kind, orphan %, N-bucket), a **hypothesis** for the root cause, and a
**suggested fix** aimed at the extractor or resolver. Apply these concrete inference rules:

- **High-orphan data-layer kinds** (`DataAccess`, `Datastore`, `Schema`, `Model`, `Repository`,
  `Entity`/ORM entities) ⟹ likely **missing `DataAccess → Datastore` READS/WRITES edges**: the
  data-access objects are extracted but never linked to what they read/write. Suggested fix:
  audit the resolver rule that binds query/repository methods to their backing table/collection.
- **High `http_endpoint` orphan rate** ⟹ either **client repos are not in the indexed group** (so
  the endpoints have no inbound callers) **or** the endpoints use **dynamic / prefixed / templated
  URLs** the resolver can't match statically. Suggested fix: confirm the caller repos are in the
  group; if so, look at URL templating (path prefixes, base-URL constants, router-mounted subpaths).
- **Async handler kinds present but 0 callers** (e.g. `message_topic` / event handlers /
  `@KafkaListener` / queue consumers exist in the kind table but their orphan rate is ~100% or the
  disposition shows no inbound edges) ⟹ **topic → handler edges are not modeled**: the producer
  side publishes to a topic string the consumer subscribes to, but the resolver doesn't join them.
  Suggested fix: model the topic-name binding between publisher and subscriber.
- **High orphan on a kind whose framework IS recognized** (framework hit count ≥ 1 in section 4
  but the kind it should annotate is orphaned) ⟹ the **detector fires but the edge-builder for
  that framework rule is missing or partial**. Suggested fix: name the framework and the specific
  annotation, and point at the edge rule that should follow from it.

Rank by **impact = orphan_rate × N-bucket weight** (a 90%-orphan kind with N in 100+ matters more
than a 100%-orphan kind with N of 11). List the top gaps, most impactful first.

#### 2. Auto-flagged report self-inconsistencies (the D-class)

Before trusting any single number, cross-check the sections against each other and flag
**collector bugs** — a metric that contradicts another metric is a bug in the *report*, not a real
gap in the user's codebase. **Do not report a self-inconsistent metric as a genuine finding** —
flag it as a `[D] collector self-inconsistency` so maintainers fix the collector, not the extractor.
Rules:

- If a completeness / extraction / coverage metric reads **0% or "none found"** for a kind, **but
  the kind distribution table shows those entities exist** (N > 0), flag it: the entities were
  extracted, so a 0% coverage reading is a **collector measurement bug**, not a missing extractor.
- If the **resolution disposition vector does not sum to 100% ± 0.1%** (and section 7 didn't
  already catch it), flag a disposition-accounting bug.
- If **framework hits ≥ 1** for a framework but **every kind that framework should produce has
  orphan rate exactly 100% at large N**, flag a possible double-counting or detector/edge-builder
  mismatch rather than assuming the whole framework is unsupported.
- If a percentage is reported against a **denominator of 0** (e.g. "0 of 0 windows complete = 0%"),
  flag it as a divide-by-zero / vacuous metric, not a quality problem.

From cajasmota/grafel, MIT.

More from cajasmota/grafel