sensitivity-check

Files (10)

• SKILL.md
• report_template.md · assets/
• lite_engines.md · references/
• pii_reference.md · references/
• secret_reference.md · references/
• pii_scan.py · scripts/
• pii_scan_lite.py · scripts/
• scan_router.py · scripts/
• secret_scan.py · scripts/
• secret_scan_lite.py · scripts/

SKILL.md (first lines)

# Sensitivity Check

Scan text or files for PII and secrets, judge false positives, and finish
with a report whose conclusion is an explicit YES/NO. Two engine families
do the scanning: preferred engines (Presidio, detect-secrets — model-based
and battle-tested, need uv) and lite engines (single-file stdlib scripts —
run anywhere python3 exists).

## Step 1 — Clarify Scope

Confirm before scanning:

1. **Target**: inline text, one file, or many files?
2. **Check type**: PII only, secrets only, or both? Default to **both**.
3. **Locales**: note which languages or regions the content involves.
   Personal data from any region is in scope — deep reads judge all of it,
   and the note also selects the `--language`/`--locale` flags below where
   the scan scripts have locale-specific detectors.

## Step 2 — Route Content

Always route first with [`scripts/scan_router.py`](scripts/scan_router.py);
reading a huge file directly can consume the entire context window.

```bash
# Preferred (exact token counts via PEP 723 tiktoken):
uv run scripts/scan_router.py --files FILE [FILE ...]

# Without uv (counts become chars/4 estimates, flagged "estimated": true):
python3 scripts/scan_router.py --files FILE [FILE ...]

# Inline text: add --text "TEXT"; custom limit: --threshold N (default 4000)
```

| Method | Meaning | Action |
|---|---|---|
| `deep` | at or under the threshold | read the content directly (Step 3) |
| `script` | over the threshold | run scan scripts (Step 4) |
| `binary` | not UTF-8 in the first 8 KB | record the path; do not scan |

Carry any `"estimated": true` flags into the report.

## Step 3 — Deep Check (items routed to `deep`)

Read the full content and judge it directly — no scripts needed. Load
[references/pii_reference.md](references/pii_reference.md) when the check
type includes PII, and
[references/secret_reference.md](references/secret_reference.md) when it
includes secrets. For each finding record entity type, location, severity,
and a short excerpt, applying the false-positive criteria from Step 5 as
you read. Deep reading is also what covers personal names, addresses, and
birth dates when the lite engines do the script scanning.

## Step 4 — Script Check (items routed to `script`)

### 4a. Select engines (once per environment)

Run `uv --version`:

- **Succeeds** → use the preferred engines,
  [`scripts/pii_scan.py`](scripts/pii_scan.py) and
  [`scripts/secret_scan.py`](scripts/secret_scan.py).
- **Fails** → use the lite engines via `python3`,
  [`scripts/pii_scan_lite.py`](scripts/pii_scan_lite.py) and
  [`scripts/secret_scan_lite.py`](scripts/secret_scan_lite.py).

Late-failure rule: if `uv run` later fails with a *dependency-class* error
(package resolution, download, import failure — typical when offline),
treat uv as unusable and switch to the lite engine for that scan type. A
*scan-class* error (bad path, analysis exception) is not a reason to
switch.

**Verify once per environment**: run `--self-test` on each script you
selected before its first real scan. The self-test only proves the script
can run in this environment — correctness is established at design time.
If a preferred engine's self-test fails, fall back to its lite engine; if
a lite self-test fails, deep-read the affected items manually. Record any
degradation in the report.

### 4b. PII scan

```bash
# Preferred engine (add --language zh or ja for Chinese/Japanese content):
uv run scripts/pii_scan.py --file FILE [FILE ...] [--language en]

# Fallback engine (locales: generic, us, gb, cn, jp; default all):
python3 scripts/pii_scan_lite.py --file FILE [FILE ...] [--locale all]
```

If the preferred engine reports a missing spaCy model, its error message
prints the exact `--with "<model> @ <wheel-url>"` flag to append to the
`uv run` command — re-run with it (first run needs the network once).

Read [references/lite_engines.md](references/lite_engines.md) when using
the lite engine: it defines the exact entity coverage and the limitations
the report must state.

### 4c. Secrets scan

```bash
uv run scripts/secret_scan.py --file FILE [FILE ...]      # preferred
python3 scripts/secret_scan_lite.py --file FILE [FILE ...] # fallback
```

Both engines report `hashed_secret` (SHA-1) instead of raw values — read
the source at the reported line to see what was actually matched.

### 4d. Review script findings

For every finding, read at least ±10 lines (or ±200 characters) around the
flagged location and apply Step 5 before recording it as confirmed. Apply
extra scrutiny to lite-engine findings and to any keyword/entropy-based
type — shape-only detection has a higher false-positive rate than the
model-based engines.

### 4e. Binary files

List each binary path in the report; do not attempt to extract or scan.

From ryan-minato/skills, Apache-2.0.

More from ryan-minato/skills