authenticity-check

Files (9)

• LICENSE
• README.md
• SKILL.md
• calibration-cases.md · references/
• common-core.md · references/
• do-not-flag.md · references/
• scoring.md · references/
• tell-patterns.md · references/
• voice-matching.md · references/

SKILL.md (first lines)

# Authenticity Check

Read a piece of text and report how authentically it reads as the work of a
person, then point at the exact spans that do not. The deliverable is a
judgment plus evidence: an authenticity band, a 0-100 score, and a list of
flagged spans with the reason each one reads as AI-generated, AI-templated, or
generically derivative. The goal is an honest read a careful human editor
would agree with, not a number tuned to beat any particular detector. This
skill names and targets none.

This is the evaluative half of a pair. The other half, the `humanizer` skill,
is a separate repository that rewrites prose. This skill diagnoses; it does
not rewrite. See "Diagnostic only" below for why that boundary is hard.

## When to use this

Use it when someone wants to *know* something about a text rather than change
it: "is this AI," "does this read like a real person," "how human does this
sound," "which parts sound machine-written," "rate the AI-ness," "audit this
draft for AI tells," or "does this still sound like me." Use it even when they
do not say "authenticity check" and even when the cue is oblique ("does this
sound like a bot," "something about this feels generated, can you tell"). It
also pairs naturally as a verification step before and after a humanizer pass.

Do not use it to help defeat a plagiarism or AI-detection system, or to tune
text against a named detector. See "Scope and intended use" below; reframe
such requests toward an honest read of quality and voice, which is what this
skill actually delivers.

## Diagnostic only: the hard boundary

This skill scores and flags. It does not rewrite, edit, paraphrase, or hand
back "improved" prose, not even a single suggested replacement sentence. If
the user wants the flagged prose fixed, that is the `humanizer` skill's job,
performed as a separate, human-judged step.

The split is intentional and load-bearing. A single tool that scores text and
then rewrites it to raise its own score is a detector-gaming loop: it optimizes
prose against a metric instead of improving it for a reader, which is exactly
the failure the humanizer skill explicitly refuses. Keeping diagnosis and
transformation in separate skills, with a human deciding between them, is what
prevents that loop from forming. So the correct shape is a pair, never a
merge:

```
authenticity-check  ->  human reads the flags and decides  ->  humanizer
   (diagnose)              (judgment, not automation)          (rewrite)
        ^                                                          |
        |__________________ re-verify, fresh read ________________|
```

The re-verify pass is a fresh diagnosis of the rewritten text, not a
continuation of the first one. This skill never carries a "target score" into
a rewrite and never instructs the humanizer skill on how to move a number.

## Core principle: uniformity is the signal

The reason AI prose reads as AI is not its vocabulary. It is its uniformity:
even sentence lengths, even rhythm, even hedging, the same shapes resolved the
same way. This is the same principle the humanizer skill is built on, read
from the other side. Humanizer's job is to add genuine structural variance;
this skill's job is to detect the absence of it.

Two consequences follow, and both matter for an honest score:

1. **Flag the signature, not the vocabulary.** A single "delve" is not a tell.
   Uniform rhythm across a whole passage is. Weigh recurrence and
   co-occurrence far above any lone word.
2. **A relocated signature still counts.** Text that swapped every "delve" for
   "explore" but kept the even rhythm is not more authentic; it just moved its
   fingerprint. Do not let surface-level word choice raise a score that the
   underlying uniformity should hold down.

## Step 0: discover the baseline (do this every run, before scoring)

Decide which mode you are in. Announce it in the output header so a wrong
guess is instantly correctable.

1. **Is this a "sounds like me" question?** If the user is asking whether the
   text reads like a specific author (themselves or a named writer) and a
   voice source is available (a pasted sample, a named well-known author, or a
   discovered profile file), enter **voice-deviation mode**. Read
   `references/voice-matching.md` for how to read a voice off a sample or
   profile; here you read it to measure *deviation from* that voice, not to
   reproduce it.
2. **Look for a profile.** Glob for a voice-profile file — see
   `references/common-core.md` for the priority order to search in. One
   strong match plus a "sounds like me" intent puts you in voice-deviation
   mode. Several plausible matches: ask one short question to pick.
3. **Otherwise, generic mode.** No voice target. Score AI-authenticity against
   the catalog and the internal-consistency heuristics only. Do not invent a
   persona to measure against; an imagined target voice is its own failure.

This discovery is filesystem-generic. It interoperates with Scriven, Pillars,
or any project that keeps a voice file, without depending on any of them, and
without assuming this skill is the only one installed.

## Step 0b: density pre-check (match scrutiny to evidence)

Before scoring, skim the whole text once and judge how heavily it is
AI-marked. Count only the dead-giveaway tells catalogued in
`references/common-core.md`, not the weak surface signals. Judge the overall
density, then set the scrutiny level and announce it in the output header:

- **Low: light scrutiny**, unless the rhythm is uniform and no human markers
  are present (the relocated-signature override below). Likely human-first
  text (a journal, rough notes, a real draft). Bias hard toward a high score
  and a near-empty flag list. Over-flagging genuine human prose is the worst
  error this skill can make; when density is low, restraint is the default.
  But low lexical density is evidence of a human only when the text also
  carries human markers or variance; clean vocabulary with even rhythm and no
  specificity is laundered prose, not a low-density human draft.
- **Medium: standard scrutiny.** Mixed authorship. Run the full catalog with
  the restraint of do-not-flag.md.
- **High: full scrutiny.** AI-first text. Run the full catalog thoroughly and
  weigh the internal-consistency heuristics fully.

This is calibration, not a number you report mechanically, not a per-100-word
count. One exception overrides density either way: any chat-UI contamination

From fbserg/skill-issue, MIT.

More from fbserg/skill-issue