jev-email-analysis

Files (31)

• README.md
• SKILL.md
• config.example.json
• controls.json · evals/
• field_check.py · evals/
• sample-api-message.json · mock-data/
• sample-paste.txt · mock-data/
• sample.eml · mock-data/
• methodology.md · references/
• schema.md · references/
• README.md · rules/
• _core.toml · rules/
• advance_fee_or_prize.toml · rules/
• callback_phishing.toml · rules/
• cold_outreach.toml · rules/
• credential_harvest.toml · rules/
• executive_impersonation.toml · rules/
• extortion.toml · rules/
• fake_invoice.toml · rules/
• legitimate_business.toml · rules/
• malware_delivery.toml · rules/
• oauth_consent_phish.toml · rules/
• payroll_diversion.toml · rules/
• qr_phishing.toml · rules/
• reconnaissance.toml · rules/
• spam_marketing.toml · rules/
• vendor_payment_fraud.toml · rules/
• analyze.py · scripts/
• ingest.py · scripts/
• jev.py · scripts/
• rules.py · scripts/

SKILL.md (first lines)

# Jev Email Analysis

> **Research proof of concept.** This skill asks TypeSafe's Jev what an email is trying to
> get someone to do. It is a **second reader for a person or another agent, not a mail
> gateway**: `malicious` means "do not act on this without verifying out of band", not
> "block the sender", and `benign` means "nothing in what you sent stood out", not "this
> message is safe". It does not check authentication, resolve domains or open attachments -
> feed it those results as evidence. Repeat the short form of this at the end of every run.

## What it does

1. **Take the email in whatever form it arrived.** The contract is the schema in
   [`references/schema.md`](references/schema.md), and **every field in it is optional**.
   `scripts/ingest.py` converts the three mechanical formats (`.eml`/mbox, JSON in any shape,
   a pasted blob). Anything else - a screenshot, a PDF, a ticket, a chat transcript - you
   transcribe yourself. Every run prints which fields it actually got.
2. **Redact, then judge.** `scripts/analyze.py` strips IBANs, card/account runs and any 6+ digit
   run **before** the request leaves the machine, then sends one request per email: a 16-way
   category choice, eight yes/no red-flag signals, and risk and pressure on 0-3 scales. Email
   text is framed to Jev as untrusted data. About 2,500 tokens and ~1.5 s for 25 emails in
   parallel.
3. **Roll up in code.** `derive()` turns the typed answers into `malicious` / `suspicious` /
   `spam` / `benign`. Category decides, risk and flags adjust, and **an identity mismatch only
   counts when identity was assessable at all** - with no `auth` and no history, code records
   `identity_unknown` and the content has to carry the verdict.
4. **Report.** You read the verdicts and tell the user what each flagged message is trying to
   do, in plain words, with the evidence.

Nothing about any category is hard-coded. Everything lives in [`rules/`](rules/README.md): one
`<category>.toml` per category (the option Jev chooses between, the verdict it rolls up to, its
regex facts) and `_core.toml` (the model, the framing, the scales, the eight shared red-flag
questions, the verdict thresholds, the facts every email gets). Pass `--rules <dir>` to add or
replace categories without forking.

## Workflow

`SKILL_DIR` means the folder containing this SKILL.md.

### 1. Understand what the user wants

- **"Is this phishing?"** - one message, usually pasted or screenshotted. Answer with the
  verdict, the category, the red flags, and what to do.
- **"Triage these"** / **"check my inbox"** - a batch. Report attention-first; do not list
  every benign message.
- **"Does it hold up on our mail?"** - an evaluation. Use `evals/field_check.py` (step 5).

If the user has a mailbox connected and asks about "my email", work out the scope with them
(how far back, which folder). Pulling a whole mailbox is rarely what they meant.

### 2. Get the email into the schema

**This skill does not retrieve mail and does not dictate how you do.** Use whatever the
environment gives you - an MCP mail server, an API, IMAP, a file the user dropped, a screenshot
they pasted. Map what you get onto the schema in `references/schema.md` and write the staged
items into your scratchpad, never into the user's working tree.

For the mechanical formats:

```bash
python3 "$SKILL_DIR/scripts/ingest.py" --eml  msg.eml          # or a directory, or an .mbox
python3 "$SKILL_DIR/scripts/ingest.py" --json export.json      # schema items, a mail API
                                                               # message, or flat records
python3 "$SKILL_DIR/scripts/ingest.py" --text pasted.txt       # "-" reads stdin
```

`--json` is deliberately tolerant: it accepts items already in the schema, a MIME-style payload
tree (`payload.headers` / `payload.parts`, as several mail APIs return), or flat records, and it
matches field names by alias (`from`/`sender`, `replyTo`/`reply_to`, `plainTextBody`/`body`, …).
If your source does not fit, do not force it - just write the schema JSON directly. That is the
supported path, not a fallback.

**For a screenshot, a photo, a PDF or anything else code cannot parse:** you read it, and you
transcribe it. Put the visible header lines and body into the `--text` form, or write the schema
object yourself. Then say what you could not capture:

```
provenance: "screenshot of a mail client; sender and authentication headers not visible"
```

**Every field is optional, and partial input is a first-class case.** A subject and a body are
enough. The two rules that make that safe:

- **Absent is not adverse.** Write `auth: "not recorded"` or leave it out. Never write `"fail"`
  or `"none"` for a check that was never performed - a NULL auth column in a log export is "not
  recorded", and calling it a failure invents a threat that is not there.
- **Never put a reference labeller's opinion in `state`** - not "Gmail marked this spam", not
  "our gateway quarantined it". That leaks the answer into the question. Keep it in your labels.

Read the coverage line `ingest.py` prints. If it warns that `auth` and `relationship` are both
absent, say so in your report: the run could not assess sender identity, and the verdicts rest
on content alone.

### 3. Check prerequisites

```bash
python3 --version   # 3.10 or newer
test -n "$TYPESAFE_API_KEY" || test -f ~/.config/typesafe/env || echo "need a TypeSafe key"
```

Nothing to install; the TypeSafe call is plain `urllib` with backoff on 429/529/5xx. Keys come
from https://console.typesafe.ai/keys. `analyze.py` exits **3** when there are no credentials, so
a caller can treat the whole step as "skipped" rather than failed.

Optionally tell Jev who the recipient is, which sharpens internal-impersonation and look-alike
judgments: `--org-name "Acme" --own-domains acme.com,acme.io`, or
`~/.config/jev-email-analysis/config.json` (see `config.example.json`). Nothing org-specific
lives in this folder.

### 4. Run

```bash
python3 "$SKILL_DIR/scripts/analyze.py" --input items.json --out results.json \
  --org-name "Acme" --own-domains acme.com

python3 "$SKILL_DIR/scripts/ingest.py" --text - | \
  python3 "$SKILL_DIR/scripts/analyze.py" --stdin        # one pasted email, no temp file
```

Before changing any criterion, run the bundled controls and keep them passing:

From SecurityMindedSolutions/ai-skills, MIT.

More from SecurityMindedSolutions/ai-skills