Skip to content
Agentmart
Submit a tool

szl-dataset-readiness

Audits a tabular dataset before training or evaluation for exact duplicate rows across train/test, the same subject, patient, cell line or sensor appearing in…

Skill by szl-holdings

IndexedApache-2.0
Write a review

Chalk Labs

Need help deploying szl-dataset-readiness?

Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.

Book a call

Install

Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install skill/szl-holdings-szl-dataset-readiness

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Audits a tabular dataset before training or evaluation for exact duplicate rows across train/test, the same subject, patient, cell line or sensor appearing in both splits, labels observed after the prediction time, and preprocessing fitted on held-out rows, all bound to actual row IDs. Use when the user is about to split data, asks 'is my test set clean', reports suspiciously high held-out accuracy, or a reviewer asks about leakage. Not a data-cleaning or imputation tool; it does not approve scientific suitability or data rights.

Use cases: testing, data, research

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this skill can do and what it asks for, from its SKILL.md and code. Checked Nov 1, 2026.

Tools
No tools found
Secrets
None asked for
Install scripts
None found
Runs as
Python package

Network hosts it talks to 1

  • codeload.github.com · found in install command

Files it touches

  • ~/.claude/skills/szl-dataset-readiness

What's inside

Skill name
szl-dataset-readiness
Description
Audits a tabular dataset before training or evaluation for exact duplicate rows across train/test, the same subject, patient, cell line or sensor appearing in both splits, labels observed after the prediction time, and preprocessing fitted on held-out rows, all bound to actual row IDs. Use when the user is about to split data, asks 'is my test set clean', reports suspiciously high held-out accuracy, or a reviewer asks about leakage. Not a data-cleaning or imputation tool; it does not approve scientific suitability or data rights.
License
Apache-2.0
Instructions
46 lines in SKILL.md · view on GitHub

Before installing

  • Ships executable scripts

Files (6)

  • SKILL.md
  • example.json · assets/
  • leakage-clean.json · assets/
  • kernel.py
  • contract.md · references/
  • run.py · scripts/

SKILL.md (first lines)

# Dataset readiness

Most impossible accuracies come from a test set that already knows the answer. This audit finds
the mechanical versions of that mistake and names the rows involved. Python 3.10+, stdlib, offline.

## Use when

- A patient, mouse, cell line, sensor or template family appears in both train and test.
- Two rows are byte-identical except for the split they landed in.
- A label (diagnosis, failure, churn) was recorded after the moment the prediction would have been made.
- A scaler, imputer or vocabulary was fitted on all rows and then "held-out" rows were scored.

## Quick start

```bash
python scripts/run.py assets/example.json
python scripts/run.py assets/leakage-clean.json
```

The first synthetic table returns `"status": "ISSUES_FOUND"`, `"readiness": "REVIEW"` with
`"issues": ["EXACT_FEATURE_DUPLICATES", "CROSS_SPLIT_FEATURE_LEAKAGE", "CROSS_SPLIT_GROUP_LEAKAGE"]`
and the offending row indexes (`[0, 2]`, one in test, one in train). The second is a clean design and
returns `NO_CHECKED_ISSUES` for every supplied check while readiness stays `REVIEW`.

## Preparing the input

Decide first what one observation is (a sample, a visit, a plate well), where predictions originate
in time, which column identifies the entity that must not straddle splits, and how the split was
designed. Then call `szl_audit_dataset` in `kernel.py` on explicitly bounded rows, or write the JSON
the CLI reads. Use content-defining feature columns that exclude row ids and split names. Add a
`leakage_spec` to check entity, temporal and preprocessing evidence; bind transform fit row ids to the
evidence you actually have, and distinguish when a label became available from when it was observed.
A `TRAIN_ONLY` declaration alone does not establish train-only fitting. Read `references/contract.md`
when preparing that manifest or interpreting missing evidence.

The helper never samples silently, never imputes missing timestamps, and never infers timezone or
column meaning. Metadata records the source revision and license as declared without verifying them.

## What it does not do

`NO_CHECKED_ISSUES` applies only to the supplied checks and selected rows; readiness remains `REVIEW`
until a human decides. Semantic near-duplicates, causal leakage through derived features, label
validity and population suitability are outside its reach and need separate investigation. Digests
are not anonymization. No network, model calls, training, upload or performance claim follows.

From szl-holdings/szl-skills, Apache-2.0.

Signals

Stars
0
Forks
0
Open issues
1
Last commit (this folder)
Oct 1, 2026 (1 month ago)
Last release
Oct 2, 2026 (1 month ago)

From GitHub (fetched Oct 24, 2026).

Details

License
Apache-2.0
First seen
Oct 24, 2026
Last checked
Oct 24, 2026

Done for you

Rather not set it up yourself?

Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.

See how we build AI agents →Read the case studies

Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.

launches shipped
50+
launches shipped
media placements
500+
media placements
founders on every account
2
founders on every account

Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block

Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.

Get szl-dataset-readiness running in your business

Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.

Goes to the Chalk Labs founders, not the newsletter. Privacy

Need help deploying szl-dataset-readiness?

CallGet help
Cite this page

szl-dataset-readiness skill. Agent Mart. https://agentmart.dev/skills/szl-holdings-szl-dataset-readiness (last updated 2026-10-24)