agentclash-challenge-pack-artifacts

Files (1)

• SKILL.md

SKILL.md (first lines)

# AgentClash Challenge Pack Artifacts

## Purpose
Make challenge pack files and evidence explicit without confusing three different surfaces:

- source assets declared in challenge-pack YAML
- case artifact references exposed to scoring evidence
- files produced by the agent and captured after execution

Use this skill when a coding agent needs exact artifact fields and evidence references without reading the AgentClash source repo.

## Use When
- A pack includes fixture files, expected reports, images, logs, datasets, or other stored assets.
- Cases need `inputs[].artifact_key`, `expectations[].artifact_key`, `artifact_refs`, or case `artifacts`.
- Scoring needs `artifact...` or `file:...` evidence references.
- Native execution should capture files from the sandbox through `post_execution_checks`.
- A reviewer needs to inspect artifacts after a run.

## Do Not Use When
- The pack only needs plain text final-output scoring; use `agentclash-challenge-pack-scoring-validators`.
- The task is selecting input sets or regression suites; use `agentclash-challenge-pack-input-sets` or `agentclash-regression-flywheel`.
- The task is configuring tools, sandbox network, or package access; use `agentclash-challenge-pack-tools-sandbox`.

## Environment
Use hosted production for CLI examples unless the user intentionally targets a local or self-hosted backend.

```bash
export AGENTCLASH_API_URL="https://api.agentclash.dev"
```

`agentclash challenge-pack validate` is a hosted API call: authenticate first and select a workspace with `agentclash link`, `--workspace`, `AGENTCLASH_WORKSPACE`, or local `.agentclash.yaml`. Use `agentclash-cli-setup` if the machine is not already logged in.

## Validation Commands
Validate after changing any asset, artifact reference, input artifact key, expectation artifact key, or file evidence target.

```bash
agentclash challenge-pack validate path/to/pack.yaml
agentclash challenge-pack validate path/to/pack.yaml --json
```

Human output prints `Challenge pack is valid` or `Challenge pack has errors`. Use `--json` for structured `valid` and `errors` fields.

## Asset Declaration Shape
Challenge-pack assets use the same field shape at `version.assets`, `challenges[].assets`, and `input_sets[].cases[].assets`.

```yaml
version:
  assets:
    - key: policy_pdf
      kind: document
      path: assets/policy.pdf
      media_type: application/pdf
      artifact_id: 00000000-0000-0000-0000-000000000000

challenges:
  - key: summarize-policy
    title: Summarize Policy
    category: document
    difficulty: medium
    assets:
      - key: challenge_notes
        path: assets/challenge-notes.md

input_sets:
  - key: policy-smoke
    name: Policy Smoke
    cases:
      - challenge_key: summarize-policy
        case_key: summary-basic
        assets:
          - key: case_notes
            path: assets/cases/summary-basic-notes.md
```

Source-backed fields:

- `key`: required and unique inside that one `assets` list.
- `path`: required, even when `artifact_id` is present.
- `kind`: optional string.
- `media_type`: optional string.
- `artifact_id`: optional UUID for an already stored artifact.

The current validator does not enforce enums for `kind` or `media_type`; use clear, conventional values and validate the pack.

## Version Assets Versus Local Assets
Only `version.assets` form the reference pool for artifact references.

These fields must reference a declared `version.assets[].key`:

- `challenges[].artifact_refs[].key`
- `input_sets[].cases[].artifacts[].key`
- `input_sets[].cases[].inputs[].artifact_key`
- `input_sets[].cases[].expectations[].artifact_key`
- `input_sets[].cases[].expectations[].source: artifact:<key>`

Challenge-level and case-level `assets` are validated for `key` and `path`, but they are not accepted as targets for `artifact_refs`, `artifacts`, `artifact_key`, or `source: artifact:...`. If a file needs to be referenced by scoring or case evidence, declare it in `version.assets`.

## Artifact References
Use `artifact_refs` on a challenge to say which version assets matter to that challenge. Use case `artifacts` to expose version assets as artifact evidence for a case.

```yaml
version:
  assets:
    - key: policy_pdf
      path: assets/policy.pdf
      media_type: application/pdf
    - key: expected_summary
      path: assets/expected-summary.json
      media_type: application/json

challenges:
  - key: summarize-policy
    title: Summarize Policy
    category: document
    difficulty: medium
    artifact_refs:
      - key: policy_pdf

input_sets:

From agentclash/agentclash, MIT.

More from agentclash/agentclash