cdf-curation-and-retrieval

Files (24)

• BENCHMARK.md
• SKILL.md
• evals.json · evals/
• calibration-config.md · references/
• capabilities.md · references/
• configuration-decision-tree.md · references/
• context-understanding.md · references/
• cosmos-curator.md · references/
• cosmos-dataset-search.md · references/
• curation-retrieval-workflow.md · references/
• data-mining.md · references/
• distribution-analysis.md · references/
• distribution-aware-curation.md · references/
• ffmpeg-sidecar.md · references/
• gotchas.md · references/
• image-curation.md · references/
• kpi-metrics.md · references/
• restrictive-curation.md · references/
• running-pipelines.md · references/
• sam3-config.md · references/
• video-curation.md · references/
• video-lake-curation.md · references/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# CDF Curator Operator Skill

GPU-accelerated video and image curation via NVIDIA Cosmos Curator inside
**Cosmos Data Factory — Curation and Retrieval**
(`cdf-curation-and-retrieval`). This skill is a short Curator index.
Embedding handoff boundaries live in
[data-mining.md](references/data-mining.md) and
[curation-retrieval-workflow.md](references/curation-retrieval-workflow.md);
mining execution is `make help` and the repository cookbooks.

- **Video**: `split`, `dedup`, `shard`.
- **Image**: `annotate` (load → filter → embed → caption → write).
- **Handoff**: Curator IV2 or CE1 parquet that downstream mining can consume.
  See [data-mining.md](references/data-mining.md) and
  [curation-retrieval-workflow.md](references/curation-retrieval-workflow.md).
- **CDS stitch**: Curator → CDS ingest/search. Operator Path D (public GA
  `1.2.0`; not supported on Blackwell GPUs): see
  [cosmos-dataset-search.md](references/cosmos-dataset-search.md).

## Purpose

Turn raw video and image collections into curated, training-ready datasets. This
skill configures and runs cosmos-curator pipelines (clip splitting, filtering,
captioning, embeddings, SAM3 event verification, dedup, WebDataset sharding,
image annotate) and supports KPI-driven, distribution-aware, and restrictive
curation.

## Instructions

1. Classify the request as advisory, config, run, TAO handoff, or CDS
   ingest/search. Do not mix those routes. A config request with no KPI output
   or samples goes through the calibration interview, which can stop with
   unresolved requirements before any YAML is written.
2. For config work, complete the mandatory pre-flight below before writing YAML.
3. For an explicit run, load
   [running-pipelines.md](references/running-pipelines.md), validate the config,
   obtain credentials only through approved injection, then execute after
   authorization.
4. For a TAO handoff, validate Curator output and the declared embedding family
   using [data-mining.md](references/data-mining.md) before mining.
5. For CDS ingest or search, confirm CDS is healthy first, then follow
   [cosmos-dataset-search.md](references/cosmos-dataset-search.md).
6. Return the Output Format below. Never print secret values.

## Examples

- Advisory: "How much SHM should I set?" → read
  [running-pipelines.md](references/running-pipelines.md) and report guidance.
  Do not run Docker.
- Config with no KPI: complete the calibration interview in
  [calibration-config.md](references/calibration-config.md), then emit YAML.
- Run: after sample clips are staged, `make run-pipeline` with the traffic
  `split-minimal` cookbook recipe only after preflight and user authorization.
- FFmpeg missing in the container: install the host sidecar
  (`make ffmpeg-install`) per
  [ffmpeg-sidecar.md](references/ffmpeg-sidecar.md).

## Inputs

Required inputs depend on the route:

- **Advisory request:** the question plus relevant repository and config context.
- **Config request:** input and output locations, domain and goal, available KPI
  output or representative samples, and hardware constraints. If neither KPI
  output nor samples exist, complete the calibration interview before writing
  YAML.
- **Run request:** reviewed config path, data and model paths, runtime, GPU,
  and SHM constraints, and explicit authorization to execute.
- **TAO handoff:** validated artifact paths and declared embedding family.

Optional inputs include target distributions, event taxonomy, prompt choices,
existing output metadata, and user-approved operational constraints.

Resolve inputs in this order: repository configuration and validated run
artifacts; explicit prompt arguments and corrections; available agent context;
then the broad user prompt. Explicit user instructions remain authoritative
unless unsafe or incompatible, in which case stop and explain the conflict.
Never infer secret values: credentials come only from approved runtime
injection.

## Prerequisites

- **Repository access**: the agent must be able to read the Makefile, the
  `configs/` and `cookbook/` trees, and every file under `references/`.
- **GPU host** with NVIDIA drivers + `nvidia-container-toolkit`; Docker. SHM
  sized from host RAM (`SHM_SIZE`, default 24gb).
- **`cosmos-curator` image**: `make pull` uses the pin configured by the
  example env file and Make. No separate product engine image. Source
  builds are developer-only; see
  [cosmos-curator.md](references/cosmos-curator.md).
- **FFmpeg host sidecar** for distributable images (`make ffmpeg-install`) —
  they do not bundle FFmpeg. See
  [ffmpeg-sidecar.md](references/ffmpeg-sidecar.md).
- **Credentials** as needed: inject S3 and captioning API keys at runtime
  through an approved secret manager or operator deployment mechanism. Never
  put secret values in repository files, commands, logs, or examples. The env
  file is for non-secret image and CDS profile overrides copied from the
  example env file.

## Mandatory pre-flight: do NOT emit a pipeline config without context

Before writing any `*.yaml` pipeline config, the agent MUST verify
that one of the following is true:

1. **KPI run output exists** -- read it and use
   [distribution-analysis.md](references/distribution-analysis.md),
   [distribution-aware-curation.md](references/distribution-aware-curation.md),
   and
   [configuration-decision-tree.md](references/configuration-decision-tree.md).
2. **KPI sample videos are available** for inspection / discovery --
   follow [context-understanding.md](references/context-understanding.md)
   Phase 1.
3. **No KPI of any kind** -- no baseline exists. Read
   [calibration-config.md](references/calibration-config.md) and complete its
   Phase 1 interview (Inputs / Domain / Goal / Hardware / Calibration)
   BEFORE emitting a config. The interview is binding, not advisory.

If none of these holds, or required paths, intent, or hardware constraints
remain unresolved, stop and report the unresolved requirements as `blocked`.

From NVIDIA/skills, CC-BY-4.0 AND Apache-2.0.

More from NVIDIA/skills