cdf-video-data-augmentation

Files (15)

• BENCHMARK.md
• SKILL.md
• openai.yaml · agents/
• variable-distribution.json · assets/
• evals.json · evals/
• airflow-direct-api.md · references/
• outputs.md · references/
• payload-contract.md · references/
• setup-and-preflight.md · references/
• troubleshooting.md · references/
• payload.py · scripts/
• summarize_results.py · scripts/
• upload_videos.py · scripts/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# Run CDF Video Data Augmentation

Run the existing Video Data Augmentation workflow end to end. Read the
[getting-started guide](../../docs/video-data-augmentation/getting-started.md) and
[payload guide](../../docs/video-data-augmentation/payload-guide.md) before authoring a payload.
Treat the checked-in DAG, payload model, and platform manifest as the source of truth if the docs
and implementation differ.

Use the bundled references progressively:

- Read [setup-and-preflight.md](references/setup-and-preflight.md) when preparing input or checking
  a controller/compute environment.
- Read [payload-contract.md](references/payload-contract.md) before rendering or changing a
  payload.
- Read [airflow-direct-api.md](references/airflow-direct-api.md) before triggering or monitoring a
  run through the API.
- Read [outputs.md](references/outputs.md) before downloading or interpreting artifacts.
- Read [troubleshooting.md](references/troubleshooting.md) when a check or run fails, or when the
  user asks to stop a run.

The workflow prepares source videos, runs the full auto-labeling and training-export chain on the
source branch, generates Cosmos Transfer variants, runs the same chain on passing generated
videos, validates storage outputs, and shuts down run-owned services. It does not produce one
combined final dataset manifest.

## Component details

Distinguish the configuration layers before authoring a payload:

| Requested work | Route to |
|---|---|
| Augmentation container input/config YAML, model parameters, distributions, or evaluator settings | `cdf-augmentation` |
| Auto-labeling container input/config, stage options, prompts, or question banks | `cdf-auto-labeling` |
| Existing VDA DagRun `conf.payload`, mapping supported component options into it, triggering, monitoring, or results | This skill and its workflow references |
| Kubernetes/Airflow controller setup, environment audit, connections, or deployment readiness | `cdf-orchestration-setup` |
| A requested new Kubernetes DAG with added, removed, or reordered stages that no existing DAG covers; or DAG parity review against a specification | `cdf-orchestration-write-dag` |

Consult the component skill before authoring or changing its container configuration, even when
that configuration is used inside a full VDA run. Component YAML and the Airflow trigger payload
are different contracts; do not put an entire container config directly into `conf.payload`.
Changing supported payload values alone does not require a new DAG. Route structural changes to
the DAG-authoring skill only when requested; setup/deployment remains separately authorized.

When the request needs component configuration or diagnosis, consult that component's
documentation, and do not invent keys or values. The links below are pinned to the latest `main`
commit of each component repository as of 2026-10-20; a component skill installed separately may
describe a different version.

| Component | Covers | Documentation at that commit |
|---|---|---|
| `cdf-augmentation` | Cosmos configuration, distributions, captioning, evaluator setup, and component errors | [`cdf-augmentation` skill][augmentation-skill] |
| `cdf-auto-labeling` | Stage configuration, prompts/question banks, stage outputs, and component errors | [`cdf-auto-labeling` skill][auto-labeling-skill] |

[augmentation-skill]: https://github.com/NVIDIA/cdf-augmentation/tree/73c861c43a4952b6060c87265bf39f4abc48199a/skills/cdf-augmentation
[auto-labeling-skill]: https://github.com/NVIDIA/cdf-auto-labeling/tree/6c75c76c52a6c76c3866a38d30885e11123b0981/skills/cdf-auto-labeling

Use only documentation at these commits; do not substitute a moving branch or another version.
If the documentation is inaccessible, report the exact reference and continue only work supported
by the local workflow guidance; do not invent component settings.

The orchestration routes are sibling skills in this repository:
[`cdf-orchestration-setup`](../cdf-orchestration-setup/SKILL.md) for environment preparation and
Airflow controller setup, and [`cdf-orchestration-write-dag`](../cdf-orchestration-write-dag/SKILL.md)
for custom DAG composition and specification parity review.

The pinned component skills own container schemas, model options, prompt authoring, and evaluator
internals; use their referenced documentation rather than maintaining local copies. This skill
owns the VDA trigger contract and the constraints imposed by its checked-in DAG. Preserve those overrides in
[payload-contract.md](references/payload-contract.md); component capabilities do not automatically
become supported DAG options. Keep execution through Airflow for a workflow request.
If a pinned component contract differs from the deployed image or DAG integration, report the
version mismatch before proposing settings; do not silently update an image or bypass an override.
Routine payload rendering, status checks, and GPU sizing do not require loading both components.

## Select the execution target

| Target | DAG ID | Service modes |
|---|---|---|
| Kubernetes | `video_data_augmentation_dag_k8s` | Internal or external inference services |
| OSMO | `video_data_augmentation_dag_osmo` | External inference and evaluator endpoints only |

List the DAGs Airflow actually loaded and check import errors before selecting a DAG. A platform
DAG is registered only when its manifest exists. Never invent a DAG ID or reinstall Airflow merely
because the requested DAG is absent; report that the existing installation does not contain it.

Use Kubernetes unless the user explicitly requests OSMO or supplies an OSMO execution context.
For OSMO, require `external_services: true`, keep every `service_lifecycle.*.enabled` value false,
and supply all VLM, LLM, Cosmos Transfer, and required Cosmos Evaluator endpoints. The OSMO
manifest cannot deploy any of these services.

## Minimum compute

For Kubernetes, let `C` be the number of enabled managed evaluator checkers, from zero to three.
Each managed checker—hallucination, attribute verification, and physical plausibility—requires one
GPU. Use `C=0` when the evaluator is external or all checks are disabled.

| Mode | Absolute GPU floor | Practical default-graph minimum |
|---|---:|---:|
| Kubernetes, external inference services | `C + 1` | `C + 2` |
| Kubernetes, internal inference services | `C + 4` | `C + 5` |
| OSMO, all endpoints external | One 1-GPU job slot | Two concurrent 1-GPU job slots |

The absolute floor reserves one GPU for a runnable batch task; the practical minimum reserves two
because source auto-labeling and Cosmos/output processing can overlap. Internal Kubernetes adds one
GPU for the shared language endpoint and two for Cosmos3-Nano Transfer. Thus the default internal
payload with all three bundled checks enabled needs **7 GPUs to make forward progress and 8 GPUs
for the default parallel graph**. With physical plausibility disabled but managed hallucination
and attribute verification enabled, the corresponding floors are **6 and 7 GPUs**. OSMO assigns
each active task 1 GPU, 11 CPUs, 100 GiB RAM, and 100 GiB storage. Read
[setup-and-preflight.md](references/setup-and-preflight.md#minimum-compute) for assumptions,
scaling, and the external-service examples.

## Manual payload entry in the Airflow UI

If the user explicitly wants to enter their own payload in the Airflow UI, confirm controller
readiness, make the UI reachable, and report the URL. Do not render, confirm, or submit a second
payload on their behalf. Once they say they triggered a run, locate it through the API and resume
monitoring from the recorded DagRun configuration.

From NVIDIA/skills, CC-BY-4.0 AND Apache-2.0.

More from NVIDIA/skills