astro-airflow

Files (2)

• SKILL.md
• af.md · references/

SKILL.md (first lines)

# astro-airflow

Read-only debugging surface for Airflow on Astro. Pair three tools:

| Need | Tool |
|---|---|
| DAG runs, task logs, import errors, health, connections, variables, pools | `af` (Airflow REST wrapper) |
| Container logs, deployment inspect, env vars, hibernate/wake | `astro deployment ...` |
| Local `astro dev start` / parse / scheduler restart | `vd:managing-astro-local-env` |
| Create or edit YAML DAGs | `vd:dag-factory` |
| User says "use Otto", long audit, Airflow 2→3 upgrade | `vd:delegating-to-otto` |
| Deep RCA after logs are in hand | `vd:debug` then `vd:fix` |
| `af` not installed or no remote instance configured | curl against `/api/v2/` (Airflow 3) or `/api/v1/` (Airflow 2) |

## Prerequisites

- `astro` CLI ≥ 1.42, logged in (`astro login`; verify with `astro context list`)
- `curl` + `jq`
- `af` optional but preferred: `uv tool install astro-airflow-mcp` (one-shot: `uvx --from astro-airflow-mcp af`)
- A Deployment / Workspace / Organization API token in gopass for curl fallback. Mint via Astro UI → Deployment → Access → API Tokens. Least-privilege: `deployment.get` + `deployment.airflow.*.get`.

Always resolve the live deployment ID first. IDs in project docs go stale.

```bash
astro deployment list
astro deployment inspect <deployment-id> --key metadata.airflow_api_url
```

## Choose the interface

1. **Named Otto / upgrade / long investigation** → `vd:delegating-to-otto`.
2. **`af` on PATH (or `uvx --from astro-airflow-mcp af`) and a configured instance** → use `af`. See [references/af.md](references/af.md).
3. **Platform / container logs / env vars** → `astro deployment ...` below.
4. **Otherwise** → curl fallback.

`af instance discover` **creates API tokens in Astro Cloud**. Always `--dry-run` first and get explicit approval before a real discover.

```bash
uvx --from astro-airflow-mcp af instance list
uvx --from astro-airflow-mcp af instance discover --dry-run
# only after the user says yes:
# uvx --from astro-airflow-mcp af instance discover astro
uvx --from astro-airflow-mcp af instance use <name>
```

Add by hand without persisting the secret. `af instance add --token` writes the value into `~/.astro/config.yaml`. Prefer a `${VAR}` reference or a one-shot env:

```bash
export AIRFLOW_AUTH_TOKEN="$(gopass show -o <path/to/deployment-token>)"
export AIRFLOW_API_URL="https://<org>.astronomer.run/<short-id>"
uvx --from astro-airflow-mcp af dags list
```

Persistent instance (token stays in the environment, not the file):

```yaml
# ~/.astro/config.yaml  (or project .astro/config.yaml)
instances:
- name: staging
  url: https://<org>.astronomer.run/<short-id>
  auth:
    token: ${AIRFLOW_AUTH_TOKEN}
```

If `af` prints `reading from the legacy ~/.af/config.yaml`, tell the user `af migrate` exists; do not run it unasked.

## Platform ops: `astro` CLI

### Container logs

Component is a **boolean flag or `--component <name>`** (CLI 1.45+). Airflow 3.3+ parse/import lives on **`--dag-processor`**, not only `--scheduler`. Airflow 3.x API is `--apiserver`; `--webserver` is Airflow 2.x.

`--keyword` is an **exact phrase**, not a regex. Do **not** combine `--error`/`--warn`/`--info` with each other or with `--keyword` - the CLI prints usage and exits. Official help examples that show `--error --info` are wrong on 1.45.x.

`--error` is not a reliable level filter: it can return `[info]` lines whose text contains "error" (DAG ids like `el_twilio__error_code`). Prefer `--keyword "ImportError"` on `--dag-processor`.

```bash
astro deployment logs <deployment-id> --scheduler --log-count 100
astro deployment logs <deployment-id> --dag-processor --keyword "ImportError"
astro deployment logs <deployment-id> --component scheduler --log-count 50   # CLI 1.45+
astro deployment logs <deployment-id> --apiserver --log-count 50
astro deployment logs <deployment-id> --triggerer --error
astro deployment logs <deployment-id> --workers --keyword "OOMKilled"
```

### Environment variables and deploy state

```bash
astro deployment variable list --deployment-id <id>            # values redacted
astro deployment variable list --deployment-id <id> -k <KEY>   # one key; secret VALUES are never retrievable
astro deployment inspect <id>
astro deployment pool list --deployment-id <id>
```

Create/update/copy variables only when the user explicitly asks. Confirm the deployment ID first; Astronomer has no undo.

### Airflow API through `astro`

`astro api airflow` accepts exactly one endpoint or operation ID argument. Pass
the deployment ID with `--deployment-id`, not as a second positional argument.
URL-encode every path value, especially DAG run IDs containing `+` or `:`.

```bash
RUN_ID_ENCODED="$(jq -rn --arg value "$RUN_ID" '$value|@uri')"
astro api airflow --deployment-id <deployment-id> \
  "/dags/<dag-id>/dagRuns/${RUN_ID_ENCODED}/taskInstances"
```

Use this wrapper when `af` is unavailable but the logged-in Astro CLI can reach
the deployment. Keep writes opt-in.

## DAG-level: prefer `af`

Once an instance points at the target deployment:

```bash
af health
af dags errors
af dags list

From vanducng/skills, MIT.

More from vanducng/skills