aws-backup-coverage-review

Files (74)

• .skilleval.yaml
• CHANGELOG.md
• README.md
• SKILL.md
• report-format.md · assets/
• .gitignore · evals/
• additional-permissions.json · evals/
• benchmark.json · evals/best-practices/v19/
• best-practices-tests-results.json · evals/best-practices/v19/iteration-1/
• best-practices-tests-results.json · evals/best-practices/v19/iteration-2/
• best-practices-tests-results.json · evals/best-practices/v19/iteration-3/
• eval_queries.json · evals/
• evals.json · evals/
• backup-context.json · evals/files/
• benchmark.json · evals/functional/v8/
• evals.json · evals/functional/v8/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-colloquial/with_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-colloquial/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-colloquial/without_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-colloquial/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-explicit/with_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-explicit/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-explicit/without_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-explicit/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-multi-region/with_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-multi-region/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-multi-region/without_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-multi-region/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-single-resource-type/with_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-single-resource-type/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/backup-coverage-review-single-resource-type/without_skill/
• journal_records.json · evals/functional/v8/iteration-1/backup-coverage-review-single-resource-type/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-1/unrelated-lambda-cold-start/with_skill/
• journal_records.json · evals/functional/v8/iteration-1/unrelated-lambda-cold-start/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-colloquial/with_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-colloquial/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-colloquial/without_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-colloquial/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-explicit/with_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-explicit/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-explicit/without_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-explicit/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-multi-region/with_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-multi-region/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-multi-region/without_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-multi-region/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-single-resource-type/with_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-single-resource-type/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/backup-coverage-review-single-resource-type/without_skill/
• journal_records.json · evals/functional/v8/iteration-2/backup-coverage-review-single-resource-type/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-2/unrelated-lambda-cold-start/with_skill/
• journal_records.json · evals/functional/v8/iteration-2/unrelated-lambda-cold-start/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-3/backup-coverage-review-colloquial/with_skill/
• journal_records.json · evals/functional/v8/iteration-3/backup-coverage-review-colloquial/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-3/backup-coverage-review-colloquial/without_skill/
• journal_records.json · evals/functional/v8/iteration-3/backup-coverage-review-colloquial/without_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-3/backup-coverage-review-explicit/with_skill/
• journal_records.json · evals/functional/v8/iteration-3/backup-coverage-review-explicit/with_skill/outputs/
• functional-tests-results.json · evals/functional/v8/iteration-3/backup-coverage-review-explicit/without_skill/
• journal_records.json · evals/functional/v8/iteration-3/backup-coverage-review-explicit/without_skill/outputs/
• …and 14 more

SKILL.md (first lines)

# AWS Backup Coverage Review

Perform a structured, read-only coverage and posture review of AWS Backup in one
account across all enabled Regions. The review answers one question precisely —
**which backup-eligible resources are actually recoverable, and which are not** —
then explains why each gap exists and how to close it.

## Output Contract — read this before doing anything else

**The only acceptable output of this skill is the full report defined in the Final
Delivery Contract below.** A conversational prose summary of the findings — however
accurate, however well organised — is a failed run.

Every response must contain, in order: **Scope** (including Regions swept and not
swept), **Coverage Rating** with a coverage percentage, **Executive Summary**,
**Coverage Matrix**, **Findings & Recommendations**, a **Check Coverage Matrix with
all 23 rows**, and **Next Steps**.

Three failure modes to avoid specifically, because all three feel natural in a chat:

- **Do not compress the report into narrative bullets** because the question was
  phrased casually. "What isn't being backed up?" requires the same full report as
  "run an AWS Backup coverage review".
- **Do not substitute a direct lookup when the question is narrow.** "Are my EBS
  volumes protected in us-east-1?" narrows the *scope* of the review; it does not
  authorise a different *output*. Narrow the sweep, then render the full report for
  that scope. See **Scoped requests** under the Final Delivery Contract — and
  **Follow-up questions in the same conversation** for the single exception, which
  applies only after a qualifying report has already been delivered in this
  conversation.
- **Do not end with an offer to investigate further or to fix anything** ("want me to
  dig into any of these?", "which gap would you like to tackle first?"). The report is
  the deliverable, complete on first response. Findings the review surfaces are
  already in it, each with a recommendation and an SLA bucket. In particular, never
  offer to "run the full coverage review" as a follow-up — if this skill loaded, the
  full review for the requested scope *is* the current response.

If you cannot complete a section, render it with the explicit status values defined
below (`AccessDenied`, `ToolingFailure`, `NotEnumerated`) — never drop it.

**Self-check before responding.** Count the rows in your Check Coverage Matrix. If
the count is not exactly 23, or if the response contains no `## Coverage Rating`
heading and no coverage percentage, the response is incomplete — fix it before
sending. Then verify the protected count and the coverage percentage are **identical
everywhere they appear** — Coverage Rating, headline, and the by-type table. A report
that states two different coverage figures is wrong regardless of which is correct.

## When to Use

Activate this skill when the user asks to:

- Find out what is not being backed up, or which resources are unprotected
- Review, audit, or assess AWS Backup coverage, posture, or configuration
- Review backup plans, backup selections, or backup vaults
- Assess data protection gaps or backup compliance without AWS Config or
  AWS Backup Audit Manager already being set up
- Check whether specific resources (volumes, databases, file systems, tables)
  are protected by AWS Backup

Do NOT activate for restoring data or recovery execution, backup/restore job
failure triage, backup storage cost optimization, or RDS-native automated
backups and manual snapshots taken outside AWS Backup. For Amazon S3 bucket
versioning, replication, and Object Lock posture, `storage-s3-resiliency-expertise`
is the correct skill.

## Why This Skill Exists

AWS Backup Audit Manager's `BACKUP_RESOURCES_PROTECTED_BY_BACKUP_PLAN` control
requires AWS Config recording, a framework, and a report plan that has already run.
Many accounts have none of that, so this skill computes the answer on demand from
read-only APIs, treating AWS Config as an optimization rather than a prerequisite.

## Architecture

- **This skill (orchestrator/analyzer):** scope resolution, routing, coverage
  model application, rating, report rendering.
- **Data collection:** [data collection reference](references/data-collection.md)
  — the read-only API allowlist, hard denials, the per-Region and
  per-resource-type call plan, and error classification. Load it at Step 2,
  before issuing any API call. Data is acquired with the agent's native `use_aws`
  tool under the assumed role in the target account. No credentials or profile
  are requested from the user.
- **Coverage logic:** [coverage logic reference](references/coverage-logic.md) —
  all 23 checks, thresholds, verdict rules, finding templates, and the rating
  roll-up. Load it at Step 6, before evaluating the checks.
- **Report template:** [report template](assets/report-format.md) — report
  structure, the coverage matrix, the check coverage matrix, severity map,
  pre-render validation. Load it at Step 8, before rendering the report.
- **Operational depth:**
  [backup best practices reference](references/backup-best-practices.md) —
  reasoning behind the thresholds, remediation guidance, and the canonical AWS
  documentation URLs. Load it when a finding needs remediation guidance or a
  documentation citation.

## The Coverage Model

Coverage is not binary. Every eligible resource resolves to exactly one of six
states. Getting this distinction right is the whole value of the review — a
resource can sit inside a backup plan and still be unrecoverable.

| State | Meaning | Severity |
|---|---|---|
| `Protected` | Has at least one recovery point, and the newest is within the plan's expected interval | ✅ |
| `Stale` | Has recovery points, but the newest is older than the plan schedule allows | ⚠️ HIGH |
| `SelectedNotProtected` | Matched by a backup selection but has zero recovery points — the plan has never successfully run for it | ❌ CRITICAL |
| `Unprotected` | Eligible, matched by no selection, zero recovery points | ❌ CRITICAL |
| `OptInBlocked` | Matched by a selection, but its resource type is **not opted in** for that Region, so AWS Backup will never protect it | ❌ CRITICAL |
| `OrphanedRecoveryPoint` | Appears in `ListProtectedResources` but the resource itself no longer exists in the account | ⚠️ MEDIUM |

`OrphanedRecoveryPoint` is resolved from the opposite direction to the other five.
`ListProtectedResources` keeps returning a resource long after it is deleted, so
**every entry it returns must be cross-checked against the live inventory**. An
entry with no matching live resource is an orphaned recovery point: it is a
retention and cost issue, not a coverage gap. Never count it as `Protected`, never
count it as `Stale`, and never include it in the coverage numerator or denominator —
a deleted resource needs no protection. Report it, with the age of its newest
recovery point, so long-abandoned recovery points in unused Regions become visible.

`OptInBlocked` is the most commonly missed real finding, because the AWS Backup

From aws/tools-for-devops-agent, Apache-2.0.

More from aws/tools-for-devops-agent