ecs-operation-review

Files (18)

• .skilleval.yaml
• CHANGELOG.md
• README.md
• SKILL.md
• TESTING.md · evals/
• eval_queries.json · evals/
• evals.json · evals/
• service-context.json · evals/files/
• alarm-thresholds.md · references/
• checks.md · references/
• common-checks-coverage.md · references/
• additional-analysis.md · references/pillars/
• observability.md · references/pillars/
• operations.md · references/pillars/
• performance.md · references/pillars/
• resiliency.md · references/pillars/
• security.md · references/pillars/
• report-format.md · references/

SKILL.md (first lines)

# ECS Operations Review Skill

## Overview
Execute a comprehensive Amazon ECS operations review across the 6 review pillars with ✓/✗/N/A observations, 7-day CloudWatch baseline metrics, and alarm threshold recommendations for IDR onboarding.

## Usage
- User mentions "ECS operations review", "ECS assessment", "ECS review"
- User provides ECS service ARN(s) for operations review
- User asks to review ECS services for a workload
- User requests ECS reliability or security review
- Delegated ECS service assessment from UOPS

## Core Concepts

- **Review Pillars**: Six assessment dimensions — Resiliency & HA, Observability, Security, Operations, Performance, Additional Analysis
- **ECS Service/Cluster**: The primary resources assessed — includes tasks, services, and cluster configuration
- **AWS API**: Public AWS service APIs used for all data collection, called read-only (describe/list/get) via the AWS CLI, an AWS SDK, or an AWS API MCP
- **Baseline Metrics**: 7-day CloudWatch metric history used to establish normal operating patterns

## Prerequisites
- Read-only AWS API access (describe/list/get) for ECS, CloudWatch, IAM, Application Auto Scaling, ELB, ECR, EC2 APIs — via AWS CLI, an AWS SDK, or an AWS API MCP
- AWS Knowledge MCP access (search_documentation, read_documentation, recommend, list_regions, get_regional_availability)
- AWS account ID and region
- CloudWatch metrics access (7-day minimum for baseline)
- ECS service ARN — format: arn:aws*:ecs:*:*:service/*/*

## Skill Files
- **references/checks.md** — Checks **index**: pillar→file map, check-ID ranges, counts, and the access-limitation / minimum-baseline rules. Read this FIRST.
- **references/pillars/resiliency.md** — Resiliency & HA checks (REL1-REL14). Read only when running the Resiliency pillar.
- **references/pillars/observability.md** — Observability checks (OBS1-OBS9). Read only when running the Observability pillar.
- **references/pillars/security.md** — Security checks (SEC1-SEC20). Read only when running the Security pillar.
- **references/pillars/operations.md** — Operations checks (OPS1-OPS9). Read only when running the Operations pillar.
- **references/pillars/performance.md** — Performance checks (PERF1-PERF11). Read only when running the Performance pillar.
- **references/pillars/additional-analysis.md** — Additional Analysis checks (ADD1-ADD7). Read only when running the Additional pillar.
- **references/alarm-thresholds.md** — Recommended CloudWatch alarm thresholds for IDR onboarding.
- **references/common-checks-coverage.md** — Crosswalk proving the review covers the shared **`review-common`** baseline (tagging, encryption, IAM least-privilege, alarms, logging, cost) via existing ECS check IDs. Read for any full review / CWR.
- **references/report-format.md** — Required report/artifact structure, severity model, finding-block format, and the **coverage gate**. Read before generating the report.

### Context Management for Checks
Do NOT read all 6 pillar files at once. Read `references/checks.md` (the index) first, then read each `references/pillars/<pillar>.md` file ONE AT A TIME as you run that pillar's checks. Agent may add checks beyond the baseline using the next sequential ID in the pillar.

## <required> Assessment Workflow

This skill is self-contained — the procedural workflow is embedded below (this copy lives in `aws-operations-review` and does not depend on any external `agent-sops/` SOP). Execute the steps in order; this skill's `references/` files supply the check definitions and alarm thresholds.

1. **Setup** — Create the output directory and a scratchpad for raw API responses. Record account ID, region, and timestamp.
2. **Parse ARN** — Validate the ECS service ARN (`arn:aws*:ecs:*:*:service/*/*`); extract cluster name and service name. Halt if the ARN region does not match the region parameter.
3. **Collect configuration data** — Call the AWS APIs following the tier order in the *API Tier Dependency Chain* below. Tier 1 (`ecs.describeServices`) MUST succeed before any other call. Save each response verbatim to the scratchpad. On non-Tier-1 access/API errors, mark dependent checks N/A and continue.
4. **Resolve compute platform** — Before grading any pillar, classify the service's compute platform from `launchType` + `capacityProviderStrategy` + `ecs.describeCapacityProviders`: **Fargate** (FARGATE/FARGATE_SPOT), **EC2 ASG capacity provider** (`autoScalingGroupProvider`), **Managed Instances** (`managedInstancesProvider`), **launchType-only EC2/Fargate** (no strategy), or **ECS Anywhere** (EXTERNAL). Mixed strategies are valid — record every platform present. This decision drives check applicability in every pillar (full rules in `references/checks.md`). Record the platform in the report header.
5. **Run pillar checks** — Read `references/checks.md` (the index) first, then for EACH pillar read its `references/pillars/<pillar>.md` file one at a time and apply its checks against the collected data, recording ✓ / ✗ / N/A with an observation. Use the exact check IDs and severities; mark platform-specific checks N/A where they don't apply (per the resolved compute platform). May add checks beyond the baseline.
6. **Generate report** — Produce the per-service review artifact following [`references/report-format.md`](references/report-format.md) exactly (Workload Details, per-pillar ✓/✗/N/A scorecards for **all 6 pillars**, Prioritized Action Plan, Detailed Findings, Recommended CloudWatch Alarms from `references/alarm-thresholds.md`, Access Limitations, Review Summary). **Before finalizing, run the report-format Coverage gate** — every check ID across all 6 pillars must appear in a scorecard as ✓/✗/N/A (including passes; no pillar dropped or truncated), the alarms table must be present, every ✗ needs a detailed finding block, and the shared **`review-common`** baseline must be accounted for per [`references/common-checks-coverage.md`](references/common-checks-coverage.md) (all eight common checks covered via their ECS equivalents or ⚪ N/A with a reason). Default to a Markdown artifact; render DOCX only if asked (build from the same content). Strip internal check IDs from the customer-facing report. Return the Review Summary with verified counts, then delete the scratchpad.

**Review Pillars:**
- **Resiliency and High Availability (REL1-REL14)** — Multi-AZ, desired count, deployment config, circuit breaker, deployment alarms, health checks, subnet AZ spread, capacity-provider managed termination protection, target-group deregistration delay, capacity provider infrastructure multi-AZ
- **Observability (OBS1-OBS9)** — Container Insights, CloudWatch alarms, logging, log retention, distributed tracing, metrics monitoring
- **Security (SEC1-SEC20)** — IAM least privilege, network mode, secrets management, ECR image scanning, security groups, VPC endpoints, private connectivity, encryption at rest, encryption in transit (TLS), VPC Flow Logs, GuardDuty Runtime Monitoring
- **Operations (OPS1-OPS9)** — Deployment controller, resource tagging, IaC-managed, platform version, ECS agent version, ECS Exec posture and session audit logging
- **Performance (PERF1-PERF11)** — Auto scaling, CPU/memory rightsizing, capacity provider strategy, managed scaling / targetCapacity headroom, CapacityProviderReservation 7-day baseline analysis, base/weight strategy design, Compute Optimizer recommendations
- **Additional Analysis & Recommendations (ADD1-ADD7)** — Graviton/ARM64, Fargate Spot, Service Connect, cost optimization, CloudWatch Logs Insights queries, ECS Managed Instances evaluation

## AWS API Summary

All calls below are public AWS API operations. Use read-only (describe/list/get) operations only, via the AWS CLI, an AWS SDK (e.g. boto3), or an AWS API MCP with least-privilege read-only credentials.

### ECS APIs (Tier 1, 2, 3)
| API | Tier | Purpose |
|-----|------|---------|
| ecs.describeServices | 1 | Foundation — service config, task def, LB, deployment, network |
| ecs.describeTaskDefinition | 2 | Container defs, CPU/memory, roles, log config, network mode |
| ecs.describeClusters | 2 | Cluster settings, Container Insights, capacity providers, Exec logging config (`include=["CONFIGURATIONS"]` for OPS9) |
| ecs.listTasks | 2 | Running task ARNs for the service |
| ecs.describeTasks | 3 | Task health, AZ spread, connectivity status |
| ecs.listContainerInstances | 2 | Container instance ARNs for EC2 launch type clusters |
| ecs.describeContainerInstances | 3 | Agent version, AMI ID, instance status (EC2 only) |
| ecs.describeCapacityProviders | 2 | Compute platform classification (ASG vs Managed Instances vs Fargate), managed termination protection (REL12), managed scaling status/targetCapacity (PERF9), MI network config (REL14) |

### Application Auto Scaling APIs (Tier 2)
| API | Tier | Purpose |
|-----|------|---------|
| applicationautoscaling.describeScalingPolicies | 2 | Auto scaling policies for the service |
| applicationautoscaling.describeScalableTargets | 2 | Min/max capacity configuration |

### ELB APIs (Tier 2)
| API | Tier | Purpose |
|-----|------|---------|
| alb.describeTargetHealth | 2 | Target health for service tasks behind ALB/NLB (skip if no LB configured) |
| alb.describeTargetGroups | 2 | Target group details including LoadBalancerArns — used to determine LB type (ALB vs NLB) by ARN path segment: `/app/` = ALB, `/net/` = NLB, for correct alarm recommendations (skip if no LB configured) |
| elbv2.describeListeners | 2 | Listener protocol/port for the LB fronting the service — HTTPS/TLS vs plaintext HTTP/TCP for encryption-in-transit (SEC20); uses LoadBalancerArns from `describeTargetGroups` (skip if no LB configured) |

### IAM APIs (Tier 4a, 4b)
| API | Tier | Purpose |
|-----|------|---------|
| iam.listAttachedRolePolicies | 4a | Managed policies on execution/task roles |
| iam.listRolePolicies | 4a | Inline policy names on execution/task roles |
| iam.getRolePolicy | 4b | Inline policy document for execution/task roles |

### ECR APIs (Tier 3)
| API | Tier | Purpose |
|-----|------|---------|
| ecr.describeRepositories | 3 | Image scanning config, tag immutability for container image repos |

### EC2/VPC APIs (Tier 2, 3, 4)
| API | Tier | Purpose |
|-----|------|---------|
| ec2.describeSecurityGroups | 2 | Security group rules for service ENIs (awsvpc mode) |
| ec2.describeSubnets | 2 | Subnet AZ distribution for service network config |
| ec2.describeVpcEndpoints | 3 | VPC endpoints for ECR, CloudWatch Logs, Secrets Manager (uses VPC ID from describeSubnets) |
| ec2.describeRouteTables | 3 | Route table entries for NAT/internet access assessment |
| ec2.describeNatGateways | 3 | NAT Gateway availability for private subnets |
| ec2.describeImages | 4 | AMI creation date for container instance AMI currency check (EC2 only, uses imageId from describeContainerInstances) |
| ec2.describeVolumes | 3 | EBS volume encryption status for task-attached / container-instance volumes (SEC17) |
| ec2.describeFlowLogs | 3 | VPC Flow Logs enablement for the service VPC (SEC18, uses VPC ID from describeSubnets) |

### CloudWatch APIs (Tier 2, 5)
| API | Tier | Purpose |
|-----|------|---------|
| cloudwatch.describeAlarms | 2 | Existing alarms for ECS service |
| cloudwatch.getMetricStatistics | 5 | 7-day baseline: CPU, memory, task count; plus CapacityProviderReservation (AWS/ECS/ManagedScaling) for EC2 ASG capacity providers (PERF10) |

From aws/tools-for-devops-agent, Apache-2.0.

More from aws/tools-for-devops-agent