bedrock-adoption-readiness

Files (5)

• CHANGELOG.md
• README.md
• SKILL.md
• eval_queries.json · evals/
• evals.json · evals/

SKILL.md (first lines)

# Bedrock Adoption Readiness Assessment

Assess an AWS account's readiness to run Amazon Bedrock at production scale. Covers four dimensions: IAM governance, data retention (ZDR), quota and capacity headroom, and operational observability. Operates across both Standard Bedrock and Mantle (OpenAI models) surfaces.

## Important: Two Surfaces

Bedrock operates across two control planes. Both must be assessed.

| | Standard Bedrock | Mantle (OpenAI models) |
|---|---|---|
| CloudWatch namespace | `AWS/Bedrock` | `AWS/BedrockMantle` |
| Metric names | `Invocations`, `InputTokenCount`, `OutputTokenCount` | `Inferences`, `TotalInputTokens`, `TotalOutputTokens` |
| CW dimensions | `ModelId` | `Model`, `Project` |
| IAM prefix | `bedrock:`, `bedrock-runtime:` | `bedrock-mantle:` |
| Cost discriminator | No marker in USAGE_TYPE | `-mantle-` substring in USAGE_TYPE |
| Cross-region inference | Yes - Geographic (`us.`, `eu.`, `apac.` prefixes) and Global (`global.` prefix) | No (in-region only) |

## When to Use

Activate this skill when the user asks to:
- Review or assess Bedrock production readiness
- Audit Bedrock IAM permissions or access governance
- Check Bedrock quota utilization or capacity planning
- Evaluate Zero Data Retention (ZDR) configuration
- Validate Bedrock operational monitoring setup
- Prepare for scaling Bedrock usage in production

## Dimension States

Every dimension reports one of three states:
- **ASSESSED**: Data collected, rules applied, findings produced
- **NOT_ASSESSED**: Collection failed or data unreachable (state the reason)
- **INSUFFICIENT_DATA**: Data returned but volume too low for meaningful analysis

A dimension producing zero findings after successful collection = GOOD. A dimension producing zero findings because collection failed = NOT_ASSESSED. These must render differently.

## Step 1: Identify Scope

Ask the user:
- Which AWS account to assess
- Which regions to review (if unknown, discover by listing metrics in us-east-1, us-east-2, us-west-2)
- Whether they have specific concerns or want a full assessment

## Step 2: Discover Active Regions and Models

For each candidate region, check both namespaces:
- List metrics in `AWS/Bedrock` namespace
- List metrics in `AWS/BedrockMantle` namespace

Any region returning metrics on either namespace is in scope. Record which ModelId/Model values appear - these identify active models for D2, D3, and later dimensions.

## Step 3: Collect Data

### 3.1 Standard Bedrock Metrics (7 days, per active region)

Query without dimensions for aggregates:
- `Invocations` (Sum)
- `InvocationThrottles` (Sum)
- `InvocationLatency` (Average)
- `InputTokenCount` (Sum)
- `OutputTokenCount` (Sum)
- `CacheReadInputTokenCount` (Sum) - NOTE: correct metric name, NOT `CacheReadInputTokens`
- `CacheWriteInputTokenCount` (Sum)
- `InvocationServerErrors` (Sum)
- `InvocationClientErrors` (Sum)

Then query WITH `ModelId` dimension for per-model quota utilization:
- `EstimatedTPMQuotaUsage` (Maximum) per ModelId discovered in Step 2

### 3.2 Mantle Metrics (7 days, per active region)

Query at zero-dimension for aggregates:
- `Inferences` (Sum)
- `TotalInputTokens` (Sum)
- `TotalOutputTokens` (Sum)
- `InferenceClientErrors` (Sum)

Then per-Model for attribution (TotalInputTokens supports Model dimension):
- `TotalInputTokens` with `Model` dimension per model discovered in Step 2
- `TotalOutputTokens` with `Model` dimension

For non-GPT-5.x models that emit `BurnDownConsumed`, query with `(Model, Project)` pair.

### 3.3 Service Quotas

List all quotas under `serviceCode: bedrock`. Paginate fully (can be 1,001+).

Separate:
- Standard Bedrock quotas: names NOT prefixed with `[bedrock-mantle endpoint]`
- Mantle quotas: names prefixed with `[bedrock-mantle endpoint]`

### 3.4 IAM (full depth)

**Step A**: List all roles in the account.

**Step B**: For each role, get attached managed policies AND inline policies.

**Step C**: For inline policies, get the policy document directly.

**Step D**: For managed policies, get the policy version document using the DefaultVersionId.

Search all policy documents for: `bedrock:`, `bedrock-runtime:`, `bedrock-mantle:`, `bedrock-agentcore:`, and bare `*` in Action fields.

Note: CDK/CloudFormation execution roles with `*` are expected. Flag as INFO, not CRITICAL.

If policy documents cannot be retrieved (only metadata returned), mark D1 as NOT_ASSESSED: "IAM policy documents not retrieved."

### 3.5 Alarms and Observability

List all CloudWatch alarms in each active region. Identify which reference Bedrock metrics.

Check model invocation logging configuration. If the API returns empty response, logging is DISABLED.

Check CloudTrail event selectors for `bedrock-runtime.amazonaws.com` data events (management events alone do not capture model invocations).

### 3.6 Guardrails

List Bedrock guardrails. Zero guardrails on a production deployment using Standard Bedrock (`bedrock-runtime`) is a finding. Note: Guardrails are NOT available on the Mantle endpoint (`bedrock-mantle`). Do not flag missing guardrails for Mantle-only workloads.

From aws/tools-for-devops-agent, Apache-2.0.

More from aws/tools-for-devops-agent