firehose-operation-review

Files (166)

• .skilleval.yaml
• CHANGELOG.md
• README.md
• SKILL.md
• report-template.md · assets/
• benchmark.json · evals/
• benchmark.json · evals/best-practices/v11/
• best-practices-tests-results.json · evals/best-practices/v11/iteration-1/
• best-practices-tests-results.json · evals/best-practices/v11/iteration-2/
• best-practices-tests-results.json · evals/best-practices/v11/iteration-3/
• eval_queries.json · evals/
• evals.json · evals/
• firehose-context.json · evals/files/
• benchmark.json · evals/functional/v3/
• evals.json · evals/functional/v3/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-pillars/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-pillars/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-pillars/without_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-pillars/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-severity-definitions/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-severity-definitions/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-severity-definitions/without_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-severity-definitions/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-trigger/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-trigger/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/firehose-review-trigger/without_skill/
• journal_records.json · evals/functional/v3/iteration-1/firehose-review-trigger/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/unrelated-bedrock-review/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/unrelated-bedrock-review/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/unrelated-cloudformation-s3/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/unrelated-cloudformation-s3/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/unrelated-kinesis-data-streams/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/unrelated-kinesis-data-streams/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/unrelated-python-script/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/unrelated-python-script/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-1/unrelated-weather/with_skill/
• journal_records.json · evals/functional/v3/iteration-1/unrelated-weather/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-pillars/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-pillars/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-pillars/without_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-pillars/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-severity-definitions/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-severity-definitions/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-severity-definitions/without_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-severity-definitions/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-trigger/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-trigger/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/firehose-review-trigger/without_skill/
• journal_records.json · evals/functional/v3/iteration-2/firehose-review-trigger/without_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/unrelated-bedrock-review/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/unrelated-bedrock-review/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/unrelated-cloudformation-s3/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/unrelated-cloudformation-s3/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/unrelated-kinesis-data-streams/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/unrelated-kinesis-data-streams/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/unrelated-python-script/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/unrelated-python-script/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-2/unrelated-weather/with_skill/
• journal_records.json · evals/functional/v3/iteration-2/unrelated-weather/with_skill/outputs/
• functional-tests-results.json · evals/functional/v3/iteration-3/firehose-review-pillars/with_skill/
• …and 106 more

SKILL.md (first lines)

# Amazon Data Firehose Operational Review

Conduct a comprehensive operational review of Amazon Data Firehose (formerly Amazon
Kinesis Data Firehose) delivery streams aligned with the
[AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html)
and [Amazon Data Firehose](https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html)
best practices.

This skill uses the **Firehose, CloudWatch, and Service Quotas APIs only** — all data
is collected through native AWS control-plane APIs and CloudWatch metrics. It performs
no data-plane record puts and reads no delivered record content.

## When to Use

Activate this skill when the user asks to:
- Review, audit, or assess Amazon Data Firehose delivery streams
- Check Firehose best-practices compliance
- Evaluate Firehose security, reliability, delivery health, performance, cost, or quotas
- Perform a Firehose operational readiness review (ORR)
- Investigate delivery lag (data freshness), delivery failures, throttling, or cost drivers

## Review Checklist

Work through these steps in order — each depends on the outputs of the ones before it.
Track progress against this checklist and do not skip a step unless its scope makes it
inapplicable (e.g. a single-stream review skips the account rollup).

- [ ] Step 1: Identify target scope (accounts, regions, streams, pillars, window)
- [ ] Step 2: Discover Firehose resources and capture per-stream config
- [ ] Step 3: Collect CloudWatch metrics (load the thresholds reference first)
- [ ] Step 4: Analyze against best practices (load the checklist reference first)
- [ ] Step 5: Generate the report (load the report template)
- [ ] Step 6: Track findings over time (recurring reviews)

## Step 1: Identify Target Scope

Ask the user which accounts and regions to review, and whether to focus on a specific
delivery stream. Accept:
- Specific account IDs and regions
- A specific delivery stream name (or "all streams")
- "all regions" for a given account
- A specific pillar or set of checks (e.g. "just reliability and cost"). Map free-text
  pillar names to this **canonical set** (accept the listed aliases, case-insensitive):
  `security`; `reliability` (aliases: resiliency, availability, durability);
  `performance` (perf); `service-quotas` (quotas, limits); `cost` (cost-optimization,
  cost optimization, finops); `operational-excellence` (ops, opex, operations);
  `sustainability` (green, efficiency). If a requested term matches none of these, ask the
  user to pick from the canonical list rather than guessing; if the run is unattended,
  default to all pillars.

If no scope is given, default to all configured account regions, all delivery streams,
and all pillars. Default the analysis window to the last 7 days unless the user
specifies a range (historical windows older than ~2 weeks may have reduced CloudWatch
resolution).

**Unattended / scheduled runs.** If the skill is invoked without an interactive user
turn — e.g. as a scheduled custom agent or an automated evaluation task — do **not** wait
for scope input. Proceed immediately with the full default scope: all configured accounts
and regions, all delivery streams, all seven pillars, and a 7-day window. Only pause to
ask for scope when there is an interactive user to answer.

## Step 2: Discover Firehose Resources

**Account scoping.** This skill operates within the accounts the DevOps Agent Space is
configured to access — it does not assume roles or enumerate an Organization itself.
"All accounts" means the accounts already associated with the Agent Space as cloud
sources; the agent invokes the read APIs below in each in-scope account/region using the
Space's configured access. If a requested account is not associated with the Space,
report it as "not accessible from this Agent Space" rather than attempting to assume into
it. Iterate account × region across the resolved scope.

Per account/region, list the delivery streams and describe each one:

```
firehose.ListDeliveryStreams                   # enumerate streams in the region
firehose.DescribeDeliveryStream                 # full config for each stream
firehose.ListTagsForDeliveryStream              # tags (cost allocation, ownership)
```

**If `ListDeliveryStreams` returns no streams for a region, record "No Firehose
delivery streams in this region — skipping analysis" and move on. Do not generate
empty findings tables for regions with no streams.**

From `DescribeDeliveryStream`, capture per stream:
- **Stream type / source**: `DirectPut`, `KinesisStreamAsSource` (with the source
  Kinesis stream ARN), or MSK as source. This determines which ingestion metrics and
  quotas apply.
- **Destination**: S3 / Extended S3, Redshift, OpenSearch Service, OpenSearch
  Serverless, Splunk, HTTP endpoint, Snowflake, or Apache Iceberg tables. A stream has
  exactly one active destination.
- **Buffering hints**: `SizeInMBs` and `IntervalInSeconds` for the destination.
- **Compression**: `CompressionFormat` (UNCOMPRESSED, GZIP, Snappy, Zip, HADOOP_SNAPPY).
- **Encryption at rest**: server-side encryption (`DeliveryStreamEncryptionConfiguration`)
  — `AWS_OWNED_CMK` vs `CUSTOMER_MANAGED_CMK` — and destination S3 `EncryptionConfiguration`
  (KMS key).
- **Error/backup config**: `S3BackupMode` (`FailedDataOnly` / `AllData` / `Disabled`),
  the backup S3 bucket, and destination `RetryOptions.DurationInSeconds`.
- **Data transformation**: `ProcessingConfiguration` — whether a Lambda transform is
  enabled and its ARN (do NOT read the Lambda's code or invoke it).
- **Format conversion**: `DataFormatConversionConfiguration` (Parquet/ORC) for S3
  destinations.
- **Dynamic partitioning**: enabled/disabled and the partitioning config.
- **CloudWatch error logging**: `CloudWatchLoggingOptions.Enabled`.
- **VPC configuration**: `VpcConfiguration` for OpenSearch/Redshift destinations.
- **Cross-account / PrivateLink destinations**: for OpenSearch, Redshift, and HTTP-endpoint
  destinations, note whether the destination ARN/endpoint is in a different account or
  reached through a VPC/PrivateLink endpoint versus the public internet — this feeds the
  Security and Reliability checks.
- **Timestamps and status**: `CreateTimestamp`, `LastUpdateTimestamp`, and
  `DeliveryStreamStatus`.

**Guard on stream status.** Only run the full metric-and-pillar analysis on streams in
`ACTIVE` status. For a stream in `CREATING`, `CREATING_FAILED`, or `DELETING` state,
record its status and skip the CloudWatch/pillar analysis — these streams have partial
or no metrics, and analyzing them produces misleading "0 IncomingRecords" findings. For
`CREATING_FAILED`, surface the failure itself as a HIGH reliability finding.

## Step 3: Collect CloudWatch Metrics (namespace `AWS/Firehose`)

From aws/tools-for-devops-agent, Apache-2.0.

More from aws/tools-for-devops-agent