aws-bedrock-integration

Files (2)

• SKILL.md
• SKILL.md.lock

SKILL.md (first lines)

<!-- bodyRef: skills/sources/aws-bedrock-integration.md -->

<!-- Generated by `harness skills bootstrap`. Edit to tune; re-run to regenerate. -->
<!-- Static skills point at live docs by design — do not hardwire version-pinned API here. -->

# AWS Bedrock integration

Integrate Amazon Bedrock **correctly and securely** for this stack: LangGraph orchestration (Python `langchain-aws` / boto3) and Next.js API routes / capture tooling (TypeScript `@aws-sdk/client-bedrock-runtime`), calling the **Converse API** for extraction/clustering/generation/evaluation and **Bedrock Guardrails** for safety. Bedrock is the *model and safety gateway* — treat every call as a credentialed, rate-limited, billable, auditable AWS API call, not a local function.

## Pull current docs first

Bedrock surfaces (model IDs, inference profiles, guardrail policy types, quotas) change often. Fetch these before writing code — do not rely on memory:

- Converse API guide: https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html
- Converse API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
- Guardrails overview: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
- Use Guardrails with Converse: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html
- ApplyGuardrail API: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ApplyGuardrail.html
- IAM / identity & access: https://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html
- Identity-based policy examples: https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html
- Data protection: https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html
- Quotas / throttling: https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html
- Cross-region inference profiles: https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html
- LangChain `ChatBedrockConverse`: https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html
- GitHub OIDC → AWS role: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

## Core concepts

**Endpoint & API choice.** Converse/ConverseStream live only on the `bedrock-runtime` endpoint (control-plane ops like creating guardrails live on `bedrock`). Prefer **Converse** over `InvokeModel` — one message shape works across models, so extraction/clustering/generation code is model-portable. Use `ConverseStream` for long generations (blog/script) to surface partial output and avoid read timeouts.

**Model access is opt-in.** A model is unusable until access is granted for it in the Bedrock console for that account+region. New account/region → request access first or every call returns `AccessDeniedException`.

**Model IDs vs inference profiles.** For current Anthropic models you generally must pass a **cross-region inference profile ID** (e.g. `us.anthropic.claude-...`), not a bare foundation-model ID. Profiles also raise effective throughput by spreading load across regions — the main lever against throttling. Don't hardwire a model string in many files; centralize it in one config.

**Auth = SigV4 over IAM, no keys in code.** The SDK signs requests with SigV4 from the ambient credential chain. **Never** hardcode or commit `AWS_ACCESS_KEY_ID`/secret. By runtime:
  - *GitHub Actions runner* (the LangGraph job): use **OIDC** with `aws-actions/configure-aws-credentials` to assume a short-lived role — no long-lived secrets in repo/Actions secrets.
  - *Vercel / Next.js API routes*: keep Bedrock calls server-side only; use a scoped role or short-lived creds in server env, never `NEXT_PUBLIC_*`. Browsers must never hold Bedrock credentials.
  - *Local dev*: profiles / SSO via the default chain.

**Least privilege.** Scope the policy to exactly the runtime actions — `bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream`, `bedrock:Converse`, `bedrock:ConverseStream`, `bedrock:ApplyGuardrail` — and restrict `Resource` to the specific foundation-model / inference-profile ARNs and the guardrail ARN. Don't grant `bedrock:*`. Add `bedrock:GetGuardrail` only if the app reads guardrail config at runtime.

**Guardrails are first-class here, not optional.** This engine generates outward-facing marketing/sales claims, so wire guardrails two ways:
  - *Inline on Converse* via `guardrailConfig` (`guardrailIdentifier`, `guardrailVersion`, `trace: "enabled"`) — filters the generation itself.
  - *Standalone `ApplyGuardrail`* (no model invocation) to screen **input evidence** (redact PII/secrets from diffs/PRs before they hit a model) and to re-check **generated artifacts** before the human gate. Useful policy types: sensitive-information (PII block/mask), denied topics, word filters, and **contextual grounding checks** — which directly support the "every claim must be evidence-backed" requirement by flagging ungrounded/hallucinated statements. Note guardrails do **not** inspect reasoning/thinking content blocks. Pin a **published numbered version** in prod; use `DRAFT` only in dev.

**Error & retry semantics.** Handle these by name, don't blanket-catch: `ThrottlingException`, `ServiceUnavailableException`, `ModelTimeoutException`, `ModelNotReadyException`, `ModelStreamErrorException` (retryable, exponential backoff + jitter); `ValidationException`, `AccessDeniedException`, `ResourceNotFoundException`, `ServiceQuotaExceededException` (not retryable — fix the request/permissions/quota). Configure the SDK retrier rather than rolling your own loop: boto3 `Config(retries={"max_attempts": N, "mode": "adaptive"})`; JS SDK v3 `retryStrategy`/`maxAttempts` (adaptive). Set generous `read_timeout`/`connect_timeout` for large generations. Let LangGraph node retries layer on top of SDK retries (and avoid double-billing — see idempotency).

**Idempotency.** Converse/ApplyGuardrail are **not idempotent** — each call re-generates and re-bills, and they take **no client request token** (only async batch *model-invocation jobs* accept `clientRequestToken`). So enforce idempotency at the app layer: key each generation on a content hash (`release_run_id` + node + input hash) persisted in Aurora, and skip/reuse if already produced. Use a LangGraph **checkpointer + stable `thread_id`** so resuming an approval gate replays committed state instead of re-invoking Bedrock. This is what keeps a retried/re-dispatched GitHub Actions run from regenerating (and re-charging for) every artifact.

**Inbound signature verification.** Bedrock has **no inbound webhooks** — there is nothing from AWS to signature-verify. The real inbound trust boundaries in this engine are: (1) **GitHub webhooks** triggering runs — verify the `X-Hub-Signature-256` HMAC against your secret before acting; (2) **dashboard → Next.js API** approval-resume calls — authenticate the reviewer and authorize the specific `thread_id`/gate before resuming a LangGraph thread. Don't conflate these with Bedrock auth.

**Test/sandbox vs production.** Bedrock has no separate sandbox endpoint. Approximate one: separate **account or region** for dev; cheap fast model (e.g. Haiku) + small `maxTokens` in dev vs the prod model via config; guardrail `DRAFT` in dev, pinned numbered version in prod; **mock the `bedrock-runtime` client** in unit tests (assert request shape, never hit the network); reserve a small set of live integration tests behind a flag. Enable **model invocation logging** (to S3/CloudWatch) for audit/repro, but treat those logs as sensitive (they can contain prompts/PII).

**Data protection.** TLS 1.2+ (1.3 preferred) is enforced; use a **VPC interface endpoint / PrivateLink** to keep traffic off the public internet from the runner; KMS CMKs for guardrails/custom resources and for the S3 evidence bucket. Bedrock does **not** use your prompts/completions to train base models, and model providers can't see your data — but your own invocation logs can, so scope and encrypt them. Never put secrets/PII in tags or free-text name fields.

## Checklist

- [ ] Pulled the current docs above; confirmed the exact model/inference-profile ID and that model access is granted for the account+region.
- [ ] All Bedrock calls are server-side (LangGraph runner / Next.js route); no credentials in client bundles or `NEXT_PUBLIC_*`.
- [ ] Credentials come from the ambient chain — OIDC-assumed role in Actions, scoped role/short-lived creds on Vercel; nothing hardcoded or committed.
- [ ] IAM policy is least-privilege: only the needed `bedrock:Converse*`/`InvokeModel*`/`ApplyGuardrail` actions, `Resource` scoped to model/profile + guardrail ARNs.
- [ ] Converse calls set `system`, `inferenceConfig` (maxTokens/temperature), and `guardrailConfig` with a pinned numbered guardrail version (+ `trace` in dev).
- [ ] `ApplyGuardrail` screens inbound evidence (PII/secret redaction) and outbound artifacts; contextual-grounding check wired to the claim-provenance requirement.
- [ ] Retryable vs terminal exceptions handled by name; SDK retrier configured (adaptive); timeouts sized for streaming generations.
- [ ] App-layer idempotency: content-hash dedupe in Aurora + LangGraph checkpointer/`thread_id` so retried runs don't regenerate/re-bill.
- [ ] Inbound GitHub webhook HMAC verified; dashboard approval-resume calls authenticated/authorized per `thread_id`.
- [ ] Dev vs prod separated (region/account, model, guardrail DRAFT vs version); `bedrock-runtime` mocked in unit tests; live tests flag-gated.
- [ ] VPC endpoint + KMS + encrypted, access-scoped invocation logs; verified no PII/secrets land in logs or tags.

## Gotchas

- **Bare foundation-model IDs throw for on-demand Anthropic models** — you must use a cross-region **inference profile** ID (`us.`/`eu.` prefix). Copy-pasting `anthropic.claude-...` from old examples fails with a validation/throughput error.
- **`AccessDeniedException` on a brand-new model is usually "model access not requested," not an IAM bug.** Grant access in the Bedrock console for that exact region before debugging the policy.
- **Guardrails are region- and version-specific.** A guardrail ID created in one region doesn't exist in another, and `DRAFT` behaves differently from a published version — pinning `DRAFT` in prod means edits silently change behavior mid-flight.
- **Guardrails skip reasoning/thinking content blocks.** Sensitive content emitted only in a model's reasoning trace is *not* filtered — don't rely on guardrails to scrub thinking output; strip it yourself before persisting/displaying.
- **Converse has no request idempotency token.** A retried LangGraph node or re-run GitHub Action will re-call and **re-bill** Bedrock unless you dedupe in Aurora — silent double-spend is the default failure mode here.
- **Throttling looks like a code bug under load.** Default on-demand TPM/RPM quotas are low; bursty parallel artifact generation hits `ThrottlingException` fast. Fix with inference profiles + adaptive retry/backoff + concurrency limits, not by retrying harder.
- **`bedrock` vs `bedrock-runtime` endpoint mix-up.** Calling Converse against the `bedrock` (control-plane) client, or creating guardrails against `bedrock-runtime`, fails confusingly — they're different clients.
- **Model invocation logging can leak the very PII you redacted.** If you enable logging for audit, it captures raw prompts/completions — encrypt the destination and lock down access, or you've recreated the data-exposure you guarded against.
- **Vercel serverless timeouts can cut off long generations** before Bedrock responds; offload heavy/streaming generation to the GitHub Actions runner (per the architecture) rather than a Next.js route.
- **Streaming errors arrive mid-stream** as `ModelStreamErrorException` after a 200 — a naive client that only checks the initial HTTP status treats a failed generation as success.
- _Anything else: none observed yet._

<!-- sources fetched at generation: https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html, https://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html, https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html, https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html, https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html, https://python.langchain.com/api_reference/aws/chat_models/langchain_aws.chat_models.bedrock_converse.ChatBedrockConverse.html -->

From ichiorca/shipsignal, Apache-2.0.

More from ichiorca/shipsignal