aws-lambda-managed-instances

Files (8)

• SKILL.md
• template.yaml · assets/sqs-processor/
• configuration-guide.md · references/
• cost-comparison.md · references/
• infrastructure-setup.md · references/
• migration-patterns.md · references/
• thread-safety.md · references/
• troubleshooting.md · references/

SKILL.md (first lines)

# AWS Lambda Managed Instances (LMI)

Runs Lambda Event Functions on EC2 instances in the user's account while AWS manages provisioning, patching, scaling, routing, and load balancing. Combines Lambda's developer experience with EC2's pricing and hardware options.

**Works best with** the [AWS MCP server](https://docs.aws.amazon.com/aws-mcp/) for sandboxed CLI execution and audit logging. All guidance also works with standard AWS CLI or SAM CLI.

**Note:** Confirm regional availability, quotas, and instance type offerings against current AWS documentation before production deployment.

## Quick Decision: Is LMI Right for This Workload?

| Signal | LMI is a strong fit | Lambda Event Functions are better |
|--------|---------------------|---------------------------|
| Traffic | Steady, predictable, 50M+ req/mo | Bursty, unpredictable, long periods of no traffic |
| Duration | Long-running asynchronous/ESM jobs that exceed 15 min (up to 90 min on LMI): ETL/data processing, media transcoding, ML inference, financial calc, web scraping | Short invocations; synchronous work needing >15 min (not supported on any Lambda) |
| Cost | Duration-heavy spend at scale | Low or sporadic invocations |
| Cold starts | Unacceptable (LMI eliminates for provisioned capacity) | Tolerable |
| Compute | Latest CPUs, specific families, high network bandwidth, GPU requirements | Lambda Event Functions memory/CPU sufficient |
| Isolation | Dedicated EC2 instances in your account, full VPC control | Shared Firecracker micro-VMs acceptable |
| Scale-to-zero | Does not scale to zero but can create custom schedules with AWS provided solutions | Required (pay nothing when idle) |
| Code readiness | Thread-safe (Node.js/Java/.NET) or any Python code | Non-thread-safe code, expensive to change |

## Routing

Read ONLY the single reference file that matches the user's task. Do not preload multiple references.

| User need | Action |
|-----------|--------|
| Cost comparison, pricing analysis, Savings Plans, Reserved Instances | Read [cost-comparison.md](references/cost-comparison.md) |
| Instance types, memory sizing, vCPU ratios, scaling tuning, capacity provider config, SOCI lazy loading for large container images | Read [configuration-guide.md](references/configuration-guide.md) |
| Thread safety, concurrency model, code review checklist, multi-concurrency readiness | Read [thread-safety.md](references/thread-safety.md) |
| Before/after code examples, runtime-specific migration, connection pooling | Read [migration-patterns.md](references/migration-patterns.md) |
| IAM roles, VPC setup, CLI commands, SAM template, CDK example | Read [infrastructure-setup.md](references/infrastructure-setup.md) |
| Errors, throttling, debugging, stuck deployments | Read [troubleshooting.md](references/troubleshooting.md) |

**Troubleshooting quick facts** (always mention when diagnosing issues):

- Capacity provider stuck in CREATING → most common cause is **private subnets missing a NAT gateway route** (instances need outbound internet for image pull and Lambda service communication)
- Function not scaling → check that a **version is published** (PublishToLatestPublished: true)
- Memory errors → LMI minimum is **2048 MB**

## Workflow

### Step 1: Assess the Workload

Gather these signals before recommending:

1. **Traffic pattern**: Steady vs bursty? Requests per second?
2. **Current costs**: Monthly Lambda spend? Existing Savings Plans?
3. **Runtime**: Node.js, Java, .NET, or Python?
4. **Memory/CPU**: How much memory? CPU-bound or I/O-bound?
5. **Execution duration**: Average and P99?
6. **Concurrency readiness**: Thread safety? Shared `/tmp` paths? Per-invocation DB connections?
7. **VPC**: Already in a VPC? Private resource access needed?

**Long-running asynchronous/ESM jobs that exceed 15 min are a positive fit** — LMI supports a function timeout up to 90 min (5400s) for asynchronous and event-source-mapping invocations (ETL/data processing, media transcoding, ML inference, financial calc, web scraping). Duration is the key differentiator here, not just cost.

When recommending LMI, ALWAYS mention: minimum 3 execution environments for AZ resiliency (cannot go below 3 in production).

### Step 2: Build the Cost Comparison

REQUIRED: Present a cost comparison before recommending LMI.

Rule of thumb: LMI becomes cost-competitive at 50-100M+ req/month with steady traffic. Use the [LMI Pricing Calculator](https://aws-samples.github.io/sample-aws-lambda-managed-instances/) for accurate comparisons.

### Step 3: Configure the Deployment

- **Instance families** (400+ types, .large and up): C-series (compute), M-series (general), R-series (memory). ARM (Graviton) for best price-performance.
- **When using Graviton instances, MUST set `Architectures: [arm64]`** in the function configuration to match.
- **Memory-to-vCPU ratios**: 2:1 (compute), 4:1 (general, default), 8:1 (memory). Min 2 GB, max 32 GB.
- **Multi-concurrency per-vCPU maximums**: Node.js 64, Java 32, .NET 32, Python 16. These are system caps — the actual setting is PerExecutionEnvironmentMaxConcurrency (per execution environment, not per vCPU).
- **For I/O-bound workloads**: use the runtime default or higher PerExecutionEnvironmentMaxConcurrency (e.g., 10 for Node.js) since each request uses minimal CPU while waiting on network.
- **For CPU-bound workloads**: set PerExecutionEnvironmentMaxConcurrency to 1-2 per vCPU since each request saturates CPU.
- **Scaling**: MinExecutionEnvironments (default 3), MaxVCpuCount (optional, default 400 — set explicitly as best practice), TargetResourceUtilization.
- **Function timeout up to 5400s (90 min)** — set via the existing `Timeout` field (CLI `update-function-configuration --timeout 5400`, SAM/CFN `Timeout: 5400`, or console). Uses the existing `Timeout` field — no separate API, property, tag, or code change is required. Applies to **asynchronous and ESM invocations only**; synchronous and On-Demand invocations stay capped at 15 min (even if `Timeout` is higher; `GetFunctionConfiguration` still reports the configured value). The Init phase is still capped at 15 min. Durable Functions: each step can run up to 90 min; a multi-step workflow up to 1 year.

### Step 4: Migrate the Code

Review code for concurrency safety. LMI runs multiple invocations concurrently per execution environment:

- **Python**: Process-based isolation — globals are NOT shared. No thread-safety changes needed. Focus on `/tmp` conflicts and memory sizing.
- **Node.js**: Worker threads — globals shared within a worker. Requires async safety.
- **Java/.NET**: OS threads/Tasks — handler shared across threads. Requires full thread safety.

### Step 5: Set Up Infrastructure

1. Create two IAM roles: execution role (for the function) and operator role (for capacity provider EC2 management)
2. Configure VPC with subnets across 3+ AZs
3. Create capacity provider with VPC config and scaling limits
4. Create a new function with the capacity provider attached
5. Publish a version (triggers instance provisioning)

### Step 6: Validate and Cut Over

1. Deploy to a non-production environment first
2. Monitor CloudWatch: CPU utilization, memory, concurrency, throttle rate
3. Gradual traffic shift at the invoking service or routing layer (10% → 50% → 100%)
4. Compare costs after 1-2 weeks of production data
5. Decommission the Lambda Event Function once stable

## Best Practices

### Pricing (always mention when discussing costs)

- **Three components**: EC2 instance hours + 15% management fee + $0.20/1M requests
- **Savings Plans**: Compute Savings Plans apply to the EC2 portion (up to 60-72% discount)
- **The 15% fee** is charged on top of EC2 cost for AWS managing provisioning, patching, scaling, lifecycle

### Scaling (always mention when discussing scaling or traffic)

- LMI absorbs a 50% traffic spike immediately and **doubles capacity within 5 minutes** — if traffic more than doubles faster, requests throttle
- Lambda Event Functions burst to 3000 instantly — LMI cannot match this
- **Pre-warm** with MinExecutionEnvironments before known spikes
- **MaxVCpuCount** (default 400) — set explicitly as a cost ceiling
- **Shape**: Reduce MinExecutionEnvironments to lower capacity during off-hours (minimum 3 for AZ resiliency)

### Large container images (SOCI lazy loading)

- For container image functions, the image pull is the largest part of the time between publishing a version or scaling out and the first invocation. **Seekable OCI (SOCI)** lets Lambda start the execution environment before the whole image is downloaded and fetch files on demand as the function reads them. The benefit is largest for multi-GB images, such as images that package ML frameworks.
- **Enable it** by pushing a SOCI index to Amazon ECR next to the image, pointing the function's image URI at the SOCI-indexed tag, and publishing a version. No other function configuration changes. Steps in [configuration-guide.md](references/configuration-guide.md).

From aws/agent-toolkit-for-aws, Apache-2.0.

More from aws/agent-toolkit-for-aws