Limits, Scaling & Runtime vs. Lambda

If your mental model of "serverless" is AWS Lambda — a stateless function that spins up, handles one request, and is gone — AgentCore Runtime will surprise you. It is serverless (you hand AWS a container, AWS runs and scales it, you never manage a fleet), but the unit of execution is different.

Every user session gets its own dedicated microVM with isolated CPU, memory, and filesystem, and AWS keeps that microVM alive across invocations for up to 8 hours. You invoke it through the InvokeAgentRuntime data-plane API using an agent runtime ARN plus a caller-supplied runtimeSessionId. Repeated invocations carrying the same session ID land on the same microVM, which still holds the in-memory reasoning context from the previous turn.

That single design choice is the whole reason Runtime exists for agents:

Agents are long-lived, stateful, and non-deterministic: they hold a reasoning trajectory, call tools mid-thought, and may run for minutes. Forcing that onto a stateless function means re-hydrating context on every call and fighting the 15-minute wall. Runtime's per-session microVM is purpose-built for it.

One caveat that follows directly from this model: the microVM is ephemeral, durability is your job. In-memory and on-disk state lives only for the microVM's lifecycle — an idle timeout or the 8-hour cap tears it down and sanitizes memory. For state that must survive, configure session storage (a persistent filesystem mount) or use AgentCore Memory for durable conversational state.

A cold start in Runtime means provisioning a fresh microVM. It happens when:

• a request opens a new session (a runtimeSessionId AWS hasn't seen, or none supplied — one is generated on first invoke), or
• a request is missing affinity to an existing microVM and gets routed to a freshly provisioned one.

The fix is to reuse the same runtimeSessionId for related invocations. That keeps your conversation context and routes you back to the same warm microVM (AWS docs call this microVM "stickiness"). Without a consistent ID, each request may hit a new microVM and pay provisioning latency again.

import json, uuid, boto3

client = boto3.client("bedrock-agentcore")          # data plane

# Create ONE session id per user/conversation and reuse it.
# Minimum 33 chars; a UUID (36 chars) satisfies that.
session_id = str(uuid.uuid4())

def ask(prompt: str) -> str:
    resp = client.invoke_agent_runtime(
        agentRuntimeArn="arn:aws:bedrock-agentcore:us-west-2:123456789012:runtime/MyAgent-XXXX",
        runtimeSessionId=session_id,                 # reuse → warm microVM + context
        payload=json.dumps({"prompt": prompt}).encode(),
        qualifier="DEFAULT",
    )
    return json.loads("".join(c.decode("utf-8") for c in resp.get("response", [])))

ask("What's the weather in Seattle?")   # may cold-start the microVM
ask("And tomorrow?")                     # same session → warm, keeps context

When you call over a raw protocol instead of boto3, the session ID rides in a header — the exact name depends on the protocol:

Always capture the session ID returned in the response and echo it on subsequent calls — your client backend, not AWS, owns the user↔session mapping.

How long is a cold start? AWS publishes no hard number — it is workload-dependent (image size, init code, framework startup) and unverified. Treat cold-start latency as something you measure for your agent, not a constant you can quote. The architectural lever you control is affinity: reuse session IDs, and keep the session warm during long async work by reporting HealthyBusy from /ping so the idle timeout doesn't reap a microVM you still want.

Runtime scales to many concurrent sessions automatically — but "automatically" is bounded by Service Quotas. Two levers matter most for capacity planning, and both are [VOLATILE] — verify the live quotas page before you size anything.

1. Concurrency — "Active session workloads per account." This caps how many sessions can be actively running at once. As researched, it is 1,000 in us-east-1 / us-west-2 and 500 in other regions, and it is adjustable via a quota-increase request.

2. Invoke throughput — InvokeAgentRuntime TPS. Per-agent invocation throughput is throttled at roughly 25 TPS (transactions per second), and this one is adjustable. The streaming and command variants share the same ballpark:

The practical reading: TPS quotas govern how fast you can start invocations; the active-session quota governs how many sessions can be in flight at once. Long agent sessions (the common case) make the concurrency cap the binding constraint far more often than raw TPS. Because every number here drifts between releases and regions, hard-code none of them — read them at runtime or check the console.

# Read the live values for your account/region instead of trusting a doc.
aws service-quotas list-service-quotas \
  --service-code bedrock-agentcore \
  --region us-west-2

# Request an increase on an adjustable quota (replace the quota code from the list above)
aws service-quotas request-service-quota-increase \
  --service-code bedrock-agentcore \
  --quota-code L-XXXXXXXX \
  --desired-value 50

Here is the limit that most often forces an architecture decision: the per-session hardware ceiling is 2 vCPU / 8 GB, and it is NOT adjustable. Unlike a Lambda where you dial memory (and proportional CPU) up to multiple GB, or an ECS/EKS task you size freely, a Runtime session is fixed. If a single session genuinely needs more than 2 vCPU / 8 GB of simultaneous compute, Runtime is the wrong host for that step — push the heavy lifting to a Built-in Tool (Code Interpreter), a separate batch/ECS job, or a downstream service, and let the agent orchestrate.

The other hard limits (all [VOLATILE], all on the quotas page) bound payloads and durations:

Two of these are configurable per agent — not via Service Quotas, but through the runtime's LifecycleConfiguration:

# Adjust idle timeout and max lifetime when creating/updating the runtime
# (control plane: bedrock-agentcore-control)
control = boto3.client("bedrock-agentcore-control")
control.update_agent_runtime(
    agentRuntimeId="MyAgent-XXXX",
    lifecycleConfiguration={
        "idleRuntimeSessionTimeout": 1800,   # seconds; default 900 (15 min)
        "maxLifetime": 28800,                # seconds; max 8 hours
    },
)

Note the contrast: session-lifetime knobs live on the runtime resource (LifecycleConfiguration), while throughput/concurrency knobs are Service Quotas. The hardware ceiling is neither — it is simply fixed.

Session storage deserves a callout: persistent filesystem is capped at 1 GB per session, with roughly 50 MB of FS metadata (~100k–200k files), 200 levels of directory depth, 255-byte filenames, and 4,095-byte symlink targets. It's for working files and checkpoints across a stop/resume — not a data lake.

Both are serverless; they solve different shapes of problem. Use this as a first-pass filter.

Reach for AgentCore Runtime when the workload is:

• Long-running — minutes to hours (async jobs up to 8h); Lambda caps at 15 min.
• Stateful / session-affine — the agent holds reasoning context across turns and benefits from the same warm microVM.
• Streaming — token-by-token SSE or WebSocket back to a UI (up to 60 min of streaming).
• Large-payload — up to 100 MB in; Lambda's synchronous payload limits are far smaller.
• Agent-framework based — you're running Strands / LangGraph / CrewAI / etc. and want the standardized HTTP/MCP/A2A contract, per-session isolation, and built-in observability without hand-rolling the plumbing.

Reach for Lambda when the workload is:

• Short, stateless, sub-15-minute request/response — a webhook, a quick transform, a tool a Gateway calls. For these, Lambda is typically cheaper and operationally simpler.

A useful sanity check on cost: Runtime bills on active vCPU-hours and GB-hours (per-second, I/O wait excluded — so an agent waiting on a model or a tool isn't burning active compute), as of writing roughly $0.0895/vCPU-hr and $0.00945/GB-hr [VOLATILE — verify the pricing page]. For a 200 ms stateless transform invoked millions of times, Lambda's per-request model wins easily. For a 20-minute multi-turn agent that streams and calls tools, Runtime's session model is both the right fit and the sane operational story. These are not mutually exclusive — a common pattern is a Runtime-hosted agent that calls Lambda-backed Gateway targets for the short, stateless tool calls.