AWS Lambda Integrations — Event Sources, Invocation Models, and IAM

The Problem

Lambda rarely runs alone. Its whole value is being wired to other AWS services — an API call, a file upload, a queue message, a scheduled event. But each of those sources invokes Lambda differently: some wait for a response, some fire and forget, some poll a stream. If you don't know which model a source uses, you get surprised by retries, lost events, timeouts, and duplicate processing. The integration model is the thing worth understanding, not the function code.

Every integration falls into one of three invocation models. Once you can place a source into the right model, its retry behavior, error handling, and concurrency all follow from that.

The Three Invocation Models

The distinction that bites people most: in the async model Lambda owns the retries and failures go to a dead-letter queue or destination, while in the stream model a failing record can block the whole shard until it succeeds or the data expires.

API Gateway — Synchronous

API Gateway invokes Lambda synchronously and relays the function's response back to the HTTP client. The client is waiting, so your function must return within the API Gateway timeout and produce a proper HTTP-shaped response.

{
  "statusCode": 200,
  "headers": { "Content-Type": "application/json" },
  "body": "{\"message\":\"ok\"}"
}

Because the caller waits, errors are returned to the client — Lambda does not retry on your behalf. Handle failures and return a sensible status code yourself.

S3 Events — Asynchronous

S3 can notify Lambda when objects are created or deleted. This is asynchronous: S3 hands the event to Lambda and moves on. Lambda retries a failed invocation twice with delays, then sends the event to a configured dead-letter queue or on-failure destination if you've set one.

aws s3api put-bucket-notification-configuration \
  --bucket my-uploads \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [{
      "LambdaFunctionArn": "arn:aws:lambda:ap-south-1:111122223333:function:process-upload",
      "Events": ["s3:ObjectCreated:*"]
    }]
  }'

SQS and SNS

SNS is pub/sub and invokes Lambda asynchronously — one message fans out to all subscribers. SQS is a queue consumed via an event source mapping: the Lambda service polls the queue, pulls messages in batches, and deletes them only after a successful invocation. SQS is the right choice when you need buffering, backpressure, and precise control over concurrency.

# SQS → Lambda is an event source mapping, not a trigger on the function
aws lambda create-event-source-mapping \
  --function-name process-orders \
  --event-source-arn arn:aws:sqs:ap-south-1:111122223333:orders \
  --batch-size 10 \
  --maximum-batching-window-in-seconds 5

With SQS, report partial batch failures so only the failed messages return to the queue — otherwise a single bad message can cause the whole batch to be reprocessed.

EventBridge — Schedules and Events

EventBridge invokes Lambda asynchronously from two kinds of rules:

• Scheduled — cron-like, for periodic jobs (nightly cleanup, hourly report).
• Event pattern — react to AWS service events or your own custom events on an event bus.

# Run a function every day at 02:00 UTC
aws events put-rule --name nightly-cleanup \
  --schedule-expression "cron(0 2   ? *)"

DynamoDB Streams — Stream Model

A DynamoDB Stream emits an ordered record for every item change. Lambda consumes it via an event source mapping, polling shards and delivering records in order per shard. Because order matters, a record that keeps failing blocks its shard until it succeeds or the record ages out — so configure a maximumRetryAttempts and an on-failure destination to avoid a stuck pipeline.

Event Source Mappings vs Triggers

This is the mental split worth keeping straight:

• Push sources (API Gateway, S3, SNS, EventBridge) call Lambda. The configuration lives on the source (or as a resource-based permission on the function).
• Poll sources (SQS, Kinesis, DynamoDB Streams) are read by the Lambda service through an event source mapping — a separate resource that controls batch size, batching window, and concurrency.

IAM — Two Different Permissions

Lambda integrations need two kinds of permission, and mixing them up is a common failure:

Execution role — what the function is allowed to do (write to DynamoDB, read a Secret). Assumed by Lambda when it runs.

Resource-based policy / event source permission — what is allowed to invoke the function (e.g. permission for S3 or EventBridge to call it), or, for poll sources, permissions in the execution role to read the queue/stream.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueAttributes"
      ],
      "Resource": "arn:aws:sqs:ap-south-1:111122223333:orders"
    }
  ]
}

For SQS, Kinesis, and DynamoDB Streams, these read/delete permissions go in the execution role because Lambda does the polling. For S3 and EventBridge, you add a resource-based permission allowing that service to invoke the function.

Concurrency Basics

• Reserved concurrency caps how many instances of a function can run at once — useful to protect a fragile downstream database from a flood of invocations.
• Provisioned concurrency keeps a pool of initialized instances warm to eliminate cold starts for latency-sensitive paths.
• With SQS, concurrency scales with the number of batches being processed; pair reserved concurrency with batch size to control the pressure on downstream systems.

Common Mistakes to Avoid

• Assuming every source retries the same way. Sync returns errors to the caller, async retries twice then DLQs, streams retry the batch until expiry. Design error handling per model.
• Ignoring the dead-letter queue for async sources. Without a DLQ or on-failure destination, failed S3/SNS/EventBridge events are silently lost after retries.
• Not reporting partial batch failures for SQS. Returning a plain error reprocesses the entire batch, including already-successful messages.
• Confusing the execution role with invoke permission. The role governs what the function can do; a separate resource-based policy governs who can invoke it.
• No concurrency limit in front of a small database. A spike of events can open thousands of connections at once and overwhelm the downstream store. Use reserved concurrency.

FAQ

What is the difference between synchronous and asynchronous Lambda invocation?

In synchronous invocation the caller (like API Gateway) waits for the function to finish and receives its response, and the caller is responsible for handling errors and retries. In asynchronous invocation (S3, SNS, EventBridge) the event is queued and the caller gets an immediate acknowledgement, while Lambda itself retries failures twice and then sends them to a dead-letter queue or destination.

What is an event source mapping?

An event source mapping is a Lambda resource that polls a stream or queue source — SQS, Kinesis, or DynamoDB Streams — and invokes your function with batches of records. It controls batch size, batching window, and concurrency, and because the Lambda service does the polling, the permissions to read the source go in the function's execution role.

Why does my S3 or EventBridge trigger need a separate permission?

Push sources like S3 and EventBridge invoke your function directly, so Lambda needs a resource-based policy granting that specific service permission to call the function. This is separate from the execution role, which only controls what the function itself may do once it runs. Both are required for the integration to work.

How do I stop a bad message from blocking a DynamoDB stream?

Set a maximumRetryAttempts and configure an on-failure destination (such as an SQS queue) on the event source mapping, and enable bisecting on error so poison records are isolated. Without these, a record that keeps failing will be retried in order until it succeeds or the stream data expires, stalling everything behind it on that shard.