AWS SQS and Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

Amazon SQS is a distributed message queuing service with at-least-once delivery guarantees for Standard Queues. When SQS messages drive billing — a usage event lands in the queue, a worker polls it and calls stripe.charges.create() — three SQS-specific behaviors introduce Stripe double-charge failure modes that standard CloudWatch billing metrics do not surface as risk.

This post covers those three failure modes with boto3 Python code, content-hash idempotency keys stable across SQS redeliveries, per-billing-period vault keys via a spend-cap proxy, and pre-flight database checks — the two-layer governance pattern that closes all three without changing your SQS queue configuration, Lambda event source mapping, or Dead Letter Queue setup.

Failure mode 1: SQS visibility timeout expires during the stripe.charges.create() HTTP call — the message is redelivered to a second billing worker before the first charge returns

When a billing worker calls sqs.receive_message(), SQS starts a visibility timeout clock for each returned message. During the visibility timeout window, the message is hidden from other receive_message calls. When the visibility timeout expires, SQS makes the message visible again — it is available for the next poll, regardless of whether the first worker has finished processing it.

SQS Standard Queue default visibility timeout is 30 seconds. Stripe’s P99 HTTP latency for stripe.charges.create() can exceed 30 seconds under load: Stripe enforces a maximum request timeout of 80 seconds on the client side, and internal retries via stripe-python’s default retry configuration (max_network_retries=2) can push total time to 3× the single-call timeout. A billing worker that receives a message at T=0, calls Stripe at T=1, and gets a response at T=32 — just 2 seconds past the default 30-second timeout — has already lost the race. SQS made the message visible at T=30, and the next polling worker received it at T=31 and called stripe.charges.create() for the same customer at T=32.

import boto3
import stripe
import os
import json

sqs = boto3.client("sqs", region_name=os.environ["AWS_REGION"])
QUEUE_URL = os.environ["SQS_QUEUE_URL"]

def poll_and_bill():
    response = sqs.receive_message(
        QueueUrl=QUEUE_URL,
        MaxNumberOfMessages=1,
        # Default VisibilityTimeout is inherited from the queue (default 30s).
        # Stripe's stripe.charges.create() can take up to 80 seconds with retries.
        # If Stripe takes longer than 30 seconds, SQS makes this message visible
        # before the worker finishes processing it.
        WaitTimeSeconds=20,
    )

    messages = response.get("Messages", [])
    if not messages:
        return

    message = messages[0]
    receipt_handle = message["ReceiptHandle"]
    event = json.loads(message["Body"])

    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    # This call can take 30-80 seconds under load.
    # If the queue's VisibilityTimeout (30s) expires before this returns,
    # SQS requeues this message. The next polling worker calls stripe.charges.create()
    # for the same customer and billing period. Stripe creates ch_B alongside ch_A.
    charge = stripe.Charge.create(
        amount=event["amount_cents"],
        currency="usd",
        customer=event["stripe_customer_id"],
        description=f"Usage billing for {event['billing_period']}",
        # no idempotency_key — second worker creates a new charge
    )

    # delete_message uses the ReceiptHandle from the ORIGINAL receive_message call.
    # If the visibility timeout expired and a second worker already received the
    # message with a new ReceiptHandle, this delete call does nothing useful:
    # the original ReceiptHandle has already expired or the message has been
    # redelivered under a different ReceiptHandle.
    sqs.delete_message(
        QueueUrl=QUEUE_URL,
        ReceiptHandle=receipt_handle,
    )
    print(f"Charged {event['customer_id']}: {charge['id']}")

The failure is invisible in SQS’s standard CloudWatch metrics. ApproximateNumberOfMessagesNotVisible shows the message as being processed (not visible) by the first worker. When the timeout expires and the message becomes visible, ApproximateNumberOfMessagesVisible increments by 1. When the second worker deletes it after processing, NumberOfMessagesDeleted increments by 1. The first worker’s delete_message call using its original ReceiptHandle either silently succeeds (if the handle was still valid) or raises ReceiptHandleIsInvalid which many implementations swallow. From SQS’s perspective, the message was processed once. From Stripe’s perspective, the customer was charged twice.

The visibility timeout window during which this can occur is not rare in production workloads. Stripe’s own status history shows P99 latency spikes to 45-90 seconds during incidents. AWS Lambda cold starts consume 1-3 seconds before the function body runs. A Lambda with reserved_concurrency=5 that receives a burst of billing messages launches 5 concurrent invocations, all polling the same SQS queue. Each invocation’s Stripe call competes for Stripe’s per-account rate limit (100 write requests per second, reduced for unverified accounts). Throttled calls retry with exponential backoff, extending total call duration well past 30 seconds. The result: in any production billing pipeline under load, the visibility timeout expiry scenario is not an edge case — it is a predictable failure mode that requires an explicit defense.

Failure mode 2: Lambda batch processing — an unhandled exception requeues the entire batch including already-charged customers

Lambda’s SQS event source mapping delivers batches of SQS messages to a single Lambda invocation. With BatchSize=10, Lambda passes up to 10 SQS messages to the handler as event["Records"]. The billing Lambda iterates over the records and calls stripe.charges.create() for each one. If any billing call raises an unhandled exception — or if the Lambda function itself times out, runs out of memory, or raises an uncaught error — Lambda does not delete any of the messages in the batch. All 10 messages become visible in SQS again and are redelivered to the next Lambda invocation.

The failure scenario: a billing Lambda with BatchSize=10 processes 10 usage events. It successfully charges customers 1 through 7 — calling stripe.charges.create() for each and receiving ch_A through ch_G. On customer 8, the billing database write fails with a ConnectionResetError. The Lambda function raises an exception and exits. Lambda did not call delete_message for any of the 10 messages (Lambda manages SQS deletions automatically for successful invocations only — it deletes the entire batch on success, does nothing on failure). All 10 messages are redelivered. The next Lambda invocation receives the same batch and calls stripe.charges.create() for customers 1 through 7 again, creating ch_A2 through ch_G2 alongside the original ch_A through ch_G.

import boto3
import stripe
import os
import json

def lambda_handler(event, context):
    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    for record in event["Records"]:
        body = json.loads(record["body"])
        customer_id = body["customer_id"]
        billing_period = body["billing_period"]

        # Called for each record in the batch.
        # If ANY record raises an exception AFTER other records have already
        # been charged, Lambda requeues the ENTIRE batch.
        # Customers already charged in this invocation are re-charged by
        # the next invocation that receives the same batch.
        charge = stripe.Charge.create(
            amount=body["amount_cents"],
            currency="usd",
            customer=body["stripe_customer_id"],
            description=f"Usage billing for {billing_period}",
            # no idempotency_key — re-charge on batch redelivery
        )

        # If this database write raises an exception (connection reset,
        # timeout, disk full), the Lambda function exits without returning
        # a success response. Lambda does NOT delete any of the batch's
        # SQS messages. All messages — including those already successfully
        # charged above — are redelivered to the next invocation.
        write_to_billing_db(customer_id, billing_period, charge["id"])

    # Lambda automatically deletes ALL batch messages from SQS only when
    # this handler returns without raising an exception. Any uncaught
    # exception prevents ALL deletions.
    return {"statusCode": 200}

AWS provides ReportBatchItemFailures to partially address this: the Lambda function returns a batchItemFailures list containing only the itemIdentifier values for messages that failed, and Lambda requeues only those messages. Successfully processed messages are deleted. This prevents re-processing already-charged records when configured correctly. The catch: ReportBatchItemFailures requires the Lambda function to catch all exceptions internally, process as many records as possible, and build the failure list explicitly — a pattern that many billing Lambda implementations do not implement because SQS event source mapping worked correctly in testing (where exceptions are rare and batches are small).

import json
import stripe
import os

def lambda_handler(event, context):
    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]
    failed_items = []

    for record in event["Records"]:
        message_id = record["messageId"]
        try:
            body = json.loads(record["body"])

            # UNSAFE: no idempotency_key — if this record is retried
            # due to a failed batch item failure report, it creates ch_B.
            # ALSO UNSAFE: no pre-flight check — even with ReportBatchItemFailures,
            # if the Lambda itself crashes (OOM, timeout) after charging customer 5
            # but before returning the batchItemFailures response, Lambda treats the
            # invocation as a full failure and requeues ALL messages.
            charge = stripe.Charge.create(
                amount=body["amount_cents"],
                currency="usd",
                customer=body["stripe_customer_id"],
                description=f"Usage billing for {body['billing_period']}",
            )
            write_to_billing_db(body["customer_id"], body["billing_period"], charge["id"])

        except Exception as e:
            # Only this specific message is retried; others in the batch are deleted.
            # But if stripe.charges.create() succeeded and write_to_billing_db() raised,
            # the retry fires stripe.charges.create() again for a customer already charged.
            failed_items.append({"itemIdentifier": message_id})

    # Lambda requeues only the messages listed in batchItemFailures.
    # BUT: if the Lambda function itself crashes before returning this response
    # (OOM kill, Lambda 15-minute timeout, SIGKILL), Lambda never receives this
    # response and treats the entire invocation as failed — all messages requeued.
    return {"batchItemFailures": failed_items}

The Lambda-timeout edge case makes ReportBatchItemFailures a necessary but insufficient defense. A billing Lambda that processes 200 customers per invocation, each with a 5-10 second Stripe call, needs 1,000-2,000 seconds of execution time. Lambda’s maximum timeout is 15 minutes (900 seconds). Any batch sized to exceed 900 seconds of processing time will hit the Lambda timeout, preventing ReportBatchItemFailures from being returned, and triggering a full batch requeue. The correct defense is an idempotency key and pre-flight check that make individual message redelivery safe — whether the redelivery comes from a batch exception, a Lambda timeout, or an OOM kill.

Failure mode 3: DLQ redrive re-fires stripe.charges.create() for charges that succeeded server-side before the timeout response was lost

SQS Dead Letter Queues (DLQs) capture messages that exceed MaxReceiveCount — the number of times a message can be received before SQS moves it to the DLQ. The standard billing implementation catches stripe.error.Timeout, raises an exception (to avoid deleting the message from SQS), and relies on SQS redelivery to retry the charge. After MaxReceiveCount retries (commonly set to 3-5), SQS moves the message to the DLQ.

The failure scenario: a billing worker receives a usage event and calls stripe.charges.create() at T=0. Stripe processes the charge internally and creates ch_A at T=28. Stripe’s HTTP response begins traveling back to the client. At T=29, the client’s socket timeout fires (29-second client timeout, slightly shorter than the 30-second SQS visibility timeout). The client receives stripe.error.Timeout and raises an exception. The Lambda function exits without calling delete_message. The message is requeued. After 3 more retry attempts that also time out (each one receiving stripe.error.Timeout because the same overloaded Stripe endpoint keeps timing out, even though ch_A was created on the first attempt), SQS moves the message to the DLQ.

An on-call engineer sees 50 messages in the DLQ after a billing run and runs start_message_move_task to redrive them back to the source queue. SQS moves all 50 messages back. The billing worker processes each one and calls stripe.charges.create() — for 50 customers where the charge already exists as ch_A on Stripe. Without an idempotency key, Stripe creates ch_B through ch_AX for all 50 customers. The redrive is intended to retry failed billing — but 50 of those “failures” were timeout failures where the charge succeeded server-side. The redrive doubles the billing for all 50.

import boto3
import stripe
import os
import json

sqs = boto3.client("sqs", region_name=os.environ["AWS_REGION"])
QUEUE_URL = os.environ["SQS_QUEUE_URL"]
DLQ_URL = os.environ["SQS_DLQ_URL"]

def poll_and_bill():
    response = sqs.receive_message(
        QueueUrl=QUEUE_URL,
        MaxNumberOfMessages=1,
        WaitTimeSeconds=20,
        AttributeNames=["ApproximateReceiveCount"],
    )
    messages = response.get("Messages", [])
    if not messages:
        return

    message = messages[0]
    receipt_handle = message["ReceiptHandle"]
    event = json.loads(message["Body"])
    receive_count = int(message.get("Attributes", {}).get("ApproximateReceiveCount", 1))

    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    try:
        charge = stripe.Charge.create(
            amount=event["amount_cents"],
            currency="usd",
            customer=event["stripe_customer_id"],
            description=f"Usage billing for {event['billing_period']}",
            # no idempotency_key — DLQ redrive creates ch_B
        )
        sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=receipt_handle)
        print(f"Charged {event['customer_id']}: {charge['id']}")

    except stripe.error.Timeout:
        # Stripe timed out. The charge may have succeeded server-side (ch_A exists
        # on Stripe) or it may not have (no charge was created yet).
        # We don't know which. Raising here keeps the message in SQS for retry.
        # After MaxReceiveCount retries, SQS moves this message to the DLQ.
        # When an operator redrives the DLQ, this billing function fires again
        # and calls stripe.charges.create() — creating ch_B if ch_A already exists.
        raise

The DLQ scenario is particularly dangerous because the redrive is a deliberate manual action that feels like a safe recovery operation. Engineers who run DLQ redrives are typically resolving a known billing failure — they expect to charge customers who were not billed. The fact that some DLQ messages contain successful charges that timed out on the response is not visible in SQS: the message body contains a usage event, not a Stripe charge ID. ApproximateReceiveCount shows 3 or 4, indicating multiple failed attempts. Nothing in the message or SQS metadata indicates whether a charge was created. Only an idempotency key and a pre-flight database check can determine the true billing state before calling Stripe again.

The fix: content-hash idempotency key + vault key + pre-flight check

All three failure modes share the same root cause: stripe.charges.create() is called more than once for the same (customer_id, billing_period) pair, and neither SQS’s message delivery layer nor Stripe’s API layer prevents the second call from creating a new charge. The fix has two independent layers that close all three modes without changing queue configuration, Lambda batch size, DLQ policies, or retry counts.

Layer 1: content-hash idempotency key stable across SQS redeliveries and DLQ redrives

The Stripe idempotency key must be derived from the billing event’s stable business fields — fields that do not change across SQS redeliveries, Lambda retries, DLQ redrives, or queue flushes. The key must not include any SQS message metadata that changes across deliveries: MessageId (unique per SQS message copy — a redriven DLQ message gets a new MessageId in the destination queue), ReceiptHandle (unique per receive call and changes on every redelivery), ApproximateReceiveCount (increments on every receive), or ApproximateFirstReceiveTimestamp (set on first delivery and resets on DLQ redrive). Including any of these produces a different idempotency key on each delivery attempt, defeating Stripe’s 24-hour deduplication window.

import hashlib
import stripe
import os
import json
import psycopg2

def make_idempotency_key(customer_id: str, billing_period: str) -> str:
    # Derived from stable business identity: customer + billing period only.
    # Must NOT include: SQS MessageId (changes on DLQ redrive), ReceiptHandle
    # (changes on every receive_message call), ApproximateReceiveCount
    # (increments each delivery), ApproximateFirstReceiveTimestamp (resets on
    # redrive), Lambda RequestId, SQS queue URL, or batch item index.
    raw = f"{customer_id}:{billing_period}:sqs-billing"
    return hashlib.sha256(raw.encode()).hexdigest()[:32]


def process_billing_event(conn, event: dict) -> dict:
    customer_id = event["customer_id"]
    stripe_customer_id = event["stripe_customer_id"]
    amount_cents = event["amount_cents"]
    billing_period = event["billing_period"]

    idempotency_key = make_idempotency_key(customer_id, billing_period)

    # Pre-flight check: was this customer already billed for this period?
    # Authoritative source: billing_records database table with a unique
    # constraint on (customer_id, billing_period).
    # This closes the DLQ redrive failure mode: if ch_A was created before
    # the timeout response was lost, and the billing worker wrote to
    # billing_records after receiving ch_A, the pre-flight check here skips
    # the customer on redrive — no second Stripe call is made.
    # This also closes the visibility timeout failure mode when the first worker
    # managed to write to billing_records before the second worker picked up
    # the redelivered message.
    with conn.cursor() as cur:
        cur.execute(
            """
            SELECT charge_id, status FROM billing_records
            WHERE customer_id = %s AND billing_period = %s
            """,
            (customer_id, billing_period),
        )
        existing = cur.fetchone()

    if existing:
        return {
            "skipped": True,
            "reason": "already_billed",
            "charge_id": existing[0],
        }

    vault_key = os.environ["KEYBRAKE_VAULT_KEY"]   # per-billing-period key with spend cap

    charge = stripe.Charge.create(
        amount=amount_cents,
        currency="usd",
        customer=stripe_customer_id,
        description=f"Usage billing for {billing_period}",
        idempotency_key=idempotency_key,
        api_key=vault_key,   # proxied through spend-cap layer
    )

    # Write to billing_records BEFORE deleting the SQS message.
    # If the delete_message call fails after this write, SQS redelivers the
    # message. The pre-flight check above skips the customer on the next delivery.
    # ON CONFLICT DO NOTHING closes the concurrent delivery race condition:
    # if two workers both passed the pre-flight check simultaneously (visibility
    # timeout race), the database unique constraint ensures only one INSERT
    # succeeds — the second is silently ignored.
    with conn.cursor() as cur:
        cur.execute(
            """
            INSERT INTO billing_records
                (customer_id, billing_period, charge_id, amount_cents, status, created_at)
            VALUES (%s, %s, %s, %s, 'charged', NOW())
            ON CONFLICT (customer_id, billing_period) DO NOTHING
            """,
            (customer_id, billing_period, charge["id"], amount_cents),
        )
    conn.commit()

    return {"charge_id": charge["id"], "status": "charged"}

Layer 2: vault key with per-billing-period spend cap

The vault key is a per-billing-period Stripe key issued through a spend-cap proxy. The vault key enforces a maximum spend for the billing run, set at 110% of the expected total. For the DLQ redrive scenario — where 50 messages are redriven and billing is called for 50 customers, some of whom were already charged — the idempotency key closes the case where the redrive occurs within Stripe’s 24-hour deduplication window. The vault key provides the hard cap for redrives that happen days or weeks later (a DLQ message retained for 14 days, then redriven after a delayed investigation), where the 24-hour deduplication window has expired and the idempotency key alone does not prevent a second charge.

import requests
import os
import datetime

def create_billing_vault_key(
    billing_period: str,
    expected_total_cents: int,
) -> str:
    resp = requests.post(
        "https://proxy.keybrake.com/vault/keys",
        headers={"Authorization": f"Bearer {os.environ['KEYBRAKE_API_KEY']}"},
        json={
            "vendor": "stripe",
            "label": f"sqs-billing-run-{billing_period}",
            "daily_usd_cap": round((expected_total_cents * 1.10) / 100, 2),
            "allowed_endpoints": ["/v1/charges", "/v1/payment_intents"],
            "expires_at": (
                datetime.datetime.utcnow() + datetime.timedelta(hours=6)
            ).isoformat() + "Z",
        },
    )
    resp.raise_for_status()
    return resp.json()["vault_key"]

# For a DLQ redrive of 50 messages (30 timeout failures, 20 actually-successful
# charges that timed out client-side):
# - Expected billing total: $5,000 (50 customers × $100)
# - 20 already charged = $2,000 already on Stripe
# - Remaining unbilled: $3,000
# - Vault key cap: $5,500 (110% of expected total)
# - Idempotency key returns ch_A for the 20 already-charged customers
#   (within 24h window) — no new Stripe charges created for those 20
# - Vault cap ensures that even if idempotency deduplication fails (redrive
#   after 24h), spend cannot exceed $5,500 — limits worst-case double-billing
# - Issue a NEW vault key per billing period, not per redrive —
#   the per-period cap accounts for the full expected total

Safe SQS billing worker with visibility timeout extension

import boto3
import stripe
import psycopg2
import os
import json
import threading

sqs = boto3.client("sqs", region_name=os.environ["AWS_REGION"])
QUEUE_URL = os.environ["SQS_QUEUE_URL"]

def extend_visibility_timeout(receipt_handle: str, stop_event: threading.Event):
    """
    Extend the visibility timeout every 20 seconds to prevent expiry
    during a long Stripe HTTP call.
    SQS allows extending the visibility timeout up to 12 hours total.
    """
    while not stop_event.wait(timeout=20):
        try:
            sqs.change_message_visibility(
                QueueUrl=QUEUE_URL,
                ReceiptHandle=receipt_handle,
                VisibilityTimeout=60,   # extend by 60 seconds from now
            )
        except Exception:
            # ReceiptHandle becomes invalid after the message is deleted.
            # After deletion, this call raises; the stop_event will also
            # be set, so the thread exits cleanly on the next iteration.
            break


def poll_and_bill():
    conn = psycopg2.connect(os.environ["DATABASE_URL"])

    response = sqs.receive_message(
        QueueUrl=QUEUE_URL,
        MaxNumberOfMessages=1,
        WaitTimeSeconds=20,
        VisibilityTimeout=60,   # explicit initial timeout longer than default
        AttributeNames=["ApproximateReceiveCount", "MessageId"],
    )
    messages = response.get("Messages", [])
    if not messages:
        return

    message = messages[0]
    receipt_handle = message["ReceiptHandle"]
    event = json.loads(message["Body"])

    # Start a background thread that extends visibility timeout every 20 seconds.
    # This prevents SQS from redelivering the message while stripe.charges.create()
    # is in-flight, even for Stripe calls that take 60-80 seconds under load.
    stop_event = threading.Event()
    extender = threading.Thread(
        target=extend_visibility_timeout,
        args=(receipt_handle, stop_event),
        daemon=True,
    )
    extender.start()

    try:
        result = process_billing_event(conn, event)

        if result.get("skipped"):
            # Pre-flight check: already billed. Delete the message — it was
            # successfully processed (nothing to do) and should not be retried.
            sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=receipt_handle)
            return

        # Write to billing_records already happened inside process_billing_event.
        # Now delete the SQS message to prevent redelivery.
        sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=receipt_handle)
        print(f"Charged {event['customer_id']}: {result['charge_id']}")

    except stripe.error.Timeout:
        # Stripe timed out. ch_A may or may not exist.
        # Do NOT delete the SQS message — leave it for SQS to redeliver.
        # On the next delivery, the idempotency key returns ch_A if it exists,
        # and the pre-flight check skips the customer if billing_records has a row
        # (which happens only if the DB write succeeded before the timeout fired).
        # If billing_records has no row (Stripe timed out before returning ch_A),
        # the next delivery retries the charge safely with the same idempotency key.
        raise

    except stripe.error.CardError as e:
        # Permanent failure: write the decline to billing_records and delete the message.
        # Deleting prevents DLQ accumulation for unrecoverable card errors.
        with conn.cursor() as cur:
            cur.execute(
                """
                INSERT INTO billing_records
                    (customer_id, billing_period, charge_id, amount_cents, status, error, created_at)
                VALUES (%s, %s, NULL, %s, 'card_declined', %s, NOW())
                ON CONFLICT (customer_id, billing_period) DO NOTHING
                """,
                (event["customer_id"], event["billing_period"],
                 event["amount_cents"], str(e)),
            )
            conn.commit()
        sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=receipt_handle)

    finally:
        stop_event.set()   # stop the visibility timeout extender thread

Safe Lambda handler with ReportBatchItemFailures and idempotency

import stripe
import psycopg2
import hashlib
import os
import json
import logging

logger = logging.getLogger(__name__)

def lambda_handler(event, context):
    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]
    conn = psycopg2.connect(os.environ["DATABASE_URL"])

    failed_items = []

    for record in event["Records"]:
        message_id = record["messageId"]
        try:
            body = json.loads(record["body"])
            result = process_billing_event(conn, body)

            if result.get("skipped"):
                logger.info("Skipped %s: already_billed", body["customer_id"])
            else:
                logger.info("Charged %s: %s", body["customer_id"], result["charge_id"])

        except stripe.error.Timeout:
            # Add to failed items — this message is retried.
            # On retry, the idempotency key returns ch_A if Stripe created it
            # before the timeout, or creates a new charge if not.
            # The pre-flight check skips the customer if billing_records was
            # written before the timeout fired (DB write after Stripe returned
            # ch_A but before the client-side timeout fired the exception).
            failed_items.append({"itemIdentifier": message_id})

        except Exception as e:
            logger.error("Failed %s: %s", message_id, e)
            failed_items.append({"itemIdentifier": message_id})

    # Lambda requeues only messages listed in batchItemFailures.
    # Successfully processed messages are deleted automatically.
    # NOTE: if the Lambda function itself crashes (OOM, 15-minute timeout,
    # SIGKILL) before returning this response, Lambda treats the invocation
    # as a full failure and requeues ALL messages in the batch.
    # The idempotency key and pre-flight check in process_billing_event()
    # make re-processing safe for customers already billed in the crashed invocation.
    return {"batchItemFailures": failed_items}

Comparison: protection levels across all three failure modes

FAQ

Should I use an SQS FIFO Queue to prevent duplicate billing?

SQS FIFO Queues provide exactly-once processing within a 5-minute deduplication window using MessageDeduplicationId. For billing pipelines, FIFO deduplication closes the at-least-once redelivery failure mode within 5 minutes — if the same MessageDeduplicationId is sent twice within 5 minutes, SQS discards the second. However, FIFO does not protect against a visibility timeout expiry (the original message is already in the queue, not a new submission) or a DLQ redrive (redriven messages are different SQS send calls and may not match the original deduplication ID depending on how the redrive is configured). FIFO throughput is also limited to 300 transactions per second per queue without high throughput mode, versus unlimited for Standard Queues. Use FIFO as an additional layer if your producer can guarantee stable MessageDeduplicationId values, but do not rely on FIFO deduplication as a substitute for idempotency keys and pre-flight checks at the billing layer.

What is the correct SQS visibility timeout for a Stripe billing worker?

Set the visibility timeout to at least 3× the expected maximum Stripe call duration, including retries. With stripe-python’s default max_network_retries=2 and a 30-second per-attempt timeout, worst-case duration is approximately 90 seconds. A visibility timeout of 300 seconds (5 minutes) provides a comfortable buffer and is a common starting point. For Lambda workers, also implement change_message_visibility extension in a background thread or goroutine to handle batches with multiple Stripe calls, where total processing time can exceed the initial visibility timeout. The extension must be sent before the timeout expires — start extending at 50% of the timeout interval, not at 90%, to account for network latency in the extension call itself.

How does the idempotency key behave when the DLQ message is redriven after 24 hours?

Stripe’s idempotency key deduplication window is 24 hours. If a DLQ message is redriven more than 24 hours after the original charge attempt, Stripe treats the stripe.charges.create() call as a new request, even with the same idempotency key, and creates ch_B alongside ch_A. This is the case where the pre-flight database check is the critical defense. If the original billing worker wrote to billing_records after receiving ch_A (the charge succeeded server-side but the HTTP response was lost after the write), the pre-flight check finds the existing row and skips the Stripe call entirely on redrive — regardless of how much time has passed. If the charge timed out before the DB write occurred (the timeout fired during the Stripe HTTP call, before ch_A was returned), neither billing_records nor Stripe can confirm whether ch_A exists; the safe behavior is to retry with the same idempotency key and inspect the result, then write to the database.

Does ReportBatchItemFailures eliminate the need for idempotency keys?

No. ReportBatchItemFailures reduces the scope of redelivery (only failed records, not the entire batch), but it does not eliminate it. A Lambda invocation that crashes due to OOM, Lambda timeout (15 minutes), or an uncaught panic before returning the batchItemFailures response triggers a full batch requeue — Lambda has no partial response to return, so all messages are requeued. Additionally, ReportBatchItemFailures only applies to Lambda SQS event source mapping; direct SQS polling workers do not have this mechanism. The idempotency key and pre-flight check are the correct layer for duplicate protection because they operate at the business logic level, independent of the delivery mechanism, batch configuration, or Lambda invocation model.

How do I distinguish a DLQ message where the charge succeeded from one where it failed, without calling Stripe?

Query the billing_records database table for (customer_id, billing_period) before calling Stripe. A row with status = 'charged' means ch_A exists and was written to the database before the failure — skip the Stripe call and delete the SQS message. No row means either the charge failed before completion or the database write failed after the charge succeeded. In that case, call stripe.charges.create() with the same content-hash idempotency key. If ch_A exists on Stripe and the call is within the 24-hour window, Stripe returns ch_A without creating ch_B. If the 24-hour window has passed, Stripe creates ch_B — a case the vault key’s spend cap limits to expected_total × 1.10, after which the proxy blocks further charges. Write the result (whether ch_A or ch_B) to billing_records and delete the SQS message.