Google Pub/Sub and Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

Google Cloud Pub/Sub is a fully-managed messaging service with at-least-once delivery guarantees for standard subscriptions. When Pub/Sub messages drive billing — a usage event is published to a topic, a subscriber receives it and calls stripe.charges.create() — three Pub/Sub-specific behaviors introduce Stripe double-charge failure modes that GCP’s subscription metrics do not surface as billing risk.

This post covers those three failure modes with google-cloud-pubsub Python code, content-hash idempotency keys stable across Pub/Sub redeliveries, per-billing-period vault keys via a spend-cap proxy, and pre-flight database checks — the two-layer governance pattern that closes all three without changing your Pub/Sub topic configuration, subscription type, or subscriber deployment.

Failure mode 1: Pub/Sub ack deadline expires while stripe.charges.create() is in-flight — the auto-lease extension thread fails independently of the billing thread

Every Pub/Sub message has an ack deadline: the window in which the subscriber must call message.ack() before Pub/Sub considers the message unacknowledged and redelivers it to any available subscriber. The default ack deadline for a subscription is 10 seconds. At the subscription level, ack_deadline_seconds can be set up to 600 seconds.

The Python Pub/Sub client’s StreamingPullFuture, returned by subscriber.subscribe(subscription_path, callback), handles this with a background lease management thread. The lease manager calls ModifyAckDeadline RPCs on behalf of any message whose callback has not yet called ack() or nack(), extending the deadline by min_duration_per_ack_extension seconds (default: 60 seconds). This is intended to prevent expiry for messages whose processing takes longer than the subscription’s ack_deadline_seconds.

The failure scenario: a billing callback receives a usage event and calls stripe.charges.create() for a 60-second Stripe call under load. The lease manager thread is supposed to extend the ack deadline every 60 seconds. But the lease manager thread makes its own network call (ModifyAckDeadline RPC to the Pub/Sub API), and that call can fail independently of the billing callback’s network activity — GCP rate limits on ModifyAckDeadline RPCs, transient VPC egress failures, subscriber pod being marked for eviction by Kubernetes, or a GCP zone-level network partition affecting the subscriber but not Stripe’s API endpoint. When the lease manager fails to extend the deadline in time, Pub/Sub marks the message as expired and redelivers it to the next available subscriber. The second subscriber receives the same usage event, starts its own billing callback, and calls stripe.charges.create() for the same customer — creating ch_B alongside ch_A that the first subscriber’s Stripe call is still waiting to receive.

from google.cloud import pubsub_v1
import stripe
import os
import json

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
    os.environ["GCP_PROJECT_ID"],
    os.environ["PUBSUB_SUBSCRIPTION_ID"],
)

def billing_callback(message):
    event = json.loads(message.data.decode("utf-8"))
    customer_id = event["customer_id"]
    billing_period = event["billing_period"]

    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    # This call can take 30-80 seconds under load with stripe-python's
    # default max_network_retries=2. The subscription's ack_deadline_seconds
    # (default 10 seconds) will expire during this call unless the lease
    # manager thread successfully extends it via ModifyAckDeadline RPC.
    # If the lease manager's own network call fails (rate limit, pod eviction,
    # zone partition), Pub/Sub redelivers this message to a second subscriber.
    # The second subscriber calls stripe.charges.create() for the same customer.
    charge = stripe.Charge.create(
        amount=event["amount_cents"],
        currency="usd",
        customer=event["stripe_customer_id"],
        description=f"Usage billing for {billing_period}",
        # no idempotency_key — second subscriber creates ch_B alongside ch_A
    )

    # message.ack() deletes the message from the subscription.
    # If reached, the message is processed. But if the Stripe call above took
    # 61 seconds and the lease manager extended the deadline at T=60 but its
    # ModifyAckDeadline RPC failed at T=9 (the initial 10-second window), Pub/Sub
    # redelivered the message at T=10 — before message.ack() was ever called.
    message.ack()
    print(f"Charged {customer_id}: {charge['id']}")


# The lease manager thread runs inside StreamingPullFuture.
# Its network calls are independent of the billing_callback thread.
# A failure in the lease manager does not raise an exception in billing_callback
# and does not cancel the in-flight Stripe call.
future = subscriber.subscribe(subscription_path, callback=billing_callback)

The failure is invisible in standard GCP subscription metrics. subscription/num_undelivered_messages (undelivered_message_count) shows 0 during the first subscriber’s processing window (the message is in-flight). When the ack deadline expires, the metric may briefly show 1 before the second subscriber picks it up and the count returns to 0. subscription/oldest_unacked_message_age shows the message age before the second subscriber picks it up and acknowledges it. From Pub/Sub’s perspective, the message was delivered and acknowledged once (by the second subscriber). From Stripe’s perspective, the customer was charged twice.

The failure window depends on the subscriber deployment. A single-instance subscriber has no second subscriber to pick up the redelivered message immediately — Pub/Sub will redeliver to the same instance on the next poll. But in a Kubernetes deployment with 3 replicas and a FlowControlSettings.max_outstanding_messages of 10 per subscriber, all three replicas are simultaneously polling the subscription. When the ack deadline expires, Pub/Sub redelivers to whichever replica polls first — potentially a different pod than the one whose Stripe call is still in-flight. In this configuration, the duplicate-charge window is exactly equal to the ack deadline expiry time, not any kind of network-level ordering guarantee.

Failure mode 2: exactly_once_delivery is not crash-safe — the gap between stripe.charges.create() returning and message.ack() being called is not covered

Google Pub/Sub introduced exactly_once_delivery as a subscription-level attribute. When enabled (enable_exactly_once_delivery=True on the subscription), Pub/Sub guarantees that a message that has been successfully acknowledged will not be redelivered to any subscriber on that subscription. For billing pipelines that suffered from Pub/Sub’s standard at-least-once duplicate deliveries, exactly_once_delivery appears to solve the double-charge problem at the infrastructure layer.

It does not. The guarantee is conditional: Pub/Sub will not redeliver a message that was acknowledged. If a subscriber crashes, is OOM-killed, or is evicted between the moment stripe.charges.create() returns ch_A and the moment message.ack() is called, the acknowledgment was never received by Pub/Sub. From Pub/Sub’s perspective, the message was never acknowledged — it must be redelivered, because Pub/Sub has no record of the ack() that the subscriber was about to send. The exactly_once_delivery attribute does not make billing callbacks atomic with Pub/Sub acknowledgment.

from google.cloud import pubsub_v1
import stripe
import os
import json

# This subscription has exactly_once_delivery enabled.
# Engineers who enabled it believe duplicate Stripe charges are now prevented
# at the Pub/Sub layer. They are not.
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
    os.environ["GCP_PROJECT_ID"],
    "billing-subscription-exactly-once",  # enable_exactly_once_delivery=True
)

def billing_callback_exactly_once(message):
    event = json.loads(message.data.decode("utf-8"))
    customer_id = event["customer_id"]
    billing_period = event["billing_period"]

    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    # stripe.charges.create() succeeds. Stripe creates ch_A.
    # ch_A is returned to the callback. The function proceeds.
    charge = stripe.Charge.create(
        amount=event["amount_cents"],
        currency="usd",
        customer=event["stripe_customer_id"],
        description=f"Usage billing for {billing_period}",
        # no idempotency_key
    )

    # === CRASH WINDOW START ===
    # The Stripe call returned ch_A. The billing is complete from Stripe's
    # perspective. But message.ack() has not been called yet.
    # If the subscriber pod is evicted here (Kubernetes rolling update,
    # node draining, OOM kill, GCP preemptible instance eviction), the
    # acknowledgment is never sent to Pub/Sub.
    # Pub/Sub's exactly_once_delivery guarantee does NOT cover this window:
    # the message was NOT acknowledged — so Pub/Sub will redeliver it.
    # The next subscriber receives the message and calls stripe.charges.create()
    # for the same customer, creating ch_B alongside ch_A.
    # === CRASH WINDOW END ===

    write_to_billing_db(customer_id, billing_period, charge["id"])

    # message.ack() is called after write_to_billing_db. If the DB write
    # also fails, the crash window extends further. But even if the DB write
    # succeeds and only message.ack() fails, Pub/Sub redelivers.
    message.ack()
    print(f"Charged {customer_id}: {charge['id']}")

The crash window is not rare in production environments. Kubernetes rolling deployments evict running pods and start new ones; the old pod may be mid-callback when it receives SIGTERM. GCP preemptible VM instances are terminated with 30 seconds of notice — a billing callback that received the notice at T=0 may have called stripe.charges.create() at T=5 (a 60-second call) and would be killed at T=30, before the Stripe call returns. GCP’s own Cloud Run and GKE Autopilot scale-down events terminate containers with a configurable grace period that may not be long enough for in-flight Stripe calls to complete. In all of these scenarios, exactly_once_delivery provides no protection: the acknowledgment was never sent.

The crash window also has an ordering dimension when message ordering is used. If the subscription has enable_message_ordering=True and the billing event carries an ordering_key (typically the customer ID), a crash between the Stripe call and message.ack() not only triggers redelivery of the crashed message — it also blocks all subsequent messages with the same ordering_key from being delivered to any subscriber until the crashed message is acknowledged. This creates a billing backlog per customer that drains sequentially after the redelivered message is processed, potentially firing multiple Stripe calls for the same customer in rapid succession as the backlog clears.

Failure mode 3: message.nack() on stripe.error.Timeout redelivers immediately and re-fires billing for charges that already succeeded server-side

The standard Pub/Sub error-handling pattern is to call message.nack() for messages that fail processing, signaling to Pub/Sub that the message should be redelivered. Unlike message.ack(), which removes the message from the subscription, nack() makes the message immediately available for redelivery — subject to the subscription’s retry policy and backoff configuration. For transient errors (database connection lost, downstream service unavailable), this is the correct pattern: nack, redeliver, retry.

For stripe.error.Timeout, nack() is the wrong response. When Stripe raises stripe.error.Timeout, the HTTP request to Stripe’s API did not receive a complete response within the client’s timeout window. But the absence of a response does not mean the charge was not created. Stripe’s processing pipeline is fully independent of the HTTP response delivery: Stripe may have created ch_A at T=28, begun sending the HTTP response at T=29, and the client’s socket timeout may have fired at T=29.5, before the response was fully received. From the client’s perspective, the call timed out. From Stripe’s perspective, ch_A was created successfully.

from google.cloud import pubsub_v1
import stripe
import os
import json

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
    os.environ["GCP_PROJECT_ID"],
    os.environ["PUBSUB_SUBSCRIPTION_ID"],
)

def billing_callback_unsafe_nack(message):
    event = json.loads(message.data.decode("utf-8"))
    customer_id = event["customer_id"]
    billing_period = event["billing_period"]

    stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

    try:
        charge = stripe.Charge.create(
            amount=event["amount_cents"],
            currency="usd",
            customer=event["stripe_customer_id"],
            description=f"Usage billing for {billing_period}",
            # no idempotency_key — each delivery attempt creates a new charge
        )
        write_to_billing_db(customer_id, billing_period, charge["id"])
        message.ack()
        print(f"Charged {customer_id}: {charge['id']}")

    except stripe.error.Timeout:
        # Stripe timed out. ch_A may already exist on Stripe (charge succeeded
        # server-side before the timeout response was lost). We don't know.
        #
        # message.nack() makes this message immediately available for redelivery.
        # The next billing callback receives the same usage event and calls
        # stripe.charges.create() with a new request — without an idempotency key,
        # Stripe creates ch_B alongside ch_A (if ch_A exists).
        #
        # With Pub/Sub's default retry policy, redelivery can happen within seconds.
        # With exponential backoff configured (minimum_backoff=10s, maximum_backoff=600s),
        # the first retry is delayed 10 seconds — but the second, third, and
        # subsequent retries each have a 90-second potential Stripe timeout, so
        # the billing callback is called multiple times with exponential delays,
        # each creating a new charge if ch_A exists and no idempotency key is used.
        message.nack()

    except stripe.error.CardError as e:
        # Card declined — permanent failure. nack() and Pub/Sub will redeliver,
        # but the card is still declined. This creates a retry loop for an
        # unrecoverable error. The correct response is to write the decline to
        # billing_records and call message.ack() to prevent further redeliveries.
        message.nack()  # WRONG: should be message.ack() after writing decline

future = subscriber.subscribe(subscription_path, callback=billing_callback_unsafe_nack)

The nack()-on-timeout failure mode is compounded by Pub/Sub’s retry behavior. A standard subscription without a dead-letter topic configured will redeliver a nacked message indefinitely, subject only to the subscription’s retry_policy.minimum_backoff and retry_policy.maximum_backoff settings (default: minimum 10 seconds, maximum 600 seconds). A billing pipeline that uses nack() on Stripe timeouts can therefore create multiple charges for the same customer across multiple redeliveries, with the charge count growing each retry until the caller either catches the pattern or the customer disputes the charges. Without an idempotency key, each delivery attempt is a unique Stripe request — Stripe has no way to know the intent was to retry a single charge.

With a dead-letter topic configured (max_delivery_attempts), a message that has been nacked max_delivery_attempts times is forwarded to the dead-letter topic and removed from the billing subscription. An operator who later processes dead-letter messages by resubmitting them to the billing topic fires stripe.charges.create() again for every message in the dead-letter topic — including messages where the charge succeeded server-side before the timeout response was lost. This is the Pub/Sub equivalent of an SQS DLQ redrive: a deliberate recovery operation that silently creates duplicate charges for a subset of dead-letter messages.

The fix: content-hash idempotency key + vault key + pre-flight check

All three failure modes share the same root cause: stripe.charges.create() is called more than once for the same (customer_id, billing_period) pair, and neither Pub/Sub’s delivery layer nor Stripe’s API layer prevents the second call from creating a new charge. The fix has two independent layers that close all three modes without changing topic configuration, subscription type, or retry policy.

Layer 1: content-hash idempotency key stable across Pub/Sub redeliveries and dead-letter resubmissions

The Stripe idempotency key must be derived from the billing event’s stable business fields — fields that do not change across Pub/Sub redeliveries, subscriber restarts, or dead-letter resubmissions. The key must not include any Pub/Sub message metadata that changes across deliveries: message_id (Pub/Sub assigns a unique message_id when the publisher calls publish(); a message resubmitted to the billing topic from a dead-letter handler gets a new message_id), ack_id (changes on every delivery attempt — the same logical message gets a different ack_id on each redelivery), delivery_attempt (the number of delivery attempts, incremented on each redelivery), or publish_time (set when the message enters the topic). Including any of these produces a different idempotency key on each delivery attempt, defeating Stripe’s 24-hour deduplication window.

import hashlib
import stripe
import os
import json
import psycopg2

def make_idempotency_key(customer_id: str, billing_period: str) -> str:
    # Derived from stable business identity: customer + billing period only.
    # Must NOT include: Pub/Sub message_id (assigned by Pub/Sub at publish time;
    # resubmitted dead-letter messages get new message_id), ack_id (changes on
    # every delivery attempt), delivery_attempt (increments on each redelivery),
    # publish_time (timestamp assigned by Pub/Sub on original publish), or
    # subscriber ID / pod name. All change across redeliveries and topic resubmissions.
    raw = f"{customer_id}:{billing_period}:pubsub-billing"
    return hashlib.sha256(raw.encode()).hexdigest()[:32]


def process_billing_event(conn, event: dict) -> dict:
    customer_id = event["customer_id"]
    stripe_customer_id = event["stripe_customer_id"]
    amount_cents = event["amount_cents"]
    billing_period = event["billing_period"]

    idempotency_key = make_idempotency_key(customer_id, billing_period)

    # Pre-flight check: was this customer already billed for this period?
    # Authoritative source: billing_records with a unique constraint on
    # (customer_id, billing_period). This closes the crash-between-charge-and-ack
    # failure mode: if the subscriber wrote to billing_records after receiving ch_A
    # but before message.ack() was called, and then crashed, the pre-flight check
    # finds the existing row and skips the Stripe call on redelivery.
    # Also closes the nack()-on-timeout case where billing_records was written
    # before the timeout fired and the message was nacked.
    with conn.cursor() as cur:
        cur.execute(
            """
            SELECT charge_id, status FROM billing_records
            WHERE customer_id = %s AND billing_period = %s
            """,
            (customer_id, billing_period),
        )
        existing = cur.fetchone()

    if existing:
        return {
            "skipped": True,
            "reason": "already_billed",
            "charge_id": existing[0],
        }

    vault_key = os.environ["KEYBRAKE_VAULT_KEY"]   # per-billing-period key with spend cap

    charge = stripe.Charge.create(
        amount=amount_cents,
        currency="usd",
        customer=stripe_customer_id,
        description=f"Usage billing for {billing_period}",
        idempotency_key=idempotency_key,
        api_key=vault_key,   # proxied through spend-cap layer
    )

    # Write to billing_records BEFORE calling message.ack().
    # Write-before-ack: if the message.ack() call fails after this write,
    # Pub/Sub redelivers the message. The pre-flight check above skips the
    # customer on the next delivery — no second Stripe call is made.
    # ON CONFLICT DO NOTHING closes the concurrent delivery race:
    # if two subscribers both passed the pre-flight check simultaneously
    # (ack deadline expired just before one subscriber's billing call returned),
    # the unique constraint ensures only one INSERT succeeds — the second is silently ignored.
    with conn.cursor() as cur:
        cur.execute(
            """
            INSERT INTO billing_records
                (customer_id, billing_period, charge_id, amount_cents, status, created_at)
            VALUES (%s, %s, %s, %s, 'charged', NOW())
            ON CONFLICT (customer_id, billing_period) DO NOTHING
            """,
            (customer_id, billing_period, charge["id"], amount_cents),
        )
    conn.commit()

    return {"charge_id": charge["id"], "status": "charged"}

Layer 2: vault key with per-billing-period spend cap

The vault key is a per-billing-period Stripe key issued through a spend-cap proxy. The vault key enforces a maximum spend for the billing run, set at 110% of the expected total. For the nack()-on-timeout failure mode — where a billing message is redelivered multiple times after each Stripe timeout, potentially creating multiple charges across exponential backoff retries — the idempotency key closes the case within Stripe’s 24-hour deduplication window. The vault key provides the hard cap when the idempotency key has expired (a dead-letter message retained for 5 days, then resubmitted to the billing topic), ensuring that even worst-case redelivery behavior cannot exceed expected_total × 1.10 in total Stripe charges.

import requests
import os
import datetime

def create_billing_vault_key(
    billing_period: str,
    expected_total_cents: int,
) -> str:
    resp = requests.post(
        "https://proxy.keybrake.com/vault/keys",
        headers={"Authorization": f"Bearer {os.environ['KEYBRAKE_API_KEY']}"},
        json={
            "vendor": "stripe",
            "label": f"pubsub-billing-run-{billing_period}",
            "daily_usd_cap": round((expected_total_cents * 1.10) / 100, 2),
            "allowed_endpoints": ["/v1/charges", "/v1/payment_intents"],
            "expires_at": (
                datetime.datetime.utcnow() + datetime.timedelta(hours=6)
            ).isoformat() + "Z",
        },
    )
    resp.raise_for_status()
    return resp.json()["vault_key"]

# For a billing run with 100 customers at $50 each = $5,000 expected total:
# - Vault key cap: $5,500 (110% of expected total)
# - If nack()-on-timeout creates 3 redeliveries for 10 customers (10 × $50 × 3 = $1,500):
#   - Idempotency key deduplicates charges within 24h of original attempt (most cases)
#   - Vault cap blocks further charges once $5,500 is reached (covers expired-idempotency edge)
#   - Per-billing-period key: issue one vault key per billing run, not per message

Safe Pub/Sub billing subscriber with write-before-ack

from google.cloud import pubsub_v1
import stripe
import psycopg2
import hashlib
import os
import json
import logging

logger = logging.getLogger(__name__)

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
    os.environ["GCP_PROJECT_ID"],
    os.environ["PUBSUB_SUBSCRIPTION_ID"],
)

def billing_callback_safe(message):
    try:
        event = json.loads(message.data.decode("utf-8"))
        conn = psycopg2.connect(os.environ["DATABASE_URL"])

        result = process_billing_event(conn, event)

        if result.get("skipped"):
            # Pre-flight check found existing billing record — customer already charged.
            # Acknowledge the message: it was processed correctly (nothing to do).
            # Do NOT nack: nacking would redeliver and hit the pre-flight check again,
            # wasting a Pub/Sub delivery attempt and incrementing delivery_attempt count.
            message.ack()
            logger.info("Skipped %s/%s: already_billed (%s)",
                        event["customer_id"], event["billing_period"],
                        result.get("charge_id"))
            return

        # billing_records write completed inside process_billing_event before we reach here.
        # Acknowledge the message — Pub/Sub will not redeliver it.
        # If message.ack() itself fails (Pub/Sub API error), Pub/Sub redelivers the message.
        # The next delivery hits the pre-flight check (billing_records row exists) and
        # skips the Stripe call, calling message.ack() again safely.
        message.ack()
        logger.info("Charged %s/%s: %s",
                    event["customer_id"], event["billing_period"],
                    result.get("charge_id"))

    except stripe.error.Timeout:
        # Stripe timed out. ch_A may or may not exist server-side.
        # Do NOT call message.nack() here — nack() triggers immediate redelivery.
        # Instead, let the ack deadline expire naturally (or extend it via
        # modify_ack_deadline). On redelivery, the idempotency key returns ch_A
        # if it exists within the 24-hour window. The pre-flight check skips
        # the customer if billing_records was written before the timeout fired.
        # If neither defense applies (charge created, DB write failed, idempotency
        # window expired), the vault key's spend cap limits worst-case overspend.
        logger.warning("Stripe timeout for %s — leaving for Pub/Sub redelivery",
                       event.get("customer_id"))
        # Do not call message.ack() or message.nack().
        # The ack deadline will expire and Pub/Sub will redeliver with backoff.

    except stripe.error.CardError as e:
        # Card declined — permanent failure. Write the decline to billing_records
        # to prevent re-charging on redelivery. Then ack the message to prevent
        # Pub/Sub from redelivering an unrecoverable error indefinitely.
        try:
            conn = psycopg2.connect(os.environ["DATABASE_URL"])
            with conn.cursor() as cur:
                cur.execute(
                    """
                    INSERT INTO billing_records
                        (customer_id, billing_period, charge_id, amount_cents, status, error, created_at)
                    VALUES (%s, %s, NULL, %s, 'card_declined', %s, NOW())
                    ON CONFLICT (customer_id, billing_period) DO NOTHING
                    """,
                    (event["customer_id"], event["billing_period"],
                     event["amount_cents"], str(e)),
                )
                conn.commit()
        finally:
            message.ack()   # do not redeliver unrecoverable card errors

    except Exception as e:
        logger.error("Billing failed for %s: %s", event.get("customer_id"), e)
        # For unknown errors, let the ack deadline expire for redelivery.
        # The idempotency key and pre-flight check make the next delivery safe.


# FlowControlSettings limits the number of messages held in memory simultaneously.
# Setting max_outstanding_messages to 1 prevents concurrent Stripe calls within
# a single subscriber instance (sequential processing per subscriber).
# Across multiple subscriber replicas, the pre-flight check + unique constraint
# closes the concurrent-billing race condition.
flow_control = pubsub_v1.types.FlowControl(max_outstanding_messages=1)
future = subscriber.subscribe(
    subscription_path,
    callback=billing_callback_safe,
    flow_control=flow_control,
)

Comparison: protection levels across all three failure modes

FAQ

Does enabling exactly_once_delivery on my Pub/Sub subscription prevent Stripe double charges?

No. exactly_once_delivery prevents Pub/Sub from redelivering a message that was successfully acknowledged. It does not make acknowledgment atomic with your billing callback: if a subscriber crashes, is evicted by Kubernetes, or receives SIGTERM between the moment stripe.charges.create() returns and the moment message.ack() is called, the acknowledgment was never sent. Pub/Sub will redeliver the message. The correct defense is an idempotency key and a pre-flight database check that make redelivery safe at the business logic layer, independent of Pub/Sub’s delivery semantics. Use exactly_once_delivery for subscriptions where exactly-once delivery of the same logical event matters (event sourcing, dedup-sensitive systems), but do not use it as a substitute for idempotent billing code.

What ack deadline should I set for a billing subscriber that calls Stripe?

Set ack_deadline_seconds to at least 3× the expected maximum Stripe call duration including retries. With stripe-python’s default max_network_retries=2 and a 30-second per-attempt timeout, worst-case is approximately 90 seconds. A subscription ack_deadline_seconds of 300 seconds (the GCP Cloud Console maximum for standard subscriptions; 600 seconds via the API) provides a comfortable buffer. Additionally, ensure the StreamingPullFuture lease manager’s min_duration_per_ack_extension is set to a value shorter than ack_deadline_seconds — the default extension of 60 seconds means the lease manager must successfully extend at least once during a 90-second Stripe call. Set min_duration_per_ack_extension to 120 seconds and monitor subscription/mod_ack_deadline_request_count to verify lease extensions are succeeding.

How does the idempotency key behave when a Pub/Sub dead-letter message is resubmitted after 24 hours?

Stripe’s idempotency key deduplication window is 24 hours. If a dead-letter message is resubmitted to the billing topic more than 24 hours after the original charge attempt, Stripe treats the stripe.charges.create() call as a new request, even with the same content-hash idempotency key, and creates ch_B alongside ch_A. This is the case where the pre-flight database check is the critical defense. If the original subscriber wrote to billing_records after receiving ch_A (charge succeeded server-side, response received, but the message.ack() call failed or the pod was evicted immediately after the DB write), the pre-flight check finds the existing row and skips the Stripe call entirely — regardless of how much time has passed since the original charge. If neither the idempotency key nor the pre-flight check applies (charge created before a timeout response was lost, and the DB write never completed), the vault key’s spend cap limits the worst-case overspend to expected_total × 1.10.

Should I use message.nack() or let the ack deadline expire for Stripe timeout errors?

Let the ack deadline expire — do not call message.nack(). nack() triggers immediate redelivery, which can result in multiple rapid-fire stripe.charges.create() calls for the same message before Pub/Sub’s backoff configuration (if any) applies. Letting the ack deadline expire allows Pub/Sub’s retry policy (retry_policy.minimum_backoff and retry_policy.maximum_backoff) to pace redeliveries. For the billing callback, the correct behavior on stripe.error.Timeout is to log the error and return without calling either ack() or nack() — Pub/Sub will redeliver when the ack deadline expires. On the next delivery, the content-hash idempotency key returns ch_A from Stripe if it was created before the timeout response was lost (within 24 hours). The pre-flight check skips the customer if billing_records was written before the timeout fired. Both defenses make the redelivery safe without explicit nack().

How do I handle message ordering and billing without creating a nack-induced queue stall?

When using enable_message_ordering=True and an ordering_key per customer, a Stripe timeout that causes the ack deadline to expire (instead of an explicit nack()) will still block subsequent messages with the same ordering key from being delivered until the expired message is reprocessed and acknowledged. To avoid cascading billing delays when a customer’s billing message times out: set a per-subscriber FlowControlSettings.max_outstanding_messages of 1 (process one message at a time per subscriber instance), and size the subscription’s ack_deadline_seconds high enough that the ack deadline does not expire during a worst-case Stripe call. For the rare case where the Stripe P99 tail exceeds even a 600-second deadline, the pre-flight check and idempotency key make eventual redelivery safe — the stall resolves when the message is redelivered and the idempotency key returns the correct ch_A. Avoid using ordering keys if your billing pipeline uses partition-based ordering at the event source: two layers of ordering can compound delivery delays without adding billing correctness guarantees.