Spring WebFlux and Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

Spring WebFlux’s reactive model makes retry deceptively easy to wire up and deceptively easy to get wrong. Mono.retryWhen(Retry.backoff(3, Duration.ofSeconds(1))) looks like a safe retry policy, but whether it produces a duplicate Stripe charge depends entirely on where in the reactive chain the idempotency key was computed: a key derived inside Mono.defer() re-evaluates on every re-subscription, a Flux.retryWhen() at the wrong abstraction level re-submits billing calls for customers who were already charged, and a @Scheduled method that returns Mono<Void> is silently discarded by Spring — the common fix of adding .subscribe() to make it run then exposes a three-pod Kubernetes race with no distributed lock.

This post covers all three failure modes with Java code, content-hash idempotency keys stable across defer re-evaluations, Flux re-subscriptions, and multi-pod scheduler firings, pre-flight PostgreSQL ON CONFLICT DO NOTHING checks — and per-billing-period vault keys via a spend-cap proxy as a hard backstop. For AOP-level retry failure modes that re-invoke entire method bodies, see the Micronaut and Stripe Integration post. For transport-level retry interceptors in RPC systems, see the gRPC and Stripe Integration post.

Failure mode 1: UUID.randomUUID() inside Mono.defer() before retryWhen() — the defer factory re-evaluates on every re-subscription — the first attempt created ch_A — the retry creates ch_B

Mono.defer() is a factory operator: its lambda argument is executed on every subscription to the returned Mono. This is what makes defer useful for creating cold publishers — lazy evaluation, fresh state per subscriber. retryWhen() works by catching errors from the upstream publisher and re-subscribing to it after each retryable failure. The combination means that any code inside the defer lambda runs again on each retry attempt. A UUID.randomUUID() call inside the lambda produces a different value each time:

// Spring WebFlux + Java — BillingService.java
// UNSAFE: UUID.randomUUID() inside Mono.defer() — re-evaluates on every subscription
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;
import reactor.util.retry.Retry;
import java.time.Duration;
import java.util.Map;
import java.util.UUID;

@Service
public class BillingService {

    private final WebClient stripeClient;
    private final String stripeApiKey;

    public BillingService(WebClient.Builder builder,
                          @Value("${stripe.api-key}") String stripeApiKey) {
        this.stripeClient = builder.baseUrl("https://api.stripe.com").build();
        this.stripeApiKey = stripeApiKey;
    }

    public Mono<ChargeResponse> chargeBilling(String customerId,
                                               String billingPeriod,
                                               long amountCents) {
        return Mono.defer(() -> {

            // UNSAFE: Mono.defer() runs this lambda on every subscription.
            // retryWhen() re-subscribes the upstream Mono on each retryable error.
            // Attempt 1: UUID = "3f7a9b2c-1d4e-4a5f-8b6c-0e2d1a3c7f9e"
            // Attempt 2: UUID = "b8d2e4f6-3a7c-4e9f-1b5d-0c8a7e2f4b6d"  ← different
            // Attempt 3: UUID = "1c9d3e5a-7f2b-4d8c-6e0f-4a3b9c1d5e7f"  ← different again
            String idempotencyKey = UUID.randomUUID().toString();

            Map<String, Object> body = Map.of(
                "amount",      amountCents,
                "currency",    "usd",
                "customer",    customerId,
                "description", "Billing period " + billingPeriod
            );

            return stripeClient.post()
                .uri("/v1/charges")
                .header("Authorization", "Bearer " + stripeApiKey)
                .header("Idempotency-Key", idempotencyKey)
                .bodyValue(body)
                .retrieve()
                .bodyToMono(ChargeResponse.class);

        }).retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
            .filter(e -> e instanceof WebClientResponseException wre
                      && wre.getStatusCode().is5xxServerError()));
    }
}

The failure scenario: the agent calls billingService.chargeBilling("cust_123", "2026-08", 9900L). The returned Mono is subscribed to (either by the agent’s reactive pipeline or a .block() call). Mono.defer()’s lambda executes. UUID.randomUUID() returns "3f7a9b2c...". The WebClient sends POST /v1/charges with Idempotency-Key: 3f7a9b2c.... Stripe’s servers receive the request and begin processing: the charge object ch_A is created and the card is authorized. Before Stripe flushes the HTTP response, its API gateway returns a transient 503 Service Unavailable — a brief overload spike on Stripe’s side, the kind that Stripe’s own documentation recommends retrying. WebClient throws a WebClientResponseException with status 503. retryWhen’s filter matches the 5xx status. After a 1-second backoff, Reactor re-subscribes to the upstream Mono. Mono.defer()’s lambda executes again. UUID.randomUUID() returns "b8d2e4f6..." — a completely different UUID. The second POST /v1/charges carries Idempotency-Key: b8d2e4f6.... Stripe has never seen this key. ch_A was already created and committed. Stripe creates ch_B. Customer 123 is charged $99 twice for August 2026.

A subtler variant of the same failure occurs when the UUID.randomUUID() call sits outside the Mono.defer() lambda but inside the billing method itself, and a higher-level retry re-invokes the method rather than re-subscribing the same Mono:

// LOOKS safe (UUID outside defer) — but only within a single Mono subscription chain.
// UNSAFE when the CALLER's reactive chain retries by calling chargeBilling() again.
public Mono<ChargeResponse> chargeBilling(String customerId,
                                           String billingPeriod,
                                           long amountCents) {
    // UUID is computed once per chargeBilling() invocation — not per subscription.
    // The retryWhen() below re-uses the same UUID across all its re-subscriptions.
    // So THIS retryWhen is safe.
    String idempotencyKey = UUID.randomUUID().toString();

    return stripeClient.post()
        .uri("/v1/charges")
        .header("Idempotency-Key", idempotencyKey)
        .bodyValue(buildBody(customerId, billingPeriod, amountCents))
        .retrieve()
        .bodyToMono(ChargeResponse.class)
        .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));
}

// UNSAFE at the call site — the caller's retryWhen re-invokes chargeBilling():
customerFlux
    .flatMap(customerId ->
        billingService.chargeBilling(customerId, billingPeriod, amountCents))
    .retryWhen(Retry.backoff(3, Duration.ofSeconds(5)));  // ← re-calls chargeBilling()
                                                          //   on re-subscription,
                                                          //   UUID.randomUUID() fires again

The key distinction: UUID.randomUUID() computed in the method body is stable across a single Mono’s internal retryWhen re-subscriptions (because the lambda captures the already-computed value). It becomes unstable when the outer reactive pipeline re-calls the method — each invocation of chargeBilling() evaluates UUID.randomUUID() afresh. This is the same failure mode in a different location in the call graph, and it produces ch_B when ch_A was created by the first invocation.

The fix for failure mode 1

The idempotency key must be derived deterministically from the business inputs of the billing operation rather than computed with UUID.randomUUID() at subscription time. A SHA-256 hash of the customer ID and billing period is stable across every defer re-evaluation and every re-invocation of the method:

// Safe: content-hash key derived from business fields — identical on every evaluation
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

public static String stableKey(String customerId, String billingPeriod) {
    try {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        byte[] hash = md.digest(
            (customerId + ":" + billingPeriod + ":spring-webflux-billing")
                .getBytes(StandardCharsets.UTF_8));
        StringBuilder sb = new StringBuilder(32);
        for (int i = 0; i < 16; i++) sb.append(String.format("%02x", hash[i]));
        return sb.toString();  // 32 hex chars, well within Stripe's 255-char limit
    } catch (NoSuchAlgorithmException e) {
        throw new IllegalStateException(e);
    }
}

// What to EXCLUDE from the key — any value that changes between evaluations:
//   UUID.randomUUID()             ← random, different per evaluation
//   System.currentTimeMillis()    ← different if retry fires 1s later
//   System.nanoTime()             ← different per JVM, per invocation
//   System.identityHashCode(obj)  ← differs per instance, per JVM
//   Thread.currentThread().getId() ← differs if retry runs on different thread

// Safe BillingService — stableKey computed before Mono.defer()
@Service
public class SafeBillingService {

    private final WebClient stripeClient;
    private final String stripeApiKey;
    private final BillingRepository billingRepository;

    public Mono<ChargeResponse> chargeBilling(String customerId,
                                               String billingPeriod,
                                               long amountCents) {
        // Safe: stableKey is computed once per method call from deterministic inputs.
        // Mono.defer() re-evaluates its lambda on each retry — but idempotencyKey
        // is captured from the outer scope, not re-computed inside the lambda.
        String idempotencyKey = stableKey(customerId, billingPeriod);

        return Mono.defer(() ->
            billingRepository.findByCustomerAndPeriod(customerId, billingPeriod)
                .flatMap(existing -> {
                    if (existing.chargeId() != null) {
                        return Mono.just(new ChargeResponse(existing.chargeId(), "succeeded"));
                    }
                    return billingRepository
                        .claimSlot(customerId, billingPeriod, idempotencyKey)
                        .flatMap(claimed -> {
                            if (!claimed) {
                                return billingRepository
                                    .findByCustomerAndPeriod(customerId, billingPeriod)
                                    .map(r -> new ChargeResponse(r.chargeId(), "succeeded"));
                            }
                            return stripeClient.post()
                                .uri("/v1/charges")
                                .header("Authorization", "Bearer " + stripeApiKey)
                                .header("Idempotency-Key", idempotencyKey)
                                .bodyValue(Map.of(
                                    "amount",      amountCents,
                                    "currency",    "usd",
                                    "customer",    customerId,
                                    "description", "Billing period " + billingPeriod
                                ))
                                .retrieve()
                                .bodyToMono(ChargeResponse.class)
                                .flatMap(resp ->
                                    billingRepository
                                        .markComplete(customerId, billingPeriod, resp.id())
                                        .thenReturn(resp));
                        });
                })
        ).retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
            .filter(e -> e instanceof WebClientResponseException wre
                      && wre.getStatusCode().is5xxServerError()));
    }
}

-- Pre-flight table: INSERT ... ON CONFLICT DO NOTHING as durable billing mutex
CREATE TABLE billing_records (
    customer_id      TEXT        NOT NULL,
    billing_period   TEXT        NOT NULL,
    idempotency_key  TEXT        NOT NULL,
    status           TEXT        NOT NULL DEFAULT 'pending',
    charge_id        TEXT,
    created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    CONSTRAINT billing_records_pk  PRIMARY KEY (idempotency_key),
    CONSTRAINT billing_records_uq  UNIQUE (customer_id, billing_period)
);

-- claimSlot(): atomic claim — returns true if this caller won the slot
INSERT INTO billing_records (customer_id, billing_period, idempotency_key)
VALUES ($1, $2, $3)
ON CONFLICT (customer_id, billing_period) DO NOTHING;
-- returns 1 row if inserted (claimed), 0 rows if conflict (already claimed)

-- markComplete(): write the charge_id after Stripe responds
UPDATE billing_records
   SET status    = 'completed',
       charge_id = $3
 WHERE customer_id    = $1
   AND billing_period = $2;

With the content-hash key, the Mono.defer() lambda captures idempotencyKey from the enclosing scope rather than recomputing it. Every re-subscription carries the same Idempotency-Key header. Stripe’s idempotency cache returns the same charge object (ch_A) for every re-attempt that carries the same key within the 24-hour window. The billing_records pre-flight provides a database-level guard that survives Stripe’s 24-hour window and cross-pod races.

Failure mode 2: Flux.retryWhen() at the wrong level wrapping a flatMap over customer IDs — the entire reactive chain re-subscribes on the first failure — customers already charged on the first subscription get billed again

A batch billing job processes a list of customers by streaming them through a flatMap. The natural place to put retryWhen seems like it should be at the outermost level so that transient failures are retried automatically. But applying retryWhen to the entire Flux rather than to individual customer billing calls means that a failure from customer 451 triggers a re-subscription of the entire pipeline — including re-processing customers 1 through 450 who were already successfully charged:

// BatchBillingService.java
// UNSAFE: retryWhen at Flux level — entire pipeline re-subscribes on any failure
@Service
public class BatchBillingService {

    private final BillingService billingService;
    private final CustomerRepository customerRepository;

    public Mono<Void> runMonthlyBilling(String billingPeriod) {
        return customerRepository.findAllActive()   // Flux<Customer> emitting 500 customers
            .flatMap(customer ->
                billingService.chargeBilling(
                    customer.id(), billingPeriod, customer.amountCents()))
            .retryWhen(Retry.backoff(3, Duration.ofSeconds(5)))  // ← WRONG LEVEL
            .then();
    }
}

The failure scenario in detail: customerRepository.findAllActive() emits 500 Customer objects. flatMap processes them concurrently with its default concurrency of 256. The first 450 customers are billed successfully — ch_1A through ch_450A exist in Stripe. On customer 451, the Stripe API returns a 500 Internal Server Error due to a transient database hiccup on Stripe’s side. The WebClientResponseException propagates up through flatMap to the outermost retryWhen operator. retryWhen classifies this as a retryable error. After a 5-second delay, Reactor re-subscribes to the entire Flux from the beginning.

Re-subscribing from the beginning means: customerRepository.findAllActive() is called again and emits all 500 customers again. flatMap calls billingService.chargeBilling() for every customer again — including cust_1 through cust_450, which were already successfully charged. If billingService.chargeBilling() generates a UUID.randomUUID() idempotency key, each re-call produces a new UUID and a new Stripe charge: ch_1B through ch_450B are created alongside ch_1A through ch_450A. Five hundred customers were charged once; 450 of them are now charged twice.

Even if billingService.chargeBilling() uses the content-hash stableKey() fix from failure mode 1, the outer retryWhen still triggers 450 redundant Stripe API calls for already-completed charges. Stripe’s idempotency cache returns the cached ch_xA responses without creating new charges — which is correct — but the 450 extra HTTP requests create unnecessary latency, consume Stripe API rate limit quota (10,000 requests per second for most accounts, but rate limits apply per key), and extend the total billing job runtime by minutes. If a network partition separating your cluster from Stripe lasts longer than 24 hours — rare, but not impossible in a prolonged incident — Stripe’s idempotency cache for the original keys expires and the re-subscribed calls create new charges anyway.

The fix for failure mode 2

Apply retryWhen at the level of the individual customer billing call, inside the flatMap mapper. A retry on a single customer’s Mono re-subscribes only that customer’s billing pipeline, not the entire customer stream:

// Safe: retryWhen inside flatMap — retries one customer's billing call
@Service
public class SafeBatchBillingService {

    public Mono<Void> runMonthlyBilling(String billingPeriod) {
        return customerRepository.findAllActive()
            .flatMap(customer ->
                billingService
                    .chargeBilling(customer.id(), billingPeriod, customer.amountCents())
                    .retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
                        .filter(e -> e instanceof WebClientResponseException wre
                                  && wre.getStatusCode().is5xxServerError()))
                    .onErrorResume(e -> {
                        // Log and continue — failed customer goes to dead-letter table
                        log.error("Billing failed for customer {}: {}", customer.id(), e.getMessage());
                        return billingRepository.markFailed(customer.id(), billingPeriod, e.getMessage())
                                                .then(Mono.empty());
                    }),
                /* concurrency = */ 10  // limit concurrent Stripe calls to avoid rate limits
            )
            .then();
        // No retryWhen at this level — let individual customer failures be handled above
    }
}

With retryWhen inside the flatMap mapper, a 503 from Stripe for customer 451 causes only customer 451’s Mono to be retried. Customers 1 through 450 are not re-subscribed. The onErrorResume catches failures that exhaust all retries and routes them to a dead-letter table for manual review rather than crashing the entire batch. The concurrency parameter on flatMap limits simultaneous Stripe API calls to a configurable number, which prevents the batch from exhausting Stripe’s rate limits when processing large customer sets.

A related mistake is applying retryWhen after a buffer() or window() operator that groups customers into batches before billing. If the batch-level retry re-subscribes, the entire batch is reprocessed — the same failure mode but at a smaller scale. The rule is consistent: retryWhen belongs at the innermost scope that corresponds to a single idempotent unit of work.

Failure mode 3: @Scheduled returning Mono<Void> — Spring discards the returned publisher — the .subscribe() fix runs the job but exposes a three-pod Kubernetes race with different UUIDs per pod

Spring’s @Scheduled annotation and its ThreadPoolTaskScheduler infrastructure were designed for blocking methods that return void. In Spring Boot 2.x applications (and Spring Boot 3.x without additional reactive scheduling configuration), if a @Scheduled method returns a Mono<Void>, Spring’s scheduler calls the method, receives the Mono object, and discards it without subscribing. The reactive chain is assembled but never executed. No billing happens, no exception is thrown, and no warning is logged. The method appears to run on schedule — the Spring scheduler does invoke it — but the reactive pipeline it returns is silently dropped:

// BillingScheduler.java — the silent trap
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import reactor.core.publisher.Mono;

@Component
public class BillingScheduler {

    private final BatchBillingService batchBillingService;

    @Scheduled(cron = "0 0 1 * * *")  // 1st of every month at 01:00 UTC
    public Mono<Void> runMonthlyBilling() {
        // TRAP: Spring's ThreadPoolTaskScheduler calls this method, gets back a Mono<Void>,
        // and discards it. The Mono is never subscribed. No billing executes.
        // No exception. No log line. Complete silence.
        return batchBillingService.runMonthlyBilling(currentBillingPeriod());
    }

    private String currentBillingPeriod() {
        return java.time.YearMonth.now().toString();  // "2026-08"
    }
}

The symptom: the monthly billing cron fires but no charge.created events appear in Stripe’s dashboard. Revenue is zero for the month. Logs show the scheduled method was invoked at the correct time. No stack traces. A developer investigating this often looks at the code, sees the method body assembles a reactive chain and returns it, and concludes the chain must be running — forgetting that in reactive programming a Mono is a description of work, not the work itself. Nothing executes until something subscribes.

The natural fix is to change the method to return void and subscribe inside the method body:

// The common "fix" — now it runs, but introduces a three-pod race
@Component
public class BillingScheduler {

    @Scheduled(cron = "0 0 1 * * *")
    public void runMonthlyBilling() {
        // Changed return type to void — Spring now "works"
        // Added .subscribe() — the reactive chain now executes
        batchBillingService.runMonthlyBilling(currentBillingPeriod())
            .subscribe(
                null,
                e -> log.error("Monthly billing failed: {}", e.getMessage())
            );
    }
}

This fix makes the billing job run. But with replicas: 3 in the Kubernetes deployment manifest, all three pods are running the same Spring application with the same cron schedule. All three pods fire runMonthlyBilling() at 01:00:00 UTC. All three pods call .subscribe(). All three pods begin streaming customers from the shared PostgreSQL database. All three pods call billingService.chargeBilling() for each customer simultaneously.

If billingService.chargeBilling() uses UUID.randomUUID() anywhere in its implementation, each pod generates different keys per customer. Pod A calls Stripe for cust_1 with key "3f7a9b2c..." (ch_1A). Pod B calls Stripe for cust_1 with key "b8d2e4f6..." (ch_1B). Pod C calls Stripe for cust_1 with key "1c9d3e5a..." (ch_1C). All three calls arrive at Stripe within milliseconds of each other. Stripe creates ch_1A, ch_1B, and ch_1C — three real charges for the same customer in the same billing period. Across 500 customers, this produces 1,500 Stripe charges where 500 were intended.

Even if billingService.chargeBilling() uses the content-hash stableKey() fix — meaning all three pods send the same idempotency key for the same customer — Stripe’s idempotency handling for concurrent requests with identical keys returns a 409 Conflict for the second and third concurrent requests: Stripe’s idempotency layer serializes concurrent requests with the same key using an internal lock, so the second pod’s request either waits for the first pod’s to complete and then returns the cached result (if the first succeeded) or returns a 409. The 409 is retryable by the pre-flight guard (both pods see DO NOTHING on the billing_records table and short-circuit). But the application is still doing three times the database and Stripe API work it should, and the duplicate reactive chains streaming all 500 customers from the database simultaneously create significant unnecessary load on the database connection pool.

The fix for failure mode 3

The complete fix requires two layers: first, the content-hash idempotency key and pre-flight database guard from failure mode 1 as the cluster-wide billing mutex; second, a distributed scheduling lock that ensures only one pod executes the billing job at all. The content-hash layer handles the case where the lock is unavailable or expires mid-job; the distributed lock eliminates the triple redundant workload under normal operation.

ShedLock is the standard library for this in Spring Boot applications. It works by writing a lock row to a shared database table at job start and releasing it at job end. Only the pod that successfully inserts the lock row runs the job; the other two pods attempt to acquire the lock, fail, and skip execution:

-- ShedLock schema (PostgreSQL)
CREATE TABLE shedlock (
    name       VARCHAR(64)  NOT NULL,
    lock_until TIMESTAMPTZ  NOT NULL,
    locked_at  TIMESTAMPTZ  NOT NULL,
    locked_by  VARCHAR(255) NOT NULL,
    PRIMARY KEY (name)
);

// build.gradle — ShedLock dependency
implementation 'net.javacrumbs.shedlock:shedlock-spring:5.13.0'
implementation 'net.javacrumbs.shedlock:shedlock-provider-jdbc-template:5.13.0'

// SchedulingConfig.java
import net.javacrumbs.shedlock.core.LockProvider;
import net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider;
import net.javacrumbs.shedlock.spring.annotation.EnableSchedulerLock;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;
import javax.sql.DataSource;

@Configuration
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "PT4H")
public class SchedulingConfig {

    @Bean
    public LockProvider lockProvider(DataSource dataSource) {
        return new JdbcTemplateLockProvider(
            JdbcTemplateLockProvider.Configuration.builder()
                .withJdbcTemplate(new org.springframework.jdbc.core.JdbcTemplate(dataSource))
                .usingDbTime()
                .build()
        );
    }
}

// Safe BillingScheduler with ShedLock
import net.javacrumbs.shedlock.spring.annotation.SchedulerLock;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

@Component
public class SafeBillingScheduler {

    private final SafeBatchBillingService batchBillingService;

    @Scheduled(cron = "0 0 1 * * *")
    @SchedulerLock(
        name            = "monthly-billing",
        lockAtMostFor   = "PT4H",    // release lock after 4h even if the job crashes
        lockAtLeastFor  = "PT30M"    // hold lock for 30m minimum to prevent rapid re-fire
    )
    public void runMonthlyBilling() {
        // Only the pod that acquires the ShedLock row reaches this line.
        // Other pods attempt the INSERT on the shedlock table, fail, and return immediately.
        batchBillingService.runMonthlyBilling(currentBillingPeriod())
            .block();  // ShedLock requires a blocking call to hold the lock until job completes
    }

    private String currentBillingPeriod() {
        return java.time.YearMonth.now().toString();
    }
}

The lockAtMostFor setting is critical: if the pod holding the lock crashes mid-job, ShedLock releases the lock after the specified duration so another pod can run the job. lockAtLeastFor prevents the scenario where a very short billing run releases the lock immediately, another pod acquires it within the same second, and the job fires twice in rapid succession. Together, lockAtMostFor = "PT4H" and lockAtLeastFor = "PT30M" mean: hold the lock for at least 30 minutes, release it automatically if the job has not completed within 4 hours.

The content-hash key and pre-flight ON CONFLICT DO NOTHING guard must remain even with ShedLock in place. ShedLock is a best-effort distributed lock backed by a database row; it provides strong consistency guarantees on a healthy database connection but does not protect against the billing job being re-triggered by an operator (kubectl exec), a deployment rollout that fires the cron twice on two overlapping pod versions during the rolling update window, or a future refactor that calls runMonthlyBilling() from a different code path. The pre-flight guard is the authoritative layer; ShedLock is the performance optimization that eliminates the triple workload under normal operation.

For applications that prefer not to add a ShedLock dependency, a raw pg_try_advisory_lock() call provides the same mutual exclusion using a PostgreSQL session lock. The lock is automatically released when the database connection is returned to the pool, which makes it safe to use inside a DataSource-backed transaction or connection borrow:

// pg_try_advisory_lock alternative — no external library required
@Scheduled(cron = "0 0 1 * * *")
public void runMonthlyBilling() {
    long lockKey = "monthly-billing".hashCode() & 0xFFFFFFFFL;  // positive long from string hash
    try {
        Boolean acquired = jdbcTemplate.queryForObject(
            "SELECT pg_try_advisory_lock(?)", Boolean.class, lockKey);
        if (!Boolean.TRUE.equals(acquired)) {
            log.info("Monthly billing lock not acquired — another pod is running it");
            return;
        }
        batchBillingService.runMonthlyBilling(currentBillingPeriod()).block();
    } finally {
        jdbcTemplate.execute("SELECT pg_advisory_unlock(" + lockKey + ")");
    }
}

Gap analysis: other Spring WebFlux billing patterns not covered above

The three failure modes above cover the most common Spring WebFlux retry surfaces. Several adjacent patterns introduce the same class of failure through different mechanisms:

ExchangeFilterFunction that adds an idempotency key header per request — a WebClient built with .filter(addIdempotencyKey()) runs the filter on every request sent through the client. If the filter computes UUID.randomUUID() inside its filter(ClientRequest, ExchangeFunction) implementation, and the client is called from inside a Mono.defer() with retryWhen, the filter runs again on each retry and generates a new UUID. The fix is the same: the filter should accept a pre-computed idempotency key as input (passed through the ClientRequest attributes map) rather than generating it internally.

R2DBC reactive transactions with serialization-failure retry — a billing operation that reads the customer’s current balance, computes the charge, and writes to Stripe inside a @Transactional(isolation = SERIALIZABLE) boundary will be retried on SerializationException by R2DBC’s transaction advisor. If the Stripe call happens inside the transaction body and the idempotency key is generated inside the reactive chain, the serialization retry produces a new key and a duplicate charge. The Stripe call must be moved outside the serializable transaction boundary, or the key must be derived from content-hash fields that exist before the transaction begins.

Spring Cloud Gateway retry filter — a gateway configured with spring.cloud.gateway.routes[*].filters[*].name: Retry retries the proxied HTTP request to a downstream billing service when the upstream pod returns 5xx or times out. The retry is at the gateway layer: the downstream billing service generates a new UUID.randomUUID() for each incoming request. The downstream service has no knowledge that the inbound request is a gateway retry — it looks like a new request. If the original request was already processed and ch_A created, the gateway retry causes the downstream service to create ch_B. The fix requires the gateway to forward a stable idempotency key from the original request (typically passed as an X-Idempotency-Key header by the caller) and the downstream service to use that header value as its Stripe idempotency key rather than generating a new UUID on each request arrival.

Spring WebFlux with Kotlin coroutines (suspend fun + retry()) — when using Kotlin coroutines on top of Spring WebFlux, the billing method is often written as a suspend fun that calls webClient.post()...awaitBody<ChargeResponse>(). A retry is implemented with Kotlin’s retry() extension on Flow, or with a plain loop and try/catch. If UUID.randomUUID() is computed inside the retry block or inside a lambda that re-runs on each iteration, the failure is identical to the Mono.defer failure mode: a new key per retry attempt. The fix is the same — compute the stable key before the retry scope and capture it in the coroutine’s closure.

Flux.interval() polling scheduler — some teams use Flux.interval(Duration.ofDays(30)).flatMap(_ -> runBillingJob()) as an alternative to @Scheduled for reactive scheduling. Flux.interval() runs on Reactor’s Schedulers.parallel() thread pool, one timer per JVM, with no cross-pod coordination. The multi-pod problem is identical to failure mode 3: all three pods fire the billing job at the same tick, each generating distinct UUIDs per customer. ShedLock does not integrate with Flux.interval() natively (ShedLock works via Spring’s @Scheduled AOP interceptor), so the pg_try_advisory_lock() pattern is the appropriate distributed mutex here.

Summary

The pattern across all three: the boundary between “key derivation scope” and “retry scope” must never overlap. If a retry mechanism can re-execute the line that generates the idempotency key — whether through Mono.defer() re-evaluation, Flux re-subscription, or pod-level re-invocation of a scheduled method — a different key is produced and a duplicate charge is created. Making the key a deterministic function of the business intent (customer ID, billing period, vendor namespace) and backing it with a database-level uniqueness constraint moves the deduplication guarantee out of ephemeral in-memory state and into durable storage that survives retries, re-subscriptions, and pod restarts. The vault key spend cap adds a hard financial boundary as a last line of defense regardless of how many retries, re-subscriptions, or pods execute concurrently.