Quarkus Hibernate Reactive Panache and Stripe Integration: How MicroProfile @Retry + @Transactional Re-invokes the CDI Method Body and Regenerates UUID, Mutiny Uni.createFrom().item(Supplier) Re-evaluates the Lazy Supplier on retry() Re-subscription, and Panache streamAll() Multi Re-streams the Entire Batch from the Database on Outer retry()

Quarkus Hibernate Reactive with Panache and SmallRye Mutiny introduces three idempotency failure patterns that are mechanically distinct from the Spring equivalents. MicroProfile Fault Tolerance’s @Retry wraps @Transactional at the CDI interceptor level, not the Spring AOP proxy level — the mechanism is different but the outcome is the same: InvocationContext.proceed() re-invokes the full method body, including UUID.randomUUID() at method entry. Mutiny’s Uni.createFrom().item(Supplier<T>) is the reactive-idiomatic lazy pattern in the Mutiny ecosystem — but like Reactor’s Mono.defer(), the supplier re-evaluates per subscription. And Panache’s streamAll() returns a cold Multi backed by a Hibernate Reactive scroll cursor — placing retry() on the outer Multi rather than inside each element’s Uni re-subscribes from the database and charges the entire batch a second time.

This post covers three failure modes specific to the Quarkus reactive stack. It is structurally distinct from the Quarkus Mutiny Retry post (which covers Uni.createFrom().emitter() re-emission, Multi.createFrom().iterable() inside flatMap with outer retry(), and @QuarkusTest + WireMock test patterns for basic Mutiny retry modes) and from the Quarkus Transactional Reactive post (which covers @Transactional + Mutiny pipeline UUID generation, Uni.createFrom().voidItem() chaining, and reactive transaction boundary semantics). The modes here are specific to how SmallRye FT’s CDI interceptor priority interacts with @Transactional, how Mutiny’s lazy item(Supplier) behaves under retry(), and how Panache’s streamAll() cold Multi semantics cause full batch re-streaming when the wrong retry scope is applied.

Background: CDI interceptor priority, Mutiny subscription semantics, and Panache reactive streaming

Quarkus uses CDI (Contexts and Dependency Injection) interceptors rather than Spring’s AOP proxy mechanism. Both achieve method interception, but the ordering mechanism differs. In Spring, proxy order is set via the @Order or Ordered interface, and the conventions for @Retryable and @Transactional are documented separately. In CDI, interceptor binding priorities use @Priority. Lower numbers run first as the outer interceptor (they wrap inner interceptors). SmallRye Fault Tolerance’s @Retry uses Priority.NORMAL + 1000 (value: 1200 in SmallRye FT 5.x by default for the FT guard interceptor), while Quarkus’s @Transactional interceptor runs at Priority.NORMAL + 50 (value: 1050). Lower priority number = outer interceptor in CDI. So @Transactional (1050) runs before (inside) @Retry (1200), making @Retry the outer interceptor. This is the same structural outcome as Spring where @Retryable is outer, but via a different mechanism.

Wait — this requires a correction. In CDI, interceptors with a lower @Priority value run first in the interceptor chain, meaning they are the outermost wrapper. An interceptor at Priority.NORMAL (1000) wraps an interceptor at Priority.NORMAL + 50 (1050), which in turn wraps the method body. SmallRye FT’s annotation-based interceptors use Priority.NORMAL + 1000 (value: 2000) in SmallRye FT 3.x and higher, while Quarkus’s @Transactional CDI interceptor is at roughly Priority.NORMAL (1000). The exact numbers depend on the Quarkus version. The critical point is: in all tested Quarkus versions that support both annotations on the same method, @Retry is the outer interceptor relative to @Transactional. This is verifiable by enabling interceptor debug logging and observing the call order. The consequence is that @Retry’s retry loop calls the complete method body (including @Transactional’s transaction management) on each retry, not just the portion after the transaction is opened.

Mutiny, Quarkus’s reactive programming library, is structurally similar to Project Reactor but uses different operator names and composition patterns. Like Reactor, Mutiny distinguishes between eager and lazy value creation. Uni.createFrom().item(T value) captures the value eagerly — the value is computed before the Uni is even subscribed to. Uni.createFrom().item(Supplier<T> supplier) defers evaluation to subscription time — the supplier is called when a subscriber subscribes to the Uni. This deferral is the Mutiny-idiomatic way to create a Uni that performs some computation per subscription, analogous to Reactor’s Mono.fromCallable() or Mono.defer(). When a retry operator re-subscribes an upstream Uni, any lazy suppliers in that upstream chain re-execute.

Panache’s streamAll() method on a reactive Panache entity returns a Mutiny Multi<Entity>. This Multi is cold — it does not start consuming from the database until a subscriber subscribes to it. Each new subscription opens a new Hibernate Reactive scroll cursor against the database and begins streaming rows. When a Mutiny retry operator causes a re-subscription, the scroll cursor is abandoned (or closed) and a new one is opened from the beginning of the result set. All rows are re-streamed. Any processing applied inside a flatMap on this Multi — including UUID generation and Stripe charges — is repeated for every re-subscribed row.

Stripe’s idempotency contract: a POST to any Stripe mutating endpoint with an Idempotency-Key header deduplicates requests with the same key per API key for 24 hours. The key is the only deduplication mechanism. Two separate keys for the same customer, same amount, and same billing period are treated by Stripe as two distinct payment intents. Both are charged.

Mode 1: MicroProfile @Retry + @Transactional on a Quarkus CDI method — InvocationContext.proceed() re-invokes the full method body

The developer is building a Quarkus billing service using Hibernate Reactive Panache. They want to add retry logic for transient Stripe failures (HTTP 500, network timeouts) and use @Transactional to wrap the Hibernate Reactive operations in a managed transaction. They annotate the method with both @Retry(maxRetries = 3, delay = 200, delayUnit = ChronoUnit.MILLIS) from MicroProfile Fault Tolerance and @Transactional from Jakarta EE.

The method computes the Idempotency-Key for Stripe at the top of the method body. The developer’s reasoning: the UUID is the first thing computed in the method — it’s outside the reactive pipeline — so it should be stable across any retry logic that operates at the reactive operator level. The problem: @Retry does not operate at the reactive operator level. SmallRye FT’s interceptor is a CDI interceptor that wraps the entire method invocation. On retry, SmallRye FT calls InvocationContext.proceed(), which re-invokes the complete CDI interceptor chain including the @Transactional interceptor and then the method body itself. UUID.randomUUID() at the top of the method body is not in any reactive pipeline — it is Java code that runs when the method is called. proceed() calls the method. UUID_B is generated.

The developer misconception takes a specific form: “MicroProfile @Retry retries the service operation at the fault tolerance level — it intercepts at the CDI boundary and retries the Stripe HTTP call. @Transactional manages the Hibernate Reactive session lifecycle — it is the inner interceptor. UUID is generated at method entry, which is outside the retry scope; only the Stripe HTTP call inside the pipeline is what gets retried.” The developer is conflating what they think @Retry retries (the Stripe call) with what it actually retries (the full CDI method invocation from the beginning, including the line that computes the UUID).

// BillingService.java — unsafe Mode 1: @Retry outer CDI interceptor + @Transactional inner
@ApplicationScoped
public class ReactiveBillingService {

    @Inject
    StripeReactiveClient stripeClient;

    // Developer's reasoning:
    //   "UUID.randomUUID() is at the top of the method body, outside the reactive pipeline.
    //    @Retry intercepts at the CDI boundary to retry the Stripe HTTP call. @Transactional
    //    manages the Hibernate Reactive session. The UUID is generated once per logical
    //    operation — it's not inside any operator that @Retry would re-execute."
    //
    // The problem:
    //   @Retry is a CDI interceptor at Priority.NORMAL+1000 (outer).
    //   @Transactional is a CDI interceptor at ~Priority.NORMAL (inner).
    //   SmallRye FT's @Retry interceptor wraps the @Transactional interceptor.
    //   On a Stripe transient failure, SmallRye FT calls InvocationContext.proceed().
    //   proceed() re-invokes @Transactional interceptor → rolls back T1 → opens T2.
    //   The method body executes again from line 1.
    //   UUID.randomUUID() generates UUID_B. ← NEW UUID
    //
    // Timeline:
    //   Invocation 1 (attempt 1):
    //     @Retry interceptor calls proceed() → @Transactional opens T1 → method body runs.
    //     UUID_A = UUID.randomUUID().toString()
    //     BillingAttempt.persist() in T1 → T1 commits (or stays open until method returns).
    //     stripeClient.charge(UUID_A) → Stripe returns 500 (transient).
    //     Stripe side effect: ch_A IS committed because 500 on PaymentIntent is ambiguous —
    //     the charge may have been processed before the 500 was returned.
    //     SmallRye FT catches the error, decides to retry.
    //
    //   Invocation 2 (attempt 2):
    //     @Retry calls proceed() again → @Transactional rolls back T1 (if open) → opens T2.
    //     Method body runs from line 1.
    //     UUID_B = UUID.randomUUID().toString() ← NEW UUID generated here
    //     BillingAttempt.persist() in T2 with UUID_B.
    //     stripeClient.charge(UUID_B) → Stripe processes ch_B. ← DUPLICATE
    //
    //   Result: ch_A and ch_B both committed. Customer charged twice.

    @Retry(maxRetries = 3, delay = 200, delayUnit = ChronoUnit.MILLIS,
           retryOn = { StripeTransientException.class })
    @Transactional
    public Uni<String> chargeCustomer(String customerId, String stripeCustomerId,
                                       long amountCents, String billingPeriod) {
        // Developer places UUID generation here, reasoning it is "outside the retry loop."
        // In reality, this line is inside the CDI method invocation that @Retry re-invokes.
        String idempotencyKey = UUID.randomUUID().toString();

        return new BillingAttempt(customerId, amountCents, billingPeriod, idempotencyKey)
                .<BillingAttempt>persistAndFlush()
                .flatMap(attempt ->
                        stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                                idempotencyKey))
                .map(paymentIntentId -> {
                    // Update attempt status — omitted for brevity
                    return paymentIntentId;
                });
    }
}

There is a secondary complication specific to Quarkus Hibernate Reactive and how @Transactional interacts with Uni-returning methods. In Quarkus, @Transactional on a method returning Uni<T> does not commit the transaction when the method returns the Uni object — it commits when the Uni terminates (i.e., emits a value or fails). This is the reactive transaction model. SmallRye FT’s @Retry also understands Uni: when the returned Uni fails (emits a failure item), SmallRye FT re-invokes the method to get a new Uni. The net result is the same: the method body re-executes, UUID_B is generated, and a new Uni is returned for subscription. But the interaction between SmallRye FT’s Uni-aware retry and Quarkus’s reactive @Transactional means that both interceptors are aware of the Uni lifecycle, and the retry does not happen until the Uni is actually subscribed and fails — which is after the transaction has attempted to commit. This makes the timing of UUID_B generation slightly different from the synchronous case but the end result identical: UUID_B is passed to Stripe on the second method invocation.

The fix: content-hash UUID in a non-@Retry facade

The fix separates UUID computation from the retried method invocation. A facade method computes a deterministic idempotency key (content-hash of the billing inputs) and passes it as a stable parameter to the @Retry + @Transactional inner method. The inner method receives a stable key argument and passes it to Stripe on every invocation — whether it is the first attempt or a retry.

// BillingService.java — safe Mode 1 fix: content-hash UUID in facade, stable parameter

@ApplicationScoped
public class ReactiveBillingService {

    @Inject
    StripeReactiveClient stripeClient;

    // Public API: facade computes stable key, does NOT have @Retry.
    public Uni<String> chargeCustomer(String customerId, String stripeCustomerId,
                                       long amountCents, String billingPeriod) {
        // Content-hash key: deterministic from billing inputs.
        // Same inputs always produce the same key within a billing window.
        // If the agent re-calls this method for the same customer+period, same key → safe.
        String idempotencyKey = Hashing.sha256()
                .hashString(customerId + ":" + billingPeriod + ":" + amountCents, UTF_8)
                .toString()
                .substring(0, 36);

        return doChargeWithKey(customerId, stripeCustomerId, amountCents,
                billingPeriod, idempotencyKey);
    }

    // Inner method: receives stable key as parameter. Safe to @Retry because key does
    // not change between invocations — the key comes from the caller, not from this body.
    @Retry(maxRetries = 3, delay = 200, delayUnit = ChronoUnit.MILLIS,
           retryOn = { StripeTransientException.class })
    @Transactional
    Uni<String> doChargeWithKey(String customerId, String stripeCustomerId,
                                 long amountCents, String billingPeriod,
                                 String idempotencyKey) {
        // idempotencyKey is a parameter — it does not change between @Retry invocations.
        return new BillingAttempt(customerId, amountCents, billingPeriod, idempotencyKey)
                .<BillingAttempt>persistAndFlush()
                .flatMap(attempt ->
                        stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                                idempotencyKey));
    }
}

Note that doChargeWithKey must be package-private or use a CDI self-injection pattern to ensure the CDI proxy intercepts the call. Calling this.doChargeWithKey() from within the same bean skips the CDI proxy and bypasses the @Retry and @Transactional interceptors. The standard Quarkus pattern is to inject the bean into itself (CDI self-injection) or use a separate class for the inner method.

// BillingService.java — safe Mode 1 fix: CDI self-injection to ensure proxy intercept

@ApplicationScoped
public class ReactiveBillingService {

    @Inject
    @Self  // Quarkus supports self-injection via @Inject on the same type
    ReactiveBillingService self;

    @Inject
    StripeReactiveClient stripeClient;

    public Uni<String> chargeCustomer(String customerId, String stripeCustomerId,
                                       long amountCents, String billingPeriod) {
        String idempotencyKey = Hashing.sha256()
                .hashString(customerId + ":" + billingPeriod + ":" + amountCents, UTF_8)
                .toString().substring(0, 36);

        // Call through self (the CDI proxy) to trigger @Retry + @Transactional interceptors.
        return self.doChargeWithKey(customerId, stripeCustomerId, amountCents,
                billingPeriod, idempotencyKey);
    }

    @Retry(maxRetries = 3, delay = 200, delayUnit = ChronoUnit.MILLIS,
           retryOn = { StripeTransientException.class })
    @Transactional
    public Uni<String> doChargeWithKey(String customerId, String stripeCustomerId,
                                        long amountCents, String billingPeriod,
                                        String idempotencyKey) {
        return new BillingAttempt(customerId, amountCents, billingPeriod, idempotencyKey)
                .<BillingAttempt>persistAndFlush()
                .flatMap(attempt ->
                        stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                                idempotencyKey));
    }
}

Mode 2: Mutiny Uni.createFrom().item(Supplier<T>) inside onFailure().retry() — lazy supplier re-evaluates per subscription

The developer is building a reactive Quarkus Panache billing service without CDI-level interceptors. They want fine-grained control over retry logic and use Mutiny’s onFailure().retry() operator directly in the reactive pipeline. They know that Uni.createFrom().item(UUID.randomUUID().toString()) evaluates the UUID eagerly — when the item() call executes in the method that builds the chain. They switch to the Supplier overload to defer UUID generation to subscription time, making it “reactive-correct” for a per-request context.

The problem is identical in structure to Reactor’s Mono.defer() issue. Uni.createFrom().item(Supplier<T>) defers the supplier call to subscription time. onFailure().retry().atMost(3) causes re-subscription on failure. On re-subscription, the supplier is called again. UUID.randomUUID() inside the supplier generates UUID_B. The downstream flatMap sends UUID_B as the Stripe Idempotency-Key header. ch_B is committed alongside ch_A.

The developer misconception takes a specific form: “Uni.createFrom().item(() -> UUID.randomUUID()) defers UUID generation to subscription time — this is correct reactive idiom. retry() re-subscribes from the failing flatMap that called Stripe, not from the item(Supplier) at the beginning of the chain. The UUID was already generated and delivered downstream when the chain was first subscribed; retry picks up from the HTTP failure point.” The developer is confusing value delivery with supplier evaluation. The supplier evaluates at subscription time. The chain re-subscribes from the beginning when retry() fires. The beginning of the chain is item(Supplier). The supplier executes again.

// ReactivePanacheBillingService.java — unsafe Mode 2: lazy Supplier inside retry() chain

@ApplicationScoped
public class ReactivePanacheBillingService {

    @Inject
    StripeReactiveClient stripeClient;

    // Developer's reasoning:
    //   "Uni.createFrom().item(UUID.randomUUID().toString()) evaluates UUID at assembly time
    //    (eager) — wrong for per-request UUID. Uni.createFrom().item(() -> UUID.randomUUID())
    //    defers evaluation to subscription time — the correct reactive idiom. retry() fires
    //    from the Stripe flatMap failure; the item(Supplier) at the chain head is upstream
    //    of the flatMap — retry() would not re-evaluate it since it was already upstream
    //    of the failure point."
    //
    // The problem:
    //   onFailure().retry().atMost(3) re-subscribes the ENTIRE upstream chain on failure.
    //   "Upstream chain" means from the source — Uni.createFrom().item(Supplier).
    //   Mutiny's retry operator creates a new subscription to the upstream Uni each time.
    //   A new subscription triggers the Supplier. UUID.randomUUID() generates UUID_B.
    //
    // Timeline:
    //   Subscription 1 (attempt 1):
    //     item(Supplier) → Supplier executes → UUID_A = UUID.randomUUID().toString()
    //     flatMap: BillingAttempt persisted with UUID_A.
    //     flatMap: Stripe called with Idempotency-Key: UUID_A → ch_A committed.
    //     Stripe returns 500. Mutiny propagates failure downstream.
    //     onFailure().retry().atMost(3) catches the failure — will retry.
    //
    //   Re-subscription (attempt 2):
    //     retry() re-subscribes from the source: Uni.createFrom().item(Supplier).
    //     Supplier executes again → UUID_B = UUID.randomUUID().toString(). ← NEW UUID
    //     flatMap: new BillingAttempt INSERT with UUID_B.
    //     flatMap: Stripe called with Idempotency-Key: UUID_B → ch_B committed. ← DUPLICATE
    //
    //   Result: ch_A and ch_B both committed. Customer charged twice.

    public Uni<String> chargeCustomer(String customerId, String stripeCustomerId,
                                       long amountCents, String billingPeriod) {
        return Uni.createFrom().item(() -> UUID.randomUUID().toString())
                // Developer uses Supplier overload for "reactive-correct" lazy UUID.
                .flatMap(key ->
                        new BillingAttempt(customerId, amountCents, billingPeriod, key)
                                .<BillingAttempt>persistAndFlush()
                                .map(v -> key))
                .flatMap(key ->
                        stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                                key))
                .onFailure(StripeTransientException.class).retry().atMost(3);
    }
}

The difference between item(T) and item(Supplier<T>) for Stripe idempotency

Understanding when each is safe requires understanding what “subscription time” means in the context of a method that returns a fresh Uni per call:

• Uni.createFrom().item(T value): the T is computed when the method body executes, before the Uni is returned to the caller. If the method is called once per request, the UUID is computed once. Re-subscribing the Uni returned by this method does not re-compute the UUID because the Uni captured the already-computed value. This is analogous to Reactor’s Mono.just().
• Uni.createFrom().item(Supplier<T> supplier): the Supplier is stored in the Uni and called each time the Uni is subscribed to. If retry() causes the Uni to be re-subscribed, the Supplier is called again. UUID_B. This is analogous to Reactor’s Mono.fromCallable() or Mono.defer().

The correct pattern for Stripe idempotency: generate the UUID at method assembly time (eager evaluation, before returning the Uni) and capture it in a local variable. Use Uni.createFrom().item(capturedKey) with the eager overload. This way, re-subscriptions caused by retry() deliver the same captured key every time.

// ReactivePanacheBillingService.java — safe Mode 2 fix: eager UUID captured in local variable

@ApplicationScoped
public class ReactivePanacheBillingService {

    @Inject
    StripeReactiveClient stripeClient;

    public Uni<String> chargeCustomer(String customerId, String stripeCustomerId,
                                       long amountCents, String billingPeriod) {
        // UUID computed at method-body assembly time — before the Uni is built.
        // This line is NOT inside any Supplier, defer block, or reactive operator.
        // Captured as a plain Java local variable. All subscriptions (including retries)
        // receive this same value — lambda capture from the enclosing scope.
        final String idempotencyKey = UUID.randomUUID().toString();

        return Uni.createFrom().item(idempotencyKey)  // ← eager item(T), not item(Supplier)
                // idempotencyKey is captured by the lambdas below — same value on every retry.
                .flatMap(key ->
                        new BillingAttempt(customerId, amountCents, billingPeriod, key)
                                .<BillingAttempt>persistAndFlush()
                                .map(v -> key))
                .flatMap(key ->
                        stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                                key))
                .onFailure(StripeTransientException.class).retry().atMost(3);
    }
}

The same safe pattern applies to Mutiny’s Uni.createFrom().deferred(Supplier<Uni<T>>) — which is Mutiny’s equivalent of Reactor’s Mono.defer(). If UUID generation is inside the deferred supplier, re-subscription caused by retry re-evaluates it. The fix is the same: compute UUID before building the deferred supplier and capture it as a stable local variable.

// Also unsafe: Uni.createFrom().deferred(Supplier<Uni<T>>) with UUID inside supplier

// UNSAFE: deferred supplier re-executes on every re-subscription (retry() or multi-subscribe)
return Uni.createFrom().deferred(() -> {
    String key = UUID.randomUUID().toString();  // ← re-evaluated per subscription
    return new BillingAttempt(customerId, amountCents, billingPeriod, key)
            .<BillingAttempt>persistAndFlush()
            .flatMap(v -> stripeClient.createPaymentIntent(stripeCustomerId, amountCents, key));
}).onFailure(StripeTransientException.class).retry().atMost(3);

// SAFE: UUID computed before the deferred supplier — captured by the lambda
final String stableKey = UUID.randomUUID().toString();  // ← computed once
return Uni.createFrom().deferred(() -> {
    return new BillingAttempt(customerId, amountCents, billingPeriod, stableKey)
            .<BillingAttempt>persistAndFlush()
            .flatMap(v -> stripeClient.createPaymentIntent(stripeCustomerId, amountCents,
                    stableKey));  // stableKey captured from outer scope — same on every retry
}).onFailure(StripeTransientException.class).retry().atMost(3);

Content-hash UUID as the safer alternative

Both the eager UUID capture and the deferred pattern above have a common limitation: if the same outer method is called twice for the same customer and billing period (e.g., by a second agent run or an operator manual retry at the service layer), a new UUID is generated for each call, and the two calls are not deduplicated at the Stripe level. Content-hash keys solve this: the key is derived from the billing inputs, so the same inputs always produce the same key regardless of how many times the method is called.

// Deterministic content-hash key — safe across both retry() re-subscriptions
// and multiple distinct calls with the same billing inputs.
final String idempotencyKey = Hashing.sha256()
        .hashString(customerId + ":" + billingPeriod + ":" + amountCents, UTF_8)
        .toString().substring(0, 36);

Mode 3: Panache Entity.streamAll() Multi + outer onFailure().retry() — cold Multi re-streams the entire batch from the database

The developer is building a batch billing job in Quarkus that charges all pending customers. They use Panache’s reactive streamAll() method to stream customers from the database, flatMap to charge each one via Stripe, and onFailure().retry().atMost(3) to handle transient Stripe failures. They place the retry() on the outer Multi at the pipeline level, reasoning that this covers all possible Stripe failures across the entire batch operation.

The problem: BillingCustomer.streamAll() returns a cold Multi<BillingCustomer>. Each subscription to this Multi opens a new Hibernate Reactive scroll cursor and begins streaming rows from the database from the beginning. Mutiny’s Multi retry operator, when applied to the outer stream, re-subscribes the entire upstream Multi on any failure from any element. The re-subscription opens a new scroll cursor. All rows are re-emitted. The flatMap inside which UUID is generated executes again for every customer. UUID_B for every customer. Every customer who received ch_A in the first subscription attempt now also receives ch_B.

The blast radius is the entire customer batch. Unlike Mode 1 and Mode 2, which affect a single customer per failed charge, Mode 3 affects all customers who were charged before the failure occurred. If customer #50 triggers the Stripe 500 that causes the outer retry() to fire, customers 1–49 (who already had their ch_A committed successfully) all receive ch_B on the re-stream.

The developer misconception takes a specific form: “retry() on a Multi retries from the element that caused the failure — Mutiny’s reactive machinery knows which item failed and re-processes only that item. Customers who successfully charged in the first pass are not re-emitted from streamAll().” The developer is conflating per-element recovery operators (which Mutiny does support) with stream-level retry (which re-subscribes the source). Mutiny’s onFailure().retry() on a Multi re-subscribes from the source, not from the failed element.

// BatchBillingJob.java — unsafe Mode 3: outer retry() on cold Multi from streamAll()

@ApplicationScoped
public class BatchBillingJob {

    @Inject
    StripeReactiveClient stripeClient;

    // Developer's reasoning:
    //   "BillingCustomer.streamAll() streams all pending customers from the database.
    //    onFailure().retry().atMost(3) on the outer Multi handles any transient Stripe
    //    failure during the batch. retry() retries the failed element — customers who
    //    successfully charged in the first pass are not re-streamed from the database."
    //
    // The problem:
    //   streamAll() returns a COLD Multi backed by a Hibernate Reactive scroll cursor.
    //   Each new subscription opens a new cursor from the beginning of the result set.
    //   onFailure().retry().atMost(3) on the outer Multi re-subscribes on any failure.
    //   Re-subscription = new cursor = all rows re-streamed from the start.
    //   UUID.randomUUID() inside the flatMap generates UUID_B for every customer.
    //
    // Timeline (50 customers, customer #50 causes a Stripe 500):
    //   Subscription 1 (attempt 1):
    //     streamAll() opens cursor C1. Begins emitting customers 1, 2, 3, ... 50.
    //     flatMap for each customer: UUID_A_N = UUID.randomUUID(); ch_A_N committed.
    //     Customer #50: Stripe returns 500. Multi propagates failure.
    //     onFailure().retry().atMost(3) catches failure. Will retry.
    //
    //   Re-subscription (attempt 2):
    //     retry() closes/abandons cursor C1 and creates a new subscription to streamAll().
    //     streamAll() opens cursor C2 from the BEGINNING of the result set.
    //     Customers 1, 2, 3, ... 50 are all re-emitted. ← ALL customers re-streamed
    //     flatMap for each customer: UUID_B_N = UUID.randomUUID(); ch_B_N committed.
    //     Customers 1–49: now have ch_A_N AND ch_B_N. ← DUPLICATES for 49 customers
    //     Customer #50: charged again with UUID_B_50 → ch_B_50.
    //
    //   Result: Every customer charged twice. Blast radius = entire batch.

    @Transactional
    public Uni<Void> chargePendingCustomers() {
        return BillingCustomer.<BillingCustomer>streamAll()
                .flatMap(customer -> {
                    // UUID generated inside the flatMap — re-generated for every re-subscription.
                    String idempotencyKey = UUID.randomUUID().toString();

                    return new BillingAttempt(customer.id, customer.amountCents,
                            customer.billingPeriod, idempotencyKey)
                            .<BillingAttempt>persistAndFlush()
                            .flatMap(attempt ->
                                    stripeClient.createPaymentIntent(customer.stripeCustomerId,
                                            customer.amountCents, idempotencyKey));
                })
                // Developer places retry() at the outer Multi level — re-subscribes streamAll().
                .onFailure(StripeTransientException.class).retry().atMost(3)
                .collect().asList()
                .replaceWithVoid();
    }
}

The fix: per-element retry scope inside the flatMap

The correct retry scope for per-element transient failures is inside the flatMap lambda, not on the outer Multi. When retry() is applied to the Uni returned by the flatMap lambda, it re-subscribes only that element’s Uni. The outer Multi is not re-subscribed. streamAll() is not re-called. Other customers’ charges are not re-executed.

// BatchBillingJob.java — safe Mode 3 fix: per-element retry() inside flatMap

@ApplicationScoped
public class BatchBillingJob {

    @Inject
    StripeReactiveClient stripeClient;

    @Transactional
    public Uni<Void> chargePendingCustomers() {
        return BillingCustomer.<BillingCustomer>streamAll()
                .flatMap(customer -> {
                    // Content-hash key: deterministic from customer inputs.
                    // Safe whether this Uni is subscribed once or retried N times.
                    final String idempotencyKey = Hashing.sha256()
                            .hashString(customer.id + ":" + customer.billingPeriod
                                    + ":" + customer.amountCents, UTF_8)
                            .toString().substring(0, 36);

                    return new BillingAttempt(customer.id, customer.amountCents,
                            customer.billingPeriod, idempotencyKey)
                            .<BillingAttempt>persistAndFlush()
                            .flatMap(attempt ->
                                    stripeClient.createPaymentIntent(customer.stripeCustomerId,
                                            customer.amountCents, idempotencyKey))
                            // Per-element retry: only THIS customer's Uni is re-subscribed.
                            // The outer Multi (streamAll()) is NOT re-subscribed.
                            .onFailure(StripeTransientException.class).retry().atMost(3);
                })
                .collect().asList()
                .replaceWithVoid();
    }
}

With per-element retry, if customer #50’s Stripe call fails, only customer #50’s Uni is re-subscribed. The UUID was computed outside the retry() scope (above the persistAndFlush().flatMap() chain) and is captured as a final local variable. Lambda capture in Java closures captures the variable binding, not a mutable value — since idempotencyKey is effectively final, every re-subscription of the per-element Uni sends the same key to Stripe.

Recognizing cold vs. hot Multi sources in Panache

Not all Panache reactive methods return cold sources, and the distinction matters for deciding where to place retry():

• Entity.streamAll() — cold. Each subscription opens a new Hibernate Reactive scroll cursor from the beginning of the result set. Do not place retry() on the outer Multi if UUID is generated inside flatMap.
• Entity.listAll() — returns Uni<List<Entity>>, not a Multi. The list is fetched eagerly on subscription. Re-subscriptions re-fetch the list. For batch billing, convert to a Multi using .onItem().transformToMulti(list -> Multi.createFrom().iterable(list)) after fetching the list once, then apply per-element retry inside flatMap.
• Multi.createFrom().iterable(preloadedList) where the list was fetched before the retry boundary — re-subscriptions re-emit the same captured list. This is effectively hot with respect to database queries. Safe for outer retry if UUID is computed outside the deferred boundary, though per-element retry is still preferred for isolated failure handling.

// Safe alternative: fetch list once (Uni), then process as a hot-source Multi
// No risk of re-querying the database on retry.

@Transactional
public Uni<Void> chargePendingCustomers() {
    return BillingCustomer.<BillingCustomer>listAll()
            // listAll() returns Uni<List> — fetched once on subscription.
            // Convert to Multi from the fetched list — not a new DB query on re-subscription.
            .onItem().transformToMulti(customers ->
                    Multi.createFrom().iterable(customers))
            .flatMap(customer -> {
                final String idempotencyKey = Hashing.sha256()
                        .hashString(customer.id + ":" + customer.billingPeriod
                                + ":" + customer.amountCents, UTF_8)
                        .toString().substring(0, 36);

                return new BillingAttempt(customer.id, customer.amountCents,
                        customer.billingPeriod, idempotencyKey)
                        .<BillingAttempt>persistAndFlush()
                        .flatMap(attempt ->
                                stripeClient.createPaymentIntent(customer.stripeCustomerId,
                                        customer.amountCents, idempotencyKey))
                        .onFailure(StripeTransientException.class).retry().atMost(3);
            })
            .collect().asList()
            .replaceWithVoid();
}

Comparison: three failure modes and their Quarkus-specific mechanisms

Detecting the failure in production: signals by mode

Test patterns: JUnit 5 + WireMock + @QuarkusTest for all three modes

Testing these failure modes requires intercepting the Stripe HTTP call at the transport layer and asserting that the Idempotency-Key header value is the same across all retry attempts. The standard pattern uses WireMock to inject a transient failure on attempt 1 and a success on attempt 2, then asserts the header stability across both HTTP interactions.

// BillingServiceRetryIdempotencyTest.java — @QuarkusTest + WireMock for all three modes

@QuarkusTest
@WireMockTest(httpPort = 8089)  // WireMock server on port 8089
class BillingServiceRetryIdempotencyTest {

    @Inject
    ReactiveBillingService billingService;  // Mode 1 + 2

    @Inject
    BatchBillingJob batchBillingJob;        // Mode 3

    // ─── Mode 1: @Retry + @Transactional CDI method ───────────────────────────

    @Test
    void mode1_retryTransactionalCDI_idempotencyKeyStableAcrossRetries(WireMockRuntimeInfo wm) {
        // Configure WireMock: fail on attempt 1, succeed on attempt 2.
        // WireMock scenario: state-machine to track attempt count.
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .inScenario("stripe-retry")
                .whenScenarioStateIs(STARTED)
                .willReturn(WireMock.serverError()
                    .withBody("{\"error\":{\"type\":\"api_error\",\"message\":\"transient\"}}"))
                .willSetStateTo("attempt-2")
        );
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .inScenario("stripe-retry")
                .whenScenarioStateIs("attempt-2")
                .willReturn(WireMock.okJson(
                    "{\"id\":\"pi_test\",\"status\":\"succeeded\",\"amount\":1000}"))
        );

        // Execute — expects Mode 1 safe fix (facade + doChargeWithKey pattern).
        billingService.chargeCustomer("cust-1", "stripe_cust_1", 1000L, "2026-10")
                .await().atMost(Duration.ofSeconds(10));

        // Verify both Stripe requests used the SAME Idempotency-Key.
        List<LoggedRequest> requests = wm.getWireMock().findAll(
                WireMock.postRequestedFor(WireMock.urlEqualTo("/v1/payment_intents")));
        assertThat(requests).hasSize(2);

        String keyAttempt1 = requests.get(0).getHeader("Idempotency-Key");
        String keyAttempt2 = requests.get(1).getHeader("Idempotency-Key");

        // If the safe fix is in place, both keys should be equal.
        // If Mode 1 bug is present, they will differ (UUID_A vs UUID_B).
        assertThat(keyAttempt1).isNotBlank();
        assertThat(keyAttempt1).isEqualTo(keyAttempt2);
    }

    // ─── Mode 2: Mutiny item(Supplier) inside retry() ─────────────────────────

    @Test
    void mode2_mutinyLazySupplierRetry_idempotencyKeyStableAcrossRetries(WireMockRuntimeInfo wm) {
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .inScenario("supplier-retry")
                .whenScenarioStateIs(STARTED)
                .willReturn(WireMock.serverError())
                .willSetStateTo("attempt-2")
        );
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .inScenario("supplier-retry")
                .whenScenarioStateIs("attempt-2")
                .willReturn(WireMock.okJson(
                    "{\"id\":\"pi_test\",\"status\":\"succeeded\",\"amount\":1000}"))
        );

        billingService.chargeCustomer("cust-2", "stripe_cust_2", 1000L, "2026-10")
                .await().atMost(Duration.ofSeconds(10));

        List<LoggedRequest> requests = wm.getWireMock().findAll(
                WireMock.postRequestedFor(WireMock.urlEqualTo("/v1/payment_intents")));
        assertThat(requests).hasSize(2);

        String keyAttempt1 = requests.get(0).getHeader("Idempotency-Key");
        String keyAttempt2 = requests.get(1).getHeader("Idempotency-Key");

        // Safe fix: eager item(T) captures key before Uni assembly. Both keys equal.
        // Bug: item(Supplier) re-evaluates UUID_B. Keys differ.
        assertThat(keyAttempt1).isEqualTo(keyAttempt2);
    }

    // ─── Mode 3: streamAll() Multi outer retry() ──────────────────────────────

    @Test
    void mode3_streamAllMultiOuterRetry_perElementRetryDoesNotRechargeOtherCustomers(
            WireMockRuntimeInfo wm) {
        // Customers 1–5 in DB. Customer #3 triggers a Stripe 500 on first attempt.
        // Safe fix: per-element retry — only customer #3's Uni is re-subscribed.
        // Customers #1, #2, #4, #5 should each receive exactly ONE Stripe request.

        // Set up WireMock: customer #3 fails first, then succeeds. Others succeed immediately.
        // Use request body matching to distinguish customers (by stripeCustomerId).
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .withRequestBody(WireMock.containing("stripe_cust_3"))
                .inScenario("cust3-retry")
                .whenScenarioStateIs(STARTED)
                .willReturn(WireMock.serverError())
                .willSetStateTo("cust3-attempt-2")
        );
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .withRequestBody(WireMock.containing("stripe_cust_3"))
                .inScenario("cust3-retry")
                .whenScenarioStateIs("cust3-attempt-2")
                .willReturn(WireMock.okJson(
                    "{\"id\":\"pi_cust3\",\"status\":\"succeeded\",\"amount\":1000}"))
        );
        wm.getWireMock().register(
            WireMock.post(WireMock.urlEqualTo("/v1/payment_intents"))
                .withRequestBody(WireMock.not(WireMock.containing("stripe_cust_3")))
                .willReturn(WireMock.okJson(
                    "{\"id\":\"pi_other\",\"status\":\"succeeded\",\"amount\":1000}"))
        );

        batchBillingJob.chargePendingCustomers()
                .await().atMost(Duration.ofSeconds(15));

        // Each of customers 1, 2, 4, 5: exactly 1 Stripe request.
        // Customer 3: exactly 2 Stripe requests (1 fail + 1 success).
        List<LoggedRequest> all = wm.getWireMock().findAll(
                WireMock.postRequestedFor(WireMock.urlEqualTo("/v1/payment_intents")));
        assertThat(all).hasSize(6);  // 4 customers × 1 + 1 customer × 2 = 6

        // Verify customer #3 used the SAME Idempotency-Key on both attempts.
        List<LoggedRequest> cust3Requests = all.stream()
                .filter(r -> r.getBodyAsString().contains("stripe_cust_3"))
                .toList();
        assertThat(cust3Requests).hasSize(2);
        assertThat(cust3Requests.get(0).getHeader("Idempotency-Key"))
                .isEqualTo(cust3Requests.get(1).getHeader("Idempotency-Key"));

        // If the Mode 3 bug were present (outer retry on Multi), each customer would
        // receive 2 requests total (one for customer #3 failure triggers re-stream).
        // That would produce 10 requests (5 customers × 2) or more.
    }
}

The common thread across all three modes

All three modes share the same root cause: the idempotency key is computed inside a code boundary that re-executes on retry. The boundary differs by mode:

• Mode 1: the CDI method body (re-invoked by InvocationContext.proceed() in SmallRye FT)
• Mode 2: the Mutiny lazy supplier (re-invoked on each subscription to the upstream Uni)
• Mode 3: the flatMap lambda that processes each batch element (re-invoked for every element on every Multi re-subscription caused by outer retry)

The fix in all three modes follows the same principle: compute the idempotency key before entering any retried boundary and pass it as a stable value into the boundary. In Mode 1, this is a facade method that computes the key and passes it as a parameter to the @Retry-annotated inner method. In Mode 2, this is a final local variable computed before the Uni is assembled. In Mode 3, this is a content-hash key computed outside the retry() operator scope (within the flatMap lambda but before the Uni that has retry() applied to it).

Content-hash keys are a stronger guarantee than UUID for all three modes: they deduplicate not only across retry re-invocations but also across independent calls with the same billing inputs. This matters when the billing job is re-run after a partial failure, when an operator manually re-triggers a charge for a specific customer, or when a second agent process starts charging the same batch before the first has fully committed all results.

Comparison with Spring equivalents

The deeper pattern: any re-invocation mechanism is a retry boundary for UUID

The three failure modes in this post, the three in the Quarkus Mutiny Retry post, the three in the Spring Data JPA post, and all other modes in this series share a single generalizable principle: wherever your code re-executes in response to a failure, that re-execution boundary is a potential UUID regeneration site. The boundary can be a CDI interceptor (proceed()), an AOP proxy (proceed()), a reactive operator (retry() causing re-subscription), a loop (while catching exceptions), or a scheduler re-running a job. The key question to ask about any retry mechanism is: where, exactly, does execution resume? If execution resumes before UUID.randomUUID(), the UUID is regenerated. If execution resumes after it, the UUID is stable.

For Quarkus Hibernate Reactive Panache specifically, the answer to “where does execution resume” is:

• @Retry CDI interceptor: at the beginning of the CDI method body (after the @Transactional interceptor opens a new transaction)
• Mutiny retry() on a Uni: at the source of the upstream Uni chain (wherever the subscription originates)
• Mutiny retry() on a Multi: at the source of the upstream Multi (including any cold source like streamAll())

With this mental model, the fix is always the same: place UUID generation above (outside) the resume point, and pass the UUID as a captured value or parameter into the retried code.

A proxy that sits between your agent and Stripe — one that enforces a policy of “the same logical operation must always use the same idempotency key” and alerts when it detects two different keys for what appears to be the same customer + amount + billing period within a short window — would catch all three of these failure modes in production. That is exactly what Keybrake is building: a scoped API-key proxy for the non-LLM SaaS APIs your agent calls, with per-vendor spend caps, audit log, and duplicate-charge detection at the proxy layer.