Micronaut HTTP Client and Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance
Micronaut HTTP Client introduces three Stripe billing failure modes that are structurally distinct from the @Retryable AOP pattern covered in the general Micronaut post. The failure modes are rooted in how Micronaut HTTP Client handles filter invocation, reactive stream re-subscription, and the scope of @Retryable boundaries: @ClientFilter runs its doFilter() method fresh on every HTTP request — including each retry invocation from @Retryable on the calling service — so UUID.randomUUID() inside the filter produces a new idempotency key per retry and Stripe creates ch_B; ReactorHttpClient.exchange() wrapped in Mono.fromCallable() with .retry() re-evaluates the callable factory on each Reactor re-subscription, so UUID inside the callable re-evaluates and the retry subscription creates ch_B; and @Retryable placed on a composite service method that encompasses both a Stripe charge and a database insert catches a database exception after the Stripe call already succeeded, re-invokes the method body with a new UUID, and Stripe creates ch_B while ch_A is already committed.
This post covers all three failure modes with Java code (Micronaut 4.x, micronaut-http-client, micronaut-reactor, Reactor Core 3.x, Micronaut Data), the filter invocation model and why @ClientFilter.doFilter() is called on every new HTTP request, Reactor re-subscription semantics and the difference between assembling a Mono from an eager value vs a deferred callable, the composite-method retry anti-pattern where a post-charge failure causes a re-charge, content-hash idempotency keys stable across filter re-invocations and Reactor re-subscriptions, pre-flight PostgreSQL ON CONFLICT DO NOTHING as the authoritative billing mutex, and per-billing-period vault keys via a spend-cap proxy as the financial backstop. For the service-layer AOP retry patterns using @Retryable on @Singleton service methods, see the Micronaut and Stripe Integration post. For the MicroProfile REST Client patterns using @RegisterRestClient and MicroProfile Fault Tolerance @Retry, see the RESTEasy Client and Stripe Integration post. The Micronaut HTTP Client failure modes documented here are structurally different from both: they arise from the HTTP client’s own filter and reactive API, not from the AOP interceptor stack.
Failure mode 1: Micronaut @ClientFilter injects UUID.randomUUID() in doFilter() — filter is invoked on every HTTP request including each @Retryable retry attempt — initial attempt creates ch_A before ReadTimeoutException — first retry creates ch_B with new UUID from filter
Micronaut HTTP Client provides @ClientFilter as a mechanism to intercept outgoing HTTP requests globally or per-host. A common pattern when integrating with Stripe is to create a single @ClientFilter that automatically injects the Idempotency-Key header for every Stripe request, centralizing key management instead of requiring every call site to supply the header. The filter implements HttpClientFilter and overrides doFilter(MutableHttpRequest<?> request, ClientFilterChain chain) to mutate the request before forwarding it to the chain.
The failure mode arises from a misunderstanding of when doFilter() runs. A developer who reasons “the filter runs once per logical operation” will write UUID.randomUUID().toString() inside doFilter() to generate the key. That reasoning is correct for a single HTTP call. But doFilter() is called on every new MutableHttpRequest that passes through the filter chain — and when @Retryable on the calling service method retries, it re-invokes the service method body, which calls the @Client proxy, which creates a new MutableHttpRequest object, which is a new HTTP request that the filter processes from scratch. The filter has no memory of the previous request’s UUID. UUID.randomUUID() evaluates again and produces a different UUID. Stripe sees a new Idempotency-Key header on the retry request and creates a new charge:
// UNSAFE: UUID.randomUUID() inside @ClientFilter.doFilter() — runs on every HTTP request.
// When @Retryable on the calling service retries, a new MutableHttpRequest is created,
// doFilter() is invoked for that new request, and UUID_B is injected.
// Initial attempt creates ch_A before ReadTimeoutException.
// Retry attempt creates ch_B — duplicate charge.
import io.micronaut.http.MutableHttpRequest;
import io.micronaut.http.annotation.Filter;
import io.micronaut.http.filter.ClientFilterChain;
import io.micronaut.http.filter.HttpClientFilter;
import org.reactivestreams.Publisher;
import java.util.UUID;
// Filter applied to all requests to api.stripe.com.
@Filter("https://api.stripe.com/**")
public class StripeIdempotencyFilter implements HttpClientFilter {
@Override
public Publisher<? extends io.micronaut.http.HttpResponse<?>> doFilter(
MutableHttpRequest<?> request,
ClientFilterChain chain) {
// UNSAFE: UUID.randomUUID() called on every invocation of doFilter().
// doFilter() is called for each new HTTP request that passes through this filter.
// When @Retryable re-invokes the calling service method, a new MutableHttpRequest
// is constructed and passed to this filter. UUID_B is generated here on the retry.
//
// Timeline:
// @Retryable attempt 1 (initial):
// service method body runs → @Client proxy creates MutableHttpRequest_1
// doFilter(MutableHttpRequest_1) → UUID_A = "7a3f1b2c-..."
// Stripe receives POST /v1/charges with Idempotency-Key: 7a3f1b2c-...
// Stripe commits ch_A (amount=$49.99)
// Stripe's response is in transit; Micronaut read-timeout fires after 10s
// ReadTimeoutException propagates to @Retryable
//
// @Retryable attempt 2 (first retry):
// service method body runs again → @Client proxy creates MutableHttpRequest_2
// doFilter(MutableHttpRequest_2) → UUID_B = "f4e3d2c1-..." ← NEW UUID
// Stripe receives POST /v1/charges with Idempotency-Key: f4e3d2c1-...
// Stripe creates ch_B ($49.99) ← DUPLICATE CHARGE
request.header("Idempotency-Key", UUID.randomUUID().toString()); // UNSAFE
return chain.proceed(request);
}
}
// Calling service with @Retryable — the @Client proxy invokes the filter on each retry.
import io.micronaut.http.HttpRequest;
import io.micronaut.http.client.annotation.Client;
import io.micronaut.retry.annotation.Retryable;
import jakarta.inject.Singleton;
@Singleton
public class BillingService {
private final StripeHttpClient stripeClient;
private final String stripeKey = System.getenv("STRIPE_SECRET_KEY");
public BillingService(StripeHttpClient stripeClient) {
this.stripeClient = stripeClient;
}
// @Retryable on the service method retries on ReadTimeoutException.
// Each retry re-invokes this method body, calls stripeClient.charge(),
// which constructs a new MutableHttpRequest and passes it through StripeIdempotencyFilter.
// doFilter() generates a new UUID per retry → ch_B.
@Retryable(attempts = "3", delay = "1s",
includes = { io.micronaut.http.client.exceptions.ReadTimeoutException.class })
public ChargeResponse chargeCustomer(String customerId, int amountCents, String billingPeriod) {
ChargeRequest req = ChargeRequest.builder()
.amount(amountCents)
.currency("usd")
.customer(customerId)
.build();
// stripeClient.charge() creates a new MutableHttpRequest.
// StripeIdempotencyFilter.doFilter() is called for that new request → new UUID.
return stripeClient.charge(req);
}
}
// Declarative @Client interface.
@Client("https://api.stripe.com")
public interface StripeHttpClient {
@Post("/v1/charges")
ChargeResponse charge(@Body ChargeRequest request);
}
There is a subtler variant: the developer moves UUID.randomUUID() to the @ClientFilter constructor, intending to generate it once per filter bean lifetime. Because @ClientFilter beans are @Singleton by default in Micronaut, the UUID is generated once when the filter is first instantiated — and the same UUID is used for every subsequent Stripe request from every thread. This “fixes” the retry duplicate by making the key stable, but introduces a different bug: every Stripe charge in the application shares the same idempotency key. On the second distinct charge from any service (even a different customer, different billing period), Stripe’s idempotency cache returns ch_A instead of creating a new charge, and the customer is silently not charged. The key must be stable across retries for the same logical operation, not globally stable across all operations.
The fix is to compute the stable content-hash key in the calling service method before the @Retryable boundary and pass it to the @Client proxy as a header parameter. The filter can read a request attribute set by the caller rather than generating its own UUID. This way the key is computed once, it travels with the MutableHttpRequest as a header, and the filter becomes a pass-through for the pre-computed value — or the filter is eliminated entirely and the caller sets the header directly on the HttpRequest:
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.util.HexFormat;
// Stable content-hash key: sha256(customerId:billingPeriod:micronaut-http-billing)[:32].
// Does not include UUID.randomUUID(), System.currentTimeMillis(), attempt counter, hostname,
// thread ID, or any other value that changes between @Retryable re-invocations.
public class StableKeyHelper {
public static String billingKey(String customerId, String billingPeriod, String service) {
try {
byte[] hash = MessageDigest.getInstance("SHA-256")
.digest((customerId + ":" + billingPeriod + ":" + service)
.getBytes(StandardCharsets.UTF_8));
return HexFormat.of().formatHex(hash).substring(0, 32);
} catch (Exception e) {
throw new RuntimeException("SHA-256 unavailable", e);
}
}
}
// SAFE: key computed before @Retryable method, passed as @Header parameter.
// @Retryable passes the same parameter values on every retry re-invocation.
// @ClientFilter can be a no-op or can validate that the header is already set.
@Singleton
public class BillingService {
private final StripeHttpClient stripeClient;
private final String stripeKey = System.getenv("STRIPE_SECRET_KEY");
public BillingService(StripeHttpClient stripeClient) {
this.stripeClient = stripeClient;
}
// SAFE: idempotencyKey computed by caller, passed as parameter.
// @Retryable passes same idempotencyKey on every context.proceed() re-invocation.
@Retryable(attempts = "3", delay = "1s",
includes = { io.micronaut.http.client.exceptions.ReadTimeoutException.class })
public ChargeResponse chargeCustomerStable(String customerId, int amountCents,
String billingPeriod, String idempotencyKey) {
// No UUID.randomUUID() here — key arrives as a stable pre-computed parameter.
ChargeRequest req = ChargeRequest.builder()
.amount(amountCents)
.currency("usd")
.customer(customerId)
.build();
return stripeClient.chargeWithKey(idempotencyKey, req);
}
}
// Calling orchestrator: compute stable key, then call the @Retryable method.
@Singleton
public class BillingOrchestrator {
private final BillingService billingService;
private final BillingRunRepository billingRunRepository;
public BillingOrchestrator(BillingService billingService,
BillingRunRepository billingRunRepository) {
this.billingService = billingService;
this.billingRunRepository = billingRunRepository;
}
public void runMonthlyBilling(List<Customer> customers, String billingPeriod) {
for (Customer customer : customers) {
// SAFE: stable key computed outside the @Retryable method.
// Same value used for initial attempt and all retries.
String idempotencyKey = StableKeyHelper.billingKey(
customer.id(), billingPeriod, "micronaut-http-billing"
);
// Pre-flight: INSERT ... ON CONFLICT DO NOTHING.
// If this pod already inserted (won the race), this returns 0 and we skip.
int inserted = billingRunRepository.insertIfAbsent(
customer.id(), billingPeriod, idempotencyKey
);
if (inserted == 0) {
continue; // already billed or billing in progress — skip
}
try {
billingService.chargeCustomerStable(
customer.id(), customer.amountCents(), billingPeriod, idempotencyKey
);
} catch (Exception e) {
log.error("Billing failed for customer {} after retries", customer.id(), e);
}
}
}
}
// Updated @Client interface: idempotencyKey is a @Header parameter, not filter-generated.
@Client("https://api.stripe.com")
public interface StripeHttpClient {
@Post("/v1/charges")
ChargeResponse chargeWithKey(
@Header("Idempotency-Key") String idempotencyKey,
@Body ChargeRequest request
);
}
// @ClientFilter becomes a validation filter, not a key generator.
@Filter("https://api.stripe.com/**")
public class StripeIdempotencyFilter implements HttpClientFilter {
@Override
public Publisher<? extends io.micronaut.http.HttpResponse<?>> doFilter(
MutableHttpRequest<?> request,
ClientFilterChain chain) {
// Validate that the caller already set the key — fail loudly if missing,
// rather than silently generating a random one that may differ per retry.
if (!request.getHeaders().contains("Idempotency-Key")) {
throw new IllegalStateException(
"Stripe request missing Idempotency-Key header — caller must supply a stable key"
);
}
return chain.proceed(request);
}
}
The pre-flight INSERT ... ON CONFLICT DO NOTHING provides a second independent layer of protection. It uses a unique constraint on (customer_id, billing_period) to ensure that only one Kubernetes pod can win the race to bill a given customer in a given period, regardless of @Retryable behavior, filter behavior, or cross-pod scheduler timing. The pre-flight insert happens before the @Retryable method is called, outside the retry boundary, so even if the service method is retried multiple times, the billing mutex was already set on the first outer call and all retries of the outer method see a pre-existing row and skip.
Failure mode 2: ReactorHttpClient.exchange() wrapped in Mono.fromCallable() with .retry() — Reactor re-subscription re-evaluates the Callable factory — UUID.randomUUID() inside the callable produces a new key per re-subscription — initial subscription creates ch_A before SocketTimeoutException — first retry re-subscription creates ch_B
Micronaut HTTP Client supports reactive programming through ReactorHttpClient (Project Reactor) and RxHttpClient (RxJava 3). When developers use the reactive client directly — not through a declarative @Client interface — they often defer request assembly inside a Mono.fromCallable() or Mono.defer() factory to keep the reactive chain compositional and avoid eagerly executing side effects at assembly time. This is good reactive practice in general. But when UUID.randomUUID() is inside the deferred factory and .retry() is chained, the key re-evaluates on each Reactor re-subscription and Stripe sees a new idempotency key on the retry request.
The mechanics: Mono.fromCallable(callable) calls the Callable exactly once per subscription. When .retry(3) is chained and the Mono emits an error, Reactor re-subscribes the upstream Mono, which calls the Callable again. If UUID.randomUUID() is inside the Callable, it evaluates again on the retry subscription. The initial subscription runs the callable (UUID_A), builds the Stripe request, Stripe receives it and creates ch_A before a SocketTimeoutException fires; .retry() catches the error and re-subscribes; the callable runs again (UUID_B); the request is built with UUID_B; Stripe sees a new idempotency key and creates ch_B:
// UNSAFE: UUID.randomUUID() inside Mono.fromCallable() — re-evaluates per Reactor re-subscription.
// .retry(3) re-subscribes the upstream Mono on each error — callable re-runs per retry.
import io.micronaut.http.HttpRequest;
import io.micronaut.http.client.ReactorHttpClient;
import io.micronaut.http.MediaType;
import jakarta.inject.Singleton;
import reactor.core.publisher.Mono;
import java.time.Duration;
import java.util.UUID;
@Singleton
public class ReactiveBillingService {
private final ReactorHttpClient httpClient;
private final String stripeKey = System.getenv("STRIPE_SECRET_KEY");
public ReactiveBillingService(ReactorHttpClient httpClient) {
this.httpClient = httpClient;
}
public Mono<ChargeResponse> chargeCustomerUnsafe(String customerId, int amountCents) {
// UNSAFE: Mono.fromCallable() runs the Callable per subscription.
// .retry(3) re-subscribes on each error — Callable re-runs — UUID_B generated.
//
// Subscription 1 (initial):
// Callable runs → UUID_A = "7a3f1b2c-..."
// HttpRequest built with Idempotency-Key: 7a3f1b2c-...
// httpClient.exchange() sends POST /v1/charges
// Stripe commits ch_A; SocketTimeoutException fires before response
//
// Subscription 2 (retry attempt 1, backoff 1s):
// Callable runs again → UUID_B = "c9d8e7f6-..." ← NEW UUID
// HttpRequest built with Idempotency-Key: c9d8e7f6-...
// httpClient.exchange() sends POST /v1/charges
// Stripe sees new key → creates ch_B ← DUPLICATE CHARGE
return Mono.fromCallable(() -> {
// UNSAFE: UUID inside fromCallable re-evaluates per Reactor re-subscription.
String idempotencyKey = UUID.randomUUID().toString();
return HttpRequest.POST(
"https://api.stripe.com/v1/charges",
"amount=" + amountCents + "¤cy=usd&customer=" + customerId
)
.header("Authorization", "Bearer " + stripeKey)
.header("Idempotency-Key", idempotencyKey) // UNSAFE: new UUID per re-subscription
.contentType(MediaType.APPLICATION_FORM_URLENCODED_TYPE);
})
.flatMap(request -> httpClient.exchange(request, ChargeResponse.class))
.map(response -> response.body())
.retry(3); // re-subscribes Mono.fromCallable() on each error — callable re-runs
}
}
The same failure occurs with Mono.defer(), Single.fromCallable() (RxHttpClient), and any other deferred factory pattern where the UUID is inside the factory lambda. The common thread is deferred evaluation: any computation inside the factory lambda runs again on each re-subscription, which is what .retry() and .retryWhen() do when the upstream Mono signals an error.
There is an important distinction from the failure mode in the general Micronaut post. That post covers @Retryable on an @Singleton service method that returns Mono<ChargeResponse> and uses Micronaut’s ReactiveRetryInterceptor — the interceptor re-invokes context.proceed() on each retry, which re-runs the method body and re-evaluates any UUID at the top of the method. This failure mode is different: the UUID is not at the method body level, it is inside a Mono.fromCallable() factory that lives within the reactive chain. The retry is a Reactor stream operator (.retry(3)), not a Micronaut AOP interceptor. The failure propagates differently: an AOP interceptor re-invokes the JVM method, while a Reactor operator re-subscribes the reactive pipeline. Both result in UUID re-evaluation, but the diagnosis and fix differ. The AOP fix is to pass the key as a method parameter before the interceptor boundary; the reactive fix is to compute the key before the reactive chain assembly and capture it as a final variable outside the lambda scope:
// SAFE: UUID computed before fromCallable() — captured as effectively-final variable.
// Reactor re-subscription re-evaluates fromCallable(), but reads the already-computed
// stable key from the final variable in the enclosing scope — same value per subscription.
@Singleton
public class ReactiveBillingService {
private final ReactorHttpClient httpClient;
private final String stripeKey = System.getenv("STRIPE_SECRET_KEY");
public ReactiveBillingService(ReactorHttpClient httpClient) {
this.httpClient = httpClient;
}
public Mono<ChargeResponse> chargeCustomerSafe(String customerId, int amountCents,
String billingPeriod) {
// SAFE: stable key computed before fromCallable() — outside the retry-susceptible scope.
// fromCallable() captures this as an effectively-final variable.
// Reactor re-subscription reads the same stableKey value — no re-evaluation.
final String stableKey = StableKeyHelper.billingKey(
customerId, billingPeriod, "micronaut-http-billing"
);
return Mono.fromCallable(() -> {
// stableKey is read from the enclosing scope — same value on every re-subscription.
return HttpRequest.POST(
"https://api.stripe.com/v1/charges",
"amount=" + amountCents + "¤cy=usd&customer=" + customerId
)
.header("Authorization", "Bearer " + stripeKey)
.header("Idempotency-Key", stableKey) // SAFE: final variable from outer scope
.contentType(MediaType.APPLICATION_FORM_URLENCODED_TYPE);
})
.flatMap(request -> httpClient.exchange(request, ChargeResponse.class))
.map(response -> response.body())
.retryWhen(reactor.util.retry.Retry
.backoff(3, Duration.ofSeconds(1))
.filter(ex -> ex instanceof io.micronaut.http.client.exceptions.ReadTimeoutException));
}
}
The rule is: any value that must be stable across .retry() re-subscriptions must be computed before the reactive chain and captured as a final (or effectively-final) variable in the enclosing scope. This applies to UUID.randomUUID(), System.currentTimeMillis() used as a key component, System.nanoTime(), counter-based keys, and any factory method that produces a fresh value per call. Computations that are inherently idempotent — reading from a @Singleton bean field, looking up a static config value, or calling a pure function with a fixed input — are safe inside the callable because they return the same value on every invocation regardless of re-subscription count.
Failure mode 3: @Retryable on a composite service method encompasses both the Stripe HTTP call and a post-charge database insert — a database exception after a successful Stripe charge triggers @Retryable with a new UUID — ch_B created while ch_A is already committed in Stripe’s ledger
Micronaut’s @Retryable is a general-purpose AOP retry annotation that applies to any exception from any code path in the annotated method body. When developers structure their billing logic as a single service method that first calls Stripe and then inserts a record into a local database, they often annotate the entire method with @Retryable to make it resilient to transient failures — typically Stripe network timeouts. What they do not account for is that @Retryable is agnostic about which line in the method body threw the exception. A database deadlock on the INSERT INTO billing_records line throws a CannotAcquireLockException, which propagates to the @Retryable boundary. @Retryable retries the entire method body. The Stripe charge call runs again with a new UUID. Stripe creates ch_B. The customer has been charged twice.
The failure sequence in detail: the service method runs, computes UUID_A, calls stripeClient.createCharge(UUID_A, req), Stripe receives the request and creates ch_A ($49.99); the Stripe call succeeds and returns the charge object; the service method continues to the database insert, INSERT INTO billing_records (customer_id, billing_period, charge_id) VALUES (?, ?, ?); PostgreSQL detects a deadlock (another billing job is also trying to insert for this customer in a batch operation) and throws a deadlock error; the deadlock exception propagates up the call stack to the @Retryable boundary; @Retryable waits one second and re-invokes the service method body; the method body computes UUID_B; stripeClient.createCharge(UUID_B, req) is called; Stripe sees a new idempotency key and creates ch_B ($49.99); the customer is charged $99.98 when the correct amount is $49.99:
// UNSAFE: @Retryable encompasses both Stripe charge and database insert.
// A database exception after a successful Stripe charge triggers @Retryable.
// @Retryable re-invokes the method body — new UUID computed — ch_B created alongside ch_A.
import io.micronaut.retry.annotation.Retryable;
import jakarta.inject.Singleton;
import io.micronaut.http.client.exceptions.ReadTimeoutException;
import org.springframework.dao.CannotAcquireLockException;
@Singleton
public class CompositeBillingService {
private final StripeHttpClient stripeClient;
private final BillingRecordRepository billingRecordRepository;
// UNSAFE: @Retryable(includes) catches both ReadTimeoutException (Stripe)
// and CannotAcquireLockException (database deadlock).
// If the Stripe call succeeds but the DB insert deadlocks, @Retryable re-invokes
// the method body — UUID_B computed — ch_B created.
@Retryable(
attempts = "3",
delay = "1s",
includes = { ReadTimeoutException.class, CannotAcquireLockException.class }
)
public ChargeResponse chargeAndRecord(String customerId, int amountCents,
String billingPeriod) {
// Step 1: Call Stripe. Computes UUID_A on initial attempt.
// On @Retryable retry attempt, computes UUID_B.
String idempotencyKey = UUID.randomUUID().toString(); // UNSAFE: re-evaluates per retry
ChargeRequest req = ChargeRequest.builder()
.amount(amountCents).currency("usd").customer(customerId).build();
// Step 1 (initial attempt): Stripe creates ch_A. Returns charge object.
ChargeResponse charge = stripeClient.chargeWithKey(idempotencyKey, req);
// Step 2: Insert billing record into local database.
// This INSERT may throw CannotAcquireLockException (deadlock) if another
// concurrent billing batch is also inserting for this customer.
billingRecordRepository.insert(
new BillingRecord(customerId, billingPeriod, charge.id(), amountCents)
);
// ↑ Deadlock thrown here on step 2!
// @Retryable catches CannotAcquireLockException.
// Re-invokes method body from step 1.
// UUID_B computed — stripeClient.chargeWithKey(UUID_B, req) — ch_B created.
// Customer charged $49.99 twice.
return charge;
}
}
A subtler variant occurs when @Retryable includes all exceptions via includes = Exception.class or omits the includes attribute entirely (which defaults to retrying on all exceptions in some Micronaut versions). Any unchecked exception from the database layer — connection pool exhaustion, SQL syntax error, constraint violation from a concurrent insert of a duplicate billing period — will trigger a full method re-invocation. The Stripe call that succeeded on the previous attempt is repeated with a new UUID, and the new charge is created regardless of whether the database constraint that failed on the previous attempt would also block this retry attempt.
Another subtle form involves @Transactional combined with @Retryable. When both annotations are present on the same method, Micronaut applies them in the order they appear on the class hierarchy, with @Transactional typically wrapping the method at a higher precedence than @Retryable in Micronaut’s default interceptor ordering. If the transaction rolls back due to a constraint violation in the database insert, the rollback does not affect the Stripe charge — Stripe charges are external side effects outside the JVM transaction boundary. @Retryable fires after the rollback, re-invokes the method in a new transaction, and re-charges via UUID_B.
The fix has two components. First, separate the Stripe HTTP call and the database insert into distinct operations with distinct retry boundaries. Apply @Retryable only to the Stripe HTTP call (which is idempotent with a stable content-hash key), not to the database insert (which must be handled by its own retry or by using an idempotent INSERT ... ON CONFLICT DO NOTHING). Second, add a pre-flight INSERT ... ON CONFLICT DO NOTHING before the Stripe call as the authoritative billing mutex. If the pre-flight insert succeeds, the current pod owns this billing slot; if it fails (returns 0 rows), billing is already in progress elsewhere and the current pod skips. This pre-flight runs outside the @Retryable scope, so even if the Stripe call is retried multiple times, the billing mutex was set before any Stripe call was made:
// SAFE: @Retryable scoped only to the Stripe HTTP call, not the composite method.
// Database insert uses ON CONFLICT DO NOTHING — idempotent, no retry needed.
// Pre-flight billing mutex set before @Retryable scope begins.
@Singleton
public class CompositeBillingService {
private final StripeHttpClient stripeClient;
private final BillingRunRepository billingRunRepository;
private final BillingRecordRepository billingRecordRepository;
// SAFE: this public method computes the stable key, checks the pre-flight mutex,
// delegates the Stripe call to a separate @Retryable method, then inserts the record.
// @Retryable is NOT on this outer method — a DB exception here does not re-charge.
public ChargeResponse chargeAndRecord(String customerId, int amountCents,
String billingPeriod) {
// Compute stable key before any retry boundary.
String stableKey = StableKeyHelper.billingKey(
customerId, billingPeriod, "micronaut-http-billing"
);
// Pre-flight billing mutex: INSERT ... ON CONFLICT DO NOTHING.
// Returns 0 if billing is already in progress or completed.
// This is the authoritative cluster-wide guard — not Stripe's 24h idempotency cache.
int inserted = billingRunRepository.insertIfAbsent(customerId, billingPeriod, stableKey);
if (inserted == 0) {
log.info("Billing already initiated for customer {} period {} — skipping",
customerId, billingPeriod);
return null; // or return existing charge from DB
}
// Step 1: @Retryable scoped only to the Stripe HTTP call.
// Database insert is NOT inside this @Retryable boundary.
ChargeResponse charge = chargeStripeOnly(customerId, amountCents, stableKey);
// Step 2: Insert billing record. Uses ON CONFLICT DO NOTHING — safe to retry
// without re-charging because the Stripe call is not re-invoked here.
billingRecordRepository.upsert(
new BillingRecord(customerId, billingPeriod, charge.id(), amountCents)
);
return charge;
}
// @Retryable scoped narrowly to the Stripe HTTP call only.
// stableKey is a parameter — @Retryable passes same value on every retry re-invocation.
// No UUID.randomUUID() inside this method — key arrives as a stable parameter.
@Retryable(
attempts = "3",
delay = "1s",
includes = { ReadTimeoutException.class }
// NOT CannotAcquireLockException — that's a DB error, not a Stripe error
)
ChargeResponse chargeStripeOnly(String customerId, int amountCents, String stableKey) {
ChargeRequest req = ChargeRequest.builder()
.amount(amountCents).currency("usd").customer(customerId).build();
return stripeClient.chargeWithKey(stableKey, req);
}
}
The separated @Retryable boundary ensures that only Stripe network errors (specifically ReadTimeoutException) trigger a retry of the Stripe call. Database exceptions that occur after the Stripe call succeeds propagate to the outer chargeAndRecord() method, which has no @Retryable, and are handled by the caller. The ON CONFLICT DO NOTHING in the billing record insert means that if the outer method is ever called again for the same customer and period (e.g., by a scheduler retry at a higher level), the database insert is a no-op and the Stripe call is skipped entirely by the pre-flight mutex.
The governance layer: per-billing-period vault keys with spend caps
Content-hash idempotency keys and pre-flight database guards address the duplicate-charge failure modes at the application layer. They are necessary. But they require every engineer on every team to understand the retry semantics, apply them consistently, avoid the composite-method retry anti-pattern, and never reach for UUID.randomUUID() as a first instinct. In a Micronaut application with multiple teams contributing billing features, that consistency is hard to maintain over time.
A complementary layer is a Stripe restricted key with a per-billing-period spend cap. Instead of using a full Stripe secret key, each billing job runs with a scoped vault key that has a daily spend cap equal to the expected total billing amount for that period, plus a small buffer (typically 1.10× the expected total). If a duplicate charge slips past the application guards, the vault key’s spend cap limits the financial damage. The second charge request fails with a cap-exceeded error instead of creating ch_B. The billing job logs the cap breach, the engineering team investigates, and the customer is not double-charged:
// Vault key creation via Keybrake API — called once before the billing job starts.
// Replace "YOUR_KEYBRAKE_API_KEY" with your vault key management token.
POST https://proxy.keybrake.com/vault/keys
Authorization: Bearer YOUR_KEYBRAKE_API_KEY
Content-Type: application/json
{
"vendor": "stripe",
"daily_usd_cap": 14700.00, // 10x $49.99 × 300 customers × 1.10 buffer
"allowed_endpoints": ["/v1/charges"],
"expires_at": "2026-09-19T23:59:59Z", // expires at end of billing day
"label": "monthly-billing-2026-09"
}
// Response:
// { "vault_key": "vk_live_abc123...", "vendor": "stripe", "cap_usd": 14700.00 }
// Use vault_key in place of the Stripe secret key for the billing job.
// If a duplicate charge is attempted, the cap is breached and Keybrake returns:
// HTTP 429 { "error": "spend_cap_exceeded", "cap_usd": 14700.00, "used_usd": 14749.99 }
// — the Stripe API is never called, and ch_B is never created.
The vault key integrates with the existing @ClientFilter-based architecture cleanly: update the filter to read the vault key from a per-job configuration property (or from a Micronaut @ConfigurationProperties bean scoped to the billing run) instead of the raw Stripe secret key. The filter sets both the Authorization header (using the vault key) and validates the Idempotency-Key header (set by the caller). The vault key’s daily cap and endpoint allowlist enforce limits at the proxy layer, outside the JVM, so they apply even if the application-layer guards are bypassed by a code change or a deploy that reverts the stable-key logic.
Summary: three Micronaut HTTP Client Stripe failure modes
| Failure mode | Root cause | When it fires | Fix |
|---|---|---|---|
@ClientFilter generates UUID per invocation |
doFilter() called for each new MutableHttpRequest — @Retryable creates new request per retry |
Any @Retryable retry after a network timeout, read timeout, or connection reset |
Compute stable content-hash key before @Retryable method; pass as @Header parameter; filter validates, does not generate |
Mono.fromCallable() + .retry() re-evaluates UUID |
Reactor re-subscription runs Callable again; UUID inside callable re-evaluates per subscription |
Any .retry() or .retryWhen() on a Mono.fromCallable() that contains UUID.randomUUID() |
Compute stable key before chain assembly; capture as final variable in enclosing scope; callable reads from outer scope |
@Retryable on composite method catches DB exception after successful Stripe charge |
@Retryable retries entire method body on any matching exception; DB exception after Stripe success re-invokes Stripe with UUID_B |
Database deadlock, connection pool exhaustion, or constraint violation on billing record insert after Stripe call succeeded | Scope @Retryable to Stripe-only method; pass stable key as parameter; handle DB insert with ON CONFLICT DO NOTHING separately |
All three failure modes share a common root: a UUID that was intended to be stable for one logical billing operation is evaluated at a point in the execution where it can be called again — either because the filter runs fresh per HTTP request, because Reactor re-subscribes a deferred factory, or because @Retryable re-invokes the method body that includes both the key generation and the Stripe call. The fix in each case is to hoist the key computation to a point that is called exactly once for a given logical billing operation, outside every retry boundary, and to pass the stable value as data rather than recomputing it.
The pre-flight INSERT ... ON CONFLICT DO NOTHING is the backstop for all three. Even if the application-layer key is accidentally unstable, the pre-flight guard ensures that only one pod can win the billing slot for a given customer and period. The vault key spend cap is the financial backstop outside the JVM: if the pre-flight guard is bypassed or a code regression reintroduces a random UUID, the cap-exceeded error from the proxy layer prevents the second charge from reaching Stripe’s ledger.
For the general Micronaut @Retryable service-level AOP failure modes — including @Retryable re-invoking method body, reactive @Retryable re-subscribing a Reactor Mono return type, and @Scheduled on multi-pod Kubernetes clusters — see the Micronaut and Stripe Integration post. For the MicroProfile REST Client patterns with @RegisterRestClient and @Retry from MicroProfile Fault Tolerance, see the RESTEasy Client and Stripe Integration post. For the Spring equivalent of the composite-method retry anti-pattern using Spring Retry’s @Retryable, see the Spring Retry and Stripe Integration post.
Cap your agent’s Stripe spend before it caps your budget
Keybrake issues scoped vault keys with per-vendor daily spend caps, endpoint allowlists, and a full audit log. A billing job that generates a duplicate charge hits the spend cap and stops — ch_B never reaches Stripe’s ledger.