Spring Cloud OpenFeign and Stripe Integration: How RequestInterceptor.apply() Is Called on Each Feign Retryer Attempt Generating a New UUID After the First Attempt Commits ch_A and the TCP Response Is Lost, Spring Cloud LoadBalancer Retry with retryOnAllOperations=true Re-sends the POST After Network Timeout While ch_A Stays on Stripe Creating ch_B, and @FeignClient fallbackFactory Generates a Fresh UUID After TimeLimiter Fires Post-Commit Calling Stripe Again with UUID_B

This post is part of the ongoing series on Stripe idempotency failure modes in JVM frameworks. Earlier posts covered vanilla OpenFeign (basic retry scope), Feign inside @Transactional (JDBC rollback leaves Stripe committed), and Spring Cloud Circuit Breaker (TimeLimiter + AOP proxy ordering + CLOSED-to-OPEN transition). This post covers three failure modes that are specific to the Spring Cloud OpenFeign project’s own annotations, interceptor pipeline, and LoadBalancer retry integration — mechanisms that are absent from vanilla Feign and from the Spring Cloud Circuit Breaker abstraction.

The three failure modes are mechanically distinct. The first arises from where in the Feign request pipeline the idempotency key is generated: inside a RequestInterceptor, which Feign calls on every attempt including retries, not just on the first attempt. The second arises from Spring Cloud LoadBalancer’s own retry logic, which sits above the Feign layer and re-issues the entire HTTP request — including a fresh interceptor pipeline run — to a different server instance when the first instance returns a connection error. The third arises from Spring Cloud OpenFeign’s fallbackFactory integration: the factory’s create(Throwable) method is invoked by the circuit breaker after a timeout that fires after Stripe has already committed the charge, and the fallback implementation generates a new UUID.randomUUID() under the assumption that the original call never reached Stripe.

The stack for examples in this post: Spring Boot 3.x, spring-cloud-starter-openfeign (Spring Cloud 2023.x / Feign 13.x), spring-cloud-starter-loadbalancer, spring-cloud-starter-circuitbreaker-resilience4j, Stripe Java SDK v24+, Testcontainers PostgreSQL 15, WireMock 3.x, JUnit 5.

Background: Feign’s request pipeline and where interceptors run

When a Spring Cloud OpenFeign @FeignClient method is called, Feign builds a Request object by merging the method template with the runtime arguments. Before the request is sent, Feign iterates over every registered RequestInterceptor and calls interceptor.apply(requestTemplate) on each one. The interceptor can add headers, mutate the URL, or transform the body. After all interceptors have run, Feign hands the request to its Client (Apache HttpClient, OkHttp, or the JDK client) and blocks on the response.

If the response does not arrive — connection timeout, socket timeout, or a network-level TCP reset — Feign throws an IOException wrapped as a RetryableException. The Retryer (configured via spring.cloud.openfeign.client.config.<clientName>.retryer or by registering a Retryer bean) decides whether to retry. If it decides to retry, Feign starts the entire request pipeline over: it rebuilds the Request, calls every RequestInterceptor again, and sends a fresh HTTP request.

This is the key architectural point: interceptors run on every attempt, not just the first. A RequestInterceptor that calls UUID.randomUUID() to populate the Idempotency-Key header will produce a different UUID on each attempt. If the first attempt committed a Stripe charge and the TCP response was lost in transit, the retry will reach Stripe with a different key and create a second charge.

Mode 1: RequestInterceptor.apply() runs on every Retryer attempt — UUID generated fresh per attempt — TCP drop after Stripe commits ch_A — retry generates UUID_B — ch_B committed alongside ch_A

The developer registers a RequestInterceptor as a Spring bean. The interceptor is responsible for adding the Idempotency-Key header to all outgoing Feign requests. The developer reasons: “putting the key in the interceptor centralizes the logic — every call gets an idempotency key automatically, and I don’t have to thread it through every caller.”

The interceptor calls UUID.randomUUID().toString() each time it runs. The developer has also configured a Retryer to handle transient network failures:

// StripeIdempotencyInterceptor.java — called by Feign before EVERY attempt
@Component
public class StripeIdempotencyInterceptor implements RequestInterceptor {

    @Override
    public void apply(RequestTemplate template) {
        // Called once for the initial attempt AND once for each retry.
        // UUID.randomUUID() produces a different value on each invocation.
        template.header("Idempotency-Key", UUID.randomUUID().toString());
    }
}

// FeignConfig.java — registers a Retryer for transient network failures
@Configuration
public class FeignConfig {

    @Bean
    public Retryer stripeRetryer() {
        // Retry up to 3 times, 100ms initial period, 500ms max
        return new Retryer.Default(100, 500, 3);
    }
}

// application.yml
spring:
  cloud:
    openfeign:
      client:
        config:
          stripe-client:
            retryer: com.example.FeignConfig    # picks up the @Bean above
            connect-timeout: 2000
            read-timeout: 10000

// StripeClient.java — Feign client interface
@FeignClient(name = "stripe-client",
             url = "${stripe.api.url}",
             configuration = FeignConfig.class)
public interface StripeClient {

    @PostMapping("/v1/charges")
    ChargeResponse createCharge(
        @RequestHeader("Authorization") String authorization,
        @RequestBody ChargeRequest request
    );
}

// BillingService.java — calls the Feign client
@Service
@Transactional
public class BillingService {

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private PaymentRepository paymentRepository;

    public Payment chargeCustomer(String customerId,
                                  int amountCents,
                                  String subscriptionId,
                                  String billingPeriod) {
        Payment payment = new Payment();
        payment.setCustomerId(customerId);
        payment.setAmountCents(amountCents);
        payment.setSubscriptionId(subscriptionId);
        payment.setBillingPeriod(billingPeriod);
        payment.setStatus(PaymentStatus.PENDING);
        paymentRepository.save(payment);

        // The interceptor adds Idempotency-Key: UUID_A for the first attempt.
        // If Stripe commits ch_A but the TCP response is lost, Feign throws
        // RetryableException. The Retryer fires. Feign calls
        // StripeIdempotencyInterceptor.apply() again → UUID_B.
        // Stripe receives a POST with Idempotency-Key: UUID_B → creates ch_B.
        ChargeResponse response = stripeClient.createCharge(
            "Bearer " + stripeApiKey,
            new ChargeRequest(customerId, amountCents)
        );

        payment.setStripeChargeId(response.getId());
        payment.setStatus(PaymentStatus.COMPLETED);
        paymentRepository.save(payment);
        return payment;
    }
}

The failure sequence:

1. chargeCustomer("cus_001", 2999, "sub_001", "2026-10") is called. The PENDING payment row is saved. stripeClient.createCharge() is called.
2. Feign runs StripeIdempotencyInterceptor.apply() → Idempotency-Key: UUID_A is set on the request template. The HTTP POST is sent to Stripe.
3. Stripe receives the POST, processes the charge, commits ch_A in Stripe’s database, and writes an HTTP 200 response to the socket. The response is in the network stack.
4. Before the HTTP 200 arrives at the JVM: a network device between Stripe and the application (NAT gateway, load balancer, firewall) resets the TCP connection. The JVM’s socket throws java.net.SocketException: Connection reset. Feign wraps this as RetryableException.
5. The Retryer.Default (period=100ms, maxPeriod=500ms, maxAttempts=3) records the failure and decides to retry. Feign calls retryer.clone() — which is a no-op for Retryer.Default since it clones internal state — and re-runs the entire request pipeline.
6. Feign calls StripeIdempotencyInterceptor.apply(requestTemplate) again. UUID.randomUUID() produces UUID_B. The request template now has Idempotency-Key: UUID_B.
7. Feign sends the POST to Stripe with Idempotency-Key: UUID_B. This is a different key from UUID_A. Stripe’s idempotency cache has no record of UUID_B — it only cached the response for UUID_A. Stripe processes this as a fresh charge request and creates ch_B.
8. Feign returns the ch_B response. BillingService saves it as COMPLETED. The customer is billed twice: ch_A and ch_B, both for $29.99.

The developer misconception is: “The Retryer retries on connection failures — if Stripe had committed the charge it would have sent a 200 response before the connection dropped.” This reasoning is wrong in two ways. First, TCP is full-duplex: the server can commit a response and begin transmitting it before the client-side connection reset reaches the application layer. The response bytes may have been in transit or in a kernel buffer when the reset propagated. Second, Stripe commits charges before writing the response to the socket — Stripe’s processing (auth, fraud checks, ledger entry) happens first, then the response is written. A connection reset in the 1–100ms window between Stripe committing and the response arriving at the JVM socket is real and not vanishingly rare under network congestion or NAT timeout conditions.

How Feign’s Retryer interacts with RequestInterceptor

Feign’s SynchronousMethodHandler.executeAndDecode() method is the core execution loop. Simplified, it looks like this:

// From Feign source: SynchronousMethodHandler.executeAndDecode (simplified)
Object executeAndDecode(RequestTemplate template, Options options)
    throws Throwable {
  Request request = targetRequest(template);
  // targetRequest() calls each RequestInterceptor in sequence.
  // This runs EVERY time this method is called — including on retries.

  try {
    Response response = client.execute(request, options);
    return decode(response);
  } catch (RetryableException e) {
    try {
      retryer.continueOrPropagate(e);   // throws if max retries exhausted
    } catch (RetryableException th) {
      throw th.getCause() != null ? th.getCause() : th;
    }
    // If continueOrPropagate returns, loop back to the top.
    // The loop calls executeAndDecode recursively (via the while loop in
    // the actual implementation), re-invoking targetRequest() which
    // re-invokes all RequestInterceptors.
  }
}

The interceptors are not called once per logical request — they are called once per attempt. Feign offers no built-in mechanism to pass state from the first attempt to subsequent retry attempts. If you need the idempotency key to be stable across attempts, you cannot compute it inside the interceptor at intercept time.

The fix: generate the key before the Feign call and pass it as a @RequestHeader parameter

The key must be computed at the call site, before the Feign client is invoked, using inputs that are stable across retries. The Feign client interface method should accept the key as a @RequestHeader parameter. The service generates the key once and passes it:

// Fixed StripeClient.java — caller passes the idempotency key explicitly
@FeignClient(name = "stripe-client",
             url = "${stripe.api.url}",
             configuration = FeignConfig.class)
public interface StripeClient {

    @PostMapping("/v1/charges")
    ChargeResponse createCharge(
        @RequestHeader("Authorization") String authorization,
        @RequestHeader("Idempotency-Key") String idempotencyKey,
        @RequestBody ChargeRequest request
    );
}

// Remove StripeIdempotencyInterceptor entirely.
// The key is now a method parameter, not a header added by an interceptor.

// Fixed BillingService.java — key derived from stable inputs before the call
@Service
@Transactional
public class BillingService {

    public Payment chargeCustomer(String customerId,
                                  int amountCents,
                                  String subscriptionId,
                                  String billingPeriod) {
        // Derive key from stable business inputs that uniquely identify
        // this billing event and do not change between retry attempts.
        String idempotencyKey = toIdempotencyKey(subscriptionId, billingPeriod, amountCents);

        Payment payment = new Payment();
        payment.setCustomerId(customerId);
        payment.setAmountCents(amountCents);
        payment.setSubscriptionId(subscriptionId);
        payment.setBillingPeriod(billingPeriod);
        payment.setStripeIdempotencyKey(idempotencyKey);   // stored before call
        payment.setStatus(PaymentStatus.PENDING);
        paymentRepository.save(payment);

        // Same key passed to every attempt — Retryer fires → same key → Stripe
        // idempotency cache returns ch_A — no ch_B.
        ChargeResponse response = stripeClient.createCharge(
            "Bearer " + stripeApiKey,
            idempotencyKey,
            new ChargeRequest(customerId, amountCents)
        );

        payment.setStripeChargeId(response.getId());
        payment.setStatus(PaymentStatus.COMPLETED);
        paymentRepository.save(payment);
        return payment;
    }

    private String toIdempotencyKey(String subscriptionId,
                                    String billingPeriod,
                                    int amountCents) {
        String input = subscriptionId + ":" + billingPeriod + ":" + amountCents;
        try {
            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            byte[] hash = digest.digest(input.getBytes(StandardCharsets.UTF_8));
            return "ik_" + HexFormat.of().formatHex(hash).substring(0, 40);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

With this fix, the @RequestHeader("Idempotency-Key") value is bound at method-call time to the string that was computed before the Feign call. Feign’s interceptor pipeline has no involvement. When the Retryer retries, the same ChargeRequest parameters are passed through to the rebuilt RequestTemplate, including the same idempotencyKey value — because it was captured in the local variable before the Feign call and passed directly. Stripe’s idempotency cache returns the cached ch_A response on the retry without creating a new charge.

Integration test for Mode 1

@SpringBootTest
@Testcontainers
class FeignRetryerIdempotencyTest {

    @Container
    static PostgreSQLContainer<?> pg =
        new PostgreSQLContainer<>("postgres:15");

    @RegisterExtension
    static WireMockExtension wm = WireMockExtension.newInstance()
        .options(wireMockConfig().dynamicPort())
        .build();

    @Autowired BillingService billingService;
    @Autowired PaymentRepository paymentRepository;

    @Test
    void feignRetryer_sameIdempotencyKey_onTCPReset() {
        // Arrange: first call triggers a WireMock fault (TCP reset),
        // second call returns ch_A successfully.
        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("stripe-retry")
            .whenScenarioStateIs(Scenario.STARTED)
            .willReturn(aResponse().withFault(Fault.CONNECTION_RESET_BY_PEER))
            .willSetStateTo("second-attempt"));

        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("stripe-retry")
            .whenScenarioStateIs("second-attempt")
            .willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/json")
                .withBody(stripeChargeJson("ch_A001"))));

        // Act
        Payment result = billingService.chargeCustomer(
            "cus_001", 2999, "sub_001", "2026-10");

        // Assert: Stripe received exactly two requests (first reset, second success)
        List<LoggedRequest> stripeRequests = wm.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        assertThat(stripeRequests).hasSize(2);

        // Assert: both requests carried the SAME Idempotency-Key
        String key1 = stripeRequests.get(0).getHeader("Idempotency-Key");
        String key2 = stripeRequests.get(1).getHeader("Idempotency-Key");
        assertThat(key1).isNotNull().startsWith("ik_");
        assertThat(key1).isEqualTo(key2);   // same key → no ch_B

        // Assert: result is ch_A001
        assertThat(result.getStripeChargeId()).isEqualTo("ch_A001");
        assertThat(result.getStatus()).isEqualTo(PaymentStatus.COMPLETED);
    }

    // Contrast test: with RequestInterceptor UUID, keys DIFFER on retry
    @Test
    void feignRetryer_interceptorUUID_differentKeysOnRetry_demonstratesBug() {
        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("stripe-retry")
            .whenScenarioStateIs(Scenario.STARTED)
            .willReturn(aResponse().withFault(Fault.CONNECTION_RESET_BY_PEER))
            .willSetStateTo("second-attempt"));

        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("stripe-retry")
            .whenScenarioStateIs("second-attempt")
            .willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/json")
                .withBody(stripeChargeJson("ch_B001"))));

        // (Simulate the buggy path where interceptor regenerates UUID)
        // This test documents the failure mode — in production this creates ch_B.
        List<LoggedRequest> requests = wm.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        // If requests had interceptor-generated UUIDs:
        // assertThat(requests.get(0).getHeader("Idempotency-Key"))
        //     .isNotEqualTo(requests.get(1).getHeader("Idempotency-Key"));
        // → ch_B created: customer double-charged
    }
}

Mode 2: Spring Cloud LoadBalancer retryOnAllOperations=true re-sends POST to next server instance after network timeout — ch_A already committed on Stripe — RequestInterceptor runs again on the re-sent request — UUID_B generated — ch_B committed alongside ch_A

Spring Cloud LoadBalancer provides a RetryLoadBalancerInterceptor (or ReactorLoadBalancerExchangeFilterFunction for WebFlux) that sits above the Feign layer in the HTTP call stack. It is configured via spring.cloud.loadbalancer.retry properties. When the initial request to a service instance fails with a connection error, the interceptor can retry the request by selecting a different server instance from the load balancer pool.

The default configuration retries only on GET requests (retryOnAllOperations=false). However, developers working with microservices that expose Stripe charging as an internal service endpoint sometimes enable retryOnAllOperations=true to handle transient instance failures — the reasoning is that internal services are supposed to be idempotent and that retrying on the next healthy instance is safe.

# application.yml — dangerous configuration for POST requests
spring:
  cloud:
    loadbalancer:
      retry:
        enabled: true
        max-retries-on-same-service-instance: 0
        max-retries-on-next-service-instance: 2
        retry-on-all-operations: true    # ← this flag makes POST retryable
        retryable-status-codes:          # also retry on these HTTP status codes
          - 502
          - 503
          - 504

In this configuration, the architecture is:

// The call stack when @FeignClient is used with Spring Cloud LoadBalancer:
//
// BillingService.chargeViaInternalService()
//   → @FeignClient proxy (Spring Cloud OpenFeign)
//       → Spring Cloud LoadBalancer RetryLoadBalancerInterceptor
//           → Load balancer: select service instance (e.g. billing-service-1:8080)
//               → Feign RequestInterceptor pipeline (all interceptors run here)
//                   → HTTP Client (Apache/OkHttp)
//                       → billing-service-1:8080 → calls Stripe → ch_A committed
//                       → Network timeout (billing-service-1 slow to respond)
//           → Load balancer: select next service instance (billing-service-2:8080)
//               → Feign RequestInterceptor pipeline runs AGAIN (new attempt)
//                   → StripeIdempotencyInterceptor.apply() → UUID_B
//                   → HTTP Client
//                       → billing-service-2:8080 → calls Stripe with UUID_B → ch_B

The specific scenario:

// Internal Feign client calling a billing microservice
@FeignClient(name = "billing-service",
             configuration = FeignConfig.class)
public interface BillingServiceClient {

    @PostMapping("/internal/charges")
    ChargeResult charge(@RequestBody ChargeCommand command);
}

// BillingServiceImpl (on billing-service-1:8080 and billing-service-2:8080)
// This service calls Stripe internally
@RestController
@RequestMapping("/internal")
public class BillingServiceImpl {

    @PostMapping("/charges")
    public ChargeResult charge(@RequestBody ChargeCommand command) {
        // This service calls Stripe with the idempotency key from the
        // ChargeCommand. The key was generated by the caller:
        // StripeIdempotencyInterceptor.apply() on the @FeignClient call.
        String idempotencyKey = command.getIdempotencyKey();

        RequestOptions options = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();
        ChargeCreateParams params = ChargeCreateParams.builder()
            .setCustomer(command.getCustomerId())
            .setAmount((long) command.getAmountCents())
            .setCurrency("usd")
            .build();

        // billing-service-1 calls Stripe — ch_A committed on Stripe
        Charge charge = stripeClient.charges().create(params, options);

        // billing-service-1 is slow to save to its DB (GC pause, disk I/O)
        // The connection from the caller times out here.
        billingResultRepository.save(new BillingResult(charge.getId(), ...));

        return new ChargeResult(charge.getId(), "COMPLETED");
    }
}

The failure sequence in detail:

1. The caller invokes billingServiceClient.charge(command). The RetryLoadBalancerInterceptor selects billing-service-1:8080.
2. Feign runs the request interceptor pipeline: StripeIdempotencyInterceptor.apply() generates UUID_A and sets it on the ChargeCommand.idempotencyKey field (or as a request body field — the mechanism varies, but in the request body scenario the command object is serialized with idempotencyKey=UUID_A).
3. The POST is sent to billing-service-1:8080/internal/charges. The billing service receives the command, calls Stripe with Idempotency-Key: UUID_A, and Stripe commits ch_A.
4. billing-service-1 is in the middle of saving the result when a full GC pause causes it to exceed the caller’s read timeout. The caller’s Feign HTTP client throws SocketTimeoutException.
5. The RetryLoadBalancerInterceptor catches the timeout, increments the retry counter, and selects the next instance: billing-service-2:8080.
6. Before sending to billing-service-2, the interceptor pipeline runs again. StripeIdempotencyInterceptor.apply() generates UUID_B. The ChargeCommand sent to billing-service-2 now contains idempotencyKey=UUID_B.
7. billing-service-2 receives the command, calls Stripe with Idempotency-Key: UUID_B. Stripe has no record of UUID_B. Stripe commits ch_B.
8. The caller receives a successful response from billing-service-2 with ch_B’s ID. From the caller’s perspective, the charge succeeded once. In Stripe’s ledger, there are two committed charges: ch_A and ch_B.

The developer misconception is: “The internal billing-service endpoints are idempotent — I can safely retry on the next instance. The timeout means the first instance never responded — the charge wasn’t committed.” This is wrong for two reasons. First, the billing service called Stripe before it timed out — Stripe’s commit is independent of whether the service had time to return a response. Second, the retryOnAllOperations=true flag makes POST requests retryable at the LoadBalancer level, which is above the Feign interceptor pipeline — every retry runs all interceptors fresh, including the one generating UUID.randomUUID().

The layered retry problem

Spring Cloud LoadBalancer retry and Feign’s own Retryer are independent and compose. If both are enabled, a single logical request can be retried by the Retryer (within a single server instance) and then retried again by the LoadBalancer (on the next instance). Each layer calls the interceptor pipeline fresh. For a Retryer.Default(100, 500, 3) and a LoadBalancer with maxRetriesOnNextServiceInstance=2, a worst-case flow produces 3 × 3 = 9 interceptor pipeline executions, each generating a distinct UUID, against up to 3 different server instances.

The fix: embed the idempotency key in the command before the Feign call

// Fixed: generate idempotency key BEFORE the Feign call and embed it in the command.
// The RequestInterceptor is removed entirely — no header injection.

@Service
public class OrderService {

    @Autowired
    private BillingServiceClient billingServiceClient;

    public ChargeResult chargeOrder(String orderId,
                                    String customerId,
                                    int amountCents) {
        // Key derived from stable business inputs BEFORE any HTTP call.
        // Same key regardless of how many LoadBalancer retries fire.
        String idempotencyKey = toIdempotencyKey(orderId, amountCents);

        ChargeCommand command = ChargeCommand.builder()
            .customerId(customerId)
            .amountCents(amountCents)
            .orderId(orderId)
            .idempotencyKey(idempotencyKey)   // embedded in command body
            .build();

        // LoadBalancer retries may send this command to multiple instances.
        // Each instance receives idempotencyKey=same_value.
        // The second instance's Stripe call uses the same key → ch_A returned.
        return billingServiceClient.charge(command);
    }

    private String toIdempotencyKey(String orderId, int amountCents) {
        String input = "order:" + orderId + ":" + amountCents;
        try {
            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            byte[] hash = digest.digest(input.getBytes(StandardCharsets.UTF_8));
            return "ik_" + HexFormat.of().formatHex(hash).substring(0, 40);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

When the idempotency key is embedded in the request body, it travels with the command to every server instance that the LoadBalancer routes the retry to. The billing service on each instance uses the same key to call Stripe. Stripe’s idempotency cache returns ch_A on subsequent requests with the same key. No matter how many instances process the retry, only one charge is created.

Integration test for Mode 2

@SpringBootTest
@Testcontainers
class LoadBalancerRetryIdempotencyTest {

    @RegisterExtension
    static WireMockExtension instance1 = WireMockExtension.newInstance()
        .options(wireMockConfig().dynamicPort())
        .build();

    @RegisterExtension
    static WireMockExtension instance2 = WireMockExtension.newInstance()
        .options(wireMockConfig().dynamicPort())
        .build();

    // Both WireMock instances represent billing-service replicas.
    // instance1 represents billing-service-1 (times out on first request).
    // instance2 represents billing-service-2 (succeeds on retry).

    @Test
    void loadBalancerRetry_sameIdempotencyKeyReachesAllInstances() {
        // billing-service-1: timeout immediately
        instance1.stubFor(post(urlEqualTo("/internal/charges"))
            .willReturn(aResponse()
                .withFixedDelay(30_000)));    // exceeds read-timeout of 5000ms

        // billing-service-2: capture the idempotency key from the request body
        instance2.stubFor(post(urlEqualTo("/internal/charges"))
            .willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/json")
                .withBody("{\"chargeId\":\"ch_A001\",\"status\":\"COMPLETED\"}")));

        // Act
        ChargeResult result = orderService.chargeOrder("order_001", "cus_001", 4999);

        // Assert: instance2 was hit with the retry
        List<LoggedRequest> instance2Requests = instance2.findAll(
            postRequestedFor(urlEqualTo("/internal/charges")));
        assertThat(instance2Requests).hasSize(1);

        // Assert: the idempotency key in the body matches what instance1 received
        // (instance1 received the original request before timing out)
        List<LoggedRequest> instance1Requests = instance1.findAll(
            postRequestedFor(urlEqualTo("/internal/charges")));
        assertThat(instance1Requests).hasSize(1);

        String key1 = extractIdempotencyKey(instance1Requests.get(0).getBodyAsString());
        String key2 = extractIdempotencyKey(instance2Requests.get(0).getBodyAsString());

        // Both requests carry the same key → Stripe sees ch_A on both → no ch_B
        assertThat(key1).isEqualTo(key2);
        assertThat(key1).startsWith("ik_");

        assertThat(result.getChargeId()).isEqualTo("ch_A001");
    }

    private String extractIdempotencyKey(String requestBody) {
        // Parse JSON body and extract idempotencyKey field
        return JsonPath.read(requestBody, "$.idempotencyKey");
    }
}

Mode 3: @FeignClient fallbackFactory generates UUID.randomUUID() after TimeLimiter fires post-commit — fallback calls Stripe again with UUID_B — ch_B committed alongside committed ch_A

Spring Cloud OpenFeign integrates with Spring Cloud Circuit Breaker via spring.cloud.openfeign.circuitbreaker.enabled=true. When enabled, each @FeignClient method is wrapped in a circuit breaker using the configured circuit breaker factory (typically Resilience4j). The @FeignClient annotation accepts a fallback or fallbackFactory attribute to provide a fallback implementation when the circuit breaker trips or when the underlying call fails.

Developers use fallbackFactory (over fallback) when they want to log or inspect the exception that caused the failure before returning the fallback response. The FallbackFactory.create(Throwable cause) method receives the exception and returns a StripeClient implementation to use as the fallback.

The failure mode occurs when the circuit breaker’s TimeLimiter fires after Stripe has committed a charge but before the response arrives at the Feign client. The fallbackFactory receives a TimeoutException, interprets it as “Stripe never responded — the charge was not committed,” and calls Stripe directly with a new UUID.randomUUID() as a recovery measure:

// application.yml
spring:
  cloud:
    openfeign:
      circuitbreaker:
        enabled: true

resilience4j:
  timelimiter:
    instances:
      stripe-client:
        timeout-duration: 3s     # fires after 3 seconds
  circuitbreaker:
    instances:
      stripe-client:
        sliding-window-size: 10
        failure-rate-threshold: 50

// StripeClient.java — @FeignClient with fallbackFactory
@FeignClient(
    name = "stripe-client",
    url = "${stripe.api.url}",
    fallbackFactory = StripeFallbackFactory.class
)
public interface StripeClient {

    @PostMapping("/v1/charges")
    ChargeResponse createCharge(
        @RequestHeader("Authorization") String auth,
        @RequestHeader("Idempotency-Key") String idempotencyKey,
        @RequestBody ChargeRequest request
    );
}

// StripeFallbackFactory.java — the factory that generates the fallback
@Component
public class StripeFallbackFactory
    implements FallbackFactory<StripeClient> {

    @Autowired
    private DirectStripeService directStripeService;

    @Override
    public StripeClient create(Throwable cause) {
        return new StripeClient() {
            @Override
            public ChargeResponse createCharge(String auth,
                                               String idempotencyKey,
                                               ChargeRequest request) {
                log.warn("Feign call to Stripe failed: {}. Retrying via " +
                         "direct SDK call.", cause.getMessage());

                // ← BUG: The developer checks for TimeoutException and assumes
                // that a timeout means Stripe never processed the charge.
                // In fact, the TimeLimiter fires after the timeout duration
                // elapses on the CALLING thread — Stripe may have committed
                // ch_A after the TimeLimiter deadline but before the response
                // byte arrived at the JVM socket buffer.
                if (cause instanceof TimeoutException
                    || cause.getCause() instanceof TimeoutException) {
                    log.warn("Timeout — assuming charge not committed. " +
                             "Retrying with new idempotency key.");

                    // Generates UUID_B — a DIFFERENT key from idempotencyKey
                    String recoveryKey = UUID.randomUUID().toString();

                    return directStripeService.charge(
                        request.getCustomerId(),
                        request.getAmountCents(),
                        recoveryKey    // ← UUID_B → creates ch_B if ch_A committed
                    );
                }

                // Non-timeout failure: return a safe failure response
                return ChargeResponse.failed("stripe_unavailable");
            }
        };
    }
}

The failure sequence:

1. BillingService calls stripeClient.createCharge(auth, idempotencyKey, request) with idempotencyKey = UUID_A.
2. The Spring Cloud Circuit Breaker wraps the call in a Resilience4j TimeLimiter with a 3-second timeout. The TimeLimiter submits the Feign call to an executor and starts a countdown timer.
3. Feign sends the POST to Stripe with Idempotency-Key: UUID_A. Stripe receives the request, processes the charge (auth, fraud check, ledger entry — ~1.8 seconds under load), and commits ch_A in Stripe’s database at t=1.8s.
4. Stripe begins writing the HTTP 200 response to the TCP socket at t=1.8s. At t=3.0s, the TimeLimiter deadline elapses on the calling thread. The TimeLimiter cancels the executor task and throws TimeoutException. The Stripe response is in the JVM’s kernel socket buffer but has not been read by the application.
5. Spring Cloud Circuit Breaker catches the TimeoutException and invokes StripeFallbackFactory.create(timeoutException). The factory returns a fallback StripeClient implementation.
6. The fallback’s createCharge method is called. It checks cause instanceof TimeoutException → true. It generates UUID_B = UUID.randomUUID() and calls directStripeService.charge(..., UUID_B).
7. directStripeService.charge calls the Stripe SDK directly: stripeClient.charges().create(params, options) with Idempotency-Key: UUID_B. Stripe has no record of UUID_B. Stripe commits ch_B.
8. The fallback returns ChargeResponse for ch_B. BillingService stores ch_B’s ID as COMPLETED. The customer is billed twice: ch_A and ch_B.

The developer misconception is: “A TimeoutException means Stripe did not respond in time — if Stripe had committed the charge, the response would have arrived before the timeout expired.” This reasoning misunderstands both what the TimeLimiter measures and what Stripe’s commit semantics imply. The TimeLimiter measures elapsed time from when the Feign call was submitted to the executor — not from when the Stripe request was sent or when Stripe began processing it. Stripe’s internal processing time (fraud scoring, ledger write, replication) is invisible to the TimeLimiter. A charge can be committed on Stripe’s side during a period in which the timer is already counting down, and the response can arrive in the socket buffer after the timer fires. “No response within N seconds” does not imply “no charge was committed.”

The fallbackFactory receives a stale idempotencyKey parameter

There is a subtlety in the Spring Cloud OpenFeign fallback invocation that compounds the problem. The FallbackFactory.create(Throwable) method creates a fallback implementation. When the circuit breaker invokes the fallback, it calls the same method on the fallback object — passing the same parameters that were passed to the original @FeignClient method call, including the original idempotencyKey value (UUID_A).

This means the fallback’s createCharge method receives idempotencyKey = UUID_A as a parameter. The developer chooses to ignore it and generate UUID_B instead — precisely because they believe UUID_A is “tainted” by the timeout and should not be reused. This is the wrong conclusion. UUID_A is exactly the key they should use: if Stripe committed ch_A under UUID_A, sending UUID_A again will return the cached ch_A response without creating a new charge.

The fix: the fallback must reuse UUID_A, not generate UUID_B

// Fixed StripeFallbackFactory.java
@Component
public class StripeFallbackFactory
    implements FallbackFactory<StripeClient> {

    @Autowired
    private DirectStripeService directStripeService;

    @Override
    public StripeClient create(Throwable cause) {
        return new StripeClient() {
            @Override
            public ChargeResponse createCharge(String auth,
                                               String idempotencyKey,
                                               ChargeRequest request) {
                log.warn("Feign call to Stripe failed: {}", cause.getMessage());

                if (cause instanceof TimeoutException
                    || cause.getCause() instanceof TimeoutException) {
                    log.warn("Timeout after deadline — Stripe may or may not have " +
                             "committed. Retrying with THE SAME idempotency key: {}",
                             idempotencyKey);

                    // Use idempotencyKey (UUID_A) — the SAME key the original
                    // call used. If Stripe committed ch_A under UUID_A, this
                    // call will receive a 200 with ch_A's data from the
                    // idempotency cache — no ch_B created.
                    // If Stripe never received or processed UUID_A, this call
                    // will create ch_A fresh — idempotent behavior.
                    return directStripeService.charge(
                        request.getCustomerId(),
                        request.getAmountCents(),
                        idempotencyKey     // ← reuse UUID_A, not UUID.randomUUID()
                    );
                }

                // Non-timeout: circuit is open or upstream is down.
                // Return a safe failure response — do NOT attempt a charge.
                return ChargeResponse.failed("stripe_circuit_open");
            }
        };
    }
}

With this fix, the fallback reuses idempotencyKey (which is UUID_A, passed as a parameter from the original invocation). If Stripe committed ch_A under UUID_A, the fallback’s retry returns the cached ch_A response. If Stripe never received or processed the original request, the fallback’s retry creates ch_A fresh. Either way, no second charge is created.

The same principle applies to fallback (non-factory) fallbacks. If the fallback class generates a new UUID, it will double-charge on timeout scenarios. The fallback must derive or receive the original idempotency key and reuse it.

Why the fallback should also check Stripe before retrying

For maximum safety, the fallback should query Stripe before retrying: if ch_A already exists for UUID_A, there is no need to call charges().create() at all. The Stripe API provides an idempotent retrieval path: a POST to /v1/charges with the same Idempotency-Key as a prior committed charge will return the cached response within the 24-hour window. But if the 24-hour window has expired (extremely rare in a fallback context), the developer should choose not to retry and instead mark the payment for manual review rather than risk a double charge.

// Defensive fallback with Stripe-side verification
@Override
public ChargeResponse createCharge(String auth,
                                   String idempotencyKey,
                                   ChargeRequest request) {
    if (cause instanceof TimeoutException
        || cause.getCause() instanceof TimeoutException) {

        // First, try to retrieve using the same idempotency key.
        // If Stripe committed ch_A, this returns ch_A without creating ch_B.
        // If Stripe never processed UUID_A (because the request never arrived),
        // this creates ch_A fresh.
        // The content-hash key (if used) makes this safe: same inputs → same
        // key → same Stripe result regardless of how many times this is called.
        try {
            return directStripeService.charge(
                request.getCustomerId(),
                request.getAmountCents(),
                idempotencyKey
            );
        } catch (StripeException se) {
            log.error("Fallback charge failed for key {}: {}",
                      idempotencyKey, se.getMessage());
            // Flag for async reconciliation — do NOT generate a new key here
            reconciliationQueue.enqueue(idempotencyKey, request);
            return ChargeResponse.pendingReconciliation(idempotencyKey);
        }
    }
    return ChargeResponse.failed("stripe_circuit_open");
}

Integration test for Mode 3

@SpringBootTest
@Testcontainers
class FallbackFactoryIdempotencyTest {

    @RegisterExtension
    static WireMockExtension wm = WireMockExtension.newInstance()
        .options(wireMockConfig().dynamicPort())
        .build();

    @Container
    static PostgreSQLContainer<?> pg =
        new PostgreSQLContainer<>("postgres:15");

    @Autowired BillingService billingService;

    @Test
    void fallbackFactory_reusesOriginalKey_noDoubleCharge_onTimeout() {
        // Arrange: simulate Stripe responding AFTER the TimeLimiter deadline.
        // WireMock delays the response by 4 seconds; TimeLimiter is set to 3s.
        // After the timeout the fallback fires and retries with the same key.
        // On the fallback's direct SDK call, WireMock returns ch_A immediately.

        // First call (via Feign) — times out after 3s (WireMock delays 4s)
        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("timeout-then-fallback")
            .whenScenarioStateIs(Scenario.STARTED)
            .willReturn(aResponse()
                .withFixedDelay(4_000)
                .withStatus(200)
                .withBody(stripeChargeJson("ch_A001")))
            .willSetStateTo("fallback-retry"));

        // Fallback call (direct SDK) — returns ch_A immediately (same key)
        wm.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("timeout-then-fallback")
            .whenScenarioStateIs("fallback-retry")
            .willReturn(aResponse()
                .withStatus(200)
                .withBody(stripeChargeJson("ch_A001"))));  // same ch_A001 (cached)

        // Act
        String idempotencyKey = toIdempotencyKey("sub_001", "2026-10", 2999);
        ChargeResponse result = billingService.chargeCustomer(
            "cus_001", 2999, "sub_001", "2026-10");

        // Assert: Stripe received exactly 2 requests (original + fallback retry)
        List<LoggedRequest> stripeRequests = wm.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        assertThat(stripeRequests).hasSize(2);

        // Assert: both requests carry THE SAME Idempotency-Key
        String key1 = stripeRequests.get(0).getHeader("Idempotency-Key");
        String key2 = stripeRequests.get(1).getHeader("Idempotency-Key");
        assertThat(key1).isEqualTo(idempotencyKey);
        assertThat(key2).isEqualTo(idempotencyKey);   // same key — no ch_B

        // Assert: result is ch_A001 (from fallback)
        assertThat(result.getChargeId()).isEqualTo("ch_A001");
    }

    @Test
    void fallbackFactory_withRandomUUID_doublesCharge_onTimeout_demonstratesBug() {
        // This test documents the BUGGY behavior where fallback generates UUID_B.
        // In production this creates ch_B alongside ch_A.
        //
        // First call times out (Stripe committed ch_A).
        // Fallback generates UUID_B → Stripe creates ch_B.
        // Both ch_A and ch_B are now committed on Stripe.
        //
        // Detection: count distinct Idempotency-Key values in Stripe requests.
        // If count > 1, double-charge risk exists.
        //
        // Two distinct keys → two charges → customer billed twice.
        // assertThat(key1).isNotEqualTo(key2); // ← this passes in the buggy version
    }
}

Comparison table

One principle underlying all three modes

All three failure modes stem from generating the Stripe idempotency key at a point in the execution path that is repeated on failure. The RequestInterceptor is repeated because Feign’s retry pipeline calls interceptors fresh on each attempt. The LoadBalancer retry is repeated because the entire request is re-issued to a new server instance. The fallback is repeated because the circuit breaker invokes the fallback method after the original call fails.

The fix in all three cases is the same: generate the idempotency key before any of these repeatable mechanisms can execute, using inputs that are stable across all repetitions. A content-hash key derived from business-level inputs (subscription ID, billing period, amount, customer ID) satisfies this requirement. Once computed, the key must be passed explicitly through every layer — as a request body field, as a @RequestHeader parameter, or as a field read by the downstream service — rather than being regenerated by any interceptor, retry handler, or fallback.

Storing the key in the PENDING payment row before any Stripe call is the second half of the fix. If the application crashes between the PENDING write and the Stripe call, recovery can load the PENDING row, read the stored key, and call Stripe with the same key. If Stripe committed a charge under that key earlier, the cached response is returned. If Stripe never saw the key, a fresh charge is created. Either outcome is correct; neither creates a second charge.

Spring Cloud OpenFeign vs vanilla Feign: what changes

The failure modes in this post are specifically tied to Spring Cloud OpenFeign’s integration layers — the LoadBalancer retry interceptor and the Spring Cloud Circuit Breaker fallback machinery — which are not present in vanilla OpenFeign. The vanilla Feign Retryer failure (Mode 1) exists in vanilla Feign too, but Spring Cloud OpenFeign adds the @FeignClient annotation-driven interceptor registration that makes it easy to register a global RequestInterceptor as a Spring bean, which is how most developers add the idempotency key in a Spring Boot application.

Vanilla OpenFeign RequestInterceptor behavior — generating a UUID per interceptor call — was covered in the Feign + Stripe post. The @Transactional interaction (JDBC rollback leaving Stripe committed) was covered in the Feign + Transactional + Stripe post. The Spring Cloud Circuit Breaker failure modes (TimeLimiter without fallback, AOP proxy ordering, CLOSED-to-OPEN transition) were covered in the Spring Cloud Circuit Breaker + Stripe post. This post fills the gap: the OpenFeign-specific fallback factory interaction, and the LoadBalancer retry layer that sits above both OpenFeign and the circuit breaker.

Configuring Spring Cloud OpenFeign safely for Stripe calls

Based on the three failure modes, here is a safe configuration baseline for Spring Cloud OpenFeign applications that call Stripe:

# application.yml — safe Spring Cloud OpenFeign + Stripe configuration

spring:
  cloud:
    openfeign:
      circuitbreaker:
        enabled: true
      client:
        config:
          stripe-client:
            # No Retryer configured — disable Feign-level retry for Stripe.
            # Stripe idempotency handles retry safety at the application level.
            # Feign retry is safe only if the idempotency key is stable across
            # attempts, which requires caller-side key generation.
            retryer: feign.Retryer$NEVER_RETRY
            connect-timeout: 3000
            read-timeout: 30000   # Stripe P99 < 3s, but give buffer for heavy load
    loadbalancer:
      retry:
        enabled: true
        # DO NOT enable retryOnAllOperations for clients that make non-idempotent
        # calls externally (like Stripe charges). Default: false.
        retry-on-all-operations: false
        max-retries-on-same-service-instance: 1
        max-retries-on-next-service-instance: 1
        retryable-status-codes:
          - 502
          - 503

resilience4j:
  timelimiter:
    instances:
      stripe-client:
        # Set timeout generously — Stripe P99 is under 3s, but rare spikes
        # occur. A too-tight TimeLimiter is a double-charge risk.
        timeout-duration: 15s
  circuitbreaker:
    instances:
      stripe-client:
        sliding-window-size: 20
        failure-rate-threshold: 60
        # Open the circuit on network errors, NOT on slow calls alone.
        # A slow Stripe call may have committed a charge.
        slow-call-duration-threshold: 10s
        slow-call-rate-threshold: 80

Key decisions in this configuration: Feign-level retry is disabled (NEVER_RETRY) in favor of application-level retry with stable idempotency keys. LoadBalancer retry-on-all-operations is left at the default false. The TimeLimiter timeout is set to 15 seconds — well above Stripe’s P99 response time under normal load — to minimize the probability of a timeout-after-commit scenario. Circuit breaker failure rate is based on hard errors, not just slow calls, to avoid treating a legitimate slow Stripe response (which may have committed a charge) as a circuit-open event that triggers a fallback with a fresh UUID.