Spring Cloud Sleuth and Stripe Integration: How @NewSpan Creates a Fresh spanId on Each @Retryable Attempt Making the Idempotency Key Non-Repeatable, a Missing LazyTraceExecutor Wrapper Causes Tracer.currentSpan() to Return a NOOP Span on the @Async Billing Thread and the Null-Safe Fallback Generates UUID_B, and a BaggageField Idempotency Key Absent from the @Scheduled Retry Context Causes the Null Guard to Call UUID.randomUUID() on Every Dead-Letter Attempt

This post is part of the ongoing series on Stripe idempotency failure modes in JVM frameworks. Earlier entries covered Spring Data JPA (JDBC transaction boundary and Stripe commit ordering), Spring Cloud Circuit Breaker (Resilience4j TimeLimiter post-commit fallback, CircuitBreakerOpenException open-state recovery), Spring Cloud OpenFeign (RequestInterceptor on every retry, retryOnAllOperations re-sends POST to new instance), Spring Data Elasticsearch (@Retryable UUID reset after VersionConflictEngineException, NRT search gap), and Spring Cloud Config (@RefreshScope bean destruction clears in-flight key cache, @Value-bound prefix changed on config refresh). This post moves to Spring Cloud Sleuth and its successor, Micrometer Tracing (Spring Boot 3.x). Distributed tracing infrastructure introduces three behavioral properties — per-invocation span creation, trace-context thread-locality, and Baggage propagation semantics — that each independently cause Stripe double-charges when developers couple idempotency key derivation to trace context without understanding how spans are scoped, how context is or is not propagated across executor boundaries, and what Baggage fields return when no active span is in scope.

The three failure modes are: the @NewSpan AOP annotation creating a fresh Span (and therefore a fresh spanId) on every method invocation, including every @Retryable attempt, so a key component derived from Tracer.currentSpan().context().spanId() is non-repeatable across retries; the trace context being thread-local and not automatically crossing executor boundaries, so a @Async billing method running on a thread pool that lacks a LazyTraceExecutor (Spring Cloud Sleuth) or a context-propagation-aware wrapper (Micrometer Tracing) receives a NOOP span from Tracer.currentSpan() and the null-safe fallback path generates a fresh UUID on every async attempt; and BaggageField values being scoped to the active trace context, so an idempotency key stored in Baggage during an HTTP-originated billing request is not in scope in a @Scheduled dead-letter reprocessor that runs without an active span, causing the null guard to generate UUID.randomUUID() on every scheduled retry.

Stack for examples: Spring Boot 2.7.x / 3.x, spring-cloud-starter-sleuth (Spring Cloud 2021.x, Brave tracer) for Spring Boot 2.x examples and micrometer-tracing-bridge-brave with io.micrometer:micrometer-tracing for Spring Boot 3.x equivalents, Spring Retry (spring-retry), spring-kafka for the dead-letter consumer example, Stripe Java SDK v24+, WireMock 3.x, JUnit 5.

Background: why developers couple Stripe idempotency keys to distributed trace context

The motivation for tying a Stripe idempotency key to a distributed trace identifier is appealing in theory: if the billing service is called as part of a traced HTTP request, the traceId (or a combination of traceId and spanId) is already a unique, request-scoped string that is stable for the duration of the request-handling pipeline. Developers already use this identifier to correlate log lines, database audit records, and downstream calls — extending that correlation to the Stripe idempotency key gives a single identifier that threads through the observability pipeline and the payment processor simultaneously. “I can look up any Stripe charge by the traceId I already have in my logs.”

The approach breaks down as soon as any of three assumptions fails: that the same trace context is active on every attempt for a given billing event, that the trace context is in scope at all on the thread executing the billing code, and that the trace context carries the same Baggage values on every execution path that might call the billing service. Spring Cloud Sleuth and Micrometer Tracing each give these assumptions an appearance of correctness that is only valid for the synchronous, single-attempt HTTP request path. The moment retries, async executors, or scheduled jobs enter the picture, all three assumptions break independently.

It is worth being explicit about the Stripe idempotency mechanism before the failure mode descriptions: Stripe deduplicates charges by the string value of the Idempotency-Key request header, scoped to the account. The string is opaque — Stripe does not parse its structure. Two calls with different Idempotency-Key strings are treated as two distinct charge requests regardless of whether they have identical amounts, customers, or currencies. If the first call committed ch_A and the retry sends a different key string, Stripe creates ch_B as a new charge. There is no “close enough” deduplication based on request parameters.

Mode 1: @NewSpan("billing-charge") creates a new Span on each @Retryable invocation — spanId differs on every attempt — UUID_B derived on the retry path — ch_B committed alongside already-committed ch_A

The developer builds a billing service that uses @NewSpan to create a dedicated span for each Stripe charge attempt, so that the Stripe call appears as a named child span in the distributed trace. The developer also derives the Stripe idempotency key from the span’s spanId, reasoning that this ties the Stripe charge to the specific span in the trace and makes the correlation unambiguous in Zipkin or Jaeger. The method is also annotated with @Retryable for Spring Retry-based automatic retry on transient network failures.

The developer’s mental model: “@NewSpan creates a span for the billing-charge operation. This span represents the billing operation itself — the same billing event, the same span. The spanId is stable for as long as I’m handling the same billing event. Since @Retryable retries are retrying the same billing event, the same span is in scope.”

This is incorrect. @NewSpan is implemented as a Spring AOP aspect (SleuthSpanCreatingBeanPostProcessor in Spring Cloud Sleuth, ObservationOrTimerCompositeKeyValuesConvention via @Observed-equivalent mechanism in Micrometer Tracing) that creates a new Span on every method invocation. When the annotated method exits — whether by returning normally or by throwing an exception — the span is closed. The @Retryable aspect catches the exception after the method exits and, after the configured back-off delay, calls the method again. Because the method is called again, @NewSpan fires again and creates a new Span with a new randomly generated spanId. The prior span, with its original spanId, is already closed and its ID is not accessible. The new span has a different spanId.

// BillingService.java — @NewSpan + @Retryable, spanId used as idempotency key component
@Service
public class BillingService {

    @Autowired
    private Tracer tracer;

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    @NewSpan("billing-charge")     // creates a new Span on every invocation
    @Retryable(
        retryFor = { StripeConnectTimeoutException.class, SocketTimeoutException.class },
        maxAttempts = 3,
        backoff = @Backoff(delay = 500, multiplier = 2)
    )
    public String chargeSubscription(String subscriptionId, String billingPeriod) {

        // Developer derives idempotency key from the current span's spanId.
        // Reasoning: "This span IS the billing-charge operation. Its spanId is
        // unique to this charge attempt, and stable for the duration of this call."
        //
        // Flaw: @NewSpan creates a NEW span on EVERY invocation of this method.
        // After @Retryable catches the exception and retries, @NewSpan creates a
        // new span with a different spanId. The key is non-repeatable across retries.
        String spanId = tracer.currentSpan().context().spanId();
        String idempotencyKey = "charge:" + spanId + ":" + subscriptionId + ":" + billingPeriod;

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        ChargeResponse charge = stripeClient.createCharge(
            subscriptionRepository.findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        return charge.getId();
    }

    @Recover
    public String recover(Exception e, String subscriptionId, String billingPeriod) {
        throw new BillingFailedException("Stripe charge failed after retries", e);
    }
}

The failure sequence:

1. chargeSubscription("sub_001", "2026-10") is called. @NewSpan starts span_A with spanId_A. The key is "charge:spanId_A:sub_001:2026-10". stripeClient.createCharge(...) is called with this key. Stripe commits ch_A. The TCP connection experiences a post-commit socket timeout — the response was in Stripe’s send buffer when the client’s read deadline expired.
2. StripeConnectTimeoutException is thrown. @NewSpan’s AOP interceptor closes span_A (the method exited via exception). The method exits.
3. @Retryable’s interceptor catches the exception. After the configured back-off delay, it calls chargeSubscription again.
4. @NewSpan fires on the second invocation. It creates span_B with a fresh spanId_B (randomly generated, independent of spanId_A). The key is "charge:spanId_B:sub_001:2026-10" — a different string.
5. stripeClient.createCharge(..., "charge:spanId_B:sub_001:2026-10") is called. Stripe has no idempotency cache entry for spanId_B. It processes the request as a new charge. ch_B is committed alongside already-committed ch_A.
6. The customer is billed twice for the same billing period.

The same failure mode applies when the developer uses tracer.currentSpan().context().traceId() in a context where @NewSpan creates a new root span. If there is no parent span in scope when the billing method is called — for example, in a @Scheduled job or a Kafka consumer listener that does not carry incoming trace context — @NewSpan creates a new root span with a fresh traceId. Both traceId and spanId are fresh on every invocation. The key is non-repeatable regardless of which identifier the developer uses.

Why the span lifecycle contradicts the developer’s mental model

The AOP-based span lifecycle in Spring Cloud Sleuth (and its Micrometer Tracing successor via @Observed or the Sleuth compatibility layer) is method-scoped, not event-scoped. A span starts when the method is entered and ends when the method exits. This mirrors the natural unit of observability for AOP — a single method call — not the unit of business logic, which may span multiple method invocations across retries.

There is no span reuse across @Retryable attempts. Spring Retry’s RetryInterceptor and Spring Cloud Sleuth’s SleuthSpanCreatingBeanPostProcessor are both AOP advisors on the same target method. Spring Retry’s advisor catches the exception after the method proxy chain has already finished for that invocation, including the span close. When Spring Retry re-invokes the method, the full proxy chain runs again from the start, including the span creation.

The fix is to not derive idempotency keys from trace context at all. Use stable business inputs. The key SHA-256(subscriptionId + billingPeriod + amountCents) is identical on every invocation of chargeSubscription for a given billing event, regardless of how many retries occur, which span is active, or whether there is a span at all. Trace context belongs to the observability layer; idempotency keys belong to the payment layer. Coupling them makes one layer’s behavior a correctness dependency of the other.

// Fixed BillingService.java — key derived from stable business inputs, no trace coupling
@Service
public class BillingService {

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    @NewSpan("billing-charge")     // kept for observability — no longer affects correctness
    @Retryable(
        retryFor = { StripeConnectTimeoutException.class, SocketTimeoutException.class },
        maxAttempts = 3,
        backoff = @Backoff(delay = 500, multiplier = 2)
    )
    public String chargeSubscription(String subscriptionId, String billingPeriod) {

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        // Key derived from stable business inputs only.
        // Same value on every @Retryable attempt, across all span IDs.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        ChargeResponse charge = stripeClient.createCharge(
            subscriptionRepository.findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        return charge.getId();
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

Test for Mode 1

@SpringBootTest
@ExtendWith(MockitoExtension.class)
class BillingServiceSpanIdKeyTest {

    @Autowired
    private BillingService billingService;

    @MockBean
    private StripeClient stripeClient;

    @MockBean
    private SubscriptionRepository subscriptionRepository;

    @Test
    void retryAttemptsSendSameIdempotencyKey() {
        when(subscriptionRepository.findAmountCents("sub_001")).thenReturn(9900);
        when(subscriptionRepository.findCustomerId("sub_001")).thenReturn("cus_001");

        // First attempt: Stripe commits, then socket times out.
        when(stripeClient.createCharge(any(), anyInt(), any(), any()))
            .thenThrow(new StripeConnectTimeoutException("timeout"))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"));

        billingService.chargeSubscription("sub_001", "2026-10");

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(stripeClient, times(2))
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        List<String> keys = keyCaptor.getAllValues();
        // Both attempts must send the same idempotency key.
        // If keys differ, the retry will produce a double charge on real Stripe.
        assertThat(keys.get(0)).isEqualTo(keys.get(1));
    }
}

This test fails with the buggy spanId-based implementation because keys.get(0) contains spanId_A and keys.get(1) contains spanId_B. It passes with the stable-key implementation because both attempts hash to the same SHA-256 value.

Mode 2: @Async billing executor not wrapped with LazyTraceExecutor — Tracer.currentSpan() returns NOOP span on the executor thread — traceId-based key falls through to UUID.randomUUID() null-safe branch — every async attempt generates a fresh UUID_B — ch_B committed

The developer builds a billing service where the Stripe charge is dispatched asynchronously to a dedicated ThreadPoolTaskExecutor for isolation from the main request thread. The billing method is annotated @Async("billingExecutor"), and it runs on the executor’s thread pool. The developer uses the traceId from the calling request as the primary idempotency key component, with a null-safe fallback to UUID.randomUUID() in case no trace context is available.

The developer’s mental model: “Sleuth automatically propagates the trace context to @Async methods because it instruments Spring’s task execution infrastructure. The traceId on the executor thread is the same as the traceId on the calling thread. The null-safe fallback is only a defensive code path for non-traced contexts like unit tests.”

The automatic propagation the developer is thinking of applies to @Async executors that Spring Cloud Sleuth has instrumented. Sleuth auto-configures a LazyTraceExecutor wrapper around the default SimpleAsyncTaskExecutor and the auto-configured ThreadPoolTaskExecutor. It does not wrap custom executor beans that the developer declares independently in a @Configuration class unless the developer either explicitly wraps them or enables Sleuth’s executor customizer.

In Micrometer Tracing (Spring Boot 3.x), the equivalent requirement is wrapping the executor with ContextExecutorService.wrap(executor, ContextSnapshotFactory.builder().build()) or using ContextPropagatingTaskDecorator on the ThreadPoolTaskExecutor. A custom executor bean declared without this wrapping runs its tasks in threads with no active trace context.

// AsyncBillingConfig.java — custom executor WITHOUT trace context propagation
@Configuration
@EnableAsync
public class AsyncBillingConfig {

    @Bean(name = "billingExecutor")
    public Executor billingExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(8);
        executor.setQueueCapacity(100);
        executor.setThreadNamePrefix("billing-");
        executor.initialize();

        // Missing: executor.setTaskDecorator(new ContextPropagatingTaskDecorator());
        // Without this, tasks submitted to this executor run with no active TraceContext.
        return executor;
    }
}

// AsyncBillingService.java — @Async billing, traceId in key, UUID fallback
@Service
public class AsyncBillingService {

    @Autowired
    private Tracer tracer;

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    @Async("billingExecutor")
    public CompletableFuture<String> chargeSubscriptionAsync(
            String subscriptionId,
            String billingPeriod) {

        // Attempt to derive key from traceId for cross-service correlation.
        // Null-safe fallback: if no trace context on this thread, use UUID.
        Span currentSpan = tracer.currentSpan();

        String traceId = (currentSpan != null && !currentSpan.isNoop())
            ? currentSpan.context().traceId()
            : null;

        // Developer expects traceId to be propagated from the calling thread.
        // On a custom executor without ContextPropagatingTaskDecorator, currentSpan()
        // returns Tracer.NOOP (isNoop() == true), traceId == null.
        // The null branch executes: UUID.randomUUID() — different on every async attempt.
        String idempotencyKey = (traceId != null)
            ? "charge:" + traceId + ":" + subscriptionId + ":" + billingPeriod
            : UUID.randomUUID().toString();   // BUG: new UUID on every async invocation

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        try {
            ChargeResponse charge = stripeClient.createCharge(
                subscriptionRepository.findCustomerId(subscriptionId),
                amountCents,
                billingPeriod,
                idempotencyKey
            );
            return CompletableFuture.completedFuture(charge.getId());
        } catch (StripeConnectTimeoutException e) {
            // Caller's retry mechanism will call chargeSubscriptionAsync again.
            // Each call to chargeSubscriptionAsync submits a new task to billingExecutor.
            // Each new task has no TraceContext. traceId == null again. UUID.randomUUID() again.
            throw new AsyncBillingException("Stripe call failed", e);
        }
    }
}

The failure sequence:

1. HTTP request handler calls chargeSubscriptionAsync("sub_001", "2026-10"). The method returns immediately with a CompletableFuture. The task is submitted to billingExecutor.
2. The executor thread picks up the task. Because billingExecutor is not wrapped with ContextPropagatingTaskDecorator, the executor thread has no active TraceContext. tracer.currentSpan() returns the NOOP span. isNoop() is true. traceId is assigned null.
3. The null branch executes: idempotencyKey = UUID.randomUUID().toString(). The key is, say, "a3f8...". stripeClient.createCharge(..., "a3f8...") is called. Stripe commits ch_A. The socket read times out after commit.
4. AsyncBillingException is thrown. The CompletableFuture completes exceptionally. The caller’s exception handler schedules a retry.
5. On the retry, chargeSubscriptionAsync("sub_001", "2026-10") is called again. A new task is submitted to billingExecutor. The executor thread again has no TraceContext. The null branch executes again: UUID.randomUUID() produces "b7c2...".
6. stripeClient.createCharge(..., "b7c2...") is called. Stripe has no cache entry for "b7c2...". ch_B is committed. The customer is billed twice.

The NOOP span in Spring Cloud Sleuth and Micrometer Tracing

Spring Cloud Sleuth (Brave tracer) returns a Span whose isNoop() method returns true when tracing is disabled or when the current thread has no active span context. The NOOP span’s context().traceId() returns "0000000000000000" (16 hex zeros for Brave’s 64-bit trace ID representation) or "00000000000000000000000000000000" for 128-bit trace IDs, depending on whether 128-bit IDs are enabled. tracer.currentSpan() may return null outright in some Brave configurations when there is no active span.

Micrometer Tracing (Spring Boot 3.x) uses a Tracer interface with the same contract: tracer.currentSpan() returns null if no span is active on the current thread. The isNoop() method is available on spans created by the NOOP tracer (used when no tracing backend is configured). In either case, the defensive check currentSpan != null && !currentSpan.isNoop() is the correct guard — but the correct response to a null/NOOP span is not UUID.randomUUID(). It is to derive the key from stable business inputs.

Two fixes: wrapping the executor, or eliminating the trace dependency

Fix A (keep trace correlation, restore propagation): Wrap the custom executor with ContextPropagatingTaskDecorator (Spring Framework 6.x / Spring Boot 3.x) or, for Spring Boot 2.x with Sleuth, wrap it as a LazyTraceExecutor or TraceableExecutorService.

// Fixed AsyncBillingConfig.java — executor with trace context propagation
@Configuration
@EnableAsync
public class AsyncBillingConfig {

    // Spring Boot 3.x / Micrometer Tracing:
    @Bean(name = "billingExecutor")
    public Executor billingExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(8);
        executor.setQueueCapacity(100);
        executor.setThreadNamePrefix("billing-");
        executor.setTaskDecorator(new ContextPropagatingTaskDecorator());  // propagates MDC + trace
        executor.initialize();
        return executor;
    }
}

// Spring Boot 2.x with Spring Cloud Sleuth / Brave:
// @Bean(name = "billingExecutor")
// public Executor billingExecutor(BeanFactory beanFactory) {
//     ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
//     executor.setCorePoolSize(4);
//     executor.setMaxPoolSize(8);
//     executor.setQueueCapacity(100);
//     executor.setThreadNamePrefix("billing-");
//     executor.initialize();
//     return new LazyTraceExecutor(beanFactory, executor);
// }

With ContextPropagatingTaskDecorator, the task decorator captures the ContextSnapshot (which includes the TraceContext, MDC map, and any Micrometer ObservationRegistry context) at task submission time and restores it on the executor thread before running the task. tracer.currentSpan() on the executor thread returns the span that was active when the task was submitted. The traceId-based key derivation now works correctly for the HTTP request path.

However, Fix A does not resolve the failure mode for requests that enter through non-HTTP paths (Kafka consumers, scheduled jobs, programmatic calls without an active span). Fix A makes the traceId-based path correct for HTTP-originated calls but leaves the UUID fallback path active for non-HTTP paths. Those paths still generate a fresh UUID on each attempt.

Fix B (eliminate the trace dependency entirely):

// Fixed AsyncBillingService.java — no trace coupling, stable key always
@Service
public class AsyncBillingService {

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    @Async("billingExecutor")
    public CompletableFuture<String> chargeSubscriptionAsync(
            String subscriptionId,
            String billingPeriod) {

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        // No trace context coupling. Key is stable regardless of executor,
        // thread pool, active span, or invocation context.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        ChargeResponse charge = stripeClient.createCharge(
            subscriptionRepository.findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        return CompletableFuture.completedFuture(charge.getId());
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

Test for Mode 2

@SpringBootTest
class AsyncBillingTraceContextTest {

    @Autowired
    private AsyncBillingService asyncBillingService;

    @MockBean
    private StripeClient stripeClient;

    @MockBean
    private SubscriptionRepository subscriptionRepository;

    @Test
    void asyncBillingRetriesSendSameIdempotencyKey() throws Exception {
        when(subscriptionRepository.findAmountCents("sub_001")).thenReturn(4900);
        when(subscriptionRepository.findCustomerId("sub_001")).thenReturn("cus_001");

        // First async attempt: Stripe commits, then socket times out.
        // Second async attempt: returns normally.
        when(stripeClient.createCharge(any(), anyInt(), any(), any()))
            .thenThrow(new StripeConnectTimeoutException("timeout"))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"));

        // Call once, then once more simulating a retry.
        try {
            asyncBillingService.chargeSubscriptionAsync("sub_001", "2026-10").get();
        } catch (ExecutionException ignored) { }

        asyncBillingService.chargeSubscriptionAsync("sub_001", "2026-10").get();

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(stripeClient, times(2))
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        List<String> keys = keyCaptor.getAllValues();
        // Both async attempts MUST send the same key.
        assertThat(keys.get(0)).isEqualTo(keys.get(1));
    }
}

Mode 3: BaggageField idempotency key populated on the HTTP request thread is absent in the @Scheduled dead-letter reprocessor — BaggageField.get(context) returns null in the scheduler — null guard calls UUID.randomUUID() on every scheduled retry — ch_B committed

The developer builds a billing system with two code paths that can call the same Stripe billing service: a synchronous HTTP request path and a @Scheduled dead-letter reprocessor that retries billing events that failed in the HTTP path. To maintain a single Stripe idempotency key per billing event across both code paths, the developer uses Micrometer Tracing’s BaggageField mechanism to propagate a pre-computed idempotency key through the distributed trace context.

In the HTTP path: the billing controller generates idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents), stores it in the BILLING_KEY_FIELD BaggageField, persists the event to a billing_events table with idempotency_key = idempotencyKey, and calls the billing service. The billing service reads the key from Baggage: BILLING_KEY_FIELD.get(tracer.currentSpan().context()).

In the scheduler path: the @Scheduled reprocessor queries SELECT * FROM billing_events WHERE status = 'PENDING' AND last_attempt_at < NOW() - INTERVAL 5 MINUTES and calls the billing service for each event, passing the subscriptionId and billingPeriod. The developer notes that the idempotency_key is in the database row and assumes it will be available in Baggage.

The flaw: the @Scheduled method runs on Spring’s scheduled task executor, which does not carry HTTP request context. There is no active trace, no Baggage, and no BaggageField values in scope. The billing service reads null from BaggageField.get(). The developer’s null-safe guard calls UUID.randomUUID() as a fallback. The key in the database row is never consulted because the billing service does not know it should look there — it expected to find the key in Baggage.

// BaggageIdempotencyService.java — reads idempotency key from BaggageField
@Service
public class BaggageIdempotencyBillingService {

    // BaggageField declaration — must be registered as a remote propagation field
    // in application.yml: spring.sleuth.baggage.remote-fields=billing-idempotency-key
    // or micrometer.tracing.baggage.remote-fields=billing-idempotency-key
    public static final BaggageField BILLING_KEY_FIELD =
        BaggageField.create("billing-idempotency-key");

    @Autowired
    private Tracer tracer;

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    public String chargeSubscription(String subscriptionId, String billingPeriod) {

        // Developer expects the idempotency key to be in Baggage,
        // set by the caller before invoking this service.
        //
        // HTTP path: BillingController sets BILLING_KEY_FIELD.updateValue(...) before calling here.
        // Scheduler path: @Scheduled reprocessor does NOT set BILLING_KEY_FIELD.
        //                 tracer.currentSpan() returns null (no active span in scheduler).
        //                 BaggageField.get(null) cannot be called safely.
        Span currentSpan = tracer.currentSpan();

        String idempotencyKey = null;
        if (currentSpan != null && !currentSpan.isNoop()) {
            idempotencyKey = BILLING_KEY_FIELD.get(currentSpan.context());
        }

        // null-safe fallback: UUID.randomUUID() — different on every scheduled retry
        if (idempotencyKey == null) {
            idempotencyKey = UUID.randomUUID().toString();   // BUG in @Scheduled context
        }

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        ChargeResponse charge = stripeClient.createCharge(
            subscriptionRepository.findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        return charge.getId();
    }
}

// BillingReprocessor.java — @Scheduled dead-letter retry
@Component
public class BillingReprocessor {

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private BaggageIdempotencyBillingService billingService;

    @Scheduled(fixedDelay = 300_000)    // every 5 minutes
    public void reprocessPendingBillingEvents() {
        List<BillingEvent> pendingEvents = billingEventRepository
            .findPendingOlderThan(Instant.now().minus(5, MINUTES));

        for (BillingEvent event : pendingEvents) {
            try {
                // Developer's intent: the billing service reads the idempotency key
                // that was generated for this event in the original HTTP request and
                // stored in the billing_events table.
                //
                // Missing: BILLING_KEY_FIELD.updateValue(tracer.currentSpan().context(),
                //              event.getIdempotencyKey());
                //
                // Without this call, billingService.chargeSubscription() reads null
                // from BaggageField and falls back to UUID.randomUUID() for EVERY retry
                // of EVERY pending event.
                billingService.chargeSubscription(
                    event.getSubscriptionId(),
                    event.getBillingPeriod()
                );
                billingEventRepository.markCompleted(event.getId());
            } catch (Exception e) {
                billingEventRepository.markFailed(event.getId(), e.getMessage());
            }
        }
    }
}

The failure sequence:

1. HTTP request path: BillingController generates idempotencyKey_A = toStableKey("sub_001", "2026-10", 9900). It stores the key in the database row for this billing event and calls chargeSubscription("sub_001", "2026-10") with BILLING_KEY_FIELD set in Baggage. BaggageIdempotencyBillingService reads idempotencyKey_A from Baggage and calls Stripe. Stripe commits ch_A. A database exception rolls back the UPDATE billing_events SET status='COMPLETED' and the billing event remains in PENDING status.
2. The @Scheduled reprocessor fires 5 minutes later. It queries the billing_events table and finds the sub_001 / 2026-10 event in PENDING status with idempotency_key = idempotencyKey_A stored in the row. The developer believes the billing service will use this key.
3. The reprocessor calls billingService.chargeSubscription("sub_001", "2026-10") without setting BILLING_KEY_FIELD in Baggage. In the billing service, tracer.currentSpan() returns null (no active span on the scheduled executor thread). idempotencyKey is null. The UUID fallback generates idempotencyKey_B = UUID.randomUUID().toString().
4. stripeClient.createCharge(..., idempotencyKey_B) is called. Stripe has no cache entry for idempotencyKey_B. ch_B is committed alongside ch_A. The customer is billed twice.
5. If the reprocessor fires multiple times while the event remains PENDING (due to ongoing database errors), each scheduled execution generates a fresh UUID and Stripe commits a new charge. The customer may be billed three, four, or more times.

Why Baggage is not available in the @Scheduled context

Micrometer Tracing’s BaggageField (and Spring Cloud Sleuth’s BaggageField) is a carrier for values that should propagate alongside the trace context through distributed calls — from a service’s inbound HTTP request to outbound HTTP requests and Kafka produces, carried in W3C baggage headers or B3 extra headers. The field is readable from tracer.currentSpan().context() on any thread that has the propagated trace context in scope.

A @Scheduled method runs on Spring’s ThreadPoolTaskScheduler or the JDK ScheduledExecutorService. There is no inbound HTTP request, no incoming trace headers, and no Baggage to propagate. The executor thread starts a new task invocation with a clean thread-local state. tracer.currentSpan() returns null. There is nothing to call BILLING_KEY_FIELD.get() on. The Baggage value from the HTTP-originated billing request that committed ch_A is not accessible in the scheduled retry context by any mechanism other than explicit restoration.

The fix: explicit Baggage restoration from the persisted key, or elimination of the Baggage coupling

Fix A (explicit Baggage restoration): The reprocessor must restore the Baggage field from the persisted idempotency_key column before calling the billing service. This requires starting a new Span, attaching the Baggage to it, and calling the billing service within that Span’s scope.

// Fixed BillingReprocessor.java — explicit Baggage restoration from database row
@Component
public class BillingReprocessor {

    @Autowired
    private Tracer tracer;

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private BaggageIdempotencyBillingService billingService;

    @Scheduled(fixedDelay = 300_000)
    public void reprocessPendingBillingEvents() {
        List<BillingEvent> pendingEvents = billingEventRepository
            .findPendingOlderThan(Instant.now().minus(5, MINUTES));

        for (BillingEvent event : pendingEvents) {
            Span span = tracer.nextSpan().name("billing-reprocess").start();
            try (Tracer.SpanInScope ws = tracer.withSpan(span)) {

                // Explicitly restore the persisted idempotency key into Baggage
                // before calling the billing service. This is the step the original
                // code was missing.
                BaggageIdempotencyBillingService.BILLING_KEY_FIELD.updateValue(
                    span.context(),
                    event.getIdempotencyKey()   // read from database row — always idempotencyKey_A
                );

                billingService.chargeSubscription(
                    event.getSubscriptionId(),
                    event.getBillingPeriod()
                );

                billingEventRepository.markCompleted(event.getId());
            } catch (Exception e) {
                billingEventRepository.markFailed(event.getId(), e.getMessage());
            } finally {
                span.end();
            }
        }
    }
}

With this fix, the billing service reads idempotencyKey_A from Baggage on every scheduled retry attempt. Stripe’s idempotency cache matches the key and returns the existing ch_A charge object, or a 200 OK with the already-committed data, without creating ch_B.

Fix B (eliminate the Baggage coupling, derive key from business inputs in the billing service): The cleanest resolution is to remove the Baggage layer entirely and derive the idempotency key from stable business inputs inside the billing service. The billing service computes SHA-256(subscriptionId + billingPeriod + amountCents) and does not depend on any propagated context.

// Fixed BaggageIdempotencyBillingService.java — no Baggage, no trace coupling
@Service
public class StableKeyBillingService {

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private SubscriptionRepository subscriptionRepository;

    public String chargeSubscription(String subscriptionId, String billingPeriod) {

        int amountCents = subscriptionRepository.findAmountCents(subscriptionId);

        // Derived from business inputs — same value on every execution path:
        // HTTP request, @Scheduled reprocessor, Kafka consumer, unit test.
        // No BaggageField, no Tracer, no thread-local dependencies.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        ChargeResponse charge = stripeClient.createCharge(
            subscriptionRepository.findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        return charge.getId();
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

Fix B does not require the caller to manage Baggage at all. The reprocessor can call the billing service with just the business-logic parameters. The key is computed identically by the HTTP path and the scheduler path because the computation depends only on subscriptionId, billingPeriod, and amountCents — all of which are available in both execution contexts from persistent storage.

Test for Mode 3

@SpringBootTest
class BaggageIdempotencyTest {

    @Autowired
    private BaggageIdempotencyBillingService billingService;

    @Autowired
    private BillingReprocessor reprocessor;

    @MockBean
    private StripeClient stripeClient;

    @MockBean
    private SubscriptionRepository subscriptionRepository;

    @MockBean
    private BillingEventRepository billingEventRepository;

    @Test
    void scheduledRetryUsesSameKeyAsOriginalHttpAttempt() {
        when(subscriptionRepository.findAmountCents("sub_001")).thenReturn(9900);
        when(subscriptionRepository.findCustomerId("sub_001")).thenReturn("cus_001");

        // Simulate original HTTP-path key
        String originalKey = "ab12cd34ef56gh78ij90kl12mn34op56";

        // Simulate the pending event with the key persisted in the database
        BillingEvent pendingEvent = new BillingEvent(
            "sub_001", "2026-10", "PENDING", originalKey
        );
        when(billingEventRepository.findPendingOlderThan(any()))
            .thenReturn(List.of(pendingEvent));

        // First Stripe call (HTTP path) committed ch_A but DB update rolled back.
        // Second Stripe call (scheduler path): should send originalKey, not UUID.
        when(stripeClient.createCharge(any(), anyInt(), any(), eq(originalKey)))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"));

        reprocessor.reprocessPendingBillingEvents();

        // Verify: scheduler path sent the persisted key, not a fresh UUID.
        verify(stripeClient).createCharge(any(), anyInt(), any(), eq(originalKey));
        verify(stripeClient, never()).createCharge(any(), anyInt(), any(),
            argThat(k -> !k.equals(originalKey)));
    }
}

Summary table

Quick diagnostic checklist for Spring Cloud Sleuth / Micrometer Tracing + Stripe deployments

1. Grep for trace context in idempotency key derivation. Search the codebase for patterns like spanId(), traceId(), currentSpan(), BaggageField.get(), or MDC.get("traceId") within the same method or class that calls Stripe. Any usage that feeds into the Idempotency-Key header string is a risk. Replace with a stable-key derivation function.
2. Check @NewSpan + @Retryable combinations. Any method annotated with both @NewSpan (or @Observed in Micrometer Tracing) and @Retryable (or any retry aspect) creates a new span on every attempt. If the method also uses Tracer.currentSpan() for key derivation, it is vulnerable to Mode 1.
3. Audit custom executor beans. Every @Bean that returns an Executor or ExecutorService used by @Async billing methods should have setTaskDecorator(new ContextPropagatingTaskDecorator()) (Spring Boot 3.x) or be wrapped as a LazyTraceExecutor (Sleuth / Spring Boot 2.x). The absence of this wrapper is silent — the executor works correctly for functional behavior but loses trace context without error.
4. Test @Scheduled paths independently. Write integration tests that invoke scheduled billing methods directly (not via HTTP) and assert that the Idempotency-Key header sent to Stripe (captured via WireMock) is identical across multiple invocations for the same billing event. The NOOP span / null Baggage failure is invisible in HTTP path tests.
5. Check null-safe fallbacks. Any conditional that falls back to UUID.randomUUID() when trace context is absent is a latent double-charge risk. The correct default behavior when trace context is unavailable is not to generate a random ID — it is to derive the key from business inputs that are always available. A UUID fallback converts a missing-context observation into a payment correctness failure.
6. Verify Baggage field registration. In Spring Cloud Sleuth 3.x, remote Baggage fields must be declared in spring.sleuth.baggage.remote-fields. In Micrometer Tracing, they must be in management.tracing.baggage.remote-fields (application-level) or registered via BaggagePropagationConfig. An unregistered field is not propagated over HTTP or Kafka, even if the sender writes it. The receiver always reads null. Baggage-based idempotency key propagation across service boundaries requires explicit registration at both sender and receiver.
7. Use WireMock to capture and assert Idempotency-Key headers across retry paths. A test that fires a Stripe call, simulates a timeout, and fires the call again through the retry path should assert that both requests sent the same Idempotency-Key. This is the only reliable test for all three failure modes simultaneously.

A note on Micrometer Tracing vs. Spring Cloud Sleuth

Spring Cloud Sleuth was deprecated as of Spring Cloud 2022.0.0 (aligned with Spring Boot 3.0) and replaced by Micrometer Tracing. In Spring Boot 3.x, the tracing API is io.micrometer.tracing.Tracer (not brave.Tracer directly) and the Baggage API is io.micrometer.tracing.BaggageField (not org.springframework.cloud.sleuth.BaggageField). The executor context propagation mechanism is ContextPropagatingTaskDecorator (Spring Framework 6.x) rather than LazyTraceExecutor (Sleuth). The failure modes described in this post apply to both generations because the underlying behavioral properties — per-invocation span creation, thread-local trace context, Baggage scoping — are identical between Brave-backed Sleuth and Brave-backed (or OpenTelemetry-backed) Micrometer Tracing. The API class names and configuration properties change between generations; the idempotency risks do not.

For teams migrating from Spring Boot 2.x (Sleuth) to Spring Boot 3.x (Micrometer Tracing), the migration is a risk point: code that worked by accident in Sleuth (because Sleuth’s executor auto-instrumentation covered a specific custom executor) may break in Micrometer Tracing if the equivalent auto-instrumentation has different coverage. All three failure modes become more likely after a migration if the idempotency key derivation was not audited as part of the migration scope.

Conclusion

Spring Cloud Sleuth and Micrometer Tracing introduce three trace context behaviors that, when coupled to Stripe idempotency key derivation, independently produce double charges. @NewSpan is method-invocation scoped, not billing-event scoped; it creates a new spanId on every @Retryable attempt. Trace context is thread-local and does not cross executor boundaries on custom thread pools that lack a context propagation wrapper; the null-safe UUID fallback in an untraced async context converts an observability gap into a payment correctness failure. BaggageField values are scoped to the active trace context; a @Scheduled reprocessor with no active span reads null and its null guard generates a fresh key on every retry, committing one new Stripe charge per scheduled execution.

The common root cause across all three modes is using distributed trace infrastructure as a source of idempotency key material. Trace identifiers are designed for observability correlation, not for payment deduplication. They have a different scope (span lifetime), a different propagation model (thread-local), and different availability guarantees (not available in all execution contexts) than what idempotency keys require (stable across retries, available in all execution contexts that may call Stripe). Idempotency keys should be derived from the business inputs that define the billing event — SHA-256(subscriptionId + billingPeriod + amountCents) — and they should be derived once before any Stripe call, stored in persistent state, and reused on every retry path regardless of what trace context is in scope. Distributed tracing can then observe the Stripe call without influencing its correctness.