Spring Cloud Bus and Stripe Integration: How a POST /actuator/busrefresh Broadcast Destroys All @RefreshScope BillingService Instances Cluster-Wide Simultaneously Clearing In-Flight Idempotency Keys After ch_A Commits, a RemoteApplicationEvent id Generated by UUID.randomUUID() at Construction Makes the Stripe Key Non-Repeatable When the RabbitMQ Queue Drops the Event on Broker Restart, and a spring.application.instance-id Random-on-Restart Component in the Key Derivation Generates a New Key After Pod Eviction

This post is part of the ongoing series on Stripe idempotency failure modes in JVM frameworks. Earlier entries covered Spring Data JPA (JDBC transaction boundary and Stripe commit ordering), Spring Cloud Circuit Breaker (Resilience4j TimeLimiter post-commit fallback, CircuitBreakerOpenException open-state recovery), Spring Cloud OpenFeign (RequestInterceptor on every retry, retryOnAllOperations re-sends POST to new instance), Spring Data Elasticsearch (@Retryable UUID reset after VersionConflictEngineException, NRT search gap, auto-generated ID lost in timeout), Spring Cloud Config (@RefreshScope bean destruction clears in-flight key cache, @Value-bound prefix changed on config refresh), Spring Cloud Sleuth (@NewSpan fresh spanId per @Retryable invocation, missing ContextPropagatingTaskDecorator on custom @Async executor, BaggageField key absent in @Scheduled context), and Spring Data Neo4j (@Transactional rollback clears COMPLETED guard and in-transaction UUID, ServiceUnavailableException after durable save() prevents ID receipt, Mono.retryWhen() re-subscribes UUID fromSupplier). This post moves to Spring Cloud Bus. The message bus layer introduces a distinct set of behavioral properties — cluster-wide simultaneous broadcast of refresh events, random UUID assignment inside RemoteApplicationEvent constructors, and instance-identity generation from JVM-startup entropy — that each independently cause Stripe double-charges when developers couple idempotency key derivation to bus infrastructure state without understanding the broadcast propagation model, the event object construction lifecycle, or the instance ID generation fallback behavior in non-Cloud-Foundry environments.

The three failure modes are: a billing service annotated with @RefreshScope that stores in-flight idempotency keys in an instance-field cache, where a POST /actuator/busrefresh causes Spring Cloud Bus to broadcast RefreshRemoteApplicationEvent to all cluster nodes simultaneously via the message broker, destroying all @RefreshScope bean instances on all nodes at the same moment, clearing the in-flight key cache everywhere, so that a retry queued after the broadcast but before a database-resident record is written generates UUID_B and commits ch_B alongside the already-committed ch_A; a developer who builds a billing saga using a custom ChargeBillingEvent extends RemoteApplicationEvent and derives the Stripe idempotency key from event.getId(), where RemoteApplicationEvent’s constructor assigns this.id = UUID.randomUUID().toString() at instantiation time, so that a re-published event (produced when the original is lost to a non-durable RabbitMQ queue on broker restart) carries a different getId() value than the original, causing the billing handler to derive key_B and commit ch_B; and a developer who includes spring.application.instance-id as a namespace component in the idempotency key derivation formula, where spring.application.instance-id defaults to UUID.randomUUID() via IdUtils.getDefaultInstanceId() on each cold JVM start in the absence of Cloud Foundry environment variables, so that a pod eviction and restart generates instanceId_B, and any retry submitted after the restart derives key_B and commits ch_B.

Stack for examples: Spring Boot 3.x, Spring Cloud Bus 4.x (spring-cloud-starter-bus-amqp, RabbitMQ), Spring Cloud Commons 4.x, Spring Retry (spring-retry), Stripe Java SDK v24+, WireMock 3.x, Testcontainers RabbitMQ 3.x, JUnit 5, Mockito.

Background: what Spring Cloud Bus does and why it interacts with billing idempotency

Spring Cloud Bus links distributed service instances across a lightweight message broker. Its primary function is broadcasting management events — most importantly RefreshRemoteApplicationEvent (which triggers ContextRefresher.refresh() on each receiving instance) and EnvironmentChangeEvent (which updates individual property entries). When a developer or a CI/CD pipeline calls POST /actuator/busrefresh on any one service instance, Spring Cloud Bus publishes a RefreshRemoteApplicationEvent to the springCloudBus exchange in RabbitMQ. Every service instance subscribed to that exchange receives the event and responds by refreshing its Spring application context.

The critical property of this mechanism for billing idempotency is that the broadcast is cluster-wide and simultaneous. A single POST /actuator/busrefresh does not target one instance — it targets all instances (unless a destination expression is specified, and even then, all matching instances receive it at the same time, within message broker delivery latency). If five billing service pods are running, all five receive the event within milliseconds of each other. All five call ContextRefresher.refresh(). All five destroy every @RefreshScope bean simultaneously.

The second property is that Spring Cloud Bus extends Spring’s application event model with RemoteApplicationEvent, a base class for custom events routed over the message broker. Every RemoteApplicationEvent instance is assigned a random UUID in its constructor. This UUID is serialized into the message payload and deserialized faithfully on the receiving side — so the same id travels from publisher to subscriber across the broker. The failure arises not in the transport but in re-publication: when the original message is lost (non-durable queue, broker restart) and the application re-publishes the same logical event by constructing a new RemoteApplicationEvent subclass instance, the constructor runs again and assigns a new random UUID. The original id is gone.

The third property is Spring Cloud Commons’s instance identity mechanism. spring.application.instance-id is used by Spring Cloud Bus to identify the origin and destination of bus events. When not explicitly configured, it is generated by IdUtils.getDefaultInstanceId(), which checks for Cloud Foundry environment variables (vcap.application.instance_id, CF_INSTANCE_INDEX) and falls back to UUID.randomUUID().toString() on every JVM start when those are absent — which is the case for most Kubernetes deployments. Any code that reads spring.application.instance-id and incorporates it into an idempotency key formula is implicitly depending on a value that changes on every cold JVM start, including every pod eviction, restart, or rolling deploy.

As in every post in this series: Stripe deduplicates charges by the string value of the Idempotency-Key request header, scoped to the Stripe account. The key is opaque to Stripe — it does not inspect amount, customer, or currency. Two calls with different key strings are two distinct charge requests. A key that changes between the original call and any retry will always produce a second charge if the first call committed.

Mode 1: POST /actuator/busrefresh broadcasts RefreshRemoteApplicationEvent cluster-wide via RabbitMQ, destroying all @RefreshScope BillingService instances on all nodes simultaneously — in-flight idempotency key cache cleared everywhere — retry generates UUID_B — ch_B committed alongside already-committed ch_A

The developer builds a billing service that is annotated with @RefreshScope to allow runtime refresh of configuration properties (billing API URL, timeout values, per-vendor rate limits). Because the service handles retries internally, the developer adds an in-flight idempotency key cache: a ConcurrentHashMap<String, String> inflightKeys instance field that stores generated idempotency keys keyed by subscriptionId + ":" + billingPeriod. The reasoning is: if a timeout forces a retry while the original charge is still in flight, the retry reuses the key from the cache rather than generating a new random UUID.

The developer’s mental model: “The in-flight cache guarantees that concurrent or near-concurrent calls for the same subscription and billing period use the same idempotency key. POST /actuator/busrefresh only fires on deploys or config pushes, which our ops team does during maintenance windows, not during billing runs. Even if it fires during a billing run, a refresh only updates the configuration values injected into the bean — it doesn’t wipe out instance state I manage myself. The Spring docs say refreshing a @RefreshScope bean re-binds its @Value fields; that’s all.”

This model has two compounding errors: ContextRefresher.refresh() always destroys and re-creates all @RefreshScope beans regardless of whether any injected property changed, and a POST /actuator/busrefresh via Spring Cloud Bus broadcasts to all cluster nodes simultaneously. Both of these are by design and are documented — but their interaction with a billing service that relies on in-memory idempotency state is lethal.

// BillingService.java — @RefreshScope with instance-field idempotency cache
@Service
@RefreshScope  // allows runtime refresh of billing.api-url, billing.timeout-seconds, etc.
public class BillingService {

    @Value("${billing.api-url:https://api.stripe.com}")
    private String billingApiUrl;

    @Value("${billing.timeout-seconds:30}")
    private int timeoutSeconds;

    // In-flight key cache.
    // Developer reasoning: "Concurrent or retry calls for the same subscription
    // reuse the key I already generated — no double-charge risk."
    //
    // Flaw: @RefreshScope bean destruction clears this map.
    // POST /actuator/busrefresh via Spring Cloud Bus destroys ALL @RefreshScope
    // BillingService beans on ALL nodes simultaneously. Any in-flight key stored
    // here at the moment of broadcast is lost cluster-wide.
    private final ConcurrentHashMap<String, String> inflightKeys =
        new ConcurrentHashMap<>();

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private BillingEventRepository billingEventRepository;

    public String chargeSubscription(String subscriptionId,
                                     String billingPeriod,
                                     int amountCents) {

        String cacheKey = subscriptionId + ":" + billingPeriod;

        // Reuse an existing in-flight key or generate a new one.
        // If the bean was just destroyed and re-created by @RefreshScope,
        // inflightKeys is a new empty ConcurrentHashMap — computeIfAbsent
        // always generates a fresh UUID.randomUUID().
        String idempotencyKey = inflightKeys.computeIfAbsent(
            cacheKey,
            k -> UUID.randomUUID().toString()
        );

        ChargeResponse charge = stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey  // UUID_A on first attempt, potentially UUID_B on retry
        );

        // Remove after successful charge; not reached if Stripe throws.
        inflightKeys.remove(cacheKey);

        // Persist to database in a separate transaction.
        billingEventRepository.save(
            new BillingEvent(subscriptionId, billingPeriod, idempotencyKey, charge.getId())
        );

        return charge.getId();
    }
}

The failure sequence:

1. At 14:03:00 UTC, Node A (one of five billing service pods) receives an HTTP request to charge subscription sub_001 for billing period 2026-10. inflightKeys is empty. computeIfAbsent generates idempotencyKey = "ik_A". The map now holds: {"sub_001:2026-10": "ik_A"}.
2. stripeClient.createCharge(..., "ik_A") is called. Stripe processes the charge and durably commits ch_A. The Stripe API then returns an HTTP 500 (internal error) or the TCP connection times out on Node A before the response is read. Node A propagates the error to its caller. The caller (a queue processor) enqueues a retry to be processed after a back-off delay.
3. At 14:03:05 UTC — while the retry is in the back-off queue — the CI/CD deployment pipeline finishes pushing a new config value to Spring Config Server and calls POST http://billing-service/actuator/busrefresh. Spring Cloud Bus publishes RefreshRemoteApplicationEvent to the springCloudBus RabbitMQ exchange.
4. All five billing service pod instances (Nodes A through E) receive the RefreshRemoteApplicationEvent message from RabbitMQ within milliseconds of each other. Each instance calls ContextRefresher.refresh(), which calls refreshScope.refreshAll(), which calls destroy() on every @RefreshScope bean in the application context. The BillingService bean instance is destroyed on all five nodes. A new BillingService instance is created lazily when the proxy next receives a call; this new instance has inflightKeys = new ConcurrentHashMap<>() — empty.
5. At 14:03:08 UTC, the retry dequeues. The load balancer routes it to Node B (Node A is under GC pressure). Node B’s BillingService proxy was refreshed in step 4 and its inflightKeys is now empty. computeIfAbsent finds no entry for "sub_001:2026-10" and generates idempotencyKey = "ik_B".
6. stripeClient.createCharge(..., "ik_B") is called. Stripe has a committed charge ch_A associated with key "ik_A". It has no record for "ik_B". It processes the request as a new charge and commits ch_B. The customer is billed twice.

The failure also occurs if the retry runs on Node A itself after the refresh. Node A’s old bean instance was destroyed; a new one was created with an empty inflightKeys. It does not matter which node handles the retry — all nodes are in the same state after the broadcast: a freshly created bean with no in-flight key state.

Why ContextRefresher.refresh() always destroys @RefreshScope beans regardless of config change

The developer’s intuition — “refresh is a no-op if nothing changed” — is contradicted by the implementation. ContextRefresher.refresh() does the following unconditionally: (1) re-bind the Environment from all property sources; (2) call refreshScope.refreshAll(), which calls destroy() on every bean currently registered in the RefreshScope cache, regardless of whether any of the properties they inject actually changed; (3) publish an EnvironmentChangeEvent containing the set of changed keys, which may be empty if nothing changed. Step 2 always runs. “Refreshing a @RefreshScope bean” means destroying the current instance and creating a new one from the bean definition — all field initializers run, all @PostConstruct methods run, all instance-field assignments run. The previous instance’s state is lost.

This is not a bug — it is the intended semantics. The @RefreshScope contract is: “On refresh, this bean is re-created from its definition.” It does not say: “On refresh with property changes, this bean is re-created.” The re-creation is unconditional. Any instance state managed by the developer (instance fields, caches, local state) that is not re-created by the bean definition is lost on every refresh cycle.

The same root cause was described in the Spring Cloud Config post (Mode 1). Spring Cloud Bus adds the cluster-wide propagation dimension: a single bus refresh event is not scoped to one pod — it hits every pod simultaneously. A billing system that relies on in-memory idempotency state in a @RefreshScope bean would lose that state cluster-wide on every deploy, every config push, and every accidental POST /actuator/busrefresh.

The fix is to never couple idempotency key state to @RefreshScope bean instance memory. Idempotency keys must be derivable from inputs that do not require in-memory state across bean lifecycle events.

// Fixed BillingService.java — key derived from stable business inputs; no instance state
@Service
@RefreshScope  // @RefreshScope can remain for config refresh — billing correctness is unaffected
public class BillingService {

    @Value("${billing.api-url:https://api.stripe.com}")
    private String billingApiUrl;

    @Value("${billing.timeout-seconds:30}")
    private int timeoutSeconds;

    @Autowired
    private StripeClient stripeClient;

    @Autowired
    private BillingEventRepository billingEventRepository;

    public String chargeSubscription(String subscriptionId,
                                     String billingPeriod,
                                     int amountCents) {

        // Key derived from stable business inputs before any Stripe or DB call.
        // No in-memory state. No dependency on bean lifecycle.
        // @RefreshScope bean destruction does not affect this key — the key is
        // computed from method arguments, not from instance fields or bean state.
        // Same inputs = same key on every attempt, even across pod restarts
        // and @RefreshScope re-creation cycles.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        // Guard: check for an existing completed charge with this key.
        // Uses the database as the source of truth, not in-memory state.
        // Survives bean destruction, pod restarts, and cluster-wide refreshes.
        Optional<BillingEvent> existing = billingEventRepository
            .findByIdempotencyKey(idempotencyKey);
        if (existing.isPresent() && "COMPLETED".equals(existing.get().getStatus())) {
            return existing.get().getChargeId();
        }

        ChargeResponse charge = stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        billingEventRepository.save(
            new BillingEvent(subscriptionId, billingPeriod, idempotencyKey, charge.getId())
        );

        return charge.getId();
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

The database-resident guard (billingEventRepository.findByIdempotencyKey()) replaces the in-memory inflightKeys cache. The database persists across pod restarts, bean re-creation, and cluster-wide refresh events. The guard is correct across all of these events because the key is the same on every attempt — it is derived from subscriptionId, billingPeriod, and amountCents, all of which are method arguments that do not change between attempts.

Targeted bus refresh does not change the analysis

Spring Cloud Bus supports destination-scoped events: POST /actuator/busrefresh?destination=billing-service:** targets only instances where spring.application.name is billing-service. A developer might reason: “If I scope the busrefresh to just the config service, it won’t hit billing-service.” This is correct for the targeted pods. But in practice, /actuator/busrefresh without a destination (the common invocation) targets all services. Any script, CI/CD pipeline, or ops runbook that calls the endpoint without a destination will broadcast to every service subscribed to the bus exchange — including billing-service. Relying on destination scoping as an operational guarantee for billing correctness is not sufficient. The billing service’s idempotency guarantee must not depend on @RefreshScope bean instance state surviving indefinitely.

Test for Mode 1

@SpringBootTest
@AutoConfigureWireMock(port = 0)
class BillingServiceBusRefreshTest {

    @Autowired
    private BillingService billingService;

    @Autowired
    private RefreshScope refreshScope;  // org.springframework.cloud.context.scope.refresh.RefreshScope

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Test
    void busRefreshBetweenAttemptAndRetryDoesNotChangeDerivedIdempotencyKey() {
        // WireMock: first call returns 500 (simulating Stripe error after ch_A committed),
        // second call returns 200 with ch_A (Stripe deduplicates correctly — same key).
        stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(serverError().withBody("{\"error\":{\"type\":\"api_error\"}}")));

        // First attempt — Stripe returns 500 (ch_A was committed but response is 500).
        // This simulates the case where Stripe commits but returns an error.
        assertThatThrownBy(() ->
            billingService.chargeSubscription("sub_001", "2026-10", 9900));

        // Simulate POST /actuator/busrefresh by calling refreshScope.refreshAll().
        // This is what Spring Cloud Bus does on every receiving instance.
        refreshScope.refreshAll();

        // Stub second Stripe call to succeed.
        resetAllRequests();
        stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(ok().withBody("{\"id\":\"ch_A\",\"status\":\"succeeded\"}")));

        // Second attempt — after busrefresh destroyed and re-created BillingService.
        billingService.chargeSubscription("sub_001", "2026-10", 9900);

        // Verify both calls used the same Idempotency-Key header.
        List<LoggedRequest> stripeRequests = getAllServeEvents().stream()
            .map(ServeEvent::getRequest)
            .collect(Collectors.toList());

        List<String> keys = stripeRequests.stream()
            .map(r -> r.getHeader("Idempotency-Key"))
            .collect(Collectors.toList());

        assertThat(keys).isNotEmpty();
        // All attempts must send the same key.
        // If keys differ, the retry generated a new charge rather than deduplicating.
        assertThat(keys).allMatch(k -> k.equals(keys.get(0)));
    }
}

Mode 2: Custom ChargeBillingEvent extends RemoteApplicationEvent used in a billing saga — event.getId() is UUID.randomUUID() assigned at construction time — non-durable RabbitMQ queue drops event on broker restart — SubscriptionService re-publishes a new ChargeBillingEvent with a different getId() — billing handler derives key_B — ch_B committed

The developer builds a billing saga where the subscription lifecycle service (SubscriptionService) publishes a ChargeBillingEvent extends RemoteApplicationEvent over Spring Cloud Bus to trigger billing processing on a designated billing service instance. The billing service handles the event and derives the Stripe idempotency key from event.getId(), reasoning: “event.getId() is a unique identifier for this specific billing event message. If the same billing event is re-delivered, it carries the same id that was assigned when it was first published. I can safely use it to anchor the Stripe idempotency key.”

This reasoning is correct for redelivery of the same message: RabbitMQ delivers the same message body (including the serialized id field) to the consumer on redelivery, and the deserialized id is preserved. The failure arises for re-publication: when the original message is lost (the RabbitMQ queue is non-durable and the broker restarts) and the publisher has to construct and send a new ChargeBillingEvent object. Constructing a new ChargeBillingEvent object means calling the constructor, which calls UUID.randomUUID(), which produces a different id.

The distinction between redelivery and re-publication is critical: redelivery happens when the broker requeues an unacknowledged message and sends it again — the serialized payload (including the id) is identical. Re-publication happens when the publisher detects that its original publish was not acknowledged by a consumer (because the queue was lost) and creates a new Java object to publish — a new constructor call, a new UUID. The billing service cannot distinguish between the two from the event alone; both arrive as a ChargeBillingEvent. But their getId() values are different.

// ChargeBillingEvent.java — custom RemoteApplicationEvent for billing saga
public class ChargeBillingEvent extends RemoteApplicationEvent {

    private String subscriptionId;
    private String billingPeriod;
    private int amountCents;

    // Required no-arg constructor for Jackson deserialization.
    protected ChargeBillingEvent() {
        super();
    }

    public ChargeBillingEvent(Object source, String originService,
                               String subscriptionId, String billingPeriod,
                               int amountCents) {
        super(source, originService);
        // RemoteApplicationEvent's constructor: this.id = UUID.randomUUID().toString()
        //
        // Developer reasoning: "super() assigns a stable ID from the source
        // service and event type — this event's getId() is a fingerprint of
        // the billing request, stable across any redelivery."
        //
        // Flaw: the id is UUID.randomUUID() called at construction time.
        // Two separate `new ChargeBillingEvent(source, origin, ...)` calls
        // for the same billing request produce two different id values,
        // even if all payload fields are identical.
        this.subscriptionId = subscriptionId;
        this.billingPeriod = billingPeriod;
        this.amountCents = amountCents;
    }

    public String getSubscriptionId() { return subscriptionId; }
    public String getBillingPeriod() { return billingPeriod; }
    public int getAmountCents() { return amountCents; }
}

// SubscriptionService.java — publishes the billing event
@Service
public class SubscriptionService {

    @Autowired
    private ApplicationEventPublisher eventPublisher;

    @Value("${spring.application.name}")
    private String serviceName;

    public void triggerBilling(String subscriptionId, String billingPeriod, int amountCents) {
        // Each call constructs a new ChargeBillingEvent object.
        // Each construction generates a new UUID via RemoteApplicationEvent's constructor.
        // If this method is called twice for the same billing request (because
        // the first event was lost), two events with different getId() are published.
        ChargeBillingEvent event = new ChargeBillingEvent(
            this, serviceName, subscriptionId, billingPeriod, amountCents
        );
        eventPublisher.publishEvent(event);
        // event.getId() at this point = "uuid-A" (first call) or "uuid-B" (second call)
    }
}

// BillingEventHandler.java — derives Stripe key from event.getId()
@Component
public class BillingEventHandler {

    @Autowired
    private StripeClient stripeClient;

    @RemoteApplicationEventListener
    public void handleChargeBillingEvent(ChargeBillingEvent event) {

        // Developer reasoning: "event.getId() is a stable fingerprint of this
        // billing request — it was assigned once at publish time and will be the
        // same on any redelivery."
        //
        // Flaw: if the original event was lost and re-published, getId() is a
        // new UUID. The key changes. Stripe processes the retry as a new charge.
        String idempotencyKey = "charge:" + event.getId() + ":" + event.getBillingPeriod();

        stripeClient.createCharge(
            findCustomerId(event.getSubscriptionId()),
            event.getAmountCents(),
            event.getBillingPeriod(),
            idempotencyKey
        );
    }
}

The failure sequence:

1. At 09:15:00 UTC, SubscriptionService.triggerBilling("sub_001", "2026-10", 9900) is called. A new ChargeBillingEvent(this, "subscription-service", "sub_001", "2026-10", 9900) is constructed. RemoteApplicationEvent’s constructor assigns this.id = UUID.randomUUID() = "uuid-A". The event is published to Spring Cloud Bus.
2. Spring Cloud Bus serializes the event as JSON: {"id": "uuid-A", "type": "ChargeBillingEvent", "subscriptionId": "sub_001", "billingPeriod": "2026-10", "amountCents": 9900, ...}. The message is published to the springCloudBus RabbitMQ exchange and routed to the billing-service queue.
3. The RabbitMQ billing-service queue is non-durable (the default for Spring Cloud Bus auto-declared queues). At 09:15:02 UTC, before the billing service consumer processes the message, the RabbitMQ node undergoes a restart (operator upgrade, OOM kill, Kubernetes pod eviction). All in-memory queues are lost. The ChargeBillingEvent message with id = "uuid-A" is gone.
4. The billing service reconnects to RabbitMQ and re-declares its queue. The queue is empty. The billing handler does not receive any message and does not process the charge.
5. At 09:15:30 UTC, the subscription service detects that the billing event was not acknowledged (via a separate idempotency check, a timeout, or a polling mechanism). It calls triggerBilling("sub_001", "2026-10", 9900) again. A new ChargeBillingEvent object is constructed. RemoteApplicationEvent’s constructor assigns a new UUID: this.id = UUID.randomUUID() = "uuid-B".
6. The new event is published to Spring Cloud Bus. The billing handler receives it. event.getId() returns "uuid-B". The idempotency key is "charge:uuid-B:2026-10". The billing handler calls Stripe with this key. Since ch_A was never actually committed (the first event was lost before the billing handler processed it), Stripe processes this as the first charge and commits ch_A — so far so good.

The above sequence where the event is lost before billing handler processes it does not produce a double-charge. The double-charge scenario requires the event to be lost after the billing handler starts processing it but before RabbitMQ receives an acknowledgment. That happens when:

1. Billing handler receives ChargeBillingEvent with id = "uuid-A".
2. Billing handler calls Stripe with key "charge:uuid-A:2026-10". Stripe commits ch_A. Stripe returns 200 OK. The billing handler processes the response.
3. Before the billing handler sends basicAck to RabbitMQ (or before Spring’s SimpleMessageListenerContainer sends the ack), the RabbitMQ connection drops (broker restart, network partition, billing service pod is OOM-killed mid-ack).
4. RabbitMQ considers the message unacknowledged and requeues it if the queue still exists (redelivery — same id = "uuid-A", safe). But if the queue is non-durable and the broker restarted, the message is gone. The queue is empty. Billing handler does not receive a redelivery.
5. Subscription service’s timeout fires. It re-publishes a new ChargeBillingEvent() — id = "uuid-B". Billing handler derives "charge:uuid-B:2026-10". Stripe commits ch_B alongside ch_A.

Why RemoteApplicationEvent.getId() is not a stable fingerprint of the payload

Looking at the Spring Cloud Bus source for RemoteApplicationEvent:

// From spring-cloud-bus RemoteApplicationEvent.java (simplified)
public abstract class RemoteApplicationEvent extends ApplicationEvent {

    private final String originService;
    private final String destinationService;
    private final String id;

    protected RemoteApplicationEvent(Object source, String originService,
                                      String destinationService) {
        super(source);
        this.originService = originService;
        this.destinationService = destinationService;
        this.id = UUID.randomUUID().toString();  // assigned at construction time
    }

    public String getId() { return this.id; }
}

this.id = UUID.randomUUID().toString() runs in the constructor. There is no hashing of the source, payload, or event type. Two separate instantiations of the same subclass with the same constructor arguments produce two different id values. The id field is a construction-time random UUID, not a content-addressed fingerprint.

The fix is to include a stable, payload-derived key in the ChargeBillingEvent itself and use that field — not getId() — for Stripe idempotency. The stable key is computed from the billing request’s business inputs by the publisher before constructing the event, and it travels in the event payload. It is preserved across both redelivery and re-publication.

// Fixed ChargeBillingEvent.java — carries a stable idempotency key as payload field
public class ChargeBillingEvent extends RemoteApplicationEvent {

    private String subscriptionId;
    private String billingPeriod;
    private int amountCents;
    private String idempotencyKey;  // stable key derived from business inputs by the publisher

    protected ChargeBillingEvent() {
        super();
    }

    public ChargeBillingEvent(Object source, String originService,
                               String subscriptionId, String billingPeriod,
                               int amountCents, String idempotencyKey) {
        super(source, originService);
        this.subscriptionId = subscriptionId;
        this.billingPeriod = billingPeriod;
        this.amountCents = amountCents;
        this.idempotencyKey = idempotencyKey;
        // event.getId() is still a random UUID — but we don't use it for Stripe
    }

    public String getIdempotencyKey() { return idempotencyKey; }
    // ... other getters
}

// Fixed SubscriptionService.java — derives key before event construction
@Service
public class SubscriptionService {

    @Autowired
    private ApplicationEventPublisher eventPublisher;

    @Value("${spring.application.name}")
    private String serviceName;

    public void triggerBilling(String subscriptionId, String billingPeriod, int amountCents) {
        // Key derived before event construction, from stable business inputs.
        // Same inputs = same key regardless of how many times this method is called
        // or how many ChargeBillingEvent objects are constructed.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        ChargeBillingEvent event = new ChargeBillingEvent(
            this, serviceName, subscriptionId, billingPeriod, amountCents, idempotencyKey
        );
        eventPublisher.publishEvent(event);
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

// Fixed BillingEventHandler.java — uses event's payload-carried idempotency key
@Component
public class BillingEventHandler {

    @Autowired
    private StripeClient stripeClient;

    @RemoteApplicationEventListener
    public void handleChargeBillingEvent(ChargeBillingEvent event) {
        // Use the payload-carried key, not event.getId().
        // This key was derived from stable business inputs by the publisher.
        // It is the same value on the original event and on any re-published event
        // with the same subscriptionId, billingPeriod, and amountCents.
        String idempotencyKey = event.getIdempotencyKey();

        stripeClient.createCharge(
            findCustomerId(event.getSubscriptionId()),
            event.getAmountCents(),
            event.getBillingPeriod(),
            idempotencyKey  // same on every delivery for the same billing request
        );
    }
}

Because both the original event and any re-published event carry the same idempotencyKey in their payload (derived from the same business inputs), the billing handler uses the same Stripe key regardless of how many events are published for the same billing request. Stripe deduplicates correctly.

Durable queues reduce the risk but do not eliminate it

Using durable RabbitMQ queues (configured via spring.cloud.bus.amqp.durable=true and a durable exchange) ensures that messages survive broker restarts. This eliminates the specific scenario where a message is lost to a broker restart. However, it does not eliminate all re-publication scenarios: the subscription service may time out waiting for an acknowledgment from the billing service (due to a slow billing service pod, a network partition, or a billing service pod restart) and re-publish a new event before receiving confirmation. In that case, two events are in the queue simultaneously — both with different getId() values — and the billing handler processes both. Durable queues are a correct operational improvement but are not a substitute for stable payload-carried idempotency keys.

Test for Mode 2

@ExtendWith(MockitoExtension.class)
class BillingEventHandlerIdempotencyTest {

    @Mock
    private StripeClient stripeClient;

    @InjectMocks
    private BillingEventHandler handler;

    @Test
    void twoEventsForSameBillingRequestUseSameIdempotencyKey() {
        // Simulate the subscription service publishing two ChargeBillingEvent objects
        // for the same billing request (e.g., the first was lost and re-published).
        //
        // In the BUGGY pattern: both events have different getId() values (random UUIDs
        // assigned at construction). The Stripe keys differ → ch_B alongside ch_A.
        //
        // In the FIXED pattern: both events carry the same idempotencyKey in their
        // payload, derived from stable business inputs. The Stripe keys are equal.

        String stableKey = SubscriptionService.toStableKey("sub_001", "2026-10", 9900);

        // First event — original publication.
        ChargeBillingEvent event1 = new ChargeBillingEvent(
            this, "subscription-service", "sub_001", "2026-10", 9900, stableKey
        );

        // Second event — re-publication (same billing request, new object, new getId()).
        ChargeBillingEvent event2 = new ChargeBillingEvent(
            this, "subscription-service", "sub_001", "2026-10", 9900, stableKey
        );

        // Verify the two event objects have DIFFERENT getId() values (they always do).
        assertThat(event1.getId()).isNotEqualTo(event2.getId());

        // Process both events.
        when(stripeClient.createCharge(any(), anyInt(), any(), any()))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"))
            .thenReturn(new ChargeResponse("ch_A", "succeeded")); // Stripe deduplicates

        handler.handleChargeBillingEvent(event1);
        handler.handleChargeBillingEvent(event2);

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(stripeClient, times(2))
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        List<String> keys = keyCaptor.getAllValues();
        // Both calls must use the same Idempotency-Key.
        // Different keys = double charge from Stripe's perspective.
        assertThat(keys.get(0)).isEqualTo(keys.get(1));
        // Both keys must equal the stable key derived from business inputs.
        assertThat(keys).allMatch(k -> k.equals(stableKey));
    }
}

Mode 3: spring.application.instance-id included in idempotency key derivation — defaults to UUID.randomUUID() via IdUtils.getDefaultInstanceId() on cold JVM start in non-Cloud-Foundry environments — pod eviction and restart generates instanceId_B — key derivation produces key_B — ch_B committed alongside ch_A

The developer builds a multi-region or multi-datacenter billing architecture where each deployment region processes charges for a subset of subscriptions. To prevent key collisions between regions — the concern being that two different regions could independently process the same subscription charge and end up with conflicting billing records — the developer namespaces idempotency keys by region and by service instance. The instance identifier is taken from spring.application.instance-id, which Spring Cloud Bus reads from the environment to identify the source of bus events and to target specific service instances for bus commands.

The developer’s mental model: “spring.application.instance-id is a stable identifier for this running service instance. In Kubernetes, it is derived from the pod’s hostname or IP address, both of which are stable for a StatefulSet pod. Including it in the idempotency key ensures that charges initiated by instance A and charges initiated by instance B are tracked under separate key namespaces, preventing false deduplication.”

This model has two problems. First, the assumption that spring.application.instance-id is stable or derived from the hostname in Kubernetes is incorrect for most default configurations. Second, even if namespacing by instance were a valid architectural goal, the correct approach would be to derive the namespace from stable, pre-configured identifiers (pod name, region name, explicit configuration), not from a JVM-startup-generated random value.

// BillingKeyGenerator.java — uses spring.application.instance-id as key namespace
@Component
public class BillingKeyGenerator {

    // Spring Cloud Bus sets spring.application.instance-id at startup.
    // Developer reasoning: "This is a stable ID for this service instance —
    // derived from the hostname/IP, stable for a Kubernetes StatefulSet pod."
    //
    // Flaw: When not explicitly configured (no vcap.application.instance_id,
    // no CF_INSTANCE_INDEX), spring.application.instance-id is set by
    // IdUtils.getDefaultInstanceId() which calls UUID.randomUUID().toString()
    // on every JVM cold start.
    // A pod eviction, restart, or rolling deploy generates instanceId_B.
    // Any retry running on the restarted pod uses a different key.
    @Value("${spring.application.instance-id}")
    private String instanceId;

    public String generate(String subscriptionId, String billingPeriod, int amountCents) {
        // Key format: "charge:{instanceId}:{subscriptionId}:{billingPeriod}"
        // On the original attempt: "charge:uuid-A:sub_001:2026-10"
        // On retry after pod restart: "charge:uuid-B:sub_001:2026-10"
        // These are different strings → different Stripe keys → ch_B alongside ch_A.
        return "charge:" + instanceId + ":" + subscriptionId + ":" + billingPeriod;
    }
}

// BillingService.java — uses BillingKeyGenerator
@Service
public class BillingService {

    @Autowired
    private BillingKeyGenerator keyGenerator;

    @Autowired
    private StripeClient stripeClient;

    public String chargeSubscription(String subscriptionId,
                                     String billingPeriod,
                                     int amountCents) {

        String idempotencyKey = keyGenerator.generate(subscriptionId, billingPeriod, amountCents);

        return stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        ).getId();
    }
}

The failure sequence:

1. At 10:00:00 UTC, Billing Service Pod A cold-starts. spring.application.instance-id is not set in application.yml or as an environment variable. Spring Cloud Commons calls IdUtils.getDefaultInstanceId(environment). No vcap.application.instance_id (not Cloud Foundry). No CF_INSTANCE_INDEX (not Cloud Foundry). The method falls back to UUID.randomUUID().toString(), producing instanceId_A = "a1b2c3d4-...". Pod A’s BillingKeyGenerator has instanceId = "a1b2c3d4-...".
2. At 10:15:00 UTC, Pod A processes a billing request for subscription sub_001, billing period 2026-10. The derived key is "charge:a1b2c3d4-...:sub_001:2026-10". Stripe is called with this key. Stripe commits ch_A. The Stripe API returns a 500 Internal Server Error (or a TCP timeout) before Pod A reads the response. Pod A enqueues a retry.
3. At 10:15:05 UTC, Pod A is evicted by Kubernetes (preempted for a high-priority workload, OOM-killed after the billing charge increased memory pressure, or evicted during a rolling deploy). The retry item in Pod A’s local queue is lost along with Pod A.
4. At 10:15:30 UTC, Kubernetes schedules a new Pod (Pod A’) on the same node. The pod starts fresh. IdUtils.getDefaultInstanceId() is called again. UUID.randomUUID().toString() produces instanceId_B = "e5f6g7h8-...". Pod A’’s BillingKeyGenerator has instanceId = "e5f6g7h8-...".
5. The retry for sub_001 / 2026-10 is dequeued from a shared retry queue (or re-triggered by the subscription service’s timeout monitor). It runs on Pod A’. The derived key is "charge:e5f6g7h8-...:sub_001:2026-10". This is a different string from "charge:a1b2c3d4-...:sub_001:2026-10". Stripe has no idempotency cache entry for this key. It processes the request as a new charge and commits ch_B alongside already-committed ch_A.

Why IdUtils.getDefaultInstanceId() produces a random UUID in Kubernetes

Spring Cloud Commons’s IdUtils.getDefaultInstanceId(Environment env) checks the following sources in order:

1. vcap.application.instance_id — set only in Cloud Foundry environments.
2. CF_INSTANCE_INDEX — set only in Cloud Foundry environments.
3. If neither is present, calls UUID.randomUUID().toString() and returns it.

In a Kubernetes deployment without explicit configuration, neither Cloud Foundry variable is present. Every JVM cold start produces a new random UUID. The pod’s hostname (e.g., billing-service-7f4b8c9d-xk9pl for a Deployment, or billing-service-0 for a StatefulSet) is not used unless the developer explicitly configures spring.application.instance-id=${HOSTNAME} (or spring.application.instance-id=${POD_NAME} via a Downward API environment variable). Without that explicit configuration, the instance ID is random on every cold start.

The fix is twofold: first, never include spring.application.instance-id (or any runtime-generated JVM identity) in an idempotency key formula; second, if instance-scoping of billing operations is a genuine architectural requirement, use an explicit, stable, pre-configured identifier (e.g., a per-region slug set in application-{region}.yml) rather than an auto-generated runtime identity.

# application.yml — explicitly configure instance-id from Kubernetes Downward API
# (For billing purposes, prefer region/cluster scoping over instance-level scoping)
spring:
  application:
    instance-id: ${POD_NAME:${HOSTNAME:${random.uuid}}}
    # POD_NAME injected via Kubernetes Downward API envFrom:
    # - name: POD_NAME
    #   valueFrom:
    #     fieldRef:
    #       fieldPath: metadata.name
    # StatefulSet pod names are stable (billing-service-0, billing-service-1, etc.).
    # Deployment pod names are NOT stable across pod recreation — don't use for billing keys.

// Fixed BillingKeyGenerator.java — derives key from stable business inputs only;
// instance identity is irrelevant to Stripe idempotency
@Component
public class BillingKeyGenerator {

    // The instance-id is still available for Spring Cloud Bus routing and observability.
    // It is NOT used in idempotency key derivation.
    @Value("${spring.application.instance-id:unknown}")
    private String instanceId;  // informational only

    // Idempotency key is derived from business inputs that are:
    //   (a) known before any network call
    //   (b) stable across JVM restarts
    //   (c) independent of any Spring Cloud Bus infrastructure state
    public String generate(String subscriptionId, String billingPeriod, int amountCents) {
        return toStableKey(subscriptionId, billingPeriod, amountCents);
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

The concern about key collision between regions is addressed not by namespacing by instance but by ensuring that each billing request has a globally unique set of business inputs (the subscriptionId is globally unique, the billingPeriod is globally scoped). If two regions both attempt to charge subscription sub_001 for period 2026-10 with the same amount, they derive the same idempotency key — which is correct behavior. Stripe deduplicates them to a single charge. Two different regions charging the same customer for the same billing period for the same amount is a billing logic error, not a key-collision problem; the idempotency layer should not paper over it by generating distinct keys.

The same failure mode for random.uuid in application configuration

A related failure occurs when developers use Spring’s random.uuid property source in their billing-related configuration. For example:

# application.yml — using random.uuid in billing configuration (dangerous)
billing:
  instance-namespace: ${random.uuid}  # evaluated at startup — new UUID per JVM start
  idempotency-prefix: billing-${billing.instance-namespace}

The random.uuid property source generates a new UUID on each property access during application startup. Properties derived from it are stable within a single JVM session but change on every restart. If the billing configuration includes a random.uuid-derived prefix, the idempotency key prefix changes on every pod restart, producing the same failure mode as spring.application.instance-id without the Cloud Foundry fallback. Any configuration property used as an input to an idempotency key formula must be explicitly set to a stable, pre-configured value — not derived from random.uuid, random.value, System.currentTimeMillis(), or any other JVM-startup-time entropy source.

Test for Mode 3

@ExtendWith(MockitoExtension.class)
class BillingKeyGeneratorStabilityTest {

    @Test
    void keyIsStableAcrossInstanceIdChanges() {
        // Simulate two different pod startups by constructing two BillingKeyGenerator
        // instances with different instanceId values (as would happen on pod restart).
        BillingKeyGenerator generatorA = new BillingKeyGenerator();
        BillingKeyGenerator generatorB = new BillingKeyGenerator();
        // Inject different instanceIds (simulating two JVM cold starts).
        ReflectionTestUtils.setField(generatorA, "instanceId", "uuid-A");
        ReflectionTestUtils.setField(generatorB, "instanceId", "uuid-B");

        String keyA = generatorA.generate("sub_001", "2026-10", 9900);
        String keyB = generatorB.generate("sub_001", "2026-10", 9900);

        // Keys must be equal regardless of instanceId.
        // If keys differ, pod restart produces a new charge key → ch_B alongside ch_A.
        assertThat(keyA).isEqualTo(keyB);

        // Key must equal the direct stable-key computation.
        assertThat(keyA).isEqualTo(
            BillingKeyGenerator.toStableKey("sub_001", "2026-10", 9900)
        );
    }

    @Test
    void keyIsStableAcrossRestartWithDifferentRandomUuid() {
        // Mimic the scenario where the application is deployed twice with different
        // random.uuid values baked into the instance-id by simulating two BillingService
        // invocations with different injected instanceIds.
        BillingService service = new BillingService();
        StripeClient mockStripe = mock(StripeClient.class);
        when(mockStripe.createCharge(any(), anyInt(), any(), any()))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"));

        ReflectionTestUtils.setField(service, "keyGenerator",
            buildGeneratorWithInstanceId("uuid-A"));
        service.chargeSubscription("sub_001", "2026-10", 9900);

        // Simulate pod restart: new generator with new random instanceId.
        ReflectionTestUtils.setField(service, "keyGenerator",
            buildGeneratorWithInstanceId("uuid-B"));
        service.chargeSubscription("sub_001", "2026-10", 9900);

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(mockStripe, times(2))
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        assertThat(keyCaptor.getAllValues().get(0))
            .isEqualTo(keyCaptor.getAllValues().get(1));
    }
}

Spring Cloud Bus vs. Spring Cloud Config: which causes the key failure

The Spring Cloud Config post described @RefreshScope bean destruction triggered by ContextRefresher.refresh() called on a single service instance. This post describes the same bean destruction triggered by Spring Cloud Bus broadcasting RefreshRemoteApplicationEvent cluster-wide. The underlying mechanism — refreshScope.refreshAll() destroying all @RefreshScope beans — is identical. The difference is the scope of impact: a direct call to /actuator/refresh on one instance affects only that instance; a call to /actuator/busrefresh (or /actuator/bus-refresh in older versions) via Spring Cloud Bus affects all instances simultaneously.

For Mode 1 of this post, the causal chain is: POST /actuator/busrefresh (HTTP) → BusRefreshEndpoint publishes RefreshRemoteApplicationEvent (Spring event) → Spring Cloud Bus serializes and publishes to RabbitMQ exchange → all subscribed instances receive the message → all instances call ContextRefresher.refresh() → all instances call refreshScope.refreshAll() → all instances destroy @RefreshScope beans. Each step is a correct and expected operation. The failure arises from combining this expected cluster-wide behavior with billing service state that is stored exclusively in @RefreshScope bean instance memory.

Developers who resolve Mode 1 from the Spring Cloud Config post by moving the key derivation out of @RefreshScope bean state are also protected against the Spring Cloud Bus variant — the fix is the same. Developers who only tested Mode 1 against a local single-instance /actuator/refresh may have tested the single-instance path correctly while leaving the cluster-wide bus path unexercised. Any test of @RefreshScope billing behavior should use refreshScope.refreshAll() (equivalent to a bus broadcast reaching all instances) rather than refreshing a single bean in isolation.

Summary table

Quick diagnostic checklist for Spring Cloud Bus + Stripe deployments

1. Audit all @RefreshScope beans in billing code paths for instance-field state. Search for classes annotated with @RefreshScope that declare instance fields of type Map, Cache, ConcurrentHashMap, or any other collection holding keys, states, or transient billing context. Any state stored in these fields is lost on every ContextRefresher.refresh() call — triggered locally by /actuator/refresh or cluster-wide by Spring Cloud Bus /actuator/busrefresh. Replace with database-resident guards keyed by deterministic idempotency keys.
2. Test refreshScope.refreshAll() (not just single-bean refresh) in billing tests. The bus variant of the failure affects all beans simultaneously. A test that calls refreshScope.refresh("billingService") (single-bean refresh) does not capture the full cluster-wide blast radius. Use refreshScope.refreshAll() to simulate a bus broadcast and assert key stability across the refresh.
3. Grep billing code for event.getId() used as an idempotency key component. Any pattern of the form "charge:" + event.getId() + ":" or stripeClient.charge(..., event.getId()) is vulnerable to Mode 2. Replace with a payload-carried field set by the publisher from stable business inputs. The pattern to search: RemoteApplicationEvent subclass instances where getId() feeds into an idempotency key formula.
4. Audit custom RemoteApplicationEvent subclasses for payload-carried idempotency keys. Every billing-related RemoteApplicationEvent subclass should carry an explicit idempotencyKey field set by the publisher before event construction. The publisher should derive this field from stable business inputs. Audit existing subclasses: if the field is absent, add it; if it is present but derived from getId() or another random source, replace the derivation.
5. Check whether spring.application.instance-id is explicitly configured in Kubernetes deployments. Run kubectl get pods -o jsonpath='{.spec.containers[*].env}' or check the application ConfigMap for an explicit spring.application.instance-id value. If it is absent, IdUtils.getDefaultInstanceId() will generate a random UUID on each pod cold start. If your billing code reads spring.application.instance-id for any purpose, confirm it is not in an idempotency key formula. If it must be stable for bus routing, configure it explicitly via the Kubernetes Downward API.
6. Grep for random.uuid, random.value, System.currentTimeMillis(), and UUID.randomUUID() in billing-related configuration and key generation code. Any of these in an idempotency key derivation path produces a key that changes on JVM restart or between method calls. Replace with deterministic derivation from stable business inputs.
7. Test re-publication of RemoteApplicationEvent subclasses by constructing two event objects from the same payload and asserting key equality. The test should verify that two separate new ChargeBillingEvent(source, originService, subscriptionId, billingPeriod, amountCents) calls (simulating re-publication) result in the same idempotency key being sent to Stripe, even though their getId() values differ.
8. Use WireMock to capture Idempotency-Key headers across all retry paths, including bus-refresh-interleaved retries. An integration test that uses Testcontainers RabbitMQ, triggers a bus refresh between the original Stripe call and a retry, and asserts key header equality is the only test that exercises all three dimensions of the Mode 1 failure: message broker delivery, cluster-wide refresh, and Stripe key stability. Combine with ArgumentCaptor on the Stripe mock client to compare keys across invocations.

AckRemoteApplicationEvent and the confirmation trap

Spring Cloud Bus supports acknowledgment events: when spring.cloud.bus.ack.enabled=true (the default in some configurations), each service instance that processes a RemoteApplicationEvent publishes an AckRemoteApplicationEvent back to the bus. The publisher or an ops tool can monitor the ack stream to confirm that all target instances received and processed the event.

A developer might implement a billing saga where the subscription service waits for all billing service instances to ack the ChargeBillingEvent before considering the billing complete. The trap: receiving an AckRemoteApplicationEvent for a ChargeBillingEvent confirms that the billing handler received the event and started processing it — not that the Stripe charge was committed or that the billing was completed. The ack is published at the start of event handling, not after the Stripe response is received. A sequence where the ack is received by the subscription service but the billing handler subsequently fails to commit (or commits and fails to persist the result) appears to the subscription service as “billing confirmed” — but the billing record may be missing from the database.

This is not a Stripe idempotency failure per se but is a common source of incorrect retry triggering: if the subscription service treats ack receipt as billing confirmation and does not re-publish when the ack is absent (e.g., because the broker restarted after the ack was published but before it was consumed), re-publication occurs for a charge that may already be committed. The combination of ack-based confirmation with re-publication that generates a new RemoteApplicationEvent (and thus a new getId()) produces Mode 2.

The correct signal for “billing completed” is not the ack but a database-resident record created by the billing handler after the Stripe response is received. The subscription service should poll this record (or subscribe to a completion event carrying the stable idempotency key) rather than relying on the bus ack as a proxy for charge success.

Conclusion

Spring Cloud Bus introduces three behavioral properties that independently cause Stripe double-charges when developers couple idempotency key derivation to bus infrastructure state. A POST /actuator/busrefresh causes Spring Cloud Bus to broadcast RefreshRemoteApplicationEvent to all cluster nodes simultaneously, which calls ContextRefresher.refresh() on every node, which calls refreshScope.refreshAll(), which destroys every @RefreshScope bean regardless of whether any injected property changed — clearing any in-flight idempotency key cache stored in the destroyed bean’s instance fields across the entire cluster at once. A RemoteApplicationEvent subclass used to route billing commands over the bus carries an id field that is assigned as UUID.randomUUID() in the constructor; re-publishing the same logical billing request by constructing a new event object produces a different id, and a billing handler that derives the Stripe key from event.getId() derives a different key for the re-published event and commits ch_B. The spring.application.instance-id used by Spring Cloud Bus for instance routing defaults to a random UUID on each JVM cold start in Kubernetes (via IdUtils.getDefaultInstanceId()’s Cloud Foundry fallback), so any idempotency key formula that includes the instance ID changes on every pod eviction or restart.

The common root cause across all three modes is coupling idempotency key derivation to bus infrastructure state: to in-memory bean state that bus events destroy, to event metadata that bus infrastructure generates randomly at construction time, or to instance identity that the bus framework generates from JVM-startup entropy. The correct approach is to derive idempotency keys exclusively from application-layer business inputs — values that exist before any network call, are known to the publisher before any event is constructed, and are invariant to any infrastructure event (bean destruction, broker restart, pod eviction). SHA-256(subscriptionId + billingPeriod + amountCents) satisfies all three properties. It is computed locally, survives any number of refreshScope.refreshAll() cycles (it was never in a @RefreshScope bean to begin with), is the same on every construction of a ChargeBillingEvent for the same billing request, and does not change when the pod restarts and gets a new spring.application.instance-id.