Keybrake

Spring Integration Solace Stripe Integration: How SolaceInboundChannelAdapter AcknowledgeMode.AUTO_ACK Fires on onReceive() Return Before ExecutorChannel Dispatches to the ServiceActivator Silently Losing the BillingRecord on DB Exception Enabling ch_B on Reconciliation Republish, How SolaceAcknowledgmentCallback.acknowledge(ACCEPT) Inside @Transactional Sends the JCSMP Settlement Unaffected by Spring Rollback Permanently Consuming the Message Without a BillingRecord, and How SolaceInboundChannelAdapter.stop() During Graceful Shutdown Closes the JCSMP FlowReceiver Without Draining In-Flight ServiceActivator Processing Generating UUID_B Committing ch_B

By Keybrake —

Spring Integration’s SolaceInboundChannelAdapter introduces three acknowledgement and lifecycle behaviors that differ structurally from both the raw JCSMP FlowReceiver and the Spring Cloud Stream Solace Binder: an AcknowledgeMode.AUTO_ACK setting whose ack timing depends on the output channel type in a way that surprises developers moving from DirectChannel to ExecutorChannel, a CLIENT_ACK mode whose SolaceAcknowledgmentCallback.acknowledge() method issues an immediate JCSMP network call rather than participating in Spring’s transaction lifecycle, and a stop() shutdown path that closes the JCSMP FlowReceiver without draining in-flight ServiceActivator work — each of which produces duplicate Stripe charges when the billing handler calls UUID.randomUUID() per invocation rather than a stable content-derived key.

This post covers all three failure modes end to end: the exact SolaceInboundChannelAdapter configuration that enables each one, the step-by-step timeline showing precisely when ch_A commits and when UUID_B is generated, the specific misconception each mode exploits about how Spring Integration Solace acknowledgement and shutdown work, and the idempotency fix that makes the billing ServiceActivator safe regardless of channel type, transactional rollback, or graceful shutdown timing. Code samples target solace-spring-integration 3.x, JCSMP 10.x, Spring Integration 6.x, Spring Boot 3.x, WireMock, Testcontainers with the Solace PubSub+ Docker image, and JUnit 5.

Background: how SolaceInboundChannelAdapter differs from the raw JCSMP subscriber and the SCS Solace Binder at the acknowledgement and lifecycle layers

To understand why these three failure modes are specific to Spring Integration Solace rather than to the raw com.solacesystems.jcsmp API or the SCS Solace Binder (covered earlier in this series), you need to understand three structural differences introduced by SolaceInboundChannelAdapter.

SolaceInboundChannelAdapter’s AcknowledgeMode.AUTO_ACK fires at the point where the message is dispatched to the Spring Integration output channel, not at the point where the downstream ServiceActivator completes processing. The raw JCSMP API in AcknowledgeMode.AUTO_ACK sends the acknowledgement when the ConsumerFlowProperties listener’s onReceive() callback returns. In SolaceInboundChannelAdapter, onReceive() calls outputChannel.send(siMessage) and the adapter’s AUTO_ACK logic fires after channel.send() returns. This distinction matters because DirectChannel.send() is synchronous — it blocks until the entire downstream handler chain returns — so AUTO_ACK fires after the ServiceActivator completes, giving the appearance of correct behavior. ExecutorChannel.send(), however, returns as soon as the message task is enqueued in the executor thread pool, before any downstream handler thread picks it up. When the team switches from DirectChannel to ExecutorChannel for throughput, AUTO_ACK silently shifts from “ack after processing” to “ack after dispatch”.

The SolaceAcknowledgmentCallback obtained from a Spring Integration Message header wraps a direct JCSMP IMessage.ackMessage() network call, not a Spring TransactionSynchronization resource. The SCS Solace Binder’s transactedSession=true mode uses a JCSMP JCSMPTransactedSession whose commit() can be ordered relative to the Spring transaction lifecycle — although it is still not an XA resource and commits sequentially rather than atomically, as the SCS Solace post in this series shows. SolaceInboundChannelAdapter in CLIENT_ACK mode does not use a transacted JCSMP session. The adapter injects a SolaceAcknowledgmentCallback into the Spring Integration Message headers via the SolaceAcknowledgmentUtils helper. When application code calls acknowledge(Status.ACCEPT), the callback immediately calls IMessage.ackMessage() on the underlying JCSMP message, which sends a direct acknowledgement control message to the Solace broker over the JCSMP session socket. This call is not deferred, not transactional, and cannot be reversed by any Spring transaction operation. JmsTransactionManager works because the JMS session is a ConnectionFactory-managed resource that Spring’s transaction manager knows how to coordinate; JCSMP has no equivalent Spring transaction resource adapter in the solace-spring-integration library.

The SmartLifecycle.stop() implementation on SolaceInboundChannelAdapter closes the JCSMP FlowReceiver at the adapter layer, not at the processing layer. A JMS MessageConsumer closed via consumer.close() while an onMessage() callback is active will block the close() call until the current onMessage() invocation returns — this is mandated by the JMS 2.0 specification (Section 8.7). JCSMP has no equivalent guarantee. When FlowReceiver.close() is called, the JCSMP session returns all messages that were received by the flow but not yet acknowledged (in CLIENT_ACK mode) to the Solace broker immediately, regardless of what the application is doing with those messages. For a DirectChannel configuration where the adapter’s JCSMP callback thread is the ServiceActivator’s processing thread, stop() is typically called from a separate shutdown thread — meaning the FlowReceiver.close() races with the in-progress handleMessage() call. For a QueueChannel configuration, the adapter thread and the ServiceActivator thread are always separate, and the race is guaranteed to occur whenever shutdown happens while a message is in the QueueChannel buffer or in a PollingConsumer dispatch.

With these three structural differences understood, each failure mode follows directly from a specific configuration choice.

Failure mode 1: AcknowledgeMode.AUTO_ACK + ExecutorChannel — JCSMP ack fires on channel dispatch before ServiceActivator runs; DB exception silently loses BillingRecord; reconciliation republish → UUID_B → ch_B

The setup

A team building a billing consumer on Solace PubSub+ starts with a simple SolaceInboundChannelAdapter wired to a DirectChannel. The configuration works correctly in load testing: messages are processed one at a time on the JCSMP subscriber thread and the AUTO_ACK fires after each ServiceActivator invocation completes. Throughput is limited to a single message at a time per consumer thread, so the team switches the output channel to an ExecutorChannel with a thread pool of 10 workers to process billing events concurrently. The ack mode stays at the default AcknowledgeMode.AUTO_ACK.

The billing handler generates its Stripe idempotency key with UUID.randomUUID() at the start of each handleMessage() invocation:

@ServiceActivator(inputChannel = "billingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String idempotencyKey = UUID.randomUUID().toString(); // new key every call

    RequestOptions options = RequestOptions.builder()
        .setIdempotencyKey(idempotencyKey)
        .build();

    ChargeCreateParams params = ChargeCreateParams.builder()
        .setAmount(event.getAmountCents())
        .setCurrency("eur")
        .setCustomer(event.getCustomerId())
        .build();

    Charge charge = Charge.create(params, options);
    billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
}

The Spring Integration configuration changes the output channel from DirectChannel to ExecutorChannel:

@Bean
public SolaceInboundChannelAdapter billingAdapter(
        @Qualifier("billingChannel") MessageChannel outputChannel,
        JCSMPSession jcsmpSession,
        SolaceConsumerDestination destination) throws JCSMPException {
    SolaceInboundChannelAdapter adapter =
        new SolaceInboundChannelAdapter(jcsmpSession, destination, "billing-events-queue");
    adapter.setOutputChannel(outputChannel);
    // AcknowledgeMode not explicitly set → defaults to AUTO_ACK
    return adapter;
}

@Bean
public MessageChannel billingChannel() {
    // Changed from new DirectChannel() for throughput
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(10);
    executor.setMaxPoolSize(20);
    executor.setQueueCapacity(200);
    executor.initialize();
    return new ExecutorChannel(executor);
}

The Solace queue is configured with the default maxDeliveryCount=255 and a dead-message queue (DMQ) enabled. The reconciliation job runs every 72 hours and compares Stripe charges against BillingRecord rows, republishing any charge without a matching record.

The failure timeline

A billing event M1 arrives for customer cust_EU_042. A transient HikariCP connection pool exhaustion causes billingRecordRepo.save() to throw CannotGetJdbcConnectionException at t=520ms, after the Stripe API has already committed ch_A.

The customer is charged twice: ch_A at t=0.500s and ch_B at t=72h+0.500s. Neither charge produced an error visible to the billing consumer. The system appears to have processed M1 correctly (ack sent, no exception logged at the adapter layer) and to have recovered M2 correctly (reconciliation found the orphaned charge and republished). The double-charge is not detectable until the customer disputes it or until a separate audit compares charge totals against subscription records.

Developer misconception

“AcknowledgeMode.AUTO_ACK fires after my ServiceActivator has finished processing the message — that’s what ‘AUTO’ means. I use an ExecutorChannel for throughput, but Spring Integration must track the in-flight executor task before firing the ack. It works like JMS AUTO_ACKNOWLEDGE which fires after MessageListener.onMessage() returns, regardless of which thread runs onMessage().”

This misconception conflates when Spring Integration fires the ack (after channel.send() returns, which is after the adapter’s onReceive() hook runs) with when the downstream ServiceActivator completes (after the executor thread finishes handleMessage(), which is decoupled from the adapter thread). The JMS analogy breaks down because JMS AUTO_ACKNOWLEDGE fires after the MessageListener.onMessage() call returns — and since the JMS MessageConsumer delivers to onMessage() synchronously (on the JMS consumer thread), “after onMessage() returns” and “after processing is done” are the same. Spring Integration decouples the two: the adapter’s onReceive() calls channel.send(), and the ack fires when channel.send() returns, not when the downstream handler finishes. For DirectChannel, these happen to be the same event. For ExecutorChannel, QueueChannel, and PriorityChannel, they are not.

The fix

Two independent fixes, both required:

Fix 1: Use AcknowledgeMode.CLIENT_ACK and send the acknowledgement only after the ServiceActivator completes successfully. With ExecutorChannel, the ack must be called from the executor thread that runs the ServiceActivator, after the handler returns without exception:

@Bean
public SolaceInboundChannelAdapter billingAdapter(
        @Qualifier("billingChannel") MessageChannel outputChannel,
        JCSMPSession jcsmpSession,
        SolaceConsumerDestination destination) throws JCSMPException {
    SolaceInboundChannelAdapter adapter =
        new SolaceInboundChannelAdapter(jcsmpSession, destination, "billing-events-queue");
    adapter.setOutputChannel(outputChannel);
    adapter.setAcknowledgeMode(SolaceAcknowledgeMode.CLIENT); // explicit CLIENT_ACK
    return adapter;
}

@ServiceActivator(inputChannel = "billingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    AcknowledgmentCallback ackCallback =
        StaticMessageHeaderAccessor.getAcknowledgmentCallback(message);

    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
        return; // already processed
    }

    try {
        RequestOptions options = RequestOptions.builder()
            .setIdempotencyKey(stableKey)
            .build();
        ChargeCreateParams params = ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("eur")
            .setCustomer(event.getCustomerId())
            .build();
        Charge charge = Charge.create(params, options);
        billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
    } catch (StripeException e) {
        // Stripe failed — requeue for retry
        ackCallback.acknowledge(AcknowledgmentCallback.Status.REQUEUE);
        throw new MessageHandlingException(message, "Stripe API failure", e);
    } catch (DataAccessException e) {
        // DB failed after Stripe committed — stable key makes retry safe
        ackCallback.acknowledge(AcknowledgmentCallback.Status.REQUEUE);
        throw new MessageHandlingException(message, "DB failure after Stripe commit", e);
    }
}

Fix 2: Use a stable content-derived idempotency key. Even with correct ack placement, the reconciliation job may still republish events under rare conditions (duplicate charge in Stripe from a different failure path, replay for audit purposes). A stable key makes every republish idempotent regardless of how much time has elapsed:

private String sha256(String input) {
    try {
        MessageDigest digest = MessageDigest.getInstance("SHA-256");
        byte[] hash = digest.digest(input.getBytes(StandardCharsets.UTF_8));
        return HexFormat.of().formatHex(hash);
    } catch (NoSuchAlgorithmException e) {
        throw new IllegalStateException(e);
    }
}

With both fixes applied: CLIENT_ACK from the executor thread ensures that a DB exception after a successful Stripe charge causes REQUEUE rather than silent loss; the Solace broker redelivers M1; the stable key causes Stripe to return ch_A from its idempotency cache (within 24 hours) or causes the existsByIdempotencyKey guard to short-circuit after the BillingRecord is eventually saved. The reconciliation path also becomes safe: any republish for any reason uses the same stable key.

Failure mode 2: SolaceAcknowledgmentCallback.acknowledge(ACCEPT) inside @Transactional — JCSMP settlement fires as immediate network call; @Transactional rollback leaves no BillingRecord; message permanently consumed; reconciliation republish → UUID_B → ch_B

The setup

A senior engineer on the platform team, with deep Spring JMS experience, is tasked with migrating the billing consumer from an ActiveMQ Artemis queue to Solace PubSub+. In the ActiveMQ version, the consumer uses DefaultMessageListenerContainer with sessionTransacted=true and JmsTransactionManager. The JMS session acknowledgement is tied to the Spring @Transactional boundary: if the DB write fails and the transaction rolls back, the JMS session also rolls back, and the message is redelivered. The engineer wants to preserve this guarantee on Solace and chooses AcknowledgeMode.CLIENT_ACK so that the billing service can control when to call acknowledge(ACCEPT).

Following the JMS pattern, the engineer calls acknowledge(ACCEPT) inside the @Transactional billing service method, after the Stripe charge succeeds and before the billingRecordRepo.save() call, reasoning that if the save throws and the transaction rolls back, the Solace ack will be rolled back too:

@Bean
public SolaceInboundChannelAdapter billingAdapter(
        @Qualifier("billingChannel") MessageChannel outputChannel,
        JCSMPSession jcsmpSession,
        SolaceConsumerDestination destination) throws JCSMPException {
    SolaceInboundChannelAdapter adapter =
        new SolaceInboundChannelAdapter(jcsmpSession, destination, "billing-events-queue");
    adapter.setOutputChannel(outputChannel);
    adapter.setAcknowledgeMode(SolaceAcknowledgeMode.CLIENT);
    return adapter;
}

@ServiceActivator(inputChannel = "billingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    AcknowledgmentCallback ackCallback =
        StaticMessageHeaderAccessor.getAcknowledgmentCallback(message);

    billingService.chargeAndRecord(event, ackCallback);
}

@Service
public class BillingService {

    @Transactional
    public void chargeAndRecord(BillingEvent event, AcknowledgmentCallback ackCallback) {
        String idempotencyKey = UUID.randomUUID().toString();

        ChargeCreateParams params = ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("eur")
            .setCustomer(event.getCustomerId())
            .build();

        Charge charge = Charge.create(params,
            RequestOptions.builder().setIdempotencyKey(idempotencyKey).build());

        // Engineer's intent: "ack inside @Transactional so rollback reverses the ack"
        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT); // ← fires immediately

        // Outbox insert: throws DataIntegrityViolationException on duplicate key
        outboxRepository.insertBillingEvent(event.getId(), idempotencyKey, charge.getId());

        billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
    }
}

The failure timeline

Customer cust_EU_077 triggers a billing event. A duplicate outbox row from a prior failed attempt (stale DB state from a crash during a previous deployment) causes the insertBillingEvent call to throw DataIntegrityViolationException.

The customer’s card now has two charges: ch_A (t=0.430s, Day 1) and ch_B (t=72h+0.430s, Day 4). The billing team notices the DataIntegrityViolationException in the Day 1 logs and investigates the outbox constraint issue. They clean up the stale row and mark the incident resolved. Three days later, a customer dispute reveals ch_B. The two issues appear unrelated in the logs because they are separated by 72 hours.

Developer misconception

“Calling acknowledge(ACCEPT) inside a @Transactional method is the same pattern as JmsTransactionManager where session.commit() is deferred to the Spring transaction boundary. If the Spring transaction rolls back, session.commit() is never called, and the JMS message is not acknowledged. I assumed SolaceAcknowledgmentCallback.acknowledge() works the same way: it queues the acknowledgement to be sent when the transaction commits and reverses it when the transaction rolls back.”

JmsTransactionManager works because it registers the JMS Session as a ConnectionFactory-backed resource with Spring’s TransactionSynchronizationManager. The session is enlisted in the transaction at the start of the @Transactional method; session.commit() fires in TransactionSynchronization.afterCommit(); if the transaction rolls back, session.rollback() fires instead. SolaceAcknowledgmentCallback in solace-spring-integration 3.x does not implement or participate in TransactionSynchronizationManager. It holds a direct reference to the JCSMP IMessage object. Calling acknowledge(ACCEPT) calls IMessage.ackMessage() immediately. There is no Solace JTA XAResource in the solace-spring-integration library; the analogy to JmsTransactionManager does not hold.

The fix

Move acknowledge(ACCEPT) out of the @Transactional method body and into a TransactionSynchronization.afterCommit() hook. This ensures the Solace ack is sent only after the database transaction commits successfully. If the transaction rolls back, the ack is not sent, the message remains unacknowledged to Solace, and the broker redelivers M1 for retry:

@Service
public class BillingService {

    @Transactional
    public void chargeAndRecord(BillingEvent event, AcknowledgmentCallback ackCallback) {
        String stableKey = "billing:" + sha256(
            event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
        );

        if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
            // Already processed — ack and return
            TransactionSynchronizationManager.registerSynchronization(
                new TransactionSynchronizationAdapter() {
                    @Override public void afterCommit() {
                        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
                    }
                });
            return;
        }

        ChargeCreateParams params = ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("eur")
            .setCustomer(event.getCustomerId())
            .build();

        Charge charge = Charge.create(params,
            RequestOptions.builder().setIdempotencyKey(stableKey).build());

        outboxRepository.insertBillingEvent(event.getId(), stableKey, charge.getId());
        billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));

        // Register ack to fire only after DB transaction commits
        TransactionSynchronizationManager.registerSynchronization(
            new TransactionSynchronizationAdapter() {
                @Override
                public void afterCommit() {
                    ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
                }
                @Override
                public void afterCompletion(int status) {
                    if (status == STATUS_ROLLED_BACK) {
                        // Do not ack — Solace will redeliver M1 after ack timeout
                        // The stable key makes redelivery safe
                    }
                }
            });
    }
}

Note that in AcknowledgeMode.CLIENT_ACK, if IMessage.ackMessage() is never called (i.e., when the transaction rolls back and no ack is sent), Solace redelivers the message after the queue’s configured redeliveryDelay (default: 0ms — immediate redelivery after disconnect or session timeout). The stable key makes the redelivery path safe: the redelivered message carries the same business payload and generates the same stable key; the existsByIdempotencyKey(stableKey) guard short-circuits if the DB write eventually succeeds; the Stripe call uses the same key and the idempotency cache returns ch_A within 24 hours. Beyond 24 hours, the guard prevents the Stripe call entirely.

Failure mode 3: SolaceInboundChannelAdapter.stop() + QueueChannel + CLIENT_ACK — FlowReceiver.close() releases unacknowledged messages to Solace; second pod processes redelivered M1; UUID_B → ch_B

The setup

A team uses SolaceInboundChannelAdapter with AcknowledgeMode.CLIENT_ACK and a QueueChannel for backpressure control. The QueueChannel buffers up to 100 messages in memory; a PollingConsumer with a fixed-rate poller dispatches messages from the queue to the billing ServiceActivator. The billing handler calls acknowledge(ACCEPT) after a successful Stripe charge and DB write, and acknowledge(REQUEUE) on failure. The team is running three replicas in Kubernetes with a terminationGracePeriodSeconds=30.

@Bean
public SolaceInboundChannelAdapter billingAdapter(
        @Qualifier("billingBuffer") MessageChannel outputChannel,
        JCSMPSession jcsmpSession,
        SolaceConsumerDestination destination) throws JCSMPException {
    SolaceInboundChannelAdapter adapter =
        new SolaceInboundChannelAdapter(jcsmpSession, destination, "billing-events-queue");
    adapter.setOutputChannel(outputChannel);
    adapter.setAcknowledgeMode(SolaceAcknowledgeMode.CLIENT);
    return adapter;
}

@Bean
public MessageChannel billingBuffer() {
    return new QueueChannel(100); // in-memory buffer
}

@Bean
public IntegrationFlow billingPollingFlow(
        @Qualifier("billingBuffer") MessageChannel billingBuffer) {
    return IntegrationFlow
        .from(billingBuffer, spec -> spec.poller(Pollers.fixedRate(50, TimeUnit.MILLISECONDS)))
        .handle("billingHandler", "handleBillingEvent")
        .get();
}

// In BillingHandler:
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    AcknowledgmentCallback ackCallback =
        StaticMessageHeaderAccessor.getAcknowledgmentCallback(message);
    String idempotencyKey = UUID.randomUUID().toString(); // new key every call

    try {
        Charge charge = Charge.create(
            ChargeCreateParams.builder()
                .setAmount(event.getAmountCents())
                .setCurrency("eur")
                .setCustomer(event.getCustomerId())
                .build(),
            RequestOptions.builder().setIdempotencyKey(idempotencyKey).build()
        );
        billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
    } catch (Exception e) {
        ackCallback.acknowledge(AcknowledgmentCallback.Status.REQUEUE);
        throw new MessageHandlingException(message, "billing failed", e);
    }
}

The failure timeline

Pod-A is processing M1 for customer cust_EU_099. The Stripe charge call is in-flight when Kubernetes sends SIGTERM to Pod-A for a rolling deployment update.

The customer’s card now has two charges: ch_A (Pod-A, committed at t=1.050s) and ch_B (Pod-B, committed at t=1.200s). The incident report shows a JCSMPSessionException on Pod-A at t=1.050s, which the team attributes to the expected behavior of the closing JCSMP session and marks as non-critical. The BillingRecord for ch_A is saved on Pod-A before the JVM exits. The BillingRecord for ch_B is saved on Pod-B. No deduplication check ran because UUID_A and UUID_B are different keys.

Developer misconception

“I use CLIENT_ACK specifically so the Solace adapter knows which messages are in-flight. When stop() is called during graceful shutdown, the adapter will not release M1 back to Solace until my ServiceActivator has either acknowledged or rejected it — that’s the whole point of CLIENT_ACK. It works like JMS MessageConsumer.close(), which per the JMS spec waits for any active onMessage() invocation to complete before returning.”

The JMS specification (Section 8.7) requires that MessageConsumer.close() blocks until any concurrently executing MessageListener.onMessage() call completes. JCSMP has no equivalent contractual guarantee for FlowReceiver.close(). The JCSMP client library documentation states that messages received but not acknowledged in CLIENT_ACK mode are released to the broker when the flow closes. Additionally, in the QueueChannel configuration, the PollingConsumer’s processing thread is entirely separate from the adapter’s JCSMP subscriber thread — even a JMS-style “drain current message” behavior on the adapter would not be aware of work queued into the QueueChannel that the PollingConsumer has already dequeued and begun processing. The message left the adapter’s visibility at the moment it was placed into the QueueChannel; the adapter has no handle on what the PollingConsumer is currently executing.

The fix

Two complementary fixes:

Fix 1: Use a stable idempotency key. This is the critical fix. With a stable key, when Pod-B processes the redelivered M1 and calls Stripe with the same key, Stripe returns ch_A from its idempotency cache (within 24 hours) rather than committing ch_B. Pod-B then saves BillingRecord(stableKey, ch_A) and the customer is charged only once. With UUID.randomUUID(), each delivery generates a new key and a new Stripe charge; with a stable key, concurrent or sequential redeliveries always yield ch_A:

public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    AcknowledgmentCallback ackCallback =
        StaticMessageHeaderAccessor.getAcknowledgmentCallback(message);

    // Stable key: same value on any re-delivery of the same billing event
    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    try {
        if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
            ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
            return;
        }

        Charge charge = Charge.create(
            ChargeCreateParams.builder()
                .setAmount(event.getAmountCents())
                .setCurrency("eur")
                .setCustomer(event.getCustomerId())
                .build(),
            RequestOptions.builder().setIdempotencyKey(stableKey).build()
        );
        billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
        ackCallback.acknowledge(AcknowledgmentCallback.Status.ACCEPT);
    } catch (Exception e) {
        ackCallback.acknowledge(AcknowledgmentCallback.Status.REQUEUE);
        throw new MessageHandlingException(message, "billing failed", e);
    }
}

Fix 2: Configure Spring Boot graceful shutdown with a drain phase longer than your Stripe p99. This reduces (but does not eliminate) the window in which a SIGTERM races with an in-flight Stripe call:

spring:
  lifecycle:
    timeout-per-shutdown-phase: 90s  # >= Stripe 3DS p99 + DB write timeout

# In application.properties:
# spring.lifecycle.timeout-per-shutdown-phase=90s

Spring Boot’s embedded Tomcat / Undertow graceful shutdown drains active HTTP request-handling threads. The SolaceInboundChannelAdapter’s JCSMP flow is not directly covered by this setting, but if the PollingConsumer is wired through a Spring Integration flow that includes an ExecutorChannelInterceptor or a TaskExecutor with a custom shutdown hook, those executors can be drained before the adapter stops. The reliable fix remains the stable idempotency key: even with a 90-second drain timeout, a SIGTERM at exactly the wrong moment (after Stripe commits, before the ack is sent) can still occur. The stable key is the only mechanism that makes that race condition safe regardless of timing.

Cross-layer comparison: ack semantics and shutdown behavior across the SI and SCS Solace stack

Layer Auto-ack timing Client-ack mechanism Shutdown behavior on in-flight message Tx integration
SI Solace (SolaceInboundChannelAdapter + DirectChannel) After ServiceActivator.handleMessage() returns (synchronous) SolaceAcknowledgmentCallback.acknowledge(ACCEPT) — immediate JCSMP IMessage.ackMessage() FlowReceiver.close() — closes flow; CLIENT_ACK unacked msgs returned to Solace immediately; JMS-style drain not guaranteed No Spring TX integration; ack is unconditional network call regardless of TX state
SI Solace (SolaceInboundChannelAdapter + ExecutorChannel) After ExecutorChannel.send() returns (before executor thread processes message) Same as above, but ack called on executor thread; requires explicit CLIENT_ACK + ack in handler Same as DirectChannel; QueueChannel items already dequeued to executor are orphaned No Spring TX integration
SCS Solace Binder (requeueRejected=false) After all SCS max-attempts retries are exhausted; routes to DMQ Not exposed; binder controls ack internally Binder-managed; messages buffered by SCS binder infrastructure transactedSession=true uses JCSMP transacted session but is NOT XA; commits sequentially after Spring TX
Raw JCSMP (FlowReceiver) N/A; developer calls IMessage.ackMessage() manually Direct IMessage.ackMessage() — immediate JCSMP network call FlowReceiver.close() — same as SI; unacked CLIENT_ACK messages returned to broker No Spring TX integration; JCSMPTransactedSession available but requires XA to coordinate with DB TX
SI GCP PubSub (PubSubInboundChannelAdapter) After handler returns with ackMode=AUTO; ack deadline timer separate issue BasicAcknowledgeablePubsubMessage.ack() — immediate HTTP POST to GCP API Subscriber client stops; in-flight ack deadline may expire; GCP redelivers on deadline expiry No Spring TX integration
JMS (DefaultMessageListenerContainer + JmsTransactionManager) After session.commit() fires in TransactionSynchronization.afterCommit() session.commit() deferred by TX manager MessageConsumer.close() — JMS 2.0 spec §8.7 mandates drain of active onMessage() before return Full Spring TX integration via JmsTransactionManager

Comparison: SCS Solace Binder failure modes vs Spring Integration Solace failure modes

The SCS Solace post in this series covers three binder-level failure modes. Each differs structurally from the three SI Solace modes above:

Mode SCS Solace Binder Spring Integration Solace (this post)
Ack on successful processing path Binder acks after all max-attempts retries; requeueRejected=false routes to DMQ; on-call re-enqueue → UUID_B → ch_B AUTO_ACK fires on channel.send() return; with ExecutorChannel this is before processing starts; DB exception → silent loss → reconciliation → UUID_B → ch_B
Transaction boundary transactedSession=true: JCSMP JCSMPTransactedSession.commit() called sequentially AFTER Spring @Transactional commits; TCP RST drops COMMIT frame; JCSMP ClosedFacilityException; Solace requeues; redelivery → UUID_B → ch_B SolaceAcknowledgmentCallback.acknowledge(ACCEPT) inside @Transactional: immediate JCSMP call; rollback doesn't reverse ack; no BillingRecord; reconciliation → UUID_B → ch_B
Redelivery / replay Solace Message Replay: 1,247 pre-outage already-charged events replayed with solace_redelivered=false (indistinguishable from first delivery); UUID.randomUUID() × 1,247 → ch_B × 1,247 SolaceInboundChannelAdapter.stop() + QueueChannel + CLIENT_ACK: FlowReceiver.close() releases unacked M1 immediately; Pod-B processes redelivered M1; UUID_B → ch_B (single immediate double-charge, not mass replay)

Diagnostic checklist: Spring Integration Solace + Stripe idempotency

  1. What is the output channel type of your SolaceInboundChannelAdapter? If ExecutorChannel, QueueChannel, or PriorityChannel, AcknowledgeMode.AUTO_ACK fires before the ServiceActivator processes the message. Switch to AcknowledgeMode.CLIENT_ACK and call acknowledge(ACCEPT) from within the handler, or use DirectChannel (synchronous, single-threaded) if throughput constraints allow.
  2. Does your billing ServiceActivator call UUID.randomUUID() at or near the Stripe API call? If yes, every re-delivery, retry, or reconciliation republish generates a new key and a new Stripe charge. Replace with a stable SHA-256(accountId + billingPeriod + amountCents) key derived from the message payload. The same value must be produced regardless of channel type, pod identity, or call count.
  3. Is SolaceAcknowledgmentCallback.acknowledge(ACCEPT) called inside a @Transactional method? If yes, the ack fires as an immediate JCSMP network call regardless of whether the Spring transaction later rolls back. Move acknowledge(ACCEPT) to TransactionSynchronizationManager.registerSynchronization().afterCommit(). Leave the REQUEUE path in the transaction’s catch block or in the afterCompletion(STATUS_ROLLED_BACK) handler.
  4. Does your application use Kubernetes rolling deployments, preStop hooks, or horizontal pod autoscaler scale-in? All three send SIGTERM. If any billing message may be mid-processing at SIGTERM time, a stable idempotency key is required because the JCSMP FlowReceiver close will redeliver the message to another pod. Configure spring.lifecycle.timeout-per-shutdown-phase ≥ Stripe p99 as a secondary defense.
  5. Is a BillingRecord row keyed on the stable business key checked before the Stripe API call? A stable Stripe idempotency key alone handles re-deliveries within Stripe’s 24-hour window. Beyond 24 hours (reconciliation republish, Solace replay, DMQ re-ingest), only the application-layer existsByIdempotencyKey() guard prevents a new charge. Both checks are required.
  6. Does an errorChannel handler republish failed messages to the same or a retry Solace queue? If yes and the billing handler uses UUID.randomUUID(), each republished message generates a new key. If the original Stripe charge succeeded and only the DB write failed, the republish will commit a second charge. The stable key prevents this; the existsByIdempotencyKey() guard handles the case where the DB write eventually succeeds on a retry.
  7. Is the Solace queue configured as exclusive or non-exclusive? Non-exclusive queues allow multiple consumer instances to receive different messages. Under a FlowReceiver close (Mode 3), any non-exclusive queue consumer can immediately pick up the released message. For exclusive queues, redelivery only occurs after the exclusive consumer reconnects or another consumer is promoted. Exclusive queue configuration reduces the Mode 3 window but does not eliminate it if multiple adapter instances compete for the exclusive flow.
  8. What is the Solace queue’s maxDeliveryCount and DMQ policy? With a stable key and REQUEUE on transient failures, redelivery count grows. If maxDeliveryCount is reached before the transient failure clears, the message routes to the DMQ. The DMQ re-ingestion path is safe with a stable key (Stripe returns ch_A from cache, or existsByIdempotencyKey() guards). Without a stable key, DMQ re-ingestion generates UUID_B and commits ch_B. Update runbook procedures to always check for an existing Stripe charge with the stable key before re-ingesting DMQ messages.

Test suite: JUnit 5 + WireMock + Testcontainers (Solace PubSub+ Docker)

Each failure mode can be verified in isolation with a test that controls: (a) the channel type and ack mode, (b) the Stripe response via WireMock, and (c) the database behavior via Testcontainers PostgreSQL or mocking.

@SpringBootTest
@Testcontainers
class SpringIntegrationSolaceStripeIdempotencyTest {

    @Container
    static final SolaceContainer solace =
        new SolaceContainer("solace/solace-pubsub-standard:latest")
            .withExposedPorts(55555, 8080);

    @RegisterExtension
    static WireMockExtension wireMock = WireMockExtension.newInstance()
        .options(wireMockConfig().dynamicPort())
        .build();

    @Autowired JCSMPSession jcsmpSession;
    @Autowired BillingRecordRepository billingRecordRepo;
    @Autowired SolaceInboundChannelAdapter billingAdapter;

    static final String BILLING_QUEUE = "billing-events-queue";

    // ── Mode 1: AUTO_ACK + ExecutorChannel → ack fires before processing ─────────

    @Test
    @DisplayName("Mode 1: AUTO_ACK + ExecutorChannel + DB exception → no BillingRecord, no redelivery (without stable key)")
    void autoAck_executorChannel_dbException_noBillingRecordAndNoRedelivery()
            throws Exception {
        // Arrange: Stripe succeeds, DB throws
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(aResponse().withStatus(200).withBody(chargeJson("ch_A"))));
        when(billingRecordRepo.save(any()))
            .thenThrow(new DataIntegrityViolationException("constraint"));

        // Publish M1 to Solace queue
        publishToBillingQueue(new BillingEvent("acct_001", "2026-10", 4900L));

        // Wait for Stripe call to complete (adapter acks immediately, executor processes)
        await().atMost(Duration.ofSeconds(5))
               .until(() -> wireMock.findAll(
                   postRequestedFor(urlEqualTo("/v1/charges"))).size() == 1);

        // BillingRecord not saved — DB threw
        assertThat(billingRecordRepo.count()).isZero();

        // No redelivery: message was acked by AUTO_ACK on channel dispatch
        // Verify no second Stripe call within 10s (which would indicate redelivery)
        Thread.sleep(3_000);
        assertThat(wireMock.findAll(postRequestedFor(urlEqualTo("/v1/charges")))).hasSize(1);
    }

    @Test
    @DisplayName("Mode 1: CLIENT_ACK + ExecutorChannel + stable key → DB exception triggers REQUEUE → Stripe returns ch_A on retry (with fix)")
    void clientAck_executorChannel_dbException_requeueAndStableKeyDeduplicates()
            throws Exception {
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(aResponse().withStatus(200).withBody(chargeJson("ch_A"))));

        AtomicInteger saveAttempt = new AtomicInteger(0);
        when(billingRecordRepo.save(any())).thenAnswer(inv -> {
            if (saveAttempt.incrementAndGet() == 1) {
                throw new DataIntegrityViolationException("constraint");
            }
            return inv.getArgument(0);
        });

        publishToBillingQueue(new BillingEvent("acct_002", "2026-10", 4900L));

        // Wait for successful second delivery (REQUEUE → Solace redelivers)
        await().atMost(Duration.ofSeconds(20))
               .until(() -> billingRecordRepo.count() == 1);

        // All Stripe calls used the same stable key
        List<LoggedRequest> requests = wireMock.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        Set<String> keys = requests.stream()
            .map(r -> r.getHeader("Idempotency-Key"))
            .collect(Collectors.toSet());
        assertThat(keys).hasSize(1); // stable key → ch_A on all calls
    }

    // ── Mode 2: acknowledge(ACCEPT) inside @Transactional → rollback doesn't un-ack ──

    @Test
    @DisplayName("Mode 2: acknowledge(ACCEPT) inside @Transactional → rollback leaves no BillingRecord (without fix)")
    void ackInsideTransactional_rollbackLeavesNoRecord() throws Exception {
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(aResponse().withStatus(200).withBody(chargeJson("ch_A"))));

        when(outboxRepository.insertBillingEvent(any(), any(), any()))
            .thenThrow(new DataIntegrityViolationException("unique constraint"));

        publishToBillingQueue(new BillingEvent("acct_003", "2026-10", 4900L));

        await().atMost(Duration.ofSeconds(5))
               .until(() -> wireMock.findAll(
                   postRequestedFor(urlEqualTo("/v1/charges"))).size() == 1);

        assertThat(billingRecordRepo.count()).isZero();
        // Message acked despite transaction rollback → no redelivery
        Thread.sleep(3_000);
        assertThat(wireMock.findAll(postRequestedFor(urlEqualTo("/v1/charges")))).hasSize(1);
    }

    @Test
    @DisplayName("Mode 2: acknowledge(ACCEPT) in afterCommit() + stable key → rollback prevents ack → Solace redelivers → one charge (with fix)")
    void ackInAfterCommit_rollbackPreventsAck_redeliveryIsIdempotent() throws Exception {
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(aResponse().withStatus(200).withBody(chargeJson("ch_A"))));

        AtomicInteger outboxAttempt = new AtomicInteger(0);
        when(outboxRepository.insertBillingEvent(any(), any(), any())).thenAnswer(inv -> {
            if (outboxAttempt.incrementAndGet() == 1) {
                throw new DataIntegrityViolationException("constraint");
            }
            return null;
        });

        publishToBillingQueue(new BillingEvent("acct_004", "2026-10", 4900L));

        await().atMost(Duration.ofSeconds(20))
               .until(() -> billingRecordRepo.count() == 1);

        List<LoggedRequest> requests = wireMock.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        Set<String> keys = requests.stream()
            .map(r -> r.getHeader("Idempotency-Key"))
            .collect(Collectors.toSet());
        assertThat(keys).hasSize(1); // stable key — ch_A on all attempts
    }

    // ── Mode 3: stop() + QueueChannel + CLIENT_ACK → FlowReceiver close redelivers ──

    @Test
    @DisplayName("Mode 3: FlowReceiver.close() with unacked QueueChannel message → Solace redelivers → UUID_B → ch_B (without stable key)")
    void flowReceiverClose_unackedQueueChannelMessage_causesDuplicateCharge()
            throws Exception {
        // First call: Stripe commits ch_A; second call: Stripe commits ch_B
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("shutdown")
            .whenScenarioStateIs(STARTED)
            .willReturn(aResponse()
                .withFixedDelay(1_000) // simulate slow Stripe call
                .withStatus(200)
                .withBody(chargeJson("ch_A")))
            .willSetStateTo("second"));
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .inScenario("shutdown")
            .whenScenarioStateIs("second")
            .willReturn(aResponse().withStatus(200).withBody(chargeJson("ch_B"))));

        publishToBillingQueue(new BillingEvent("acct_005", "2026-10", 4900L));

        // Wait for first Stripe call to start, then simulate shutdown
        await().atMost(Duration.ofSeconds(3))
               .until(() -> wireMock.findAll(
                   postRequestedFor(urlEqualTo("/v1/charges"))).size() >= 1);
        // Simulate FlowReceiver.close() — releases unacked M1 to Solace
        billingAdapter.stop();

        // Wait for second delivery to complete
        await().atMost(Duration.ofSeconds(10))
               .until(() -> wireMock.findAll(
                   postRequestedFor(urlEqualTo("/v1/charges"))).size() >= 2);

        // Two distinct keys → two charges (without stable key)
        List<LoggedRequest> requests = wireMock.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        Set<String> keys = requests.stream()
            .map(r -> r.getHeader("Idempotency-Key"))
            .collect(Collectors.toSet());
        assertThat(keys).hasSize(2); // UUID_A and UUID_B
    }

    @Test
    @DisplayName("Mode 3: stable key → FlowReceiver.close() redelivery is safe → Stripe returns ch_A on both calls (with fix)")
    void stableKey_flowReceiverCloseRedelivery_onlyOneCharge() throws Exception {
        wireMock.stubFor(post(urlEqualTo("/v1/charges"))
            .willReturn(aResponse()
                .withFixedDelay(1_000)
                .withStatus(200)
                .withBody(chargeJson("ch_A"))));

        publishToBillingQueue(new BillingEvent("acct_006", "2026-10", 4900L));

        await().atMost(Duration.ofSeconds(3))
               .until(() -> wireMock.findAll(
                   postRequestedFor(urlEqualTo("/v1/charges"))).size() >= 1);
        billingAdapter.stop();

        await().atMost(Duration.ofSeconds(10))
               .until(() -> billingRecordRepo.count() == 1);

        List<LoggedRequest> requests = wireMock.findAll(
            postRequestedFor(urlEqualTo("/v1/charges")));
        Set<String> keys = requests.stream()
            .map(r -> r.getHeader("Idempotency-Key"))
            .collect(Collectors.toSet());
        assertThat(keys).hasSize(1); // one stable key → ch_A on all redeliveries
    }

    private void publishToBillingQueue(BillingEvent event) throws JCSMPException {
        TextMessage msg = JCSMPFactory.onlyInstance().createMessage(TextMessage.class);
        msg.setText(objectMapper.writeValueAsString(event));
        Queue queue = JCSMPFactory.onlyInstance().createQueue(BILLING_QUEUE);
        Producer producer = jcsmpSession.createProducer(null);
        producer.send(msg, queue);
        producer.close();
    }

    private String chargeJson(String chargeId) {
        return """
            {
              "id": "%s",
              "object": "charge",
              "amount": 4900,
              "currency": "eur",
              "status": "succeeded"
            }
            """.formatted(chargeId);
    }
}

Run with SolaceContainer from the testcontainers-solace module or the community solace/solace-pubsub-standard Docker image. The Solace PubSub+ Standard Edition image provides full JCSMP support for queue, ack, and guaranteed message delivery behavior in an isolated container. The Mode 3 test simulates FlowReceiver.close() by calling billingAdapter.stop() directly; in production, this is triggered by Spring’s graceful shutdown on SIGTERM.

How this relates to Keybrake

The three failure modes in this post share the same root shape as every other post in this series: the billing consumer reaches Stripe and commits a charge, then an infrastructure or application event causes a second Stripe call with a different idempotency key. The modes are different each time — ack timing on channel dispatch, transactional rollback after JCSMP ack, FlowReceiver close on shutdown — but the double-charge is the same outcome.

Keybrake adds a proxy-layer audit trail at the point where the charge is committed. When your SolaceInboundChannelAdapter’s billing handler routes through proxy.keybrake.com/stripe/v1/charges, every charge call is logged with its idempotency key, the Stripe charge ID, the timestamp, and the customer ID. If you see two audit entries with two different idempotency keys for the same customer and billing period — the fingerprint of all three failure modes in this post — you have an actionable alert before the customer files a dispute. Keybrake’s per-customer daily spend cap also provides a circuit-breaker for the worst-case scenario (an auto-requeue loop on a transient DB failure with UUID.randomUUID()): once the customer hits the daily cap, all subsequent charge attempts are rejected at the proxy layer, capping the blast radius at a configurable dollar amount rather than at whatever the DB failure window turns out to be.

Catch duplicate Stripe charges before your customers do

Keybrake proxies your agent’s Stripe calls with per-customer spend caps, idempotency-key audit log, and one-click revoke. One config line — no Stripe account changes needed.

Summary

Spring Integration’s SolaceInboundChannelAdapter introduces three failure modes that produce duplicate Stripe charges, each arising from a behavior that differs structurally from the raw JCSMP FlowReceiver, the SCS Solace Binder, and from JMS-based messaging systems that developers commonly migrate from:

The stable idempotency key is the fix for all three modes: a deterministic SHA-256(accountId + billingPeriod + amountCents) of the billing event’s business fields, producing the same string on every re-delivery, reconciliation republish, and concurrent duplicate delivery. Combined with a BillingRecord guard in the application layer, it makes the billing ServiceActivator safe under any re-invocation scenario that Spring Integration, Solace JCSMP, and Stripe’s 24-hour idempotency cache can produce.

The next post in this series covers Spring Integration IBM MQ + Stripe idempotency failure modes — how IBM MQ’s JMS provider, XA transaction manager integration, and message backout queue behavior interact with UUID.randomUUID() to produce duplicate charges in ways specific to the MQ JMS layer.