Spring Integration IBM MQ Stripe Integration: How JmsInboundChannelAdapter SESSION_TRANSACTED Commits the IBM MQ Transaction on JmsTemplate.receive() Return Before ExecutorChannel Dispatches to the ServiceActivator Silently Losing the BillingRecord on DB Exception, How ChainedTransactionManager Commits the JmsTransactionManager Phase Before DataSourceTransactionManager Permanently Consuming M1 Without a BillingRecord on DB Commit Failure, and How IBM MQ BACKOUT_THRESHOLD Moves M1 to BACKOUT_QUEUE_NAME Where a Separate Consumer Generates UUID_B Committing ch_B Alongside ch_A from the Retry Path

Spring Integration’s IBM MQ integration introduces three transaction and message-lifecycle behaviors that differ structurally from both Spring Integration’s push-mode DefaultMessageListenerContainer adapter and the ActiveMQ, RabbitMQ, and Solace adapters covered earlier in this series: a polling JmsInboundChannelAdapter whose JmsTemplate-based session management commits the IBM MQ local transaction on channel.send() return rather than on handler completion, a ChainedTransactionManager composition that commits IBM MQ’s JMS local transaction before the database transaction in a way that cannot be rolled back on DB commit failure, and IBM MQ’s broker-side BACKOUT_THRESHOLD mechanism that moves messages to a BACKOUT_QUEUE independently of what the application’s retry logic believes — each producing duplicate Stripe charges when the billing handler uses UUID.randomUUID() per invocation.

This post covers all three failure modes end to end: the exact Spring Integration + IBM MQ configuration that enables each one, the step-by-step timeline showing precisely when ch_A commits and when UUID_B is generated, the specific misconception each mode exploits, and the idempotency fix that makes the billing ServiceActivator safe regardless of channel type, transaction manager composition, or IBM MQ broker-side redelivery behavior. Code samples target ibm-mq-spring-boot-starter 3.x, com.ibm.mq.jakarta.client 9.x, Spring Integration 6.x, Spring Boot 3.x, WireMock, Testcontainers with the IBM MQ Docker image, and JUnit 5.

Background: three IBM MQ-specific behaviors that differ from RabbitMQ, ActiveMQ, and Solace at the transaction and message-lifecycle layers

To understand why these three failure modes are specific to Spring Integration IBM MQ rather than to other Spring Integration JMS adapters, you need to understand three structural differences introduced by the combination of IBM MQ’s JMS client, Spring Integration’s polling adapter, and IBM MQ’s broker-side backout mechanism.

JmsInboundChannelAdapter in polling mode uses JmsTemplate internally, and JmsTemplate manages the JMS session transaction as a single unit scoped to the execute(SessionCallback) call. The JmsInboundChannelAdapter operates in two modes: push mode (backed by DefaultMessageListenerContainer) and polling mode (backed by JmsTemplate). In polling mode, the adapter calls JmsTemplate.receiveAndConvert() or JmsTemplate.execute(new ReceiveCallback()) on each polling interval. JmsTemplate.execute(SessionCallback) opens a JMS session, passes it to the callback, and — if sessionTransacted=true — commits the session when the callback returns. The dispatch to the output channel happens inside the callback, before the commit. For DirectChannel, channel.send() is synchronous and blocks until the entire downstream handler chain returns, meaning the ServiceActivator completes before JmsTemplate’s session commits. For ExecutorChannel, channel.send() enqueues the task and returns immediately, so JmsTemplate commits the IBM MQ transaction before any ServiceActivator thread has started. This is a fundamentally different commit scope from DefaultMessageListenerContainer in push mode, where the session commit is tied to the onMessage() callback completion regardless of channel type.

ChainedTransactionManager commits its constituent transaction managers in registration order, not in a 2PC manner. ChainedTransactionManager (from spring-data-commons) is a common pattern for coordinating JMS and database transactions without a full JTA XA implementation. It creates a composite transaction that, on commit, calls each constituent manager’s commit() in the order they were registered, and on rollback calls each manager’s rollback() in reverse order. This means that if JmsTransactionManager is registered first (which is the common pattern to ensure JMS acks are rolled back on DB failure), JMS commits happen before DB commits. In the vast majority of cases — normal execution, runtime exceptions — this is fine: if the ServiceActivator throws, the exception triggers a rollback before any commit occurs, and both managers roll back in reverse order. The failure case is a commit-phase failure: the ServiceActivator completes without exception, JmsTransactionManager.commit() executes successfully (IBM MQ message permanently consumed), and then DataSourceTransactionManager.commit() fails (DB connection reset, timeout, or optimistic lock at commit time). At that point, the IBM MQ message is gone and cannot be recovered; the BillingRecord was never persisted. This is a well-known limitation of ChainedTransactionManager — its Javadoc explicitly warns that “if the second resource fails to commit, the first will not be rolled back” — but teams migrating from single-resource transactions often miss the implication for IBM MQ billing consumers.

IBM MQ’s BACKOUT_THRESHOLD is a broker-side property that moves messages to a designated BACKOUT_QUEUE based on MQMD.BackoutCount, independently of the application’s retry logic or Spring Integration’s error handling. When a message is rolled back (via JMS session.rollback() or DefaultMessageListenerContainer exception handling with sessionTransacted=true), IBM MQ increments the MQMD.BackoutCount field in the message descriptor. If BackoutCount >= BOTHRESH (BackoutThreshold, configured as a queue attribute), IBM MQ will not redeliver the message on the next session.receive() call; instead, at the point of the next attempted delivery, it moves the message to the queue named in BOQNAME (BackoutQueueName). Critically, this BOTHRESH check and the move to BOQNAME happen at the IBM MQ broker layer, not at the Spring Integration layer. If a team has set up a separate Spring Integration pipeline consuming from the BOQNAME queue as a “recovery handler,” that handler may run even when the main-queue consumer ultimately succeeded on an earlier attempt but the message was redelivered multiple times before success. The BOQNAME handler, if it calls Stripe with UUID.randomUUID(), generates a second Stripe charge alongside the one already committed by the main-queue pipeline.

With these three structural differences understood, each failure mode follows directly from a specific configuration choice.

Failure mode 1: JmsInboundChannelAdapter polling + SESSION_TRANSACTED + ExecutorChannel — JmsTemplate commits IBM MQ transaction on channel.send() return before ServiceActivator runs; DB exception silently drops BillingRecord; reconciliation republish → UUID_B → ch_B

The setup

A team building a billing consumer on IBM MQ chooses JmsInboundChannelAdapter in polling mode because their IBM MQ deployment uses a specific TLS mutual-authentication configuration that works reliably with JmsTemplate’s connection management but has historically caused reconnection issues with DefaultMessageListenerContainer’s background listener threads. The adapter is initially wired to a DirectChannel, which processes billing events one at a time on the polling thread. To improve throughput, the team switches the output channel to an ExecutorChannel backed by a thread pool.

The billing handler generates its Stripe idempotency key with UUID.randomUUID() at the start of each invocation:

@ServiceActivator(inputChannel = "billingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String idempotencyKey = UUID.randomUUID().toString(); // new key on every call

    ChargeCreateParams params = ChargeCreateParams.builder()
        .setAmount(event.getAmountCents())
        .setCurrency("gbp")
        .setCustomer(event.getCustomerId())
        .build();

    Charge charge = Charge.create(params,
        RequestOptions.builder().setIdempotencyKey(idempotencyKey).build());

    billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
}

The Spring Integration configuration switches from DirectChannel to ExecutorChannel and uses a transacted JmsInboundChannelAdapter:

@Bean
public JmsInboundChannelAdapter billingAdapter(
        @Qualifier("billingChannel") MessageChannel outputChannel,
        ConnectionFactory connectionFactory) {
    JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
    jmsTemplate.setSessionTransacted(true); // IBM MQ local JMS transaction
    jmsTemplate.setDefaultDestinationName("BILLING.EVENTS.QUEUE");

    JmsInboundChannelAdapter adapter =
        new JmsInboundChannelAdapter(jmsTemplate);
    adapter.setOutputChannel(outputChannel);
    adapter.setAutoStartup(true);
    return adapter;
}

@Bean
public MessageChannel billingChannel() {
    // Switched from DirectChannel for throughput
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(8);
    executor.setMaxPoolSize(16);
    executor.setQueueCapacity(100);
    executor.initialize();
    return new ExecutorChannel(executor);
}

The IBM MQ queue BILLING.EVENTS.QUEUE is configured with BOTHRESH(5) and BOQNAME(BILLING.EVENTS.BACKOUT). The reconciliation job runs every 72 hours and compares Stripe charges against BillingRecord rows, republishing any orphaned charge.

The failure timeline

A billing event M1 arrives for customer cust_GB_019. A transient HikariCP pool exhaustion during a load spike causes billingRecordRepo.save() to throw CannotGetJdbcConnectionException at t=520ms, after the Stripe API has already committed ch_A.

The customer is charged twice: ch_A at t=0.500s and ch_B at t=72h+0.500s. No error is visible at the IBM MQ layer — the message was consumed cleanly. The IBM MQ BOTHRESH mechanism is never triggered, so the operations team has no dead-letter evidence that anything went wrong. The double charge appears as a reconciliation “successful recovery” in the billing logs.

Developer misconception

“SESSION_TRANSACTED=true on the JmsTemplate means the IBM MQ transaction does not commit until my ServiceActivator has finished processing the message. JmsTemplate must hold the session open across the channel.send() dispatch so that any exception in the ServiceActivator can trigger a session.rollback(). I’ve used DefaultMessageListenerContainer with sessionTransacted=true for years and it always rolls back on exception — I assumed the polling adapter worked the same way.”

This misconception conflates two different architectures. DefaultMessageListenerContainer in push mode calls onMessage() synchronously inside the JMS session scope — the session does not commit until onMessage() returns (or the DMLC exception handler decides to rollback). The JMS session is kept open for the entire duration of message processing. JmsInboundChannelAdapter in polling mode uses JmsTemplate.execute(SessionCallback), which scopes the session to a single execute() call. The adapter calls channel.send() inside that session scope, but JmsTemplate has no knowledge of what the output channel does downstream — it only knows when channel.send() returns. For DirectChannel, those happen to be the same point in time. For ExecutorChannel, channel.send() returns as soon as the task is queued, decoupling the session scope from handler execution. JmsTemplate’s design was never intended to hold sessions open across asynchronous downstream work; the adaptation is the application developer’s responsibility.

The fix

Two independent fixes, both required:

Fix 1: Switch to push mode (DefaultMessageListenerContainer backed adapter) when using ExecutorChannel. DMLC’s AbstractMessageListenerContainer processes messages inside the JMS session callback, so the session commit is tied to the listener completion. With sessionTransacted=true and a concurrent DMLC listener, each listener thread has its own session, and the session commits only after onMessage() returns on that thread — which, for a DirectChannel wired to a concurrent-consumer DMLC, means after the ServiceActivator completes:

@Bean
public JmsMessageDrivenEndpoint billingEndpoint(
        @Qualifier("billingChannel") MessageChannel outputChannel,
        ConnectionFactory connectionFactory) {
    DefaultMessageListenerContainer container =
        new DefaultMessageListenerContainer();
    container.setConnectionFactory(connectionFactory);
    container.setDestinationName("BILLING.EVENTS.QUEUE");
    container.setSessionTransacted(true); // IBM MQ local JMS transaction
    container.setConcurrentConsumers(4);
    container.setMaxConcurrentConsumers(16);

    ChannelPublishingJmsMessageListener listener =
        new ChannelPublishingJmsMessageListener();
    listener.setRequestChannel(outputChannel);

    return new JmsMessageDrivenEndpoint(container, listener);
}

With DMLC, each concurrent consumer thread holds its own Session across the full onMessage() execution. If handleBillingEvent() throws, DMLC catches the exception and calls session.rollback() — IBM MQ increments BackoutCount for M1 and redelivers it.

Fix 2: Use a stable content-derived idempotency key. DMLC + sessionTransacted=true handles the transactional case, but the reconciliation job and any BACKOUT_QUEUE handler also need to be safe. A stable key derived from the business payload ensures every redelivery — from DMLC rollback, from reconciliation republish, or from the BACKOUT_QUEUE consumer — produces the same Stripe idempotency key:

@ServiceActivator(inputChannel = "billingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
        return; // Stripe already charged; BillingRecord already saved
    }

    ChargeCreateParams params = ChargeCreateParams.builder()
        .setAmount(event.getAmountCents())
        .setCurrency("gbp")
        .setCustomer(event.getCustomerId())
        .build();

    Charge charge = Charge.create(params,
        RequestOptions.builder().setIdempotencyKey(stableKey).build());

    billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
}

private String sha256(String input) {
    try {
        MessageDigest digest = MessageDigest.getInstance("SHA-256");
        byte[] hash = digest.digest(input.getBytes(StandardCharsets.UTF_8));
        return HexFormat.of().formatHex(hash);
    } catch (NoSuchAlgorithmException e) {
        throw new IllegalStateException(e);
    }
}

With both fixes applied: DMLC ensures that a DB exception rolls back the JMS session and IBM MQ redelivers M1 (incrementing BackoutCount); the stable key ensures that every redelivery — including the reconciliation republish M2 — uses the same Stripe idempotency key; Stripe returns ch_A from its cache within 24 hours, and the existsByIdempotencyKey guard short-circuits after the BillingRecord is saved.

Failure mode 2: ChainedTransactionManager + JmsTransactionManager-first ordering — IBM MQ message committed before DB transaction commits; DB commit-phase failure leaves no BillingRecord; message permanently consumed; reconciliation republish → UUID_B → ch_B

The setup

A platform team is asked to add transactional guarantees to a billing consumer that uses DefaultMessageListenerContainer in push mode. They want to ensure that a DB exception during billingRecordRepo.save() triggers both a database rollback and an IBM MQ JMS session rollback (so M1 is redelivered). Rather than using a full JTA XA implementation (which requires IBM MQ’s XA client and a JTA provider), they use ChainedTransactionManager from spring-data-commons — a well-documented pattern for “best-effort 1.5 phase commit”.

The team registers JmsTransactionManager first (on the reasoning that JMS transactions should be the “outer” resource, so DB failures cause JMS rollback) and DataSourceTransactionManager second:

@Bean
public PlatformTransactionManager transactionManager(
        ConnectionFactory connectionFactory,
        DataSource dataSource) {
    JmsTransactionManager jmsTm = new JmsTransactionManager(connectionFactory);
    DataSourceTransactionManager dbTm = new DataSourceTransactionManager(dataSource);
    // JmsTm registered first → commits first → rolls back last
    return new ChainedTransactionManager(jmsTm, dbTm);
}

@Bean
public DefaultMessageListenerContainer billingContainer(
        ConnectionFactory connectionFactory,
        PlatformTransactionManager transactionManager) {
    DefaultMessageListenerContainer container =
        new DefaultMessageListenerContainer();
    container.setConnectionFactory(connectionFactory);
    container.setDestinationName("BILLING.EVENTS.QUEUE");
    container.setSessionTransacted(false); // managed by ChainedTransactionManager
    container.setTransactionManager(transactionManager);
    container.setConcurrentConsumers(4);
    return container;
}

@ServiceActivator(inputChannel = "billingChannel")
@Transactional
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String idempotencyKey = UUID.randomUUID().toString();

    ChargeCreateParams params = ChargeCreateParams.builder()
        .setAmount(event.getAmountCents())
        .setCurrency("gbp")
        .setCustomer(event.getCustomerId())
        .build();

    Charge charge = Charge.create(params,
        RequestOptions.builder().setIdempotencyKey(idempotencyKey).build());

    billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
}

Under normal conditions this works: when billingRecordRepo.save() throws a runtime exception, the @Transactional proxy triggers a rollback on the ChainedTransactionManager. The chained manager rolls back in reverse order — DataSourceTransactionManager first, then JmsTransactionManager — so IBM MQ increments BackoutCount for M1 and redelivers it.

The failure timeline

Customer cust_GB_033 triggers a billing event. The ServiceActivator executes without runtime exception — the Stripe charge succeeds and the DB write completes within the transaction. The @Transactional proxy initiates a commit. At commit time, a DB connection to the PostgreSQL primary is reset mid-commit (a brief network partition between the application pod and the database pod during a Kubernetes node drain): DataSourceTransactionManager.commit() throws TransactionSystemException wrapping PSQLException: connection reset.

The double charge is caused by a commit-phase DB failure — an event that is rare, transient, and invisible in application logs as a processing error. The TransactionSystemException logged at t=0.435s looks like a transient infrastructure problem, not a billing data-integrity issue. The reconciliation job three days later looks like a successful recovery. The two log entries are separated by 72 hours and attributed to different incident tickets.

Developer misconception

“ChainedTransactionManager provides near-XA guarantees for JMS + DB. If DB fails, JMS rolls back. I’ve tested this: when billingRecordRepo.save() throws during normal execution, IBM MQ redelivers the message. The JMS-first registration order means JMS is the ‘outer’ resource — if the inner resource (DB) fails, the outer resource rolls back too.”

This misconception misidentifies when ChainedTransactionManager rolls back vs. when it is too late to roll back. During the execution phase (inside the @Transactional method body), any exception triggers a rollback before any commit is attempted — both managers roll back in reverse order, IBM MQ redelivers, and the system behaves correctly. This covers the common case (DB constraint violation, CannotGetJdbcConnectionException) and is what developers test. The commit phase is different: once ChainedTransactionManager starts committing, it cannot roll back a manager that has already committed. JMS-first registration means JMS commits first during the commit phase. A DB commit failure after the JMS commit is a window that cannot be closed by the ChainedTransactionManager design. The Javadoc for ChainedTransactionManager states explicitly: “In case of failure it is not guaranteed to roll back the first resource even if it participated in the transaction.” Most developers who read this treat it as a theoretical concern and do not account for commit-phase DB failures under infrastructure stress.

The fix

Two independent fixes, both required:

Fix 1: Register DataSourceTransactionManager first in ChainedTransactionManager. This reverses the commit order: DB commits first, then JMS commits. If the DB commits and the JMS commit fails (a much rarer event with IBM MQ’s reliable client), the IBM MQ message is not acknowledged and IBM MQ will redeliver it on the next poll — the stable idempotency key (Fix 2) makes this redelivery safe. If the DB commit fails, the JMS commit has not yet occurred, so the JMS transaction rolls back and IBM MQ redelivers M1 for a retry:

@Bean
public PlatformTransactionManager transactionManager(
        ConnectionFactory connectionFactory,
        DataSource dataSource) {
    DataSourceTransactionManager dbTm = new DataSourceTransactionManager(dataSource);
    JmsTransactionManager jmsTm = new JmsTransactionManager(connectionFactory);
    // DB registered first → DB commits first → DB rolls back last
    // If DB commit fails, JMS has not yet committed → JMS rolls back → IBM MQ redelivers
    // If JMS commit fails (rare), DB already committed → stable key prevents double-charge
    return new ChainedTransactionManager(dbTm, jmsTm);
}

Fix 2: Use a stable content-derived idempotency key. Even with DB-first ordering, there is a residual window: DB commits, JMS commit fails, IBM MQ redelivers M1, and the billing handler runs again. Without a stable key, the retry generates UUID_B → ch_B. With a stable key, Stripe returns ch_A from cache (within 24 hours) or the existsByIdempotencyKey guard short-circuits (beyond 24 hours):

@ServiceActivator(inputChannel = "billingChannel")
@Transactional
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
        return; // Idempotent short-circuit
    }

    ChargeCreateParams params = ChargeCreateParams.builder()
        .setAmount(event.getAmountCents())
        .setCurrency("gbp")
        .setCustomer(event.getCustomerId())
        .build();

    Charge charge = Charge.create(params,
        RequestOptions.builder().setIdempotencyKey(stableKey).build());

    billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
}

With both fixes applied: DB-first commit ordering minimizes the commit-phase window (DB failures before JMS commit → IBM MQ redelivers); stable key makes all redeliveries safe (DMLC rollback redeliveries, residual JMS-commit-failure redeliveries, reconciliation republishes); the BACKOUT_QUEUE handler described in Mode 3 also becomes safe for the same reason.

Failure mode 3: IBM MQ BACKOUT_THRESHOLD + BACKOUT_QUEUE_NAME consumer using UUID.randomUUID() — retry exhaustion moves M1 to BACKOUT_QUEUE; BACKOUT consumer generates UUID_B; Stripe commits ch_B alongside ch_A from the final successful retry on the main queue

The setup

A team configures the IBM MQ billing queue with a backout threshold and a dedicated backout queue:

DEFINE QLOCAL(BILLING.EVENTS.QUEUE) BOTHRESH(5) BOQNAME(BILLING.EVENTS.BACKOUT) DEFPSIST(YES)
DEFINE QLOCAL(BILLING.EVENTS.BACKOUT) DEFPSIST(YES)

The main-queue consumer uses DefaultMessageListenerContainer with sessionTransacted=true. The billing ServiceActivator generates Stripe idempotency keys with UUID.randomUUID(). A separate Spring Integration pipeline consumes from BILLING.EVENTS.BACKOUT — the team’s “recovery handler” that re-attempts billing for messages the main queue gave up on:

// Main-queue consumer (on BILLING.EVENTS.QUEUE)
@ServiceActivator(inputChannel = "mainBillingChannel")
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String idempotencyKey = UUID.randomUUID().toString();

    Charge charge = Charge.create(
        ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("gbp")
            .setCustomer(event.getCustomerId())
            .build(),
        RequestOptions.builder().setIdempotencyKey(idempotencyKey).build()
    );
    billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
}

// BACKOUT_QUEUE consumer (on BILLING.EVENTS.BACKOUT)
@ServiceActivator(inputChannel = "backoutBillingChannel")
public void handleBackoutBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    log.warn("Billing event in BACKOUT queue, re-attempting: {}", event.getId());

    String idempotencyKey = UUID.randomUUID().toString(); // ← new random key

    Charge charge = Charge.create(
        ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("gbp")
            .setCustomer(event.getCustomerId())
            .build(),
        RequestOptions.builder().setIdempotencyKey(idempotencyKey).build()
    );
    billingRecordRepo.save(new BillingRecord(event, idempotencyKey, charge.getId()));
}

The team’s mental model is that the BACKOUT_QUEUE handler only processes messages that the main queue completely failed to process — it is a safety net for catastrophic failure, not a parallel processing path.

The failure timeline

A billing event M1 arrives for customer cust_GB_058 during a period of intermittent DB connection pool exhaustion. The main-queue consumer retries multiple times before succeeding. IBM MQ’s BACKOUT_THRESHOLD=5 means IBM MQ moves M1 to BILLING.EVENTS.BACKOUT after the BackoutCount reaches 5.

The net result is five Stripe charges from Attempt 1 through 4 (each with a different random key, each a real charge because Stripe’s 24-hour cache only protects retries with the same key within 24 hours), plus the successful ch_A5, plus the BACKOUT_QUEUE handler’s ch_B. The operations team’s “proactive backout” script was designed to protect the system, but without stable idempotency keys it created additional duplicate charges on top of the retry loop’s own duplicates.

Developer misconception

“The BACKOUT_QUEUE handler only processes messages that the main-queue consumer gave up on after exhausting all retries. If the main-queue consumer eventually succeeded, IBM MQ would have committed M1 and removed it from the queue before it was moved to the BACKOUT_QUEUE. The two paths cannot be active simultaneously for the same message. The BACKOUT_QUEUE handler is a safety net that runs after the main queue has given up — not a parallel charge path.”

This is incorrect in two ways. First, IBM MQ’s BOTHRESH check is not an application-level mutex — there is no exclusive lock that prevents a message from being on both the main queue and the BACKOUT_QUEUE simultaneously. Manual moves, monitoring scripts, and application-layer BACKOUT_QUEUE logic operate independently of IBM MQ’s own BOTHRESH logic. Second, even when relying only on IBM MQ’s native BOTHRESH behavior (without manual moves), the BOTHRESH check happens at the next attempted delivery after BackoutCount reaches the threshold. If the application’s retry loop has multiple concurrent consumer threads, it is possible for one thread to commit M1 successfully (consuming it from the main queue) while another thread is processing a message it received before the successful commit completed. More importantly: the main-queue retry loop itself generates multiple Stripe charges for M1 across attempts 1 through N-1 (before the first successful attempt), because each attempt uses UUID.randomUUID(). The BACKOUT_QUEUE handler generates one more. Any of these charges represents a real double charge to the customer.

The fix

Three independent fixes, all required:

Fix 1: Use a stable content-derived idempotency key in both the main-queue and BACKOUT_QUEUE handlers. A stable key means every retry attempt — whether on the main queue or the BACKOUT_QUEUE — produces the same Stripe idempotency key. Stripe returns the cached charge from the first successful attempt on every subsequent call:

// Main-queue consumer
@ServiceActivator(inputChannel = "mainBillingChannel")
@Transactional
public void handleBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
        return; // Already processed; ack the message without re-charging
    }

    Charge charge = Charge.create(
        ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("gbp")
            .setCustomer(event.getCustomerId())
            .build(),
        RequestOptions.builder().setIdempotencyKey(stableKey).build()
    );
    billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
}

// BACKOUT_QUEUE consumer — same stable key derivation
@ServiceActivator(inputChannel = "backoutBillingChannel")
@Transactional
public void handleBackoutBillingEvent(Message<BillingEvent> message) {
    BillingEvent event = message.getPayload();
    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    if (billingRecordRepo.existsByIdempotencyKey(stableKey)) {
        log.info("Backout handler: event already billed, acking without re-charge: {}", event.getId());
        return;
    }

    // If not yet billed (e.g. main queue truly exhausted without success), charge now
    Charge charge = Charge.create(
        ChargeCreateParams.builder()
            .setAmount(event.getAmountCents())
            .setCurrency("gbp")
            .setCustomer(event.getCustomerId())
            .build(),
        RequestOptions.builder().setIdempotencyKey(stableKey).build()
    );
    billingRecordRepo.save(new BillingRecord(event, stableKey, charge.getId()));
    log.warn("Backout handler: billed via backout path for event {}, charge {}", event.getId(), charge.getId());
}

Fix 2: Eliminate manual proactive-move scripts; rely on IBM MQ’s native BOTHRESH mechanism exclusively. Manual move scripts that fire at BackoutCount = BOTHRESH - 1 race with the active DMLC retry and can move a message to the BACKOUT_QUEUE while it is being successfully committed on the main queue. If you need proactive intervention, use IBM MQ’s BOTHRESH and BOQNAME queue attributes exclusively, and set BOTHRESH low enough that the main-queue retry loop is unlikely to succeed — do not run parallel move scripts that violate the assumption that main and BACKOUT paths are mutually exclusive at any point in time.

Fix 3: Use idempotent BACKOUT_QUEUE handling as an alert, not a re-billing path. The BACKOUT_QUEUE handler should check existsByIdempotencyKey(stableKey) before any Stripe call. If a BillingRecord already exists, the event was successfully processed on the main queue — log at INFO, ack the BACKOUT_QUEUE message, and exit. Only charge via Stripe if no BillingRecord exists. This makes the BACKOUT_QUEUE handler safe for both concurrent-processing races and the case where the main queue truly exhausted without billing successfully:

// Configure BACKOUT_QUEUE DMLC
@Bean
public DefaultMessageListenerContainer backoutBillingContainer(
        ConnectionFactory connectionFactory,
        PlatformTransactionManager transactionManager) {
    DefaultMessageListenerContainer container = new DefaultMessageListenerContainer();
    container.setConnectionFactory(connectionFactory);
    container.setDestinationName("BILLING.EVENTS.BACKOUT");
    container.setTransactionManager(transactionManager);
    container.setConcurrentConsumers(1); // low concurrency for backout processing
    container.setMaxConcurrentConsumers(2);
    return container;
}

With all three fixes applied: the stable key makes every retry on the main queue and every BACKOUT_QUEUE handler call idempotent; the existsByIdempotencyKey guard prevents re-charging regardless of timing; BACKOUT_QUEUE monitoring provides operational visibility without creating duplicate charge risk.

Cross-layer comparison: Spring Integration IBM MQ vs other adapters in the series

Behavior SI IBM MQ (JmsInboundChannelAdapter polling) SI IBM MQ (DMLC push) SI Solace (SolaceInboundChannelAdapter) SI GCP PubSub (PubSubInboundChannelAdapter) SI ActiveMQ (DMLC push)
Ack/commit scope with DirectChannel After ServiceActivator returns (JmsTemplate commit after SessionCallback returns; DirectChannel.send() is synchronous) After onMessage() returns (DMLC controls session commit) After ServiceActivator returns for CLIENT_ACK with correct placement After ServiceActivator returns for MANUAL with correct placement After onMessage() returns (DMLC controls session commit)
Ack/commit scope with ExecutorChannel After channel.send() returns (async enqueue) — before ServiceActivator runs After onMessage() returns (DMLC controls session commit; each concurrent consumer has its own session) AUTO_ACK: after channel.send() returns (same hazard). CLIENT_ACK: after explicit call from executor thread ackMode=AUTO: deadline timer runs from broker delivery, not handler completion After onMessage() returns (DMLC session scope)
TX + DB coordination ChainedTransactionManager: commit-order determines which resource commits first; not XA ChainedTransactionManager or JmsTransactionManager alone; same commit-order risk No Spring TX resource adapter; acknowledge() is a direct JCSMP network call No Spring TX resource adapter; ack() is a direct HTTP call to GCP Pub/Sub API JmsTransactionManager; ActiveMQ supports XA via ActiveMQXAConnectionFactory
Retry on exception Polling: JmsTemplate commits on return; retry is application responsibility (no DMLC-style rollback) DMLC: sessionTransacted=true → session.rollback() on exception → IBM MQ redelivers; BackoutCount incremented CLIENT_ACK: REQUEUE status causes Solace redelivery; AUTO_ACK: no retry ackMode=AUTO: Pub/Sub redelivers after ack deadline expires; ackMode=MANUAL: nack causes redelivery DMLC: session.rollback() on exception → ActiveMQ redelivers; DLQ after maximumRedeliveries
Broker-side dead-message handling IBM MQ BOTHRESH / BOQNAME: broker moves message to BACKOUT_QUEUE after BackoutCount ≥ BOTHRESH; independent of application retry logic Same as polling; DMLC rollbacks increment BackoutCount Solace DMQ: broker moves message to DMQ after maxDeliveryCount is exceeded GCP Dead Letter Topic: Pub/Sub moves message to DLT after maxDeliveryAttempts ActiveMQ DLQ: broker moves message to DLQ after maximumRedeliveries
Shutdown / graceful stop behavior JmsTemplate polling: stop() halts polling thread; in-flight executor tasks continue independently DMLC: stop() waits for active onMessage() calls to complete (JMS 2.0 spec §8.7 for MessageConsumer.close()) stop() closes JCSMP FlowReceiver without draining QueueChannel buffer; unacked messages returned to broker immediately stop() terminates streaming pull subscription; in-flight messages have ack deadline for Pub/Sub to redeliver DMLC: stop() waits for active onMessage() per JMS spec; ActiveMQ returns unacked messages on disconnect

ChainedTransactionManager commit-order behavior table

Event JMS-first: new ChainedTransactionManager(jmsTm, dbTm) DB-first: new ChainedTransactionManager(dbTm, jmsTm)
Runtime exception in ServiceActivator Rollback: DB rolls back first, then JMS rolls back → IBM MQ redelivers M1 (correct behavior) Rollback: JMS rolls back first, then DB rolls back → IBM MQ redelivers M1 (correct behavior)
DB commit-phase failure (e.g., connection reset on COMMIT) JMS already committed → M1 permanently consumed; no BillingRecord → double-charge risk on reconciliation DB commits first → fails → JMS has not committed yet → JMS rolls back → IBM MQ redelivers M1 → stable key makes retry safe
JMS commit-phase failure (IBM MQ network issue on session.commit()) JMS commit fails → exception; DB not yet committed → ChainedTransactionManager rolls back DB → M1 on IBM MQ (not committed yet) → IBM MQ redelivers; stable key makes retry safe DB already committed → JMS commit fails → ChainedTransactionManager tries to roll back DB (may fail if connection lost) → M1 on IBM MQ (not committed) → IBM MQ redelivers → stable key prevents double-charge
Recommendation Do not use for IBM MQ billing consumers where DB commit failures under load are possible Preferred: DB-first commit minimizes IBM MQ permanent-consumption window; stable key handles residual JMS-commit-failure redeliveries

IBM MQ BACKOUT_THRESHOLD behavior and Spring Integration retry interaction table

Configuration DMLC sessionTransacted=true exception behavior IBM MQ BackoutCount tracking BOTHRESH trigger BACKOUT_QUEUE handler risk with UUID.randomUUID()
DMLC + sessionTransacted=true + no retry advice + random key Each ServiceActivator exception → DMLC calls session.rollback() → IBM MQ increments BackoutCount → redelivers M1 Increments on every rollback After N rollbacks, IBM MQ stops delivering to DMLC; moves M1 to BACKOUT_QUEUE High: each DMLC retry generates a new Stripe charge; BACKOUT_QUEUE handler generates one more
DMLC + sessionTransacted=true + RequestHandlerRetryAdvice (in-handler retries) + random key Retries within ServiceActivator; if all retries fail, exception propagates to DMLC → rollback; if one retry succeeds, session commits BackoutCount increments only on DMLC-level rollback (not on in-handler retries) Only after DMLC-level rollback; in-handler retries do not increment BackoutCount High within ServiceActivator (multiple Stripe charges per DMLC delivery); one more from BACKOUT_QUEUE on final exhaustion
DMLC + sessionTransacted=true + no retry advice + stable key Each DMLC rollback causes redelivery; stable key → Stripe cache hit on retry; existsByIdempotencyKey guard on BillingRecord save Increments on every rollback (normal) IBM MQ moves M1 to BACKOUT_QUEUE after BOTHRESH rollbacks None: BACKOUT_QUEUE handler uses same stable key → Stripe returns cached charge; existsByIdempotencyKey short-circuits if BillingRecord already saved
JmsInboundChannelAdapter polling + SESSION_TRANSACTED + ExecutorChannel + random key JmsTemplate commits before ServiceActivator runs; no DMLC-style rollback available Never incremented (message committed before exception occurs) Never triggered N/A for BACKOUT_QUEUE (BackoutCount never increments); but reconciliation path generates second random key → ch_B

Diagnostic checklist for Spring Integration IBM MQ + Stripe billing consumers

  1. Identify polling vs. push mode. Is the adapter a JmsInboundChannelAdapter (polling via JmsTemplate) or a JmsMessageDrivenEndpoint / DefaultMessageListenerContainer (push)? Polling mode does not support JMS-session-scoped transaction rollback on downstream handler exceptions — use DMLC push mode for transactional billing consumers.
  2. Check output channel type. Is the output channel a DirectChannel, ExecutorChannel, QueueChannel, or PriorityChannel? For JmsInboundChannelAdapter polling mode, only DirectChannel keeps the handler within the JmsTemplate session scope. All other channel types decouple processing from the session commit.
  3. Audit ChainedTransactionManager registration order. Is JmsTransactionManager registered first or second? First means JMS commits before DB — DB commit-phase failures cause permanent IBM MQ message loss. Second (DB-first) is safer for billing consumers.
  4. Check whether a separate BACKOUT_QUEUE consumer exists. Does the BACKOUT_QUEUE handler call Stripe.Charge.create()? Does it use UUID.randomUUID() or a stable content-derived key? Does it check existsByIdempotencyKey before charging?
  5. Verify IBM MQ BOTHRESH and BOQNAME queue attributes. What is the BOTHRESH value on the billing queue? What is the BOQNAME? Is there a Spring Integration consumer on that queue? Does the BOQNAME consumer have idempotency-safe Stripe integration?
  6. Check for manual BackoutCount monitoring scripts. Are there operational scripts that move messages from the main billing queue to the BACKOUT_QUEUE based on BackoutCount, outside of IBM MQ’s native BOTHRESH mechanism? These scripts race with active DMLC processing and can deliver the same billing event to both main and BACKOUT handlers concurrently.
  7. Audit Stripe idempotency key derivation across all billing paths. Does every code path that calls Stripe.Charge.create() — main queue, BACKOUT_QUEUE, reconciliation job, manual reprocessing scripts — use the same stable content-derived key for the same billing event?
  8. Test the reconciliation job’s Stripe key derivation. When the reconciliation job finds a Stripe charge without a matching BillingRecord and republishes the event, does the republished message carry the same business payload fields (accountId, billingPeriod, amountCents) as the original? If the reconciliation job reconstructs the event from Stripe metadata and any field differs, the stable key will differ and a duplicate charge will occur.

Test patterns for Spring Integration IBM MQ + Stripe idempotency

Mode 1: polling adapter + ExecutorChannel + DB exception after Stripe commit

@Test
@DisplayName("JmsInboundChannelAdapter polling + ExecutorChannel: JmsTemplate commits before ServiceActivator runs; DB failure after Stripe commit does not cause redelivery; stable key prevents ch_B on reconciliation republish")
void pollingAdapterSessionTransactedExecutorChannel_dbExceptionAfterStripeCommit_noDuplicateCharge() throws Exception {
    BillingEvent event = BillingEvent.builder()
        .accountId("acc_GB_019")
        .billingPeriod("2026-10")
        .amountCents(4900L)
        .customerId("cust_GB_019")
        .build();

    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    // First delivery: Stripe succeeds, DB fails; with polling adapter, JmsTemplate already committed
    // so M1 won't be redelivered by IBM MQ
    stubFor(post(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey))
        .inScenario("billing-retry")
        .whenScenarioStateIs(STARTED)
        .willReturn(aResponse()
            .withStatus(200)
            .withHeader("Content-Type", "application/json")
            .withBody("""
                {"id":"ch_A","amount":4900,"currency":"gbp","status":"succeeded"}
            """))
        .willSetStateTo("stripe-charged"));

    // Simulate DB failure on first attempt
    doThrow(new CannotGetJdbcConnectionException("pool exhausted"))
        .doNothing()
        .when(billingRecordRepo).save(any(BillingRecord.class));

    // Put M1 on IBM MQ queue (via Testcontainers IBM MQ)
    ibmMqHelper.sendMessage("BILLING.EVENTS.QUEUE", event);

    // Wait for executor to process
    Thread.sleep(2000);

    // Verify Stripe called exactly once (first delivery; JmsTemplate committed M1)
    verify(1, postRequestedFor(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey)));

    // Verify no BillingRecord (DB failed; with stable key, reconciliation republish will be idempotent)
    assertThat(billingRecordRepo.findByIdempotencyKey(stableKey)).isEmpty();

    // Simulate reconciliation republish M2 (same business payload)
    ibmMqHelper.sendMessage("BILLING.EVENTS.QUEUE", event);
    Thread.sleep(2000);

    // Stripe called again with same stableKey — returns ch_A from idempotency cache
    verify(2, postRequestedFor(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey)));

    // Now BillingRecord saved (DB recovered on second attempt)
    assertThat(billingRecordRepo.findByIdempotencyKey(stableKey))
        .isPresent()
        .map(BillingRecord::getChargeId)
        .hasValue("ch_A"); // Same charge ID from Stripe cache — not ch_B
}

Mode 2: ChainedTransactionManager DB-first ordering + JMS commit failure redelivery

@Test
@DisplayName("ChainedTransactionManager DB-first: JMS commit failure causes IBM MQ redelivery; stable key prevents duplicate charge on retry")
void chainedTransactionManagerDbFirst_jmsCommitFailure_ibmMqRedelivers_stableKeyPreventsDoubleCharge()
        throws Exception {
    BillingEvent event = BillingEvent.builder()
        .accountId("acc_GB_033")
        .billingPeriod("2026-10")
        .amountCents(9900L)
        .customerId("cust_GB_033")
        .build();

    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    // Stripe responds successfully on both attempts
    stubFor(post(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey))
        .willReturn(aResponse()
            .withStatus(200)
            .withHeader("Content-Type", "application/json")
            .withBody("""
                {"id":"ch_A","amount":9900,"currency":"gbp","status":"succeeded"}
            """)));

    // Simulate JMS commit failure on first attempt (IBM MQ network blip)
    doThrow(new JmsException("IBM MQ commit failed: network reset") {})
        .doNothing()
        .when(jmsTransactionManager).commit(any());

    ibmMqHelper.sendMessage("BILLING.EVENTS.QUEUE", event);
    Thread.sleep(3000); // Allow redelivery

    // DB committed successfully on first attempt; JMS commit failed; IBM MQ redelivered M1
    // Second attempt: DB finds existsByIdempotencyKey = true (BillingRecord saved in first DB commit)
    // ServiceActivator short-circuits; Stripe NOT called on second delivery
    verify(1, postRequestedFor(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey)));

    assertThat(billingRecordRepo.findByIdempotencyKey(stableKey))
        .isPresent()
        .map(BillingRecord::getChargeId)
        .hasValue("ch_A");
}

Mode 3: BACKOUT_QUEUE concurrent delivery with main-queue success

@Test
@DisplayName("BACKOUT_QUEUE: concurrent delivery with main-queue success; stable key in BACKOUT handler prevents ch_B")
void backoutQueueConcurrentWithMainSuccess_stableKeyPreventsDoubleCharge() throws Exception {
    BillingEvent event = BillingEvent.builder()
        .accountId("acc_GB_058")
        .billingPeriod("2026-10")
        .amountCents(14900L)
        .customerId("cust_GB_058")
        .build();

    String stableKey = "billing:" + sha256(
        event.getAccountId() + ":" + event.getBillingPeriod() + ":" + event.getAmountCents()
    );

    // Stripe responds with ch_A on first call; returns ch_A from cache on subsequent calls with same key
    stubFor(post(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey))
        .willReturn(aResponse()
            .withStatus(200)
            .withHeader("Content-Type", "application/json")
            .withBody("""
                {"id":"ch_A","amount":14900,"currency":"gbp","status":"succeeded"}
            """)));

    // Send M1 to both main queue and BACKOUT_QUEUE simultaneously
    // (simulating the scenario where a move script placed a copy on BACKOUT_QUEUE
    // before the main queue committed successfully)
    ibmMqHelper.sendMessage("BILLING.EVENTS.QUEUE", event);
    ibmMqHelper.sendMessage("BILLING.EVENTS.BACKOUT", event);

    Thread.sleep(4000);

    // Stripe called at most twice (once from main, once from backout)
    // but ALWAYS with the same stableKey → Stripe returns ch_A both times
    verify(lessThanOrExactly(2), postRequestedFor(urlEqualTo("/v1/charges"))
        .withHeader("Idempotency-Key", equalTo(stableKey)));

    // Only one BillingRecord exists (second save is idempotent via existsByIdempotencyKey guard)
    assertThat(billingRecordRepo.countByIdempotencyKey(stableKey)).isEqualTo(1);
    assertThat(billingRecordRepo.findByIdempotencyKey(stableKey))
        .isPresent()
        .map(BillingRecord::getChargeId)
        .hasValue("ch_A");
}

What Keybrake adds to Spring Integration IBM MQ billing pipelines

The three failure modes described above all share a structural root cause: the Stripe idempotency key is generated at handler invocation time (UUID.randomUUID()) rather than derived from the billing event’s business identity. A stable content-derived key prevents duplicate charges across all three paths — polling adapter retries, ChainedTransactionManager commit-phase failures, and BACKOUT_QUEUE concurrent processing. The fixes described above implement this at the application layer.

At the infrastructure layer, Keybrake adds a complementary defense: a scoped API-key proxy that sits between your Spring Integration billing consumer and the Stripe API, enforcing per-account and per-billing-period daily spend caps. When a stuck retry loop or an unexpected BACKOUT_QUEUE re-processing path generates multiple Stripe Charge.create() calls for the same customer in a short window, Keybrake’s policy layer blocks the second call and logs it as a policy violation — providing an audit trail that is visible in the Keybrake dashboard before the customer disputes the charge. The proxy-layer enforcement is separate from the idempotency key fix, not a substitute for it, but it creates a second defense line that catches the cases where stable keys are correct but retry logic, backout handlers, or reconciliation jobs produce unexpected call volumes.

Audit your agent’s Stripe calls before the next BACKOUT_QUEUE incident

Keybrake gives your Spring Integration billing consumer a scoped proxy key with per-account spend caps, endpoint allowlists, and a real-time audit log of every Stripe call — so you catch retry storms and BACKOUT_QUEUE double-charges in the dashboard before your customers see them on their cards.

Related posts in this series