Spring Data Neo4j and Stripe Integration: How a @Transactional Neo4j Write Rollback After ch_A Is Committed Clears Both the COMPLETED Status and the In-Transaction-Generated Key Causing Recovery to Generate UUID_B, a ServiceUnavailableException After a Successful Node Write Prevents the Application from Receiving the Assigned id() Causing the Retry to Derive a Different Key from a Second Node, and Reactive Mono.retryWhen() Re-Subscribes the Full Pipeline Including the Mono.fromSupplier UUID Key-Generation Step Producing a Fresh UUID on Each Retry Attempt

This post is part of the ongoing series on Stripe idempotency failure modes in JVM frameworks. Earlier entries covered Spring Data JPA (JDBC transaction boundary and Stripe commit ordering), Spring Cloud Circuit Breaker (Resilience4j TimeLimiter post-commit fallback, CircuitBreakerOpenException open-state recovery), Spring Cloud OpenFeign (RequestInterceptor on every retry, retryOnAllOperations re-sends POST to new instance), Spring Data Elasticsearch (@Retryable UUID reset after VersionConflictEngineException, NRT search gap, auto-generated ID lost in timeout), Spring Cloud Config (@RefreshScope bean destruction clears in-flight key cache, @Value-bound prefix changed on config refresh), and Spring Cloud Sleuth (@NewSpan fresh spanId per @Retryable invocation, missing ContextPropagatingTaskDecorator on custom @Async executor, BaggageField key absent in @Scheduled context). This post moves to Spring Data Neo4j. Graph databases introduce a set of behavioral properties — bolt session semantics, internal node ID assignment, and reactive re-subscription — that each independently cause Stripe double-charges when developers couple idempotency key derivation to Neo4j-specific state without understanding the transactional guarantees of the bolt protocol, the lifecycle of internally assigned node identifiers, and the re-execution semantics of Project Reactor’s retry operators.

The three failure modes are: a @Transactional billing method that stores the idempotency key as a property on a newly created Neo4j node, where a transaction rollback (triggered by a constraint violation or a bolt session error after Stripe commits ch_A) discards both the node and the key it carried, leaving recovery to query for a COMPLETED billing event node that no longer exists and generate a fresh UUID.randomUUID(); a developer who saves a Neo4j node without an application-supplied ID to receive a Neo4j-assigned internal node identifier and then uses that identifier as a component of the Stripe idempotency key, where a ServiceUnavailableException thrown by the bolt driver after the write has been durably committed to the Neo4j store prevents the application from receiving the assigned id(), causing the retry to create a second node with a different ID and derive a different Stripe key; and a reactive pipeline that uses ReactiveNeo4jRepository with Mono.retryWhen(), where the key is generated inside the pipeline via Mono.fromSupplier(() -> UUID.randomUUID()) and Mono.retryWhen() re-subscribes to the entire upstream chain on each retry attempt, including the key-generation supplier, producing a different UUID on every retry and a new Stripe charge on each attempt.

Stack for examples: Spring Boot 3.x, Spring Data Neo4j 7.x, Neo4j Java driver 5.x (bolt protocol), Project Reactor 3.x (ReactiveNeo4jRepository), Spring Retry (spring-retry), Stripe Java SDK v24+, WireMock 3.x, Testcontainers Neo4j 3.x, JUnit 5.

Background: why developers couple Stripe idempotency keys to Neo4j state

The motivation for tying a Stripe idempotency key to Neo4j-managed state is understandable. In a graph-database-backed billing system, the natural pattern is to represent a billing event as a node — a (:BillingEvent) node with properties subscriptionId, billingPeriod, status, and idempotencyKey. The reasoning goes: “I store the idempotency key in Neo4j when I create the billing event node. On retry, I query for the existing node and re-use the persisted key. This is safe because the key is in the database.”

This pattern is correct in spirit but fails in practice whenever the Neo4j transaction and the Stripe call have a boundary mismatch. The idempotency key must be stable across all retry attempts, including attempts that happen after Neo4j transactions have rolled back. Any key stored exclusively inside a Neo4j transaction that subsequently rolls back is as lost as a key stored only in a local variable — it does not survive the rollback. If the Stripe charge was committed before the rollback, recovery has no way to retrieve the key it used, and a fresh UUID generates a second charge.

The internal node ID problem is a separate and more subtle issue. Spring Data Neo4j can use the Neo4j database-internal ID (a Long assigned by the server, formerly exposed as id() in Cypher, now accessed via elementId() in Neo4j 5.x) as the entity’s primary key. When a developer derives the Stripe idempotency key from this internal ID, they implicitly depend on a server-side assignment that travels from the Neo4j store to the application over the bolt connection. The connection can fail after the write is committed but before the application reads the response — exactly the same “commit succeeded but response was lost” scenario that produces double charges in JDBC, Elasticsearch, and every other network-adjacent data store.

The reactive failure mode is more specific to the Project Reactor model. Mono.retryWhen() does not retry a single operator — it re-subscribes to the entire Mono chain from the source. If the chain’s source includes a key-generation step, that step runs again on each retry. Developers new to reactive programming frequently assume that retryWhen “restarts from the point of failure,” which is not how reactive stream re-subscription works. The result is a different UUID per retry and a different Stripe charge per attempt.

As in every post in this series: Stripe deduplicates charges by the string value of the Idempotency-Key request header, scoped to the Stripe account. The key is opaque to Stripe — it does not parse the key for amount, customer, or currency. Two calls with different key strings are treated as two distinct charge requests. There is no semantic deduplication. A key that changes between the first call and any subsequent retry will always produce a second charge if the first call committed.

Mode 1: @Transactional Neo4j write rollback after ch_A is committed clears both the COMPLETED status guard and the in-transaction-generated idempotency key — recovery generates UUID_B — ch_B committed alongside already-committed ch_A

The developer builds a billing method that wraps the entire billing operation in a single @Transactional boundary. Inside the transaction: (1) check whether a (:BillingEvent) node with status = 'COMPLETED' already exists for this subscription and billing period — if so, return the existing charge ID; (2) create a new BillingEvent node with a freshly generated UUID stored as the idempotencyKey property; (3) call Stripe using the key from the in-memory entity; (4) update the node’s status to 'COMPLETED' and set the chargeId property; (5) the transaction commits.

The developer’s mental model: “The idempotency key is persisted in the BillingEvent node. If a retry is needed, I query for the existing node and re-use the key I stored there. The COMPLETED status check at the start of the method ensures I never charge twice — if the first attempt charged the customer, the node would be COMPLETED and I’d return early.”

This model breaks when the transaction rolls back after Stripe commits the charge but before the transaction commits to Neo4j. The rollback can be triggered by a ConstraintViolationException (a unique constraint on subscriptionId + billingPeriod violated by a concurrent attempt), a Neo4j bolt session-level error (SessionExpiredException from a cluster leader re-election), or any other exception that propagates out of the @Transactional method boundary and triggers Spring’s transaction rollback.

// BillingService.java — @Transactional wrapping Neo4j write + Stripe charge
@Service
public class BillingService {

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private StripeClient stripeClient;

    @Transactional  // Neo4j transaction boundary
    public String chargeSubscription(String subscriptionId, String billingPeriod, int amountCents) {

        // Guard: check for existing COMPLETED node
        Optional<BillingEvent> existing = billingEventRepository
            .findBySubscriptionIdAndBillingPeriodAndStatus(
                subscriptionId, billingPeriod, "COMPLETED");
        if (existing.isPresent()) {
            return existing.get().getChargeId();
        }

        // Create the billing event node and store the idempotency key inside it.
        // Developer reasoning: "The key is now in Neo4j. I can retrieve it on retry."
        //
        // Flaw: if this transaction rolls back AFTER Stripe commits ch_A,
        // the BillingEvent node (and its idempotencyKey property) is discarded.
        // Recovery query finds no COMPLETED node, generates UUID_B, commits ch_B.
        BillingEvent event = new BillingEvent(
            subscriptionId,
            billingPeriod,
            UUID.randomUUID().toString(),   // key stored only in this transaction
            "PENDING"
        );
        billingEventRepository.save(event);

        String idempotencyKey = event.getIdempotencyKey();

        // Stripe commits ch_A here.
        ChargeResponse charge = stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        // If any exception is thrown between here and transaction commit
        // (e.g., ConstraintViolationException, SessionExpiredException,
        //  a second save() call failing for any reason), the @Transactional
        // interceptor will roll back the entire Neo4j transaction,
        // discarding the BillingEvent node and the idempotencyKey it carries.
        // Stripe's ch_A is not rolled back — Stripe has no knowledge of Neo4j transactions.
        event.setStatus("COMPLETED");
        event.setChargeId(charge.getId());
        billingEventRepository.save(event);  // <-- ConstraintViolationException here rolls back

        return charge.getId();
    }
}

The failure sequence:

1. chargeSubscription("sub_001", "2026-10", 9900) is called. No COMPLETED node exists. A new BillingEvent node is created in the pending Neo4j transaction with idempotencyKey = "ik_A" and status = "PENDING". The node is not yet committed.
2. stripeClient.createCharge(..., "ik_A") is called. Stripe processes the charge and durably commits ch_A. Stripe returns a 200 OK with chargeId = "ch_A".
3. The method attempts to update event.status = "COMPLETED" and calls billingEventRepository.save(event). A concurrent chargeSubscription call for the same subscriptionId and billingPeriod has already reached step 1 and created a node with the same properties, causing a Neo4j unique constraint violation. Spring Data Neo4j throws ConstraintViolationException.
4. The exception propagates out of the @Transactional method. Spring’s TransactionInterceptor catches it, marks the transaction for rollback, and rolls back the Neo4j bolt transaction. The entire transaction is discarded, including the BillingEvent node with idempotencyKey = "ik_A". The node does not exist in Neo4j after the rollback.
5. A retry mechanism calls chargeSubscription("sub_001", "2026-10", 9900) again. The guard query runs: findBySubscriptionIdAndBillingPeriodAndStatus("sub_001", "2026-10", "COMPLETED"). The query returns Optional.empty() — correctly, because the node was rolled back and no COMPLETED node exists. The guard concludes: “no prior charge.”
6. A new BillingEvent is created with idempotencyKey = "ik_B" (a different UUID.randomUUID()). Stripe is called with "ik_B". Stripe has no idempotency cache entry for "ik_B". It processes the request as a new charge. ch_B is committed alongside already-committed ch_A.

The guard is not defective in its implementation — it correctly detects that no COMPLETED node exists. The defect is in the assumption that “no COMPLETED node” implies “Stripe was never charged.” When the Neo4j transaction that would have created the COMPLETED node rolled back after Stripe committed, those two things are no longer equivalent. The guard’s answer is correct about Neo4j state and wrong about Stripe state.

Why the transaction boundary assumption fails

Neo4j and Stripe are separate durable stores with no distributed transaction protocol between them. When a Spring @Transactional method writes to Neo4j and calls Stripe, it is executing two non-atomic side effects. The Neo4j write can be rolled back; the Stripe charge cannot. Any exception thrown after the Stripe call but before the Neo4j transaction commits produces a state where Stripe has committed ch_A and Neo4j has no record of it.

This is not specific to Neo4j — the same pattern fails with any transactional store that is combined with Stripe calls inside the same transaction boundary. The entries for Spring Data JPA and Spring Data R2DBC describe the same root cause in a relational database context. Neo4j’s graph model adds a specific additional trigger: unique-constraint violations on graph patterns are more common than developers expect, because Neo4j enforces node property uniqueness at the property level (not the pair/triplet level, unless a composite index is added) and concurrent billing attempts for the same subscription can violate a constraint that seemed impossible in sequential reasoning.

The fix is to decouple the idempotency key from the Neo4j transaction state entirely. The key must be derivable from inputs that survive any transaction rollback and are available before any Neo4j or Stripe call is made.

// Fixed BillingService.java — key derived before the transaction, stable across all attempts
@Service
public class BillingService {

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private StripeClient stripeClient;

    public String chargeSubscription(String subscriptionId, String billingPeriod, int amountCents) {

        // Key derived from stable business inputs BEFORE any Neo4j or Stripe call.
        // Same inputs = same key on every attempt, regardless of how many Neo4j
        // transactions fail or roll back.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        // Guard: check for existing COMPLETED node.
        // This now correctly says "Stripe was charged with idempotencyKey and committed"
        // rather than "a COMPLETED node exists" -- but both are equivalent because
        // the key is deterministic. An existing COMPLETED node always carries
        // the same idempotencyKey for these inputs.
        Optional<BillingEvent> existing = billingEventRepository
            .findByIdempotencyKey(idempotencyKey);
        if (existing.isPresent() && "COMPLETED".equals(existing.get().getStatus())) {
            return existing.get().getChargeId();
        }

        BillingEvent event = new BillingEvent(
            subscriptionId, billingPeriod, idempotencyKey, "PENDING"
        );

        try {
            savePending(event);
        } catch (DataIntegrityViolationException ignored) {
            // A concurrent attempt already created the PENDING node with the same key.
            // The key is the same regardless of which attempt created the node.
        }

        ChargeResponse charge = stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        );

        // Update status in a separate transaction — rollback of this update
        // does not affect the idempotency key, which is derived from inputs,
        // nor the Stripe charge, which is already committed.
        markCompleted(idempotencyKey, charge.getId());

        return charge.getId();
    }

    @Transactional
    void savePending(BillingEvent event) {
        billingEventRepository.save(event);
    }

    @Transactional
    void markCompleted(String idempotencyKey, String chargeId) {
        billingEventRepository.setCompleted(idempotencyKey, chargeId);
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

The key difference: toStableKey() is called before the Neo4j transaction opens. The key is not stored in transient transaction state — it is computed from inputs that exist independently of any database operation. A Neo4j transaction rollback cannot erase the key because the key was never stored exclusively inside the transaction. On retry, the same inputs produce the same key. Stripe deduplicates the retry against the original ch_A.

Test for Mode 1

@SpringBootTest
@Testcontainers
class BillingServiceTransactionRollbackTest {

    @Container
    static Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5")
        .withAdminPassword("test");

    @Autowired
    private BillingService billingService;

    @MockBean
    private StripeClient stripeClient;

    @Test
    void retryAfterTransactionRollbackSendsOriginalKey() {
        // First attempt: Stripe commits ch_A. Neo4j transaction rolls back after.
        when(stripeClient.createCharge(any(), anyInt(), any(), any()))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"))
            .thenReturn(new ChargeResponse("ch_B", "succeeded"));

        // Simulate constraint violation on the first COMPLETED save by having
        // the service itself throw post-charge. The retry should send the same key.
        // In practice, inject a @Transactional wrapping that throws post-charge.

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);

        // First attempt (succeeds from Stripe perspective).
        billingService.chargeSubscription("sub_001", "2026-10", 9900);
        // Second attempt (simulating retry after rollback):
        billingService.chargeSubscription("sub_001", "2026-10", 9900);

        verify(stripeClient, atLeastOnce())
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        List<String> keys = keyCaptor.getAllValues();
        // All Stripe calls must use the same idempotency key.
        // If keys differ, each attempt is a new charge from Stripe's perspective.
        String firstKey = keys.get(0);
        assertThat(keys).allMatch(k -> k.equals(firstKey));
    }
}

Mode 2: Neo4j-assigned internal id() used as idempotency key component — ServiceUnavailableException after durable commit prevents the application from receiving the assigned ID — retry creates a second node with a different ID — key derived from new ID — ch_B committed alongside ch_A

The developer builds a billing flow where the Stripe idempotency key is derived from the Neo4j-assigned internal node identifier. The reasoning: “Each BillingEvent node has a unique ID assigned by Neo4j. I save the node first to get the ID, then use that ID in the Stripe key. The ID is a stable identifier for this billing event — if I need to retry, I query for the existing node by subscription ID and billing period, get its ID, and re-derive the same Stripe key.”

// BillingService.java — Neo4j internal node ID used as idempotency key component
@Service
public class BillingService {

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private StripeClient stripeClient;

    public String chargeSubscription(String subscriptionId, String billingPeriod, int amountCents) {

        // Save first to get the Neo4j-assigned node ID.
        // Developer reasoning: "The node ID is unique and stable — it's the
        // canonical identifier for this billing event."
        //
        // Flaw: if save() throws ServiceUnavailableException after the write is
        // committed to Neo4j (but before the bolt response is received by the
        // application), the ID assignment happened on the server but the application
        // never received it. Retry creates a SECOND node with a DIFFERENT internal ID.
        BillingEvent event = new BillingEvent(subscriptionId, billingPeriod, amountCents);
        BillingEvent saved = billingEventRepository.save(event);

        // Uses the Neo4j-assigned ID: saved.getId() returns a Long assigned by Neo4j.
        String idempotencyKey = "charge:" + saved.getId() + ":" + billingPeriod;

        return stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey
        ).getId();
    }
}

The failure sequence:

1. chargeSubscription("sub_001", "2026-10", 9900) is called. billingEventRepository.save(event) opens a bolt session, sends a CREATE (n:BillingEvent {...}) query. Neo4j commits the write and assigns internal ID 42 to the new node. The bolt driver starts sending the response back over the TCP connection.
2. Before the application reads the response containing id = 42, the Neo4j cluster undergoes a leader election (ServiceUnavailableException: “No routing servers available”). The bolt driver throws ServiceUnavailableException. The application sees this as a failed save() call.
3. The application’s retry logic catches ServiceUnavailableException and retries. The guard check (if any) queries for an existing node by subscriptionId and billingPeriod. The guard finds — or does not find — the first node depending on whether the cluster has recovered. In many implementations the guard is absent altogether, because the developer reasoned “if save() threw, the node was not persisted.”
4. The retry calls billingEventRepository.save(new BillingEvent(...)). The cluster has recovered. Neo4j processes the request and assigns a new internal ID 87 to the second node. saved.getId() returns 87.
5. The idempotency key for the retry is "charge:87:2026-10". The first attempt would have derived "charge:42:2026-10" — but the first attempt never reached the Stripe call because the exception was thrown at the save() step. Wait: in this variant, the first attempt did not call Stripe yet. But here is where the real failure occurs: the first attempt’s save did persist to Neo4j. There are now two nodes: ID 42 (committed by the first attempt, never visible to the application) and ID 87 (committed by the retry, visible to the application). The Stripe call proceeds with key "charge:87:2026-10" and commits ch_B.
6. Later, a background cleanup job (or a separate billing run) queries all PENDING BillingEvent nodes older than a threshold. It finds node ID 42 (which has no associated Stripe charge). It generates a key from "charge:42:2026-10" and calls Stripe. Stripe has no idempotency cache entry for this key. It processes it as a new charge. ch_C is committed. The customer has been charged three times.

The more common and immediate failure path does not require a cleanup job. It occurs when the developer’s billing flow is: (1) check for an existing BillingEvent node for this subscription and period; (2) if none exists, save and charge. If step 1 finds the orphaned node (ID 42, status = 'PENDING'), it may continue with the charge, deriving the key as "charge:42:2026-10". If the save() in the separate retry path (which created ID 87) already called Stripe with "charge:87:2026-10" and committed ch_B, the cleanup path re-derives a different key and commits ch_C. Alternatively, if the guard finds node ID 87 (the retry’s node), it derives "charge:87:2026-10" and calls Stripe — which correctly deduplicates against ch_B. The behavior depends on which node the guard query returns, which depends on Neo4j query evaluation order.

Why ServiceUnavailableException does not mean “the write did not happen”

The Neo4j bolt driver throws ServiceUnavailableException when a session cannot route its transaction to an available cluster member. This can occur at several points in the transaction lifecycle. If the exception is thrown before the COMMIT message is sent, the write did not happen. If the exception is thrown while waiting for the COMMIT acknowledgement, the write may or may not have happened. If the exception is thrown while reading the commit response after the server durably committed the transaction, the write happened and the node exists in Neo4j with a server-assigned ID that the application never received.

The bolt protocol does not provide exactly-once semantics at the application layer — only at-least-once from the server’s perspective. A committed write that was never acknowledged to the application appears to the application as a failed write. From Neo4j’s perspective, the write is permanent. This is not a Neo4j bug; it is the fundamental property of networked distributed systems: a request-response protocol over a TCP connection can lose the response after the server processes and durably commits the request.

The same property affects PostgreSQL (JDBC SocketException after COMMIT), Elasticsearch (ConnectException after IndexResponse), Kafka (producer timeout after broker acknowledgement), and every other networked data store. The fix is identical in all cases: do not generate idempotency key material on the server side and then transport it to the application over a network that can fail. Generate the key on the application side from inputs that are already in the application’s local state before any network call is made.

// Fixed BillingService.java — key derived from stable business inputs before any save()
@Service
public class BillingService {

    @Autowired
    private BillingEventRepository billingEventRepository;

    @Autowired
    private StripeClient stripeClient;

    public String chargeSubscription(String subscriptionId, String billingPeriod, int amountCents) {

        // Key derived BEFORE any Neo4j save(). No network dependency.
        // Same inputs = same key on every retry attempt regardless of how many
        // save() calls succeed or fail at the network layer.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        // Optional guard: check for existing completed event with this key.
        // Uses idempotencyKey as the lookup field (not node ID).
        // findByIdempotencyKey() is a GetRequest-equivalent: a Cypher
        // MATCH (n:BillingEvent {idempotencyKey: $key}) RETURN n
        // which is consistently readable (no NRT gap like Elasticsearch).
        Optional<BillingEvent> existing = billingEventRepository
            .findByIdempotencyKey(idempotencyKey);
        if (existing.isPresent() && "COMPLETED".equals(existing.get().getStatus())) {
            return existing.get().getChargeId();
        }

        // Supply the idempotency key as a node property BEFORE saving.
        // The node is now identified by idempotencyKey, not by the Neo4j internal ID.
        // The @Id field on BillingEvent should be the idempotencyKey (a String),
        // not a Neo4j-assigned Long.
        BillingEvent event = new BillingEvent(
            subscriptionId, billingPeriod, idempotencyKey, amountCents, "PENDING"
        );

        try {
            billingEventRepository.save(event);
        } catch (ServiceUnavailableException | DataAccessException e) {
            // save() may or may not have persisted. The key is known regardless.
            // The MERGE-based save (if repository is configured with MERGE semantics)
            // will be idempotent on the next retry.
        }

        ChargeResponse charge = stripeClient.createCharge(
            findCustomerId(subscriptionId),
            amountCents,
            billingPeriod,
            idempotencyKey  // same key on every attempt
        );

        billingEventRepository.setCompleted(idempotencyKey, charge.getId());

        return charge.getId();
    }
}

The entity’s @Id field should be the application-supplied idempotencyKey string, not the Neo4j internal Long. Spring Data Neo4j supports both @Id with @GeneratedValue (server-assigned) and @Id without @GeneratedValue (application-supplied). Using the application-supplied key means there is no server-assigned ID to transport over the network, and save() can use MERGE semantics on the property — creating the node if it does not exist and doing nothing if it already does. Multiple concurrent or retry save() calls for the same key converge to a single node.

Test for Mode 2

@SpringBootTest
@Testcontainers
class BillingServiceNodeIdKeyTest {

    @Container
    static Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5")
        .withAdminPassword("test");

    @Autowired
    private BillingService billingService;

    @MockBean
    private StripeClient stripeClient;

    @MockBean
    private BillingEventRepository billingEventRepository;

    @Test
    void twoSaveAttemptsResultInSameIdempotencyKey() {
        // Simulate first save() throwing ServiceUnavailableException after durable commit.
        // Second save() succeeds.
        BillingEvent savedOnFirstAttempt = new BillingEvent(
            "sub_001", "2026-10", /* idempotencyKey = */ null, 9900, "PENDING"
        );
        // On first attempt the save throws, simulating lost response:
        doThrow(new ServiceUnavailableException("No routing server"))
            .doReturn(savedOnFirstAttempt)
            .when(billingEventRepository).save(any());

        when(billingEventRepository.findByIdempotencyKey(any()))
            .thenReturn(Optional.empty());

        when(stripeClient.createCharge(any(), anyInt(), any(), any()))
            .thenReturn(new ChargeResponse("ch_A", "succeeded"))
            .thenReturn(new ChargeResponse("ch_B", "succeeded"));

        // First attempt (save throws):
        assertThatThrownBy(() ->
            billingService.chargeSubscription("sub_001", "2026-10", 9900));

        // Second attempt (save succeeds):
        billingService.chargeSubscription("sub_001", "2026-10", 9900);

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(stripeClient, atLeastOnce())
            .createCharge(any(), anyInt(), any(), keyCaptor.capture());

        // If the key is derived from stable business inputs (not from Neo4j-assigned ID),
        // all captured keys are equal.
        assertThat(keyCaptor.getAllValues()).allMatch(k -> k.equals(keyCaptor.getValue()));
    }
}

Mode 3: Reactive ReactiveNeo4jRepository with Mono.retryWhen() re-subscribes the full pipeline including Mono.fromSupplier(() -> UUID.randomUUID()) — fresh UUID_B on each retry — ch_B committed alongside ch_A

The developer uses the reactive Spring Data Neo4j API (ReactiveNeo4jRepository) and builds a reactive billing pipeline that chains a key-generation step, a Neo4j node creation step, and a Stripe call step inside a single Mono chain. The chain is retried with Mono.retryWhen() on transient Neo4j or Stripe network errors.

The developer’s mental model: “Mono.retryWhen() retries the failed operation. If the Stripe call failed, the retry restarts from the Stripe call. The key-generation step and the Neo4j save step already succeeded — they won’t be repeated.”

This is incorrect. Mono.retryWhen() works by re-subscribing to the upstream source. In Project Reactor, every operator in a Mono chain is a factory that creates a new publisher instance on each subscription. Re-subscribing to the chain means starting from the root source and re-evaluating every operator. There is no concept of “retry from the middle” or “replay already-emitted elements.” Every element in the chain runs again on every retry.

// ReactiveBillingService.java — Mono.fromSupplier UUID inside retried pipeline
@Service
public class ReactiveBillingService {

    @Autowired
    private ReactiveNeo4jTemplate neo4jTemplate;

    @Autowired
    private ReactiveStripeClient stripeClient;

    public Mono<String> chargeSubscription(String subscriptionId,
                                            String billingPeriod,
                                            int amountCents) {

        return Mono
            // Step 1: Generate the idempotency key.
            // Developer reasoning: "This runs once when the pipeline is assembled
            // and the UUID is captured by the downstream flatMaps."
            //
            // Flaw: Mono.fromSupplier is a lazy operator. It re-executes the
            // supplier on EVERY subscription, including every retryWhen re-subscription.
            // Each retry produces a different UUID.
            .fromSupplier(() -> UUID.randomUUID().toString())
            // Step 2: Save the BillingEvent node to Neo4j.
            .flatMap(idempotencyKey -> neo4jTemplate
                .save(new BillingEventNode(subscriptionId, billingPeriod, idempotencyKey))
                .map(saved -> idempotencyKey))
            // Step 3: Call Stripe with the key from Step 1.
            .flatMap(idempotencyKey -> stripeClient
                .chargeReactive(
                    subscriptionId,
                    amountCents,
                    billingPeriod,
                    idempotencyKey)
                .map(ChargeResponse::getId))
            // Retry on transient Neo4j or Stripe connectivity errors.
            // This re-subscribes to the ENTIRE pipeline from Mono.fromSupplier.
            // UUID.randomUUID() fires again on every retry attempt.
            .retryWhen(Retry.backoff(3, Duration.ofMillis(500))
                .filter(e ->
                    e instanceof ServiceUnavailableException ||
                    e instanceof StripeConnectTimeoutException));
    }
}

The failure sequence:

1. A subscriber calls chargeSubscription("sub_001", "2026-10", 9900). Reactor subscribes to the chain. Mono.fromSupplier executes: UUID.randomUUID() returns UUID_A = "uuid-A".
2. neo4jTemplate.save(new BillingEventNode(..., "uuid-A")) succeeds. A (:BillingEvent) node with idempotencyKey = "uuid-A" is persisted in Neo4j.
3. stripeClient.chargeReactive(..., "uuid-A") is called. Stripe processes the charge and durably commits ch_A. The bolt connection experiences a StripeConnectTimeoutException (or a TCP read timeout) after the Stripe server sends the 200 OK response but before the client reads it.
4. The error signal propagates upstream through the Mono chain to retryWhen. retryWhen matches StripeConnectTimeoutException against the filter, increments the retry counter, and applies the back-off delay. After the delay, it re-subscribes to the upstream Mono.
5. Re-subscription means starting from Mono.fromSupplier(() -> UUID.randomUUID()) again. The supplier executes: UUID.randomUUID() returns UUID_B = "uuid-B". A second (:BillingEvent) node with idempotencyKey = "uuid-B" is saved to Neo4j. stripeClient.chargeReactive(..., "uuid-B") is called. Stripe has no idempotency cache entry for "uuid-B". It processes the request as a new charge. ch_B is committed alongside already-committed ch_A.
6. The customer is billed twice. Neo4j contains two (:BillingEvent) nodes for the same subscription and billing period, with different idempotencyKey values.

Why Mono.retryWhen() re-subscribes the source, not just the failing operator

In Project Reactor, the retryWhen operator is implemented by creating a new subscription to the upstream when the retry signal fires. The upstream here is the entire chain assembled above retryWhen. There is no mechanism to re-subscribe to only the last operator in the chain. Reactor’s design principle is that operators are stateless factories — a Mono.fromSupplier(supplier) is a recipe that says “call supplier.get() every time this publisher is subscribed.” Re-subscribing always calls the supplier again.

This is not a defect in Reactor — it is the intended behavior and it is correct for most use cases. For a pure function pipeline where all steps are safe to re-execute, re-subscription from the root is the correct retry behavior. The defect is in combining a non-idempotent operation (UUID.randomUUID()) with a retry operator that assumes the pipeline is safe to re-execute. The UUID generation is not safe to re-execute because the first execution’s output was used to call Stripe and that Stripe call may have committed.

The fix is to move the key generation outside the retried portion of the pipeline. The key should be a constant value with respect to the pipeline — a Mono.just(stableKey) that resolves to the same value on every subscription. The stable key is derived before the pipeline is assembled.

// Fixed ReactiveBillingService.java — key derived outside the retried pipeline
@Service
public class ReactiveBillingService {

    @Autowired
    private ReactiveNeo4jTemplate neo4jTemplate;

    @Autowired
    private ReactiveStripeClient stripeClient;

    public Mono<String> chargeSubscription(String subscriptionId,
                                            String billingPeriod,
                                            int amountCents) {

        // Key derived OUTSIDE the pipeline, before Mono assembly.
        // Mono.just() wraps a constant value — re-subscription returns the same value.
        // No UUID generation on retry.
        String idempotencyKey = toStableKey(subscriptionId, billingPeriod, amountCents);

        return Mono
            .just(idempotencyKey)   // constant — same key on every re-subscription
            .flatMap(key -> neo4jTemplate
                .save(new BillingEventNode(subscriptionId, billingPeriod, key))
                .map(saved -> key))
            .flatMap(key -> stripeClient
                .chargeReactive(subscriptionId, amountCents, billingPeriod, key)
                .map(ChargeResponse::getId))
            .retryWhen(Retry.backoff(3, Duration.ofMillis(500))
                .filter(e ->
                    e instanceof ServiceUnavailableException ||
                    e instanceof StripeConnectTimeoutException));
    }

    static String toStableKey(String subscriptionId, String billingPeriod, int amountCents) {
        try {
            MessageDigest sha = MessageDigest.getInstance("SHA-256");
            byte[] hash = sha.digest(
                (subscriptionId + ":" + billingPeriod + ":" + amountCents)
                    .getBytes(StandardCharsets.UTF_8));
            return HexFormat.of().formatHex(hash).substring(0, 36);
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

An alternative fix that keeps key generation inside the pipeline is to use Mono.defer(() -> Mono.just(toStableKey(...))) instead of Mono.fromSupplier(() -> UUID.randomUUID()). Mono.defer defers pipeline assembly to subscription time, but if the deferred factory returns Mono.just(stableKey) where stableKey is derived from stable business inputs, every subscription returns the same value. This has the same correctness property as hoisting the key computation above the pipeline — the key is stable because its inputs are stable, not because it is a captured constant.

The important invariant: any value that feeds into the Idempotency-Key header sent to Stripe must resolve to the same string on every re-subscription of the reactive pipeline. Values derived from UUID.randomUUID(), from System.currentTimeMillis(), from a random nonce, or from any non-deterministic source violate this invariant and produce a new Stripe charge on every retry.

The same failure mode with Flux.retryWhen() on a batch billing pipeline

The same re-subscription behavior applies to Flux.retryWhen() in a batch billing pipeline. If a developer processes a list of subscriptions as a Flux and retries the entire flux on a transient error, all key-generation steps for all subscriptions in the flux are re-executed on every retry, not just the step for the subscription that failed. This produces a new idempotency key for every subscription in the batch on every retry attempt, regardless of which subscription triggered the retry.

// Buggy batch billing — Flux.retryWhen() re-executes UUID generation for all items
public Flux<String> chargeAllPendingSubscriptions(List<String> subscriptionIds,
                                                   String billingPeriod,
                                                   int amountCents) {
    return Flux.fromIterable(subscriptionIds)
        .flatMap(subId -> Mono
            .fromSupplier(() -> UUID.randomUUID().toString())  // re-runs for ALL subIds on retry
            .flatMap(key -> chargeOne(subId, billingPeriod, amountCents, key)))
        .retryWhen(Retry.backoff(3, Duration.ofMillis(500)));  // re-subscribes the entire Flux
}

The fix is to derive all keys before the Flux pipeline is assembled, using a Map<String, String> from subscription ID to stable key:

// Fixed batch billing — keys computed outside the pipeline
public Flux<String> chargeAllPendingSubscriptions(List<String> subscriptionIds,
                                                   String billingPeriod,
                                                   int amountCents) {
    // All keys computed before pipeline assembly. Stable across re-subscriptions.
    Map<String, String> keys = subscriptionIds.stream()
        .collect(Collectors.toMap(
            id -> id,
            id -> toStableKey(id, billingPeriod, amountCents)
        ));

    return Flux.fromIterable(subscriptionIds)
        .flatMap(subId -> chargeOne(subId, billingPeriod, amountCents, keys.get(subId)))
        .retryWhen(Retry.backoff(3, Duration.ofMillis(500)));
}

Test for Mode 3

@ExtendWith(MockitoExtension.class)
class ReactiveBillingServiceRetryTest {

    @Mock
    private ReactiveNeo4jTemplate neo4jTemplate;

    @Mock
    private ReactiveStripeClient stripeClient;

    @InjectMocks
    private ReactiveBillingService billingService;

    @Test
    void retryWhenSendsSameIdempotencyKeyOnEachAttempt() {
        BillingEventNode savedNode = new BillingEventNode("sub_001", "2026-10", "any-key");

        when(neo4jTemplate.save(any(BillingEventNode.class)))
            .thenReturn(Mono.just(savedNode));

        // First attempt: Stripe commits, then connectivity error.
        // Second attempt: succeeds.
        when(stripeClient.chargeReactive(any(), anyInt(), any(), any()))
            .thenReturn(Mono.error(new StripeConnectTimeoutException("timeout")))
            .thenReturn(Mono.just(new ChargeResponse("ch_A", "succeeded")));

        StepVerifier.create(billingService.chargeSubscription("sub_001", "2026-10", 9900))
            .expectNextMatches(id -> id.equals("ch_A"))
            .verifyComplete();

        ArgumentCaptor<String> keyCaptor = ArgumentCaptor.forClass(String.class);
        verify(stripeClient, times(2))
            .chargeReactive(any(), anyInt(), any(), keyCaptor.capture());

        List<String> keys = keyCaptor.getAllValues();
        // Both Stripe calls must have used the same idempotency key.
        // If keys differ, the retry produced a new charge from Stripe's perspective.
        assertThat(keys.get(0)).isEqualTo(keys.get(1));
    }
}

Neo4j 5.x element ID vs. legacy internal ID: the failure mode is identical

Neo4j 5.x deprecates the id() Cypher function in favor of elementId(), which returns a string in the format "4:<database-uuid>:<node-id>" rather than a plain Long. Spring Data Neo4j 7.x reflects this change: entity fields annotated with @Id @GeneratedValue may be of type String (element ID) rather than Long (internal ID). The Mode 2 failure mode applies identically to both representations. The element ID is still assigned server-side, transported over the bolt connection, and subject to loss if the bolt response is not received by the application. The format of the identifier changes; the correctness risk does not.

Teams migrating from Neo4j 4.x to Neo4j 5.x (and from Spring Data Neo4j 6.x to 7.x) face a specific transition risk: code that used entity.getId() returning a Long must be changed to use entity.getElementId() returning a String. If the idempotency key derivation used the Long form and the migration changes the field type without reviewing the key derivation logic, the key format changes across the migration boundary. Any pending billing events with keys derived from the old Long format will not match keys derived from the new String element ID format — they are different strings, and Stripe treats them as different charges. This is a third-generation of the same failure mode that activates specifically during the Neo4j 5.x migration.

The fix for the migration scenario is the same as for the base case: derive keys from stable application-layer business inputs (subscription ID, billing period, amount) that have no dependency on Neo4j’s internal identifier format. Application-layer inputs do not change across a Neo4j version migration. The key is invariant across Neo4j 4.x and 5.x deployments.

Summary table

Quick diagnostic checklist for Spring Data Neo4j + Stripe deployments

1. Audit idempotency key derivation for transaction-scoped state. Search for patterns where UUID.randomUUID() is called inside a @Transactional method and the UUID is stored as a Neo4j node property. If the transaction can roll back after the Stripe call (any exception after stripeClient.create*() returns), the UUID is lost. Replace with key derivation from inputs that exist before the transaction opens.
2. Audit @Id field type on billing-related entities. If any entity used in the billing flow has @Id @GeneratedValue with a Long or String field whose value is assigned by Neo4j (rather than by the application), and if that ID feeds into a Stripe idempotency key, the billing code is vulnerable to Mode 2. Switch to application-supplied @Id fields (a deterministic UUID string derived from business inputs) for entities used in billing flows.
3. Grep for Mono.fromSupplier or Mono.fromCallable with UUID or random generation inside retried pipelines. Any fromSupplier(() -> UUID.randomUUID()), fromCallable(() -> generateKey()), or equivalent that feeds into a downstream Stripe call, when wrapped by retryWhen, is vulnerable to Mode 3. Move the key computation above the pipeline.
4. Check Flux.retryWhen() on batch billing pipelines. Batch billing is more likely to combine Flux.retryWhen() with per-item key generation inside the flux. A single transient error on one item causes the entire flux to re-subscribe, re-generating keys for all items. Compute all keys before the Flux is assembled.
5. Test Neo4j constraint violation scenarios explicitly. Unique constraint violations in Neo4j can trigger transaction rollbacks at unexpected points (e.g., on the second save() in a method that calls save() twice). Test billing methods by injecting ConstraintViolationException at each save() call and asserting that the Stripe idempotency key (captured via WireMock) is the same value as in a non-exception path.
6. Verify behavior during Neo4j cluster leader election. ServiceUnavailableException is the common exception during Neo4j cluster topology changes. Billing code should be tested against this exception at the save() call site, the findBy* query site, and the markCompleted() call site. The key must remain stable in all three cases.
7. Check the Neo4j 5.x migration path if upgrading from Neo4j 4.x. If @Id @GeneratedValue fields changed type from Long to String (element ID format), audit any idempotency key derivation that uses those fields. Keys derived from the old Long format and keys derived from the new String format are different strings. Pending billing events recorded before the migration have keys in the old format; post-migration retries derive keys in the new format; Stripe sees them as different charges.
8. Use WireMock to capture and assert Idempotency-Key headers across all retry paths. An integration test that exercises the full billing method (including Neo4j writes via Testcontainers and Stripe via WireMock) and asserts key stability across a simulated ConstraintViolationException, a simulated ServiceUnavailableException, and a simulated Stripe timeout is the only test that catches all three failure modes simultaneously.

Applying MERGE semantics in Neo4j Cypher as a defense in depth

Spring Data Neo4j’s save() method on a repository with an application-supplied @Id (not @GeneratedValue) uses Cypher MERGE rather than CREATE under the hood. MERGE matches an existing node by the @Id property and creates it only if it does not exist. This makes the write operation idempotent at the Neo4j level: calling save(billingEvent) twice with the same idempotencyKey produces a single node.

This property is valuable as a defense in depth but is not a substitute for stable key derivation. If the @Id is a random UUID (the application-supplied version of a non-deterministic key), MERGE still creates duplicate nodes on retries because the key is different on each attempt — there is no matching node for MERGE to find. MERGE is only idempotent if the property being merged on is a stable, deterministic value. When the idempotency key is derived from stable business inputs (subscription ID, billing period, amount), MERGE on that key correctly deduplicates concurrent and retry writes at the Neo4j layer, complementing Stripe’s deduplication at the payment layer.

The recommended pattern for billing event nodes in Spring Data Neo4j:

1. Use @Id (without @GeneratedValue) with a String field populated by the application from stable business inputs.
2. Ensure a unique constraint exists on the idempotencyKey property for the BillingEvent label: CREATE CONSTRAINT FOR (n:BillingEvent) REQUIRE n.idempotencyKey IS UNIQUE.
3. Use repository.save(event) with the stable key as the @Id; Spring Data Neo4j will use MERGE ON CREATE SET / ON MATCH SET under the hood, making concurrent writes safe.
4. Use a separate query (findByIdempotencyKey(key)) as the guard, with immediate consistency (Neo4j property index lookups are consistent within the transaction, not near-real-time like Elasticsearch).

Conclusion

Spring Data Neo4j introduces three behavioral properties that independently cause Stripe double-charges when developers couple idempotency key derivation to Neo4j-specific state. A @Transactional boundary that wraps both a Neo4j write and a Stripe call will roll back the Neo4j write on any exception after the Stripe commit, discarding the key that was stored in the transaction and leaving recovery to generate a new key. A Neo4j-assigned internal node ID used as a key component is transported over the bolt connection and can be lost to a ServiceUnavailableException thrown after the server durably commits the write; the retry creates a second node with a different ID and a different key. A reactive pipeline that generates a UUID inside Mono.fromSupplier and retries with Mono.retryWhen() re-evaluates the supplier on each re-subscription, producing a different UUID and a new Stripe charge on every retry attempt.

The common root cause across all three modes is generating or transporting key material at a point in the billing flow that is either discarded by a rollback, lost by a network failure, or re-evaluated by a retry mechanism. Idempotency keys must be generated from application-side stable business inputs — values that exist before any network call, survive any transaction rollback, and are invariant to any runtime infrastructure event. SHA-256(subscriptionId + billingPeriod + amountCents) satisfies all three properties: it is computed locally, survives rollbacks (it was never inside a transaction to begin with), and is a pure function of inputs that do not change between the original attempt and any retry. The stable key can then be used safely with Neo4j MERGE semantics, Stripe’s idempotency cache, and Project Reactor’s re-subscription model.