Spring Data Redis and Stripe Integration: How a Redis Idempotency Key TTL Expiry Between the Stripe Commit and a Retry Billing Job Generates UUID_B Producing ch_B, a setIfAbsent Connection Timeout Returns null Hiding Whether the Guard Was Stored After ch_A Commits, and a Redis Sentinel Failover Promotes a Replica Missing the Guard Key Causing the New Primary to Return Not Found After ch_A Commits

This post is part of the ongoing series on Stripe idempotency failure modes in JVM frameworks. Earlier entries covered Spring Data JPA (JDBC transaction boundary and Stripe commit ordering), Spring Data Cassandra (WriteTimeoutException treated as write failure, LWT IF NOT EXISTS timeout, ConsistencyLevel.ONE guard read after LOCAL_QUORUM write), Spring Cloud Config (@RefreshScope bean destruction clears in-flight key cache), Spring Cloud Bus (busrefresh cluster-wide broadcast), and Spring Cloud Kubernetes (ConfigMap Watch cluster-wide @RefreshScope refresh, Fabric8 GC pause dual-active leader, rolling deployment key-formula split-brain). This post covers Spring Data Redis. Where a relational database stores idempotency keys in a table with no automatic expiry, Redis keys have a configurable TTL that can silently evict a guard key before the retry window closes. Where a database write either commits or rolls back with a clear outcome observable from the calling thread, a Redis SET NX under a connection failure returns an ambiguous null that does not distinguish between “the command never reached Redis” and “the command executed but the acknowledgment was lost.” And where a relational primary failover with synchronous replication commits only when the replica has the write, Redis Sentinel failover uses asynchronous replication by default, creating a window during which a write acknowledged by the primary may not yet exist on the replica that will become the new primary after failover.

The three failure modes are: a billing service that stores idempotency keys in Redis with a TTL chosen for cache hygiene, where the retry window of the billing job scheduler — a @Scheduled task or a message queue with dead-letter redeliver delay — can exceed the key TTL by hours or days, so that when a retry fires after TTL expiry Redis returns nil, the billing code generates UUID_B, and Stripe commits ch_B alongside the already-committed ch_A; a developer who uses RedisTemplate.opsForValue().setIfAbsent(key, value, duration) as an atomic check-and-set guard, where a transient network partition between the Spring Boot pod and the Redis primary causes the TCP connection to drop after the SET NX EX command bytes are transmitted but before the Redis acknowledgment is received, so that Spring Data Redis throws RedisConnectionFailureException or returns null, and the developer’s guard logic interprets null as “key not present, safe to proceed” when in fact the key may have been set on the Redis side before the connection dropped; and a production deployment that uses Redis Sentinel for high availability, where the primary fails, Sentinel promotes the least-lagged replica, but that replica had a replication offset 80 ms behind the primary at the moment of failure, so that the idempotency key written by the first billing attempt was not yet replicated, and after failover the new primary responds with nil on a GET, causing a billing rerun to generate UUID_B and commit ch_B.

Stack for examples: Spring Boot 3.x, Spring Data Redis 3.x (spring-boot-starter-data-redis), Lettuce 6.x (the default Spring Boot Redis client), Redis 7.x (standalone, Sentinel, and Cluster), Spring Retry (spring-retry), Stripe Java SDK v24+, Testcontainers (org.testcontainers:redis), WireMock 3.x, JUnit 5.

Background: what Spring Data Redis adds and why it interacts with billing idempotency differently from a relational database

Spring Data Redis wraps the Redis client (Lettuce or Jedis) in a template API and provides repository abstractions for Redis data structures. For billing idempotency purposes, developers commonly use Redis because of three properties that seem well-suited to the problem: high throughput key lookups, atomic SET NX EX for check-and-set without a separate lock, and TTL-based automatic cleanup without a background job.

Each of these properties has a counterpart failure mode when applied to Stripe idempotency.

The first is TTL as a semantic guarantee vs. TTL as operational hygiene. Developers set TTLs on Redis idempotency keys because “we don’t want stale keys to build up forever.” This is operationally reasonable. But a TTL set for cache hygiene reasons is not the same as a TTL that accurately models the retry window of every consumer of that key. Stripe’s own idempotency key documentation specifies that keys are deduplicated for 24 hours after the first request. A billing job that retries failed charges over a 48-hour window — common in subscription billing to handle card declines across multiple bank processing cycles — must have its Redis key TTL set to at least 48 hours, not 1 hour or 24 hours. The mismatch between key TTL and retry window is the core of Mode 1.

The second is SET NX command delivery ambiguity under connection failure. A relational database transaction is either committed (and the commit acknowledgment is received by the calling thread) or not committed (and the connection failure results in a rollback). When Spring Data JPA catches DataAccessException from a failed transaction, the developer knows the write did not happen. Redis does not offer this guarantee. SET NX EX is a single command, not a transaction, and TCP is not a reliable request-response protocol under connection failure: it is possible for the command bytes to be delivered to the Redis server, for the command to execute (setting the key), and for the TCP connection to drop before the server’s +OK or $-1 response reaches the client. Spring Data Redis then throws RedisConnectionFailureException (or returns null depending on the error path and template configuration). The developer who treats this as a definitive “key was not set” signal will regenerate the idempotency key and call Stripe again. This is Mode 2.

The third is Redis Sentinel asynchronous replication and failover data loss. Redis Sentinel provides automatic failover with a configurable quorum. The default Redis replication mode is asynchronous: the primary writes data to its local AOF/RDB and acknowledges the write to the client before waiting for replica acknowledgment. Replicas receive the write stream asynchronously. When a primary fails, Sentinel selects the replica with the highest replication offset to minimize data loss, but that replica may still be seconds behind the primary’s last acknowledged writes. Any idempotency key written to the primary in that replication lag window is invisible on the new primary after failover. This is Mode 3.

Mode 1: Redis idempotency key TTL expires between ch_A commit and retry billing job — retry generates UUID_B, Stripe commits ch_B

Setup: the billing service

The developer builds a subscription billing service that stores idempotency keys in Redis. The chosen TTL is 24 hours, matching Stripe’s deduplication window documentation. The billing logic is:

@Service
public class BillingService {

    private final RedisTemplate<String, String> redis;
    private final StripeClient stripe;

    public String chargeSubscription(String subscriptionId,
                                     String stripeCustomerId,
                                     long amountCents) {

        String key = "idem:" + subscriptionId + ":current";
        String idempotencyKey = redis.opsForValue().get(key);

        if (idempotencyKey == null) {
            idempotencyKey = UUID.randomUUID().toString();
            // TTL: 24 hours — matches Stripe's deduplication window
            redis.opsForValue().set(key, idempotencyKey, Duration.ofHours(24));
        }

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(amountCents)
            .setCurrency("usd")
            .setCustomer(stripeCustomerId)
            .setConfirm(true)
            .build();

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        return pi.getId();
    }
}

The billing pipeline is driven by a @Scheduled cron that runs the monthly billing job for all subscriptions in the PENDING state. A subscription moves to FAILED if it encounters a transient Stripe error (network timeout, HTTP 500), and a separate recovery job rescheduled by an exponential backoff runner retries FAILED subscriptions. The retry policy is designed around card-decline patterns: many bank-issued cards that decline on the 1st of the month succeed on the 3rd after the cardholder’s paycheck clears. The retry window is therefore configured to 72 hours.

The failure path

At t=0, the billing job runs for subscription sub_abc. Redis has no key for idem:sub_abc:current. The service generates UUID_A and stores it in Redis with a 24-hour TTL. The Stripe call succeeds: ch_A is committed with idempotencyKey=UUID_A. The billing job marks the subscription SUCCEEDED.

Then, at t+2h, an operator discovers that a data pipeline miscalculation caused a batch of subscriptions — including sub_abc — to be billed for the wrong amount. The operator resets the affected subscriptions to PENDING for a corrected re-billing run.

The developer’s intent is: “if we re-run billing for this subscription within 24 hours, the Redis key will still be present, chargeSubscription will reuse UUID_A, and Stripe will return the cached response for ch_A rather than creating a new charge.” This reasoning is correct for the 24-hour window.

However, the corrected billing run is scheduled for t+26h — after the 24-hour overnight audit period that the operator’s process requires before re-billing corrected amounts. At t+26h, Redis has evicted the key for idem:sub_abc:current. The billing service generates UUID_B and stores it with a new 24-hour TTL. The Stripe call creates a new charge: ch_B. The customer is billed twice — once with the incorrect amount (ch_A) and once with the corrected amount (ch_B).

Developer misconception

The developer’s model is: “I set the TTL to 24 hours, which matches Stripe’s idempotency deduplication window, so I am safe.”

The flaw is that Stripe’s 24-hour deduplication window is a property of Stripe’s own key cache: if you call Stripe twice with the same key within 24 hours, Stripe deduplicates the second call. But the Stripe deduplication guarantee is conditional on the client sending the same key both times. If the Redis TTL expires between calls, the client sends a different key on the retry, and Stripe has no way to know that the second call represents the same billing intent as the first.

The correct TTL is not Stripe’s deduplication window. The correct TTL is the maximum retry horizon of every consumer of that idempotency key — the longest time that can pass between the first billing attempt and a legitimate retry of the same billing intent. For a billing pipeline with a 72-hour retry window, the TTL must be at least 72 hours. For a billing system with a manual re-run pathway that an operator might trigger days later, the TTL may need to be 7 days or more.

The fix: stable key derivation independent of Redis TTL

The root cause is that the idempotency key is generated once, stored in Redis with a TTL, and retrieved on retry. This design makes the idempotency guarantee dependent on Redis liveness and TTL configuration. The fix is to derive the idempotency key deterministically from the billing intent itself — inputs that are stable across retries — so that Redis is not on the critical path:

@Service
public class BillingService {

    private final StripeClient stripe;

    public String chargeSubscription(String subscriptionId,
                                     String billingPeriod,   // e.g. "2026-10"
                                     String stripeCustomerId,
                                     long amountCents) {

        // Stable key derived from billing inputs — no Redis TTL dependency
        String raw = subscriptionId + ":" + billingPeriod + ":" + amountCents;
        String idempotencyKey = "kb_" + sha256Hex(raw);

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(amountCents)
            .setCurrency("usd")
            .setCustomer(stripeCustomerId)
            .setConfirm(true)
            .build();

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        return pi.getId();
    }

    private static String sha256Hex(String input) {
        try {
            MessageDigest md = MessageDigest.getInstance("SHA-256");
            byte[] hash = md.digest(input.getBytes(StandardCharsets.UTF_8));
            StringBuilder sb = new StringBuilder(64);
            for (byte b : hash) sb.append(String.format("%02x", b));
            return sb.toString();
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }
}

When the corrected re-billing run fires at t+26h, the stable key derived from subscriptionId + "2026-10" + amountCents_corrected will differ from the original key only if the amount changed. If the intent is genuinely to charge a different amount, a different key is correct: the corrected charge is a new billing event, not a retry of the incorrect one. If the intent is to retry the exact same charge (network failure, not data error), the key will be identical on retry, and Stripe will deduplicate within its own 24-hour window without any Redis dependency.

Redis may still be used as a short-circuit cache to avoid re-deriving and re-calling Stripe for recently committed charges. But that cache must be explicitly separated from the idempotency mechanism: a Redis cache miss means “derive the key deterministically and call Stripe,” not “generate a new UUID and call Stripe.”

Testing for TTL-driven key rotation

@Test
void ttl_expiry_does_not_produce_second_stripe_charge() {
    // Pre-condition: first charge committed with stable key
    String subscriptionId = "sub_test_001";
    String billingPeriod   = "2026-10";
    long   amountCents     = 2999L;

    String firstKey = billingService.deriveKey(subscriptionId, billingPeriod, amountCents);
    wireMock.stubFor(post(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(firstKey))
        .willReturn(okJson("{\"id\":\"pi_A\",\"status\":\"succeeded\"}")));

    billingService.chargeSubscription(subscriptionId, billingPeriod, "cus_x", amountCents);

    // Simulate TTL expiry by flushing Redis
    redisTemplate.delete("idem:" + subscriptionId + ":current");

    // Second call (retry after TTL expiry) must use the same key
    billingService.chargeSubscription(subscriptionId, billingPeriod, "cus_x", amountCents);

    // Stripe must have been called exactly once (second call returns cached pi_A)
    wireMock.verify(1, postRequestedFor(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(firstKey)));
}

Mode 2: setIfAbsent connection timeout returns null — command delivery to Redis is ambiguous — UUID_B generated after ch_A commits

Setup: atomic guard with setIfAbsent

The developer wants an atomic check-and-set guard that prevents two concurrent threads from generating different idempotency keys for the same subscription. They use RedisTemplate.opsForValue().setIfAbsent(key, value, duration), which maps to SET key value NX EX seconds in Redis — a single atomic command that sets the key only if it does not already exist:

@Service
public class BillingService {

    private final RedisTemplate<String, String> redis;
    private final StripeClient stripe;

    public String chargeSubscription(String subscriptionId,
                                     String stripeCustomerId,
                                     long amountCents) {

        String key   = "idem:" + subscriptionId + ":lock";
        String newKey = UUID.randomUUID().toString();

        // Atomic: set only if absent; returns true if set, false if already existed
        Boolean wasSet = redis.opsForValue()
            .setIfAbsent(key, newKey, Duration.ofHours(48));

        String idempotencyKey;
        if (Boolean.TRUE.equals(wasSet)) {
            // We set the key — use this UUID
            idempotencyKey = newKey;
        } else {
            // Key already existed — retrieve the existing UUID
            idempotencyKey = redis.opsForValue().get(key);
            if (idempotencyKey == null) {
                // Defensive: key expired between setIfAbsent and get
                idempotencyKey = newKey;
            }
        }

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(amountCents)
            .setCurrency("usd")
            .setCustomer(stripeCustomerId)
            .setConfirm(true)
            .build();

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        return pi.getId();
    }
}

This looks correct: if two threads call chargeSubscription concurrently, only one will receive Boolean.TRUE from setIfAbsent; the other will receive Boolean.FALSE and retrieve the existing key. The charge will be submitted with a consistent idempotency key regardless of which thread executes first.

The failure path

The failure occurs not from concurrent threads but from a network event between the billing pod and the Redis primary.

At t=0, thread T1 calls chargeSubscription for subscription sub_xyz. T1 generates UUID_A as the candidate key. T1 sends SET idem:sub_xyz:lock UUID_A NX EX 172800 to Redis over a Lettuce connection from the connection pool.

The Redis server receives the command, evaluates NX (the key does not exist), sets the key, and prepares the +OK response. At this exact moment, a transient network event — a cloud provider maintenance operation, a brief NIC reset, a TCP packet drop that exceeds the Lettuce socket timeout — drops the TCP connection before the +OK bytes are delivered to T1.

Lettuce detects the broken connection. Depending on whether the command was acknowledged at the TCP level before the connection dropped, Lettuce may either:

• Throw RedisConnectionFailureException wrapping a java.net.SocketException, or
• Return null from setIfAbsent because the response decode phase threw an exception that was swallowed by the template’s error handling.

In the code above, the developer’s guard handles Boolean.TRUE (set) and Boolean.FALSE (already existed). Neither case catches null (ambiguous). The Boolean.TRUE.equals(wasSet) check returns false for null, routing execution to the else branch. The else branch calls redis.opsForValue().get(key).

But here is the key detail: the TCP connection dropped, and Lettuce has already reconnected to Redis using its auto-reconnect logic. The GET on the reconnected connection returns UUID_A — the key that was actually set on the Redis side before the connection dropped. The developer’s code retrieves UUID_A and calls Stripe with it. ch_A commits. So far, no double charge.

However, the developer has a Spring Retry annotation on the method:

@Retryable(retryFor = RedisConnectionFailureException.class,
           maxAttempts = 3,
           backoff = @Backoff(delay = 500, multiplier = 2))
public String chargeSubscription(...) { ... }

Spring Retry intercepts the RedisConnectionFailureException thrown by setIfAbsent before the developer’s null-check code runs. The retry re-invokes the method from the top. On the retry invocation, setIfAbsent returns Boolean.FALSE because the key already exists in Redis (it was set on the first attempt). The else branch retrieves UUID_A. Stripe is called with UUID_A, which deduplicates against the already-committed ch_A. No double charge.

Except: there is a race condition. If the Redis TTL on the key is shorter than the retry backoff total (which it is not in this example, but which is a common configuration error), the key may expire between the first attempt and the retry. More critically, if the developer’s error handling catches RedisConnectionFailureException and returns a fallback UUID rather than rethrowing, the retry mechanism is bypassed entirely.

The more direct failure path is the following specific variant of the code: the developer catches RedisConnectionFailureException and treats it as a non-fatal Redis outage, falling back to UUID generation:

Boolean wasSet;
try {
    wasSet = redis.opsForValue()
        .setIfAbsent(key, newKey, Duration.ofHours(48));
} catch (RedisConnectionFailureException e) {
    // Redis is down — fall back to UUID, accept the risk
    log.warn("Redis unavailable for idempotency guard — proceeding with new key");
    wasSet = true;  // treat as if we just set the key
}

In this pattern, the developer’s fallback logic generates UUID_A (the newKey variable), calls Stripe, and ch_A commits. The key was actually set in Redis as UUID_A before the connection dropped. A retry of the billing job (triggered by a monitoring system that sees the RedisConnectionFailureException in logs and marks the subscription for re-processing) calls chargeSubscription again. This time, Redis is available. setIfAbsent returns Boolean.FALSE. The existing key in Redis is UUID_A. Stripe deduplicates. No double charge.

But now consider a slightly different scenario: the connection drops, the Redis command was not yet fully transmitted (the TCP buffer was flushed but the full command was not received by the Redis server), and Redis did not set the key. setIfAbsent throws RedisConnectionFailureException. The developer’s fallback sets wasSet = true and calls Stripe with UUID_A. ch_A commits. A billing job retry later calls setIfAbsent again. Redis returns Boolean.TRUE (key was not set, because the original command never reached Redis). The retry is treated as the first attempt. The retry generates UUID_B (a new UUID.randomUUID() in the local variable newKey). Stripe commits ch_B alongside ch_A.

Developer misconception

The developer’s model is: “If setIfAbsent throws a connection exception, Redis was unavailable and therefore the key was not set.”

This is the TCP delivery ambiguity problem. A RedisConnectionFailureException arising from a connection drop after the command bytes were sent does not distinguish between “Redis received the command and set the key (but dropped the connection before replying)” and “Redis never received the command (the bytes did not arrive before the connection dropped).” Both cases produce the same exception from the Spring Data Redis perspective. The developer cannot determine from the exception alone which case occurred.

The correct interpretation of a RedisConnectionFailureException from setIfAbsent is: the result of the SET NX EX command is unknown. The safe response is to treat this as “key may already be set with some value unknown to me” and to not proceed with a new UUID. Either abort the billing attempt (leaving the subscription for a retry that will see the key if it was set, or regenerate and set if it was not), or attempt a GET on the key to observe what Redis actually has.

The fix: read-back on connection failure before generating a new key

@Service
public class BillingService {

    private final RedisTemplate<String, String> redis;
    private final StripeClient stripe;

    public String chargeSubscription(String subscriptionId,
                                     String stripeCustomerId,
                                     long amountCents) {

        String key      = "idem:" + subscriptionId + ":lock";
        String candidate = UUID.randomUUID().toString();

        String idempotencyKey = resolveIdempotencyKey(key, candidate);

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(amountCents)
            .setCurrency("usd")
            .setCustomer(stripeCustomerId)
            .setConfirm(true)
            .build();

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        return pi.getId();
    }

    private String resolveIdempotencyKey(String key, String candidate) {
        try {
            Boolean wasSet = redis.opsForValue()
                .setIfAbsent(key, candidate, Duration.ofHours(48));

            if (Boolean.TRUE.equals(wasSet)) {
                return candidate;
            }
            // FALSE: key existed — retrieve it
            String existing = redis.opsForValue().get(key);
            // existing should not be null here (key has a long TTL);
            // if it is null (race with expiry), use candidate as a new SET was done
            return existing != null ? existing : candidate;

        } catch (RedisConnectionFailureException e) {
            // Unknown NX result — attempt read-back on fresh connection
            log.warn("setIfAbsent connection failure — attempting read-back for key {}", key);
            try {
                String existing = redis.opsForValue().get(key);
                if (existing != null) {
                    // Key was set before the connection dropped — reuse it
                    return existing;
                }
                // Key was not set (command did not reach Redis) — set with candidate
                redis.opsForValue().set(key, candidate, Duration.ofHours(48));
                return candidate;
            } catch (RedisConnectionFailureException e2) {
                // Redis is genuinely down — use stable derived key as fallback
                log.error("Redis unavailable — falling back to derived key for {}", key);
                throw new BillingRedisUnavailableException(
                    "Cannot resolve idempotency key — Redis unavailable", e2);
            }
        }
    }
}

The read-back on a fresh connection resolves the ambiguity: if the key is present in Redis, the SET NX EX command reached the server and the key was set; reuse that value. If the key is absent, the command did not reach the server; proceed with a new SET. The only remaining failure case is if the read-back connection also fails, which indicates a genuine Redis outage rather than a transient connection drop. In that case, the correct action is to throw a retryable exception — not to silently proceed with a new UUID.

Better approach: stable key derivation eliminates the race entirely

The read-back approach handles the connection failure case but adds complexity and an extra Redis round-trip. The cleaner fix is the same as for Mode 1: derive the idempotency key deterministically from billing inputs. If the key is always SHA-256(subscriptionId:billingPeriod:amountCents), the developer does not need to store it in Redis at all. Concurrent threads and retry attempts naturally converge on the same key without any coordination primitives. Redis can still be used as a performance cache (to avoid re-hashing on every call), but a Redis miss means “derive and call Stripe,” never “generate new UUID and call Stripe.”

Testing for setIfAbsent ambiguity

@Test
void setIfAbsent_connection_failure_does_not_produce_second_stripe_charge() {
    // Simulate: setIfAbsent throws on first call, key was actually set in Redis
    String subscriptionId = "sub_test_002";
    String key = "idem:" + subscriptionId + ":lock";

    // Pre-populate Redis with UUID_A (simulating command reached Redis before drop)
    String uuidA = UUID.randomUUID().toString();
    redisTemplate.opsForValue().set(key, uuidA, Duration.ofHours(48));

    // Stub Stripe to succeed for UUID_A
    wireMock.stubFor(post(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(uuidA))
        .willReturn(okJson("{\"id\":\"pi_A\",\"status\":\"succeeded\"}")));

    // Now call — setIfAbsent will return FALSE (key exists), code retrieves UUID_A
    String result = billingService.chargeSubscription(subscriptionId, "cus_y", 999L);

    assertThat(result).isEqualTo("pi_A");
    // Stripe must have been called exactly once with UUID_A
    wireMock.verify(1, postRequestedFor(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(uuidA)));
}

Mode 3: Redis Sentinel failover promotes replica with replication lag — guard key missing on new primary after ch_A commits

Setup: Redis Sentinel for high availability

The developer deploys Redis with Sentinel for high availability: one primary, two replicas, three Sentinel processes. Lettuce is configured as the Spring Data Redis client with RedisSentinelConfiguration. Lettuce transparently reconnects to the new primary after Sentinel announces a failover. From the developer’s perspective, Redis Sentinel provides HA with no application code changes.

The billing service uses Redis to store the idempotency key for each active billing attempt. The key is written before the Stripe call (to prevent concurrent duplicate attempts) and is read on retry to reuse the same idempotency key:

@Service
public class BillingService {

    private final StringRedisTemplate redis;
    private final StripeClient stripe;

    @Transactional
    public String chargeSubscription(BillingJob job) {
        String key = "idem:" + job.getId();

        // Write the key before the Stripe call (prevents concurrent duplicates)
        redis.opsForValue().setIfAbsent(
            key,
            job.getIdempotencyKey(),
            Duration.ofHours(72)
        );

        String idempotencyKey = redis.opsForValue().get(key);

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(job.getAmountCents())
            .setCurrency("usd")
            .setCustomer(job.getStripeCustomerId())
            .setConfirm(true)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        job.setChargeId(pi.getId());
        job.setStatus(BillingJobStatus.SUCCEEDED);
        return pi.getId();
    }
}

The idempotency key is stored in the BillingJob entity and pre-generated when the job is created. This means the same key is always associated with a given billing job, and the developer correctly separates key generation from key storage. The Redis key is a convenience store to prevent concurrent execution, not the source of the idempotency key value.

The failure path

The billing job for customer cus_sentinel_test with job ID job_999 runs at t=0. The service calls redis.opsForValue().setIfAbsent("idem:job_999", "key_A", 72h). The command is sent to and acknowledged by the Redis primary. The key is now stored in the primary’s memory. The replication stream is propagated asynchronously to replicas R1 and R2.

At t=0 + 80ms, the Stripe HTTP call is in flight. The Redis primary experiences a hardware failure (OOM kill from the OS, or a cloud hypervisor maintenance event that pauses and then terminates the VM). The primary disappears from the network.

Redis Sentinel processes detect the primary failure after the down-after-milliseconds timeout (default: 30 seconds; often tuned to 5–10 seconds in production to reduce failover latency). After quorum (at least 2 of 3 Sentinel processes agree the primary is down), Sentinel promotes the replica with the highest replication offset. In this deployment, that is R1.

R1’s replication offset at the time of primary failure was 80 ms behind the primary. The write of idem:job_999 = key_A was part of those 80 ms of unreplicated writes. R1 was promoted without those writes. From the new primary’s perspective, the key idem:job_999 does not exist.

Lettuce detects the failover via its Sentinel subscription and reconnects to R1 (now the new primary) transparently. The Stripe HTTP call that was in flight eventually returns. The billing job marks itself SUCCEEDED and stores ch_A as the charge ID.

Three days later, the billing operations team runs a monthly audit that cross-references billing jobs with Stripe charges and finds job_999 with status SUCCEEDED and charge ID ch_A. However, a separate data integrity check finds that ch_A was for a different amount than the subscription’s current plan (an upgrade had been processed between when the job was created and when it was executed). The billing team concludes that job_999 was billed at the wrong amount and creates a corrected re-billing job job_999_corrected with a new pre-generated idempotency key key_B and the correct amount.

This is a different billing event and correctly uses a different key. No double charge in this scenario.

The double charge scenario occurs when the following additional detail is present: the billing service wraps the entire chargeSubscription call in a @Transactional annotation that rolls back the BillingJob database record on any exception thrown by the Stripe SDK. The Stripe HTTP call that was in flight during the Sentinel failover — the one that eventually committed ch_A — was also running inside a Lettuce connection that got disconnected when the primary went down. Lettuce throws RedisConnectionFailureException. Spring’s transaction manager catches the unchecked exception (or the developer’s billing orchestrator catches it) and rolls back the BillingJob to FAILED status.

The billing retry scheduler picks up the FAILED job and re-runs chargeSubscription. The new primary (formerly R1) has no record of idem:job_999. setIfAbsent returns Boolean.TRUE (key not present). The billing service reads back key_A (the pre-generated key from the BillingJob entity). The Stripe call uses key_A. Stripe’s own idempotency cache for key_A is still valid (within 24 hours). Stripe returns the cached response for ch_A. No double charge in the happy case.

The double charge occurs in the following variation: the billing team’s retry policy uses a different code path for retried jobs. Instead of re-running the same BillingJob entity (which preserves the pre-generated key_A), the retry creates a new BillingJob entity with a newly generated idempotency key key_B = UUID.randomUUID(). The retry job is a “resubmit” rather than a “retry” in the developer’s terminology: the old failed job is marked CANCELLED and a new job is created and queued. The new job runs chargeSubscription with key_B. Stripe commits ch_B alongside the already-committed ch_A.

Developer misconception

The developer’s model is: “Redis Sentinel provides high availability with automatic failover. My application reconnects to the new primary transparently. Write failures during failover will throw exceptions and my retry logic will re-run the billing job with the same idempotency key.”

There are two flaws:

First, Redis Sentinel asynchronous replication means that writes acknowledged by the primary may not be present on the promoted replica. The developer assumes that Sentinel failover is loss-free; it is loss-minimizing, but not loss-free. Writes in the replication lag window are silently discarded. The key may have been written to the primary, the primary may have acknowledged it, and the key may not exist on the new primary. There is no error raised on the write path to indicate this; the write was successful from the client’s perspective.

Second, the “resubmit vs. retry” distinction in the billing team’s operational runbook creates a pathway for a new idempotency key to be generated for what is logically the same billing intent. If the pre-generated idempotency key is stored only in the failed BillingJob entity, and the resubmit pathway creates a new entity with a new key, the safety of the system depends on the developer ensuring that a resubmit is never done for a charge that may already have committed. After a Sentinel failover where ch_A was committed but the BillingJob was rolled back to FAILED, a resubmit creates exactly this scenario.

The fix: stable key + database-resident guard

The idempotency key must be stored in the same transactional store as the billing job outcome. If the BillingJob is stored in PostgreSQL, the idempotency key must be stored in PostgreSQL too — not in Redis. The Redis store can be an additional cache for fast lookup, but it must not be the authoritative record. The authoritative record must be stored in a system that commits synchronously and can be read reliably after failover.

@Entity
public class BillingJob {
    @Id
    private String id;

    // Pre-generated at job creation — stable for the lifetime of this job
    @Column(nullable = false, unique = true)
    private String idempotencyKey;

    private String chargeId;
    private BillingJobStatus status;
    // ... other fields
}

@Service
public class BillingService {

    private final BillingJobRepository repo;
    private final StripeClient stripe;

    @Transactional
    public String chargeSubscription(String jobId) {
        BillingJob job = repo.findById(jobId)
            .orElseThrow(() -> new IllegalArgumentException("Unknown job: " + jobId));

        if (job.getStatus() == BillingJobStatus.SUCCEEDED) {
            return job.getChargeId(); // idempotent short-circuit
        }

        // Use the pre-generated key from the database — no Redis dependency
        String idempotencyKey = job.getIdempotencyKey();

        PaymentIntentCreateParams params = PaymentIntentCreateParams.builder()
            .setAmount(job.getAmountCents())
            .setCurrency("usd")
            .setCustomer(job.getStripeCustomerId())
            .setConfirm(true)
            .build();

        RequestOptions opts = RequestOptions.builder()
            .setIdempotencyKey(idempotencyKey)
            .build();

        PaymentIntent pi = stripe.paymentIntents().create(params, opts);
        job.setChargeId(pi.getId());
        job.setStatus(BillingJobStatus.SUCCEEDED);
        // Committed atomically with the job status update
        repo.save(job);
        return pi.getId();
    }
}

With this design, a Redis Sentinel failover loses no idempotency state because the idempotency key is not in Redis. The BillingJob.idempotencyKey field is written once at job creation and is never updated. Any retry of the job, whether via a retry loop or via a resubmit, reads job.getIdempotencyKey() from the database and sends that key to Stripe. Within Stripe’s 24-hour deduplication window, the second call returns the cached result. Outside the window, the second call is a new charge — which is correct, because billing jobs older than 24 hours represent a genuinely new billing event, not a retry of the original.

Sentinel-specific mitigations

For cases where Redis is used for other critical data beyond idempotency keys, several configuration options reduce the replication lag window:

• min-slaves-to-write 1 and min-slaves-max-lag 10 (Redis config): the primary refuses writes if fewer than 1 replica has acknowledged within 10 seconds. This prevents writes from committing on a primary that has no replicas, at the cost of write availability during single-replica failure.
• WAIT numreplicas timeout command: issues a synchronous wait after the write until at least numreplicas replicas have acknowledged. Spring Data Redis 3.x exposes this via RedisTemplate.execute(connection -> connection.sync().wait(1, 0)). A timeout of 0 means “wait indefinitely,” which provides synchronous replication semantics at the cost of latency.
• Redis 7.0+ LMPOP and stream-based idempotency: Redis Streams with consumer groups provide at-least-once delivery semantics with message ID-based deduplication. Message IDs are replicated synchronously in some deployment configurations.

None of these eliminate the window entirely for the default configuration. The only complete fix for critical idempotency state is to store it in a system with synchronous, durable replication — which for most billing applications means the relational database that already stores the billing job record.

Testing for Sentinel failover data loss

@Test
void sentinel_failover_replication_lag_does_not_produce_second_charge() {
    // Simulate: billing job has pre-generated key stored in database
    BillingJob job = new BillingJob("job_sentinel_1", "cus_z", 4999L);
    String keyA = job.getIdempotencyKey(); // pre-generated at job creation
    billingJobRepo.save(job);

    // Stub Stripe to succeed for keyA
    wireMock.stubFor(post(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(keyA))
        .willReturn(okJson("{\"id\":\"pi_sentinel_A\",\"status\":\"succeeded\"}")));

    // First call: sets key in Redis, calls Stripe, ch_A commits
    billingService.chargeSubscription("job_sentinel_1");

    // Simulate Sentinel failover: flush Redis (new primary has no keys)
    redisTemplate.getConnectionFactory().getConnection().flushDb();

    // Simulate billing job rollback due to Lettuce exception during failover:
    // reset job to FAILED in the database (the @Transactional rollback scenario)
    job = billingJobRepo.findById("job_sentinel_1").get();
    job.setStatus(BillingJobStatus.FAILED);
    job.setChargeId(null);
    billingJobRepo.save(job);

    // Retry: Redis has no key, but job still has keyA in database
    billingService.chargeSubscription("job_sentinel_1");

    // Stripe must have been called exactly once with keyA (second call deduped)
    wireMock.verify(1, postRequestedFor(urlPathEqualTo("/v1/payment_intents"))
        .withHeader("Idempotency-Key", equalTo(keyA)));

    BillingJob finalJob = billingJobRepo.findById("job_sentinel_1").get();
    assertThat(finalJob.getStatus()).isEqualTo(BillingJobStatus.SUCCEEDED);
    assertThat(finalJob.getChargeId()).isEqualTo("pi_sentinel_A");
}

Summary: why Redis changes the idempotency guarantee

A developer choosing Redis for idempotency key storage typically has three goals: fast lookup, atomic check-and-set, and automatic cleanup. All three are achievable with Redis, but each comes with an assumption that can be violated in production:

All three failure modes share the same root: the idempotency key is derived from a mutable Redis value rather than from the immutable properties of the billing intent itself. The fix for all three is the same: derive the idempotency key deterministically from billing intent fields (subscriptionId, billingPeriod, amountCents) that are stable across retries, store the key derivation result in the same durable transactional store as the billing job outcome, and use Redis only as an optional performance cache — one whose absence must route to derivation, never to new UUID generation.

Diagnostic checklist for Spring Data Redis + Stripe billing

1. What is the maximum retry horizon for a billing intent? Count every retry pathway: Spring Retry, @Scheduled requeue, message queue dead-letter redeliver, manual operator resubmit. The Redis TTL must exceed the maximum of all of these.
2. Is the Redis TTL set shorter than 24 hours? Stripe’s own deduplication window is 24 hours. Any Redis key TTL shorter than 24 hours creates a window where Redis expires the key before Stripe expires its own deduplication cache, defeating the purpose.
3. Does any catch block on RedisConnectionFailureException proceed with a new UUID? Connection failures from setIfAbsent are ambiguous. Any code path that catches the exception and generates a new UUID.randomUUID() is a double-charge risk. Replace with: read-back after reconnection, or throw a retryable exception that will re-execute the method with the same pre-generated key.
4. Is the idempotency key stored outside Redis? If the only copy of the key is in Redis, any Redis data loss (TTL expiry, cluster flush, Sentinel failover, AOF corruption) produces a new UUID on the next attempt. The key must have a durable home in the relational database.
5. Does the billing retry create a new entity or reuse the existing one? If “retry” means “create a new BillingJob with a new UUID.randomUUID(),” the idempotency guarantee is broken every time a retry fires. Retry must always reuse the pre-generated key from the original job entity.
6. Is WAIT used for critical Redis writes? If Redis is used for any purpose beyond a volatile cache (e.g., as the authoritative guard store), use WAIT 1 0 after writes to ensure at least one replica has the write before acknowledging to the caller.
7. Is min-slaves-to-write 1 configured? Without it, writes can succeed on a primary with no live replicas, and any data written in that window is lost on failover.
8. Are there integration tests with Testcontainers Redis that simulate TTL expiry and connection failure? TTL expiry is testable by calling redisTemplate.expire(key, Duration.ZERO) in a test to force immediate eviction. Connection failure is testable by pausing/stopping the Redis container between operations using the Docker SDK via Testcontainers.

The same pattern across all three modes

Across all three Spring Data Redis failure modes, the observable double-charge sequence is identical:

1. Stripe call 1 commits ch_A with idempotency key key_A.
2. An event removes key_A from Redis or makes it inaccessible (TTL expiry, connection failure ambiguity, Sentinel failover data loss).
3. A retry call generates or receives key_B ≠ key_A.
4. Stripe call 2 commits ch_B.

The event in step 2 is always a Redis data loss or ambiguity event. The fix is always the same: eliminate the dependency of key_A on Redis by deriving it deterministically from the billing intent and storing it durably outside Redis. Redis becomes a fast-path cache whose absence causes derivation, not key rotation.

The Keybrake proxy enforces this at the infrastructure level: the idempotency key for each agent-to-Stripe call is derived from the proxy request fingerprint (agent ID, endpoint, payload hash) and stored in a SQLite audit record that persists independently of Redis liveness. Whether or not a Redis TTL expires, whether or not a Sentinel failover occurs, the proxy can always retrieve the original key for a retry and ensure Stripe sees the same key it saw on the first call.