Keybrake

http4k and Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

By Keybrake —

http4k’s “Server as a Function” model defines typealias Filter = (HttpHandler) -> HttpHandler and typealias HttpHandler = (Request) -> Response. The entire client stack is assembled by composing these function types with .then(). This model introduces three Stripe billing failure modes that have no equivalent in any Java HTTP client covered in this series, because the failures arise from the composition semantics of the model itself — not from any library-specific API quirk: (1) filter composition order: retryFilter.then(uuidFilter).then(httpClient) places the UUID-generating filter inside the retry boundary — when RetryFilter retries by re-invoking its inner HttpHandler, that handler is uuidFilter.then(httpClient), which calls UUID.randomUUID() again on each retry and produces UUID_B, creating ch_B alongside the already-committed ch_A; (2) ClientFilters.SetHeader() Kotlin expression evaluation timing: ClientFilters.SetHeader("Idempotency-Key", UUID.randomUUID().toString()) evaluates the UUID expression at the Kotlin call site — at client construction time — producing a single static string that becomes a permanent header on every request from that client instance, making all billing calls share one key and causing Stripe’s idempotency cache to return ch_A for every customer; (3) Kotlin coroutines withTimeout race: wrapping a billing call in withTimeout(5000L) { } fires TimeoutCancellationException on the client side while Stripe has already received the full request body and committed ch_A — a catch block that retries by calling the billing function again generates UUID_B inside the function and creates ch_B.

This post covers all three failure modes with http4k Filter typealias mechanics, .then() composition call-flow tracing, Kotlin expression evaluation timing in ClientFilters.SetHeader() vs dynamic filter lambdas, withTimeout TCP-level race conditions, content-hash idempotency keys stable across all retry paths — and per-billing-period vault keys via a spend-cap proxy as a hard financial backstop. For Ktor-specific failure modes (Kotlin multiplatform HTTP client with HttpRequestBuilder, install(HttpRequestRetry), and BearerTokens plugin), see the Ktor and Stripe Integration post. For Spring WebFlux WebClient patterns including Mono.defer() factory re-evaluation and Flux.retryWhen() level errors, see the Spring WebFlux and Stripe Integration post. For Armeria (another JVM HTTP client with service decoration patterns), see the Armeria and Stripe Integration post.

Failure mode 1: UUID-generating Filter placed inside RetryFilter in the .then() composition chain — RetryFilter re-invokes its inner HttpHandler per retry — UUID.randomUUID() inside uuidFilter generates UUID_B — Stripe committed ch_A before the error — retry with UUID_B creates ch_B

http4k’s Filter is defined as typealias Filter = (HttpHandler) -> HttpHandler. A filter takes the next handler in the chain as input and returns a new handler that wraps it. Composing two filters with filterA.then(filterB) produces a new Filter whose output handler calls filterA’s body, which in turn calls filterB’s body, which in turn calls the final client. Applying the composed filter to a client with .then(httpClient) produces the final HttpHandler.

The composition order determines which function calls are “inside” any given filter. A RetryFilter works by re-invoking its inner HttpHandler when the wrapped call fails. Its inner HttpHandler is everything that was passed to RetryFilter’s .then() call. If a UUID-generating filter is composed to the right of RetryFilter in the chain, it is part of that inner HttpHandler and re-executes on every retry:

// BillingService.kt — UNSAFE: uuidFilter placed INSIDE RetryFilter in .then() chain.
import org.http4k.client.JavaHttpClient
import org.http4k.core.*
import org.http4k.filter.ClientFilters
import org.http4k.resilience4j.RetryFilter
import io.github.resilience4j.retry.RetryConfig
import java.util.UUID

val uuidFilter = Filter { next ->
    { request ->
        // This lambda body IS the HttpHandler that RetryFilter re-invokes per retry.
        // BUG: UUID.randomUUID() is called inside this lambda, which executes once
        // per request that flows through this filter — including retried requests.
        // First attempt: UUID_A set as Idempotency-Key.
        // RetryFilter catches the IOException and re-invokes next(request) below,
        // which means it re-invokes uuidFilter's inner handler, calling UUID.randomUUID()
        // again and producing UUID_B.
        val keyed = request.header("Idempotency-Key", UUID.randomUUID().toString())
        next(keyed)
    }
}

val retryConfig = RetryConfig.custom<Response>()
    .maxAttempts(3)
    .retryOnException { e -> e is java.io.IOException }
    .build()

// WRONG ORDER: RetryFilter is the outermost wrapper.
// uuidFilter.then(JavaHttpClient()) is the inner HttpHandler that RetryFilter retries.
// uuidFilter runs again on every retry, producing UUID_B, UUID_C on each attempt.
val billingClient: HttpHandler = RetryFilter(retryConfig)
    .then(uuidFilter)          // INSIDE RetryFilter — re-executes per retry
    .then(JavaHttpClient())

fun chargeCustomer(customerId: String, period: String): Response {
    val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
        .header("Authorization", "Bearer $stripeKey")
        .header("Content-Type", "application/x-www-form-urlencoded")
        .body("customer=$customerId&amount=2999¤cy=usd")
    return billingClient(request)
}

The call flow: billingClient(request) invokes RetryFilter’s handler. RetryFilter calls its inner handler — uuidFilter.then(JavaHttpClient()) — with the original request. uuidFilter’s lambda body runs. UUID.randomUUID().toString() produces UUID_A. keyed = request.header("Idempotency-Key", UUID_A) creates a new Request object with the key set. next(keyed) calls JavaHttpClient() with keyed. JavaHttpClient sends the HTTP request. Stripe receives it, begins processing, commits ch_A, and then a TCP reset occurs before the response bytes arrive. JavaHttpClient throws IOException. The exception propagates back to RetryFilter’s handler.

RetryFilter’s retry logic fires. It re-invokes its inner handler: uuidFilter.then(JavaHttpClient()). Note carefully what “inner handler” means here: it is the function that was passed to RetryFilter’s .then() call, which is uuidFilter.then(JavaHttpClient()). This is not a memoized closure over a previously computed request — it is a fresh invocation of the uuidFilter lambda body. UUID.randomUUID().toString() produces UUID_B. keyed is a new Request object with Idempotency-Key: UUID_B. Stripe receives UUID_B, finds no entry in its idempotency cache (UUID_A was the key for ch_A, not UUID_B), and creates ch_B. The customer is charged twice.

The reason http4k makes this error easy to introduce is that .then() reads naturally in a “first retry, then add the key, then call the client” mental model — “retry, then set idempotency key, then call” sounds like the correct order. But in the Filter = (HttpHandler) -> HttpHandler model, “then” means the left side wraps the right side. retryFilter.then(uuidFilter) means retryFilter receives uuidFilter.then(client) as its inner handler. The UUID-generating filter is therefore executed by retryFilter on every retry. The correct mental model is not “steps in sequence” but “layers of onion wrapping”: the outermost layer is called first and calls the next layer inside it, and the retry layer re-calls everything inside it from scratch.

The subtler variant: developer writes an inline HttpHandler lambda as the target of RetryFilter.then() — the lambda is the inner handler RetryFilter retries — UUID.randomUUID() inside the lambda generates a new key on each attempt

Rather than writing a separate uuidFilter function, a developer might inline the handler construction:

// BillingService.kt — UNSAFE: UUID generated inside the inline HttpHandler lambda
// passed directly as the inner target of RetryFilter.
// The developer writes this as a single expression, not realising the lambda
// body is the function that RetryFilter re-calls on every retry.

val billingClient: HttpHandler = RetryFilter(retryConfig).then { request ->
    // This lambda IS the HttpHandler that RetryFilter passes to retry.
    // It is not a "closure over a single UUID" — it is a function that
    // evaluates UUID.randomUUID() every time it is called.
    // First retry: UUID_A. Second retry: UUID_B. Third retry: UUID_C.
    val keyed = request
        .header("Idempotency-Key", UUID.randomUUID().toString()) // BUG: per-call
        .header("Authorization", "Bearer $stripeKey")
        .header("Content-Type", "application/x-www-form-urlencoded")
    JavaHttpClient()(keyed)
}

The inline form makes the bug invisible in code review. The developer who writes RetryFilter(config).then { request -> val keyed = request.header(..., UUID.randomUUID()...) ... } may read this as “build a retrying client whose handler sets a UUID”, not as “build a retrying client that generates a new UUID on every attempt”. In Kotlin, a lambda expression is a function value; it is called every time it is invoked, regardless of how it was written. The lambda body does not “capture” the result of UUID.randomUUID() at the time the lambda is written — it evaluates UUID.randomUUID() at the time the lambda is called. When RetryFilter calls the lambda three times across three retry attempts, UUID.randomUUID() evaluates three times and produces three distinct values.

Fix: place the UUID-generating filter as the outermost layer in the .then() chain, wrapping RetryFilter — RetryFilter receives a request that already carries the stable key and retries with that same request unchanged

// BillingService.kt — FIXED.
import java.nio.charset.StandardCharsets
import java.security.MessageDigest

// Content-hash key: same value for a given (customerId, period) on every JVM
// instance, every retry, every session. Not UUID.randomUUID().
fun stableIdempotencyKey(customerId: String, period: String): String {
    val input = "$customerId:$period:http4k-billing"
    val md = MessageDigest.getInstance("SHA-256")
    val hash = md.digest(input.toByteArray(StandardCharsets.UTF_8))
    return hash.joinToString("") { "%02x".format(it) }.take(32)
}

// idempotencyFilter is OUTSIDE RetryFilter in the .then() chain.
// When billingClient(request) is called:
//   1. idempotencyFilter runs — computes stable key from request metadata — adds header.
//   2. RetryFilter receives the keyed request.
//   3. RetryFilter calls JavaHttpClient() with the keyed request.
//   4. On retry, RetryFilter calls JavaHttpClient() AGAIN with the SAME keyed request.
//   idempotencyFilter does NOT run again on retry.
fun idempotencyFilter(customerId: String, period: String) = Filter { next ->
    { request ->
        val key = stableIdempotencyKey(customerId, period)
        next(request.header("Idempotency-Key", key))
    }
}

// CORRECT ORDER: idempotencyFilter wraps everything including RetryFilter.
fun buildBillingClient(customerId: String, period: String): HttpHandler =
    idempotencyFilter(customerId, period)
        .then(RetryFilter(retryConfig))   // OUTSIDE idempotencyFilter — key already set
        .then(JavaHttpClient())

fun chargeCustomer(customerId: String, period: String): Response {
    val client = buildBillingClient(customerId, period)
    val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
        .header("Authorization", "Bearer $stripeKey")
        .header("Content-Type", "application/x-www-form-urlencoded")
        .body("customer=$customerId&amount=2999¤cy=usd")
    return client(request)
}

Three properties of this fix: (1) idempotencyFilter(customerId, period) is built with the specific customer and period baked into the closure — the same filter instance produces the same stable key for that (customerId, period) pair on every invocation because stableIdempotencyKey() is deterministic; (2) idempotencyFilter is the leftmost (outermost) element in the .then() chain — it runs exactly once per call to chargeCustomer(), before RetryFilter receives the request — RetryFilter only ever sees a request that already carries the stable key and passes that same Request object to JavaHttpClient() on every retry attempt; (3) buildBillingClient() creates a per-operation client rather than a shared singleton client, which is correct because the idempotency key must be per-operation, not per-client-instance — different billing operations for different customers require different keys.

Stop a runaway retry loop before it charges your customers twice

Keybrake issues a scoped vault key per billing run with a daily USD cap. When a misconfigured retry loop hits the cap, the proxy returns 429 — the charge never reaches Stripe. Enter your email to try it on your next http4k deployment.

Failure mode 2: ClientFilters.SetHeader("Idempotency-Key", UUID.randomUUID().toString()) evaluates the UUID expression at Kotlin call time — produces a single static string applied to every request from the client instance — all billing calls share one key — Stripe returns ch_A for every customer — developer “fixes” with a dynamic filter but composes it inside the retry boundary — UUID_B on retry — ch_B

ClientFilters.SetHeader(name: String, value: String) is an http4k built-in filter that adds a static header to every outbound request. It takes two String parameters. In Kotlin, all function arguments are evaluated before the function is called — this is standard eager evaluation. When you write ClientFilters.SetHeader("Idempotency-Key", UUID.randomUUID().toString()), the Kotlin compiler evaluates UUID.randomUUID().toString() at the call site, passes the resulting String to SetHeader, and SetHeader stores it as a permanently fixed string. The filter it returns will set Idempotency-Key to that one string on every request it processes, for the entire lifetime of the client:

// BillingService.kt — UNSAFE: UUID.randomUUID() evaluated ONCE at client construction time.
// Every billing call from this client sends the same static Idempotency-Key.

import org.http4k.filter.ClientFilters

object BillingService {

    // UUID.randomUUID().toString() is evaluated HERE, when this object is initialised.
    // It produces, say, "3f4a9b2c-e1d7-4a8f-b0c3-9e5f2a1d7b6e".
    // That string is stored inside the filter returned by SetHeader().
    // Every POST /v1/charges sent through billingClient carries:
    //   Idempotency-Key: 3f4a9b2c-e1d7-4a8f-b0c3-9e5f2a1d7b6e
    //
    // Consequence:
    //   chargeCustomer("cus_A", "2026-01") → Stripe creates ch_A (key: 3f4a...)
    //   chargeCustomer("cus_B", "2026-01") → Stripe returns CACHED ch_A (same key!)
    //   chargeCustomer("cus_A", "2026-02") → Stripe returns CACHED ch_A (same key!)
    //   chargeCustomer("cus_C", "2026-01") → Stripe returns CACHED ch_A (same key!)
    //
    // No new charges are created after the first. Customer B, C, D are never billed.
    // The billing service "succeeds" without errors from its perspective —
    // Stripe returns 200 with ch_A's JSON for every call.
    private val billingClient: HttpHandler =
        ClientFilters.SetHeader("Idempotency-Key", UUID.randomUUID().toString()) // BUG
            .then(ClientFilters.SetHeader("Authorization", "Bearer $stripeKey"))
            .then(ClientFilters.SetHeader("Content-Type", "application/x-www-form-urlencoded"))
            .then(RetryFilter(retryConfig))
            .then(JavaHttpClient())

    fun chargeCustomer(customerId: String, period: String): Response {
        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .body("customer=$customerId&amount=2999¤cy=usd")
        return billingClient(request)
    }
}

The mechanics: Stripe’s idempotency cache is keyed on the combination of the Idempotency-Key header value and the API endpoint. The first billing call — chargeCustomer("cus_A", "2026-01") — sends Idempotency-Key: 3f4a... and Stripe creates ch_A, caches the result under that key. All subsequent billing calls from billingClient also send Idempotency-Key: 3f4a.... Stripe recognises the key as matching a completed request and returns the cached response: the JSON representation of ch_A. The billing service receives a 200 OK response for every call. No exception is thrown. No error is logged. The application code parses the response and believes each billing call succeeded. But only one charge — ch_A for customer A’s January billing — was ever created at Stripe. Every other customer in the billing run is uncharged, silently.

This failure mode is the inverse of the duplicate charge problem. Instead of charging customers twice, it charges the first customer once and silently skips every subsequent customer. The silence makes it difficult to detect without explicit cross-checking between the billing service’s records and Stripe’s charge list.

The subtler variant: developer realises the bug and moves to a dynamic filter written as a Kotlin lambda — composes it as RetryFilter.then(dynamicFilter).then(JavaHttpClient()) — dynamicFilter is inside the retry boundary — UUID_B on retry — ch_B

After discovering that ClientFilters.SetHeader("Idempotency-Key", UUID.randomUUID().toString()) produces a static key, a developer writes a dynamic filter that generates a fresh UUID per request:

// BillingService.kt — PARTIALLY FIXED but still broken.
// Developer correctly replaced static SetHeader with a dynamic filter.
// BUG: dynamicFilter is composed INSIDE RetryFilter in the .then() chain.
// RetryFilter retries by re-invoking its inner handler, which includes dynamicFilter.
// dynamicFilter generates UUID_B on the first retry — ch_B alongside committed ch_A.

// The dynamic filter correctly generates a new UUID per request invocation.
val dynamicIdempotencyFilter = Filter { next ->
    { request ->
        // This generates a fresh UUID per invocation of this filter's inner lambda.
        // Per-invocation is correct if this filter is OUTSIDE RetryFilter.
        // If INSIDE RetryFilter, per-invocation means per-retry-attempt — BUG.
        next(request.header("Idempotency-Key", UUID.randomUUID().toString()))
    }
}

object BillingService {

    // WRONG ORDER: dynamicFilter is inside RetryFilter.
    // RetryFilter.then(dynamicFilter) means RetryFilter receives
    // dynamicFilter.then(JavaHttpClient()) as its inner handler.
    // On retry, RetryFilter calls dynamicFilter again — UUID_B.
    private val billingClient: HttpHandler =
        ClientFilters.SetHeader("Authorization", "Bearer $stripeKey")
            .then(ClientFilters.SetHeader("Content-Type", "application/x-www-form-urlencoded"))
            .then(RetryFilter(retryConfig))        // wraps everything below it
            .then(dynamicIdempotencyFilter)         // INSIDE RetryFilter — UUID per retry
            .then(JavaHttpClient())

    fun chargeCustomer(customerId: String, period: String): Response {
        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .body("customer=$customerId&amount=2999¤cy=usd")
        return billingClient(request)
    }
}

The composition trace: billingClient(request) calls SetHeader("Authorization")’s handler, which calls SetHeader("Content-Type")’s handler, which calls RetryFilter’s handler. RetryFilter’s inner handler is dynamicIdempotencyFilter.then(JavaHttpClient()). On the first attempt, dynamicIdempotencyFilter generates UUID_A, JavaHttpClient() sends the request, Stripe commits ch_A, a network error occurs. RetryFilter retries: re-calls dynamicIdempotencyFilter.then(JavaHttpClient()). dynamicIdempotencyFilter generates UUID_B. Stripe creates ch_B. Two charges exist for one billing operation.

This is the progression from failure mode 2a (static UUID → all customers share one key) to failure mode 2b (dynamic UUID placed inside retry → UUID_B on retry). A developer who debugs 2a and moves to a dynamic filter has fixed the wrong bug if they simultaneously introduce 2b. The correct arrangement is to place the dynamic filter — or better, the stable content-hash filter — outside RetryFilter.

Fix: compute a stable content-hash key per-operation before composing the filter chain; close over it in a per-operation filter that is the outermost layer

// BillingService.kt — FIXED.
// Idempotency key computed from (customerId, period) as a deterministic hash.
// The filter that sets it is the outermost layer, wrapping RetryFilter.
// RetryFilter only retries JavaHttpClient() — the keyed request is already built.

class BillingService(private val stripeKey: String) {

    // Shared base client: static headers only, no idempotency key.
    // RetryFilter is inside here — it only ever sees requests that already carry
    // the stable key, added by the per-operation filter outside.
    private val baseClient: HttpHandler =
        ClientFilters.SetHeader("Authorization", "Bearer $stripeKey")
            .then(ClientFilters.SetHeader("Content-Type", "application/x-www-form-urlencoded"))
            .then(RetryFilter(retryConfig))
            .then(JavaHttpClient())

    fun chargeCustomer(customerId: String, period: String): Response {
        // Stable key for this billing operation.
        val key = stableIdempotencyKey(customerId, period)

        // Per-operation filter: wraps the shared baseClient as the outermost layer.
        // This filter runs ONCE when chargeCustomer() is called.
        // On retry, RetryFilter re-calls JavaHttpClient() with the already-keyed request;
        // this per-operation filter is NOT part of what RetryFilter retries.
        val client: HttpHandler = Filter { next ->
            { request -> next(request.header("Idempotency-Key", key)) }
        }.then(baseClient)

        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .body("customer=$customerId&amount=2999¤cy=usd")
        return client(request)
    }
}

// stableIdempotencyKey: deterministic SHA-256 truncated to 32 hex chars.
// Same output for a given (customerId, period) on every JVM instance.
fun stableIdempotencyKey(customerId: String, period: String): String {
    val md = MessageDigest.getInstance("SHA-256")
    val hash = md.digest("$customerId:$period:http4k-billing".toByteArray(StandardCharsets.UTF_8))
    return hash.joinToString("") { "%02x".format(it) }.take(32)
}

The critical structural property: Filter { next -> { request -> next(request.header("Idempotency-Key", key)) } }.then(baseClient) produces an HttpHandler whose execution flow is: (1) the outer filter lambda runs — it takes the Request object, adds the stable key header, and calls next(keyedRequest) where next = baseClient; (2) baseClient(keyedRequest) calls the SetHeader("Authorization") filter, then SetHeader("Content-Type"), then RetryFilter, then JavaHttpClient(); (3) when RetryFilter retries, it re-calls JavaHttpClient(keyedRequest) with the same keyedRequest that was passed to RetryFilter from step 2 above — the outer filter’s lambda does not re-run on retry, because RetryFilter’s inner handler is SetHeader("Authorization").then(SetHeader("Content-Type")).then(JavaHttpClient()), not the outer filter that sets the idempotency key.

Failure mode 3: Kotlin coroutines withTimeout { } fires TimeoutCancellationException while Stripe committed ch_A — catch block retries chargeCustomer() — new UUID.randomUUID() inside chargeCustomer() generates UUID_B — ch_B

http4k’s JavaHttpClient() and other backend clients (OkHttp(), ApacheClient()) are synchronous blocking HttpHandler implementations. When called from Kotlin coroutines — either wrapped in a withContext(Dispatchers.IO) { } block or used inside a runBlocking { } scope — the call blocks the coroutine’s thread. Kotlin’s withTimeout(millis) { } coroutine builder adds a deadline to a coroutine scope: if the scope does not complete within the specified duration, TimeoutCancellationException is thrown at the next cancellation point inside the scope.

The problem is that TimeoutCancellationException is a client-side event. At the moment it fires, the following is true at the network level:

// BillingService.kt — UNSAFE: withTimeout + chargeCustomer() retry generates UUID_B.
import kotlinx.coroutines.*
import org.http4k.client.JavaHttpClient
import org.http4k.core.*
import java.util.UUID

class BillingService(private val stripeKey: String) {

    private val httpClient = JavaHttpClient()

    // BUG: UUID generated inside chargeCustomer(), called per invocation.
    // If the caller retries chargeCustomer() after a timeout, a new UUID is generated.
    suspend fun chargeCustomer(customerId: String, period: String): ChargeResult {
        val idempotencyKey = UUID.randomUUID().toString() // BUG: new UUID per call

        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .header("Authorization", "Bearer $stripeKey")
            .header("Content-Type", "application/x-www-form-urlencoded")
            .header("Idempotency-Key", idempotencyKey)
            .body("customer=$customerId&amount=2999¤cy=usd")

        val response = withContext(Dispatchers.IO) { httpClient(request) }
        return parseChargeResult(response)
    }
}

// Calling code — monthly billing job:
suspend fun runMonthlyBilling(customers: List, period: String) {
    val service = BillingService(stripeKey)
    for (customerId in customers) {
        try {
            // withTimeout fires TimeoutCancellationException if the coroutine
            // does not complete within 5 seconds. At that point, Stripe may
            // have already committed the charge for this customer.
            val result = withTimeout(5_000L) {
                service.chargeCustomer(customerId, period)
            }
            log.info("Charged $customerId: ${result.chargeId}")
        } catch (e: TimeoutCancellationException) {
            log.warn("Timeout for $customerId, retrying...")
            // BUG: retry calls chargeCustomer() again — generates UUID_B inside.
            // Stripe committed ch_A during the timeout period.
            // This retry with UUID_B creates ch_B.
            delay(1_000)
            val result = service.chargeCustomer(customerId, period) // UUID_B — ch_B
            log.info("Retry charged $customerId: ${result.chargeId}")
        }
    }
}

The charge-doubling sequence: withTimeout(5_000L) { service.chargeCustomer(customerId, period) } starts the coroutine. Inside, UUID.randomUUID() generates UUID_A. JavaHttpClient() sends the request to Stripe synchronously on the IO dispatcher. Stripe receives the request and begins processing. Five seconds pass. Stripe has committed ch_A but the response has not yet been delivered to the client thread. TimeoutCancellationException is thrown. The catch block fires. delay(1_000) waits one second. service.chargeCustomer(customerId, period) is called again. Inside, UUID.randomUUID() generates UUID_B. The retry request carries Idempotency-Key: UUID_B. Stripe sees UUID_B as a new request — it has no idempotency cache entry for UUID_B, only for UUID_A. Stripe creates ch_B. The customer is charged twice.

The subtler variant: developer uses async { }.await() with coroutine job cancellation — the cancelled async block sent the request to Stripe before cancellation — a subsequent coroutine with UUID.randomUUID() generates UUID_B — ch_B

An alternative async pattern uses async { }.await() with an explicit timeout and job cancellation:

// BillingService.kt — UNSAFE: async + cancel + new coroutine with UUID_B.
suspend fun chargeWithManualTimeout(customerId: String, period: String): ChargeResult {
    val job = CoroutineScope(Dispatchers.IO).async {
        // UUID generated inside the async block — this block may be cancelled
        // after Stripe received the request body and committed ch_A.
        val idempotencyKey = UUID.randomUUID().toString() // BUG
        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .header("Idempotency-Key", idempotencyKey)
            .header("Authorization", "Bearer $stripeKey")
            .header("Content-Type", "application/x-www-form-urlencoded")
            .body("customer=$customerId&amount=2999¤cy=usd")
        httpClient(request)
    }

    return try {
        withTimeout(5_000L) { ChargeResult(job.await()) }
    } catch (e: TimeoutCancellationException) {
        job.cancel()
        delay(2_000)
        // Developer's reasoning: "The previous job was cancelled.
        // I'll start a fresh one with a new UUID to avoid idempotency conflicts."
        // This reasoning is wrong. The cancelled job may have already sent ch_A.
        // "To avoid idempotency conflicts" should mean REUSING the same key,
        // not generating a new one. UUID_B creates ch_B alongside ch_A.
        val retryJob = CoroutineScope(Dispatchers.IO).async {
            val idempotencyKey = UUID.randomUUID().toString() // UUID_B — BUG
            val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
                .header("Idempotency-Key", idempotencyKey)
                .header("Authorization", "Bearer $stripeKey")
                .header("Content-Type", "application/x-www-form-urlencoded")
                .body("customer=$customerId&amount=2999¤cy=usd")
            httpClient(request)
        }
        ChargeResult(retryJob.await())
    }
}

The developer’s comment — “new UUID to avoid idempotency conflicts” — reveals the misunderstanding. Idempotency keys do not cause conflicts between different billing operations; they prevent duplication within the same billing operation. Generating a new UUID for the retry is exactly the wrong response to a timeout: a new UUID sends a request that Stripe cannot match to the original, and therefore creates a new charge. The correct response to a timeout is to retry with the same UUID so that if Stripe committed ch_A, the retry gets the cached result rather than triggering ch_B. Only when you know with certainty that Stripe did not process the original request should you use a new UUID — and a client-side timeout provides no such certainty.

Fix: compute the stable key outside the coroutine scope and pass it as a parameter — same key used in the initial call and any timeout-triggered retry — Stripe’s idempotency cache returns ch_A on the retry rather than creating ch_B

// BillingService.kt — FIXED.
class BillingService(private val stripeKey: String) {

    private val httpClient: HttpHandler =
        ClientFilters.SetHeader("Authorization", "Bearer $stripeKey")
            .then(ClientFilters.SetHeader("Content-Type", "application/x-www-form-urlencoded"))
            .then(JavaHttpClient())

    // Stable key passed as a parameter — computed by the caller before any retry boundary.
    // chargeCustomer() no longer generates UUID internally.
    // Same key reused on any retry triggered by a timeout at the call site.
    suspend fun chargeCustomer(
        customerId: String,
        period: String,
        idempotencyKey: String   // caller-provided stable key
    ): ChargeResult {
        val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
            .header("Idempotency-Key", idempotencyKey)
            .body("customer=$customerId&amount=2999¤cy=usd")

        val response = withContext(Dispatchers.IO) { httpClient(request) }
        return parseChargeResult(response)
    }
}

// Calling code — stable key computed BEFORE timeout boundary.
suspend fun runMonthlyBilling(customers: List, period: String) {
    val service = BillingService(stripeKey)
    for (customerId in customers) {
        // Stable key computed once per billing operation, outside all retry logic.
        // sha256(customerId:period:http4k-billing)[:32] — same value on every JVM
        // instance, every session, every retry attempt for this (customerId, period) pair.
        val key = stableIdempotencyKey(customerId, period)

        var result: ChargeResult? = null
        var attempts = 0
        while (result == null && attempts < 3) {
            attempts++
            try {
                result = withTimeout(10_000L) {
                    // Same key on every attempt — if Stripe committed ch_A during a
                    // previous timeout, this retry gets the cached response for UUID_A
                    // rather than creating ch_B.
                    service.chargeCustomer(customerId, period, key)
                }
            } catch (e: TimeoutCancellationException) {
                log.warn("Timeout attempt $attempts for $customerId (key=$key)")
                if (attempts < 3) delay(2_000L * attempts) // exponential backoff
            }
        }
        if (result == null) {
            // After 3 timeouts: query Stripe for the charge by the known key
            // before concluding the billing failed.
            val existing = queryStripeForIdempotencyKey(key)
            log.error("Billing failed or timed out for $customerId, existing=$existing")
        }
    }
}

Two properties of this fix: (1) the stable key is computed outside chargeCustomer() and passed in as a parameter — this makes the key visible at the call site, where retry logic lives, rather than hidden inside the function being retried; (2) the same key value is used in every iteration of the retry loop — if Stripe processed the original request during a timeout period, subsequent retries with the same key receive Stripe’s cached response for ch_A rather than triggering ch_B; if Stripe did not process the original request (e.g., the TCP connection was dropped before Stripe read the request body), the retry with the same key creates one fresh charge, which is correct.

A note on timeout configuration: Stripe’s P99 response latency for charge creation is typically under 2 seconds on production infrastructure but can reach 10–20 seconds during degraded conditions. A 5-second withTimeout means timing out on a significant fraction of valid-but-slow Stripe responses. A timeout of 10–15 seconds provides better balance: it cancels genuine hangs (TCP established, Stripe silent for 15 seconds) without triggering spurious timeouts on slow-but-valid responses that would arrive at 6–8 seconds. Spurious timeouts are safe when the idempotency key is stable (the retry gets the cached result), but they increase Stripe API call volume unnecessarily.

Pre-flight database guard: the authoritative layer beneath all three fixes

The three fixes above eliminate the primary causes of duplicate charges in http4k: composition-order-induced UUID regeneration, construction-time static key sharing, and timeout-retry UUID rotation. But stable keys alone do not eliminate all duplicate charge scenarios. Two additional failure scenarios require the database pre-flight guard.

Scenario 1: Concurrent billing triggers. A monthly billing job fires from two instances simultaneously (two Kubernetes pods with un-synchronised clocks, a duplicate cron trigger, a manual backfill overlapping with the scheduled run). Both instances call stableIdempotencyKey("cus_A", "2026-09") and produce the same key. Both send POST /v1/charges to Stripe with that key. If the two requests arrive before either has completed at Stripe, Stripe may return a 409 idempotency_key_in_use error for the second request, indicating that the first is still processing. The 409 does not mean the charge failed — it means the first is in flight. The billing job must handle 409 by retrying with the same key after a delay.

Scenario 2: Stripe’s idempotency window expiry. Stripe stores idempotency key results for 24 hours. A billing job that fails and retries after 24 hours with the same stable key loses the idempotency guarantee: Stripe no longer recognises the key and creates a new charge. A pre-flight database guard catches this: if ch_A was committed and the billing job recorded the stripe_charge_id in the database before the job crashed, a pre-flight SELECT before the Stripe call finds the existing record and skips the call entirely.

The complete pre-flight guard in PostgreSQL with R2DBC or any JDBC driver:

-- billing_records table (create once at application startup):
CREATE TABLE billing_records (
    id               BIGSERIAL PRIMARY KEY,
    customer_id      TEXT        NOT NULL,
    period           TEXT        NOT NULL,      -- e.g. "2026-09"
    idempotency_key  TEXT        NOT NULL,      -- sha256(customerId:period:http4k-billing)[:32]
    stripe_charge_id TEXT,                      -- filled after Stripe confirmation
    created_at       TIMESTAMPTZ DEFAULT NOW(),
    CONSTRAINT billing_records_unique UNIQUE (customer_id, period)
);

-- Pre-flight: attempt to insert the billing record.
-- If INSERT succeeds (returns a row), no prior charge exists: proceed to Stripe.
-- If INSERT returns nothing (ON CONFLICT DO NOTHING), a prior charge exists: skip Stripe.
INSERT INTO billing_records (customer_id, period, idempotency_key)
VALUES ($1, $2, $3)
ON CONFLICT (customer_id, period) DO NOTHING
RETURNING id;

-- After successful Stripe response:
UPDATE billing_records
SET stripe_charge_id = $4
WHERE customer_id = $1 AND period = $2;

-- At billing job start, if stripe_charge_id IS NOT NULL, skip both pre-flight and Stripe call:
SELECT stripe_charge_id
FROM billing_records
WHERE customer_id = $1 AND period = $2;

In http4k code, this guard is placed before the filter chain invocation:

// BillingService.kt — COMPLETE with pre-flight guard.
suspend fun chargeCustomer(customerId: String, period: String): ChargeResult {
    val key = stableIdempotencyKey(customerId, period)

    // Pre-flight: try to insert. If row already exists (charge previously attempted),
    // skip Stripe and return the existing charge result from the database.
    val existingCharge = db.queryOptional(
        "SELECT stripe_charge_id FROM billing_records WHERE customer_id = ? AND period = ?",
        customerId, period
    )
    if (existingCharge?.stripeChargeId != null) {
        return ChargeResult(existingCharge.stripeChargeId, source = "db_cache")
    }

    // Insert the record before calling Stripe.
    // ON CONFLICT DO NOTHING handles the concurrent-billing-job race.
    val inserted = db.update(
        "INSERT INTO billing_records (customer_id, period, idempotency_key) " +
        "VALUES (?, ?, ?) ON CONFLICT (customer_id, period) DO NOTHING",
        customerId, period, key
    )

    if (inserted == 0) {
        // Another instance inserted the record concurrently; poll for stripe_charge_id.
        return pollForChargeCompletion(customerId, period, maxWaitMs = 30_000)
    }

    // Call Stripe with the stable key.
    val request = Request(Method.POST, "https://api.stripe.com/v1/charges")
        .header("Idempotency-Key", key)
        .body("customer=$customerId&amount=2999¤cy=usd")
    val response = withContext(Dispatchers.IO) {
        billingHttpClient(request)
    }
    val result = parseChargeResult(response)

    // Record the charge ID.
    db.update(
        "UPDATE billing_records SET stripe_charge_id = ? WHERE customer_id = ? AND period = ?",
        result.chargeId, customerId, period
    )

    return result
}

Vault keys and spend caps: the proxy layer backstop

The fixes above eliminate the duplicate charge risk from the three http4k failure modes. A residual financial risk remains: any code path that successfully reaches Stripe can, if the billing logic is sufficiently broken, exceed the intended billing amount for a run. The pre-flight database guard prevents duplicate charges for a given (customerId, period) pair, but it does not prevent a billing loop that iterates over a synthetic or externally-injected customer list from creating an unbounded number of distinct charges.

The structural solution is a per-billing-run vault key issued by a spend-cap proxy. The three properties relevant to http4k deployments:

The http4k client that targets the proxy rather than Stripe directly is a one-line change: replace JavaHttpClient() with a client configured to target proxy.keybrake.com/stripe using the vault key as the Authorization header. The filter chain, retry logic, and idempotency key handling remain unchanged. The proxy layer provides the financial cap independently of the application code.

Summary table

Failure mode Root cause Stripe outcome Fix
1. Filter composition order retryFilter.then(uuidFilter) places UUID generation inside the retry boundary — RetryFilter re-invokes uuidFilter per retry — UUID_B UUID_A on first attempt, UUID_B on retry → ch_B alongside committed ch_A UUID-generating filter must be the outermost layer: uuidFilter.then(retryFilter).then(client)
2a. ClientFilters.SetHeader() static UUID Kotlin eager evaluation: UUID.randomUUID().toString() evaluated once at SetHeader() call site — static string applied to every request from the client instance All billing calls share one key — Stripe returns cached ch_A for every customer — no new charges created Use a dynamic per-operation filter with stableIdempotencyKey(customerId, period) computed per call
2b. Dynamic filter inside retry Developer fixes 2a with a dynamic filter but places it inside RetryFilter in the .then() chain UUID_B on retry → ch_B Dynamic filter must be outermost: dynamicFilter.then(retryFilter).then(client)
3. withTimeout + coroutine retry Client-side timeout fires after Stripe committed ch_A; retry invokes chargeCustomer() again, which calls UUID.randomUUID() internally, producing UUID_B ch_A committed + ch_B from retry = two charges Compute stable key outside chargeCustomer(); pass it as parameter; reuse same key on every retry attempt

Per-run vault keys with spend caps for Stripe

Keybrake issues a vault_key_xxx per billing run. You set the USD cap and endpoint allowlist. When the run exceeds the cap or hits a blocked endpoint, the proxy stops forwarding — no code changes needed in the http4k filter chain. Join the waitlist to add Keybrake to your Kotlin billing stack.

Further reading