Relevance AI Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

Relevance AI makes it easy to build AI agents that call external APIs as Tools. The same design that makes it easy also makes billing governance invisible — until a Tool retry fires a second Stripe charge, a bulk run doubles your customer billing, or a shared workspace key lets every agent in the project spend against one Stripe account with no cap.

Relevance AI is a no-code platform for building AI agents with custom Tools — HTTP requests, code execution, LLM transforms, and data lookups chained into multi-step agent workflows. Its built-in Agent builder lets non-engineers deploy autonomous pipelines that read CRMs, draft emails, and call payment APIs like Stripe without writing application code. The power is real. So is the billing risk when those agents handle money.

This post covers three failure modes specific to Relevance AI's architecture, concrete configuration examples for each, and the two-layer governance pattern — content-hash idempotency keys plus per-run vault keys via a spend-cap proxy — that closes all three.

Failure mode 1: Tool step retry re-fires a completed Stripe charge

When an agent run fails partway through a multi-step Tool chain, Relevance AI can be configured to retry from the failed step or from the beginning of the sequence. The problem is that "retry" means re-executing the HTTP request block — which may re-send a Stripe POST /v1/charges request that already succeeded before the failure occurred downstream.

A typical pattern looks like this: Step 1 calls Stripe to create a charge. Step 2 updates a record in a CRM. Step 3 sends a confirmation email via Resend. If the Resend call fails with a 500 error and the run is retried, the retry re-executes from Step 1. The Stripe charge from the original run already succeeded — the customer was billed. The retry bills them again.

The sequence matters because Stripe's response from Step 1 is not available to a fresh retry execution. The agent has no memory that Step 1 already completed. From its perspective, it is starting a fresh run with the same inputs, and Step 1 is just a Stripe HTTP request that needs to fire.

The Relevance AI Tool for the charge step might look like this:

# Relevance AI HTTP Tool — charge_customer (without idempotency key)
POST https://api.stripe.com/v1/charges
Authorization: Bearer sk_live_EXPOSED_KEY
Content-Type: application/x-www-form-urlencoded

amount={{amount_cents}}
currency=usd
customer={{stripe_customer_id}}
description=Monthly subscription {{billing_period}}

Without an idempotency key, every POST to /v1/charges creates a new charge object, regardless of whether an identical charge was created seconds before. A retry on a transient email API failure turns into a duplicate billing event.

The fix is a stable idempotency key derived from the billing parameters, not from the run:

# Relevance AI HTTP Tool — charge_customer (with idempotency key)
POST https://proxy.keybrake.com/stripe/v1/charges
Authorization: Bearer {{vault_key}}
Content-Type: application/x-www-form-urlencoded
Idempotency-Key: {{idempotency_key}}

amount={{amount_cents}}
currency=usd
customer={{stripe_customer_id}}
description=Monthly subscription {{billing_period}}

The idempotency_key input to this Tool is generated by a preceding Code Tool step:

# Relevance AI Code Tool — generate_idempotency_key
import hashlib

customer_id = params["customer_id"]
amount_cents = str(params["amount_cents"])
billing_period = params["billing_period"]

raw = f"{customer_id}:{amount_cents}:{billing_period}:relevance-billing"
return {"idempotency_key": hashlib.sha256(raw.encode()).hexdigest()[:32]}

Because the key is derived from customer_id + amount_cents + billing_period, it is identical across all retries of the same billing intent. Stripe returns the existing charge object rather than creating a new one. The customer is billed exactly once regardless of how many times the Tool step executes.

Failure mode 2: Bulk agent trigger runs execute in parallel with no cross-run deduplication

Relevance AI allows bulk triggering an agent across a list of inputs — for example, a list of customers to bill for the current month. Each customer becomes an independent agent run. Runs execute concurrently.

The billing risk emerges in two scenarios. First, if the same bulk trigger is fired twice (a button click, an API call, a webhook that retried because the acknowledgment was slow), Relevance AI creates two sets of independent runs — one for each trigger event — with identical input data. Both sets reach the Stripe billing Tool step at roughly the same time. Stripe receives two POST /v1/charges requests for each customer, seconds apart, with no idempotency key. Each customer in the list is billed twice.

Second, within a single bulk trigger, if the customer list contains a duplicate (a data cleaning bug, a pagination artifact, a test record that matches production), both entries create agent runs that execute in parallel and both call the billing Tool with the same customer ID and amount. Again, two charges, one customer.

Content-hash idempotency keys handle both scenarios. Because the key is derived from billing parameters rather than from the run or trigger event, two runs with identical billing intent produce the same key. Stripe deduplicates at the API layer — the second request returns the existing charge object with HTTP 200, not a new charge.

The same Code Tool step that generates the key for single-run retries works identically for parallel bulk runs:

# Works for both retry dedup and parallel bulk run dedup
raw = f"{customer_id}:{amount_cents}:{billing_period}:relevance-billing"
idempotency_key = hashlib.sha256(raw.encode()).hexdigest()[:32]

No changes to the bulk trigger configuration or the agent workflow are needed. The idempotency key is the deduplication mechanism — it works at the Stripe API layer, independent of Relevance AI's execution model.

Failure mode 3: Workspace Tool definitions share one Stripe key across all agents

Relevance AI Tools store their configuration — including API keys — at the workspace level. Every agent in the workspace that uses the Stripe HTTP Tool accesses the same credential. There is no per-agent key scoping in the Tool definition itself.

The consequence is that a runaway agent, a misconfigured test workflow, or an intern who accidentally enables the wrong agent can drain the Stripe account's balance or exhaust a daily transaction budget. A stuck billing loop in one agent affects every agent in the workspace because they all share the key. Revoking the key to stop the runaway loop breaks every other agent simultaneously.

More specifically, the shared key has no endpoint allowlist. Any agent with access to the Stripe Tool can call any Stripe endpoint the key permits — not just POST /v1/charges, but also POST /v1/refunds, DELETE /v1/subscriptions/{id}, or any other endpoint the key has permission for. A well-intentioned refund tool and a billing tool sharing the same workspace key means a prompt-injection in the refund agent can reach the billing scope.

The solution is vault keys at the proxy layer. Instead of embedding a real Stripe key in the Tool definition, you embed a vault key scoped to exactly the endpoint and spend limit the agent is allowed to use:

# Relevance AI Tool — billing agent's charge_customer step
POST https://proxy.keybrake.com/stripe/v1/charges
Authorization: Bearer krv_billing_agent_Abc123
#                         ^^^ vault key: POST /v1/charges only, $500/day cap

# Relevance AI Tool — audit agent's get_charge_status step
GET https://proxy.keybrake.com/stripe/v1/charges/{{charge_id}}
Authorization: Bearer krv_audit_agent_Xyz789
#                         ^^^ vault key: GET /v1/charges/* only, read-only

Each vault key is issued at proxy.keybrake.com with a policy attached: {"allowed_endpoints": ["POST /stripe/v1/charges"], "daily_usd_cap": 500, "expires_at": "2026-07-01"}. The billing vault key literally cannot call POST /v1/refunds — the proxy returns a 403 before the request reaches Stripe. The audit vault key cannot create charges. A runaway billing loop exhausts the $500/day cap and stops; the audit agent is unaffected because it uses a separate key with a separate cap.

Revoking a runaway agent's key is a one-click action at the proxy dashboard. The real Stripe key never changes. Every other agent continues operating normally.

Combined governance pattern for Relevance AI

The three fixes above work together in a single agent workflow:

1. Code Tool (Step 1): Generate a stable idempotency key from customer_id + amount_cents + billing_period. Pass it as an output to subsequent steps.
2. HTTP Tool (Step 2): POST to proxy.keybrake.com/stripe/v1/charges with Authorization: Bearer {{billing_vault_key}} and Idempotency-Key: {{idempotency_key}} headers. The vault key is stored as a Relevance AI secret (not embedded in the Tool definition directly); the billing_vault_key is scoped to POST /v1/charges with a daily USD cap equal to the maximum expected single billing run.
3. Code Tool (Step 3, optional): Check the proxy audit log to confirm the charge was created, not deduped. If the idempotency key matched an existing charge, log it as a detected duplicate rather than a billing error — this is the correct behavior.

The workflow changes are minimal. The governance layer is additive: Relevance AI's visual workflow stays the same; the only change is the URL (Stripe → proxy) and the credential (real key → vault key with policy).

Comparison: Relevance AI vs. other no-code platforms

Enforcement tests

import hashlib
import pytest

def generate_idempotency_key(customer_id: str, amount_cents: int, billing_period: str) -> str:
    raw = f"{customer_id}:{amount_cents}:{billing_period}:relevance-billing"
    return hashlib.sha256(raw.encode()).hexdigest()[:32]

def test_key_is_stable_across_retries():
    k1 = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    k2 = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    assert k1 == k2

def test_different_customers_get_different_keys():
    k1 = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    k2 = generate_idempotency_key("cus_XYZ", 9900, "2026-06")
    assert k1 != k2

def test_different_billing_periods_get_different_keys():
    k1 = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    k2 = generate_idempotency_key("cus_ABC", 9900, "2026-07")
    assert k1 != k2

def test_parallel_bulk_runs_produce_same_key():
    # Two concurrent runs with the same customer data must produce the same key
    run_a = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    run_b = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    assert run_a == run_b, "Parallel bulk runs must not produce duplicate charges"

def test_key_length_is_stripe_safe():
    k = generate_idempotency_key("cus_ABC", 9900, "2026-06")
    assert len(k) <= 255  # Stripe's Idempotency-Key max length

Gap analysis

Relevance AI's "Knowledge" retrieval can surface prior billing records and trigger re-execution

Relevance AI agents can query a Knowledge base (vector store) as a Tool step. If a billing agent's Knowledge base contains records of past charges — customer IDs, amounts, billing periods — a poorly scoped retrieval query can surface them as context during the current run. If the LLM interprets prior billing records as "pending" rather than "completed," it may call the charge Tool with parameters from a prior period. Content-hash idempotency keys prevent the charge from creating a new Stripe object (the prior period key differs from the current period key), but the failed attempt still fires an HTTP request and generates a proxy audit log entry that looks like an attempt to double-bill the prior period. Exclude completed billing records from the Knowledge base, or scope Knowledge queries to only retrieve customer metadata, not transaction history.

Long-running agents accumulate conversation history that includes prior billing tool calls

Relevance AI's multi-step agents maintain conversation context across steps in a single run. In an agent that handles both billing and customer service in a single workflow, prior billing Tool calls and their responses appear in the conversation as completed actions. On a follow-up trigger or a re-run with a similar prompt, the LLM may interpret the accumulated history as an instruction to repeat the billing action. The content-hash idempotency key with an explicit billing_period parameter prevents cross-period duplicates; however, within the same period, a second charge attempt with identical parameters and the same key returns the same Stripe charge object — which the agent may misinterpret as a successful new charge rather than a Stripe dedup. Always log the Stripe Request-Id and check whether it matches the prior run's charge ID to detect dedup events.

Webhook triggers with Relevance AI's trigger delay setting create time-window duplicates

Relevance AI agents triggered by webhooks can be configured with a trigger delay — a wait period before execution begins. If the upstream webhook sender retries before the delay expires (e.g., because the Relevance AI acknowledgment was slow), a second independent agent run is created with the same trigger data. Both runs execute after their respective delays, with identical inputs. Content-hash idempotency keys handle this correctly — both runs produce the same key for the billing step and only one charge is created. But the proxy audit log will show two billing attempts from two different Relevance AI run IDs. Use this signal to detect upstream webhook sender misconfiguration (retry intervals too short) and fix the trigger delivery settings.

Code Tool execution environment has access to installed packages

Relevance AI Code Tools run Python in a sandboxed environment with certain packages available. If stripe is available in that environment, a Code Tool can call stripe.Charge.create() directly — bypassing the HTTP Tool governance pattern entirely. The idempotency key and proxy URL are configured in the HTTP Tool step; a Code Tool that calls Stripe directly via the Python SDK uses neither. Audit your Relevance AI workspace for any Code Tool that imports stripe or references sk_live_ strings, and replace them with HTTP Tool steps that route through the proxy.

FAQ

Can I use the Relevance AI run ID as the idempotency key?

No. Each retry and each bulk trigger event generates a new run ID. Using the run ID as the idempotency key means the second run produces a different key — Stripe creates a second charge. The key must be derived from the billing parameters (customer, amount, period), which are stable across all run IDs that represent the same billing intent. A content-hash key survives retries; a run ID key doesn't.

What if I need to charge the same customer twice in the same billing period?

Include a charge type in the idempotency key material: customer_id + amount_cents + billing_period + charge_type, where charge_type is "setup" for a one-time setup fee and "recurring" for the subscription charge. The two keys are different, so both charges are created. Stripe only deduplicates requests with identical idempotency keys — different keys create different charges as expected.

My daily USD cap is exhausted mid-bulk-run. What happens to the remaining customers?

The proxy returns a 402 for any request that would exceed the cap. The Relevance AI HTTP Tool receives a 402 response and marks the step as failed. The bulk run shows some agents as completed (below the cap limit) and some as failed (at and above the cap). Do not configure automatic retry on 402 — the cap is genuinely exhausted and retrying won't help. Raise the cap at the proxy dashboard and re-trigger the bulk run with the failed customers' data. The content-hash idempotency keys ensure that customers already charged are not charged again on the re-run.

Does routing through the proxy add latency to my Relevance AI agent runs?

The proxy adds sub-10ms per request for the policy lookup and key validation. Stripe's own API latency is typically 200–500ms, so the proxy overhead is less than 5% of the total Stripe step duration. For a bulk run processing 200 customers, the additional latency is under 2 seconds for the entire batch. Relevance AI's Agent runs are already network-bound; the proxy overhead is negligible in practice.

How do I issue a vault key scoped to Relevance AI's billing agent specifically?

Create a vault key at proxy.keybrake.com/dashboard with a policy scoped to the billing agent's actual scope: {"vendor": "stripe", "allowed_endpoints": ["POST /v1/charges"], "daily_usd_cap": 500, "expires_at": "2026-07-01"}. Store the vault key in Relevance AI's Secrets manager (Settings → Secrets) as KEYBRAKE_BILLING_VAULT_KEY. Reference it in the HTTP Tool as {{KEYBRAKE_BILLING_VAULT_KEY}} in the Authorization header. The real Stripe key never appears in any Relevance AI Tool definition — only the vault key does, and it can be revoked without affecting your Stripe account or any other agent.

Is there a way to test the idempotency and spend cap behavior before a live bulk run?

Use Stripe test mode with a test-mode vault key at the proxy. Issue a vault key with a low daily cap (e.g., $10) and run a small test bulk trigger (5–10 customers). Then manually re-trigger the same bulk run. The second run should show all agents completing with the same Stripe charge IDs as the first run — no new charges. The proxy audit log confirms that the second run's billing requests matched the first run's idempotency keys and returned existing charge objects. Once this passes in test mode, swap to a production vault key with the appropriate daily cap.