Azure Durable Functions AI agent API key: scoping vendor calls in durable orchestrations

How Durable Functions AI agent orchestrations call vendor APIs

A typical agent billing orchestrator uses Task.all() to fan out charge activities across all customers:

// orchestrator.ts
import * as df from "durable-functions";

const orchestrator = df.orchestrator(function* (context) {
  // Fetch customer list
  const customers: Customer[] = yield context.df.callActivity("FetchCustomers", {
    planId: context.bindingData.input.planId
  });

  // Fan-out: charge all customers concurrently
  const tasks = customers.map((customer) =>
    context.df.callActivityWithRetry(
      "ChargeCustomer",
      new df.RetryOptions(30_000, 3),
      {
        customer,
        stripeKey: process.env.STRIPE_SECRET_KEY  // ← same key for all activities
      }
    )
  );

  return yield context.df.Task.all(tasks);  // N simultaneous Stripe calls
});

df.app.orchestration("BillingOrchestrator", orchestrator);

This pattern has two compounding risks. First, Task.all() schedules all activity tasks concurrently — for 3,000 customers, this dispatches 3,000 simultaneous activity function executions each calling Stripe with the same STRIPE_SECRET_KEY. Azure's per-Function App concurrency limit is the only constraint; there is no dollar-spend stop condition at the orchestrator level. Second, callActivityWithRetry re-schedules failed activities up to MaxNumberOfAttempts times. If an activity threw an error after Stripe had applied the charge, the retry re-calls Stripe without an idempotency key, creating duplicate charges at the rate of your retry policy.

Three gaps Durable Functions' native tooling doesn't fill for vendor spend control

The fan-out/fan-in amplification risk

Task.all() is the canonical fan-out pattern in Durable Functions and the primary spend amplifier for agent billing workflows. Unlike sequential activity calls (where you can check a running total between calls), Task.all() schedules all activities as concurrent queue messages — each becomes an independent activity function execution with no coordination between instances. A Task.all() over 500 customer records dispatches 500 concurrent Stripe calls. The orchestrator resumes only when the last activity completes — but the vendor charges have already been applied by every activity that succeeded before a cap would have been enforced.

The retry amplification compounds this. callActivityWithRetry with a RetryOptions(30_000, 3) policy reschedules the activity up to 3 additional times on failure. If the activity threw an exception after Stripe had applied the charge (e.g. a network timeout after a successful Stripe API call), the retry re-calls Stripe without an idempotency key — each retry independently charges the customer. An activity that fails after being retried 3 times can produce 4 separate charges for the same customer.

Scoping vault keys per Durable Functions orchestration

// orchestrator.ts — with vault key
import * as df from "durable-functions";

interface VaultKeyResult {
  vault_key: string;
  key_id: string;
  expires_at: string;
}

const orchestrator = df.orchestrator(function* (context) {
  const instanceId = context.df.instanceId;

  // Step 1: Issue a vault key scoped to this orchestration
  const vault: VaultKeyResult = yield context.df.callActivity("IssueVaultKey", {
    vendor: "stripe",
    daily_usd_cap: context.bindingData.input.budgetUsd ?? 500,
    allowed_endpoints: ["POST /v1/payment_intents"],
    expires_in: "2h",
    agent_run_label: `azure-durable/${instanceId}`
  });

  // Step 2: Fetch customers
  const customers: Customer[] = yield context.df.callActivity("FetchCustomers", {
    planId: context.bindingData.input.planId
  });

  // Step 3: Fan-out — all activities share the same vault key and cap
  const tasks = customers.map((customer, itemIndex) =>
    context.df.callActivityWithRetry(
      "ChargeCustomer",
      new df.RetryOptions(30_000, 3, 1, 2, ["CapExhausted"]),  // don't retry CapExhausted
      {
        customer,
        vault_key: vault.vault_key,
        orchestration_id: instanceId,
        item_index: itemIndex
      }
    )
  );

  return yield context.df.Task.all(tasks);
});

df.app.orchestration("BillingOrchestrator", orchestrator);

// charge-customer-activity.ts
import * as df from "durable-functions";

const KEYBRAKE_BASE = "https://proxy.keybrake.com";

df.app.activity("ChargeCustomer", {
  handler: async (input: ChargeInput) => {
    const { customer, vault_key, orchestration_id, item_index } = input;
    const idempotencyKey = `${orchestration_id}-${item_index}`;

    const res = await fetch(`${KEYBRAKE_BASE}/stripe/v1/payment_intents`, {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${vault_key}`,
        "Idempotency-Key": idempotencyKey,
        "Content-Type": "application/json"
      },
      body: JSON.stringify({
        amount: customer.amount_cents,
        currency: "usd",
        customer: customer.id
      })
    });

    if (res.status === 429) {
      const body = await res.json();
      if (body.code === "cap_exhausted") {
        // Throw a named class so RetryOptions can exclude it
        class CapExhausted extends Error {
          constructor(message: string) { super(message); this.name = "CapExhausted"; }
        }
        throw new CapExhausted(body.message);
      }
    }

    if (!res.ok) throw new Error(`Stripe error: ${res.status}`);
    return res.json();
  }
});

df.app.activity("IssueVaultKey", {
  handler: async (input: VaultKeyInput): Promise<VaultKeyResult> => {
    const res = await fetch(`${KEYBRAKE_BASE}/vault/keys`, {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${process.env.KEYBRAKE_API_KEY}`,
        "Content-Type": "application/json"
      },
      body: JSON.stringify(input)
    });
    if (!res.ok) throw new Error(`Keybrake error: ${res.status}`);
    return res.json();
  }
});

The IssueVaultKey activity issues the vault key once at the start of the orchestration. The orchestrator passes vault.vault_key along with instanceId and itemIndex into each fan-out activity's input. All concurrent ChargeCustomer activities receive the same vault key and the same cap accumulates atomically across all concurrent executions. The idempotency key orchestration_id-item_index is stable across retries: Durable Functions replays the orchestrator deterministically with the same instanceId, and the same itemIndex is computed from the same array position. RetryOptions with a handle exclusion list prevents cap exhaustion errors from being retried.

How Keybrake fits

Keybrake is the proxy layer between your charge activity functions and Stripe, Twilio, or Resend. The vault key issued in IssueVaultKey replaces the full-access Stripe key previously loaded from Key Vault into activity function process memory. The real Stripe secret stays in Keybrake — it never appears in Application Insights telemetry, Durable Functions execution history, or activity function environment variables. For Task.all() fan-out, all concurrent activities receive the same vault key in their input and the same cap accumulates atomically. Revoking a runaway orchestration is a single DELETE /vault/keys/{key_id} call — effective on the next proxied request from any activity, with no terminateAsync call, no Key Vault rotation, and no impact on other orchestrations running different vault keys.

Get early access

Related questions

Further reading

• Temporal AI agent API key — similar per-workflow vault key pattern; Temporal Activities map to Durable Functions activities and the same idempotency-key-from-workflow-ID pattern applies.
• AWS Step Functions AI agent API key — the AWS equivalent of Durable Functions fan-out, with Map state replacing Task.all() and the same vault-key-via-ItemSelector pattern.
• AI agent idempotency — why orchestration-ID-based idempotency keys are essential when callActivityWithRetry re-executes failed activities that may have already charged Stripe.
• AI agent spend reporting — the four SQL queries that give per-orchestration cost visibility that Application Insights doesn't provide natively.