Snowflake Tasks Stripe Integration: Restricted API Keys, Spend Caps, and Agent Governance

By Keybrake · July 16, 2026

Snowflake Tasks are Snowflake's native job scheduler: SQL or Snowpark stored procedures that run on a cron schedule or when a stream detects new data. Teams wire Tasks into Stripe billing to close the loop between data-warehouse usage aggregates and SaaS charges — the Task reads from a billing queue table populated by upstream transformations, calls the Stripe API for each customer row via a Snowpark Python stored procedure, and writes the charge results back into a billing results table in the warehouse. Three Snowflake-specific behaviors introduce Stripe double-charge failure modes that Snowflake's Task History and Error Notifications do not surface as billing risk.

This post covers those three failure modes with Snowpark Python stored procedure code and the two-layer governance pattern — content-hash idempotency keys plus per-task-run vault keys via a spend-cap proxy — that eliminates all three without restructuring your Task DAG.

Failure mode 1: `ALLOW_OVERLAPPING_EXECUTION = TRUE` lets two billing task instances charge the same customers simultaneously

Snowflake Tasks have a parameter called ALLOW_OVERLAPPING_EXECUTION. The default is FALSE, which means Snowflake will skip a scheduled run if the previous run has not yet completed — protecting against concurrent execution. The setting is typically changed to TRUE by teams whose Tasks are slow to initialize a Snowpark Python environment or whose billing stored procedures process large customer sets and occasionally run past the next scheduled start time. The intent is to avoid skipped runs. The consequence is concurrent billing executions.

When ALLOW_OVERLAPPING_EXECUTION = TRUE and a billing Task takes longer than expected — say, a billing period with 10× the normal customer volume, or a Snowpark compute cluster that was cold and took two minutes to spin up — the next scheduled run starts while the first is still executing. Both task instances call SELECT * FROM billing_queue WHERE status = 'pending' and read the same set of unbilled customer rows. Both iterate over those rows. Both call stripe.charges.create() for each customer. The first task marks rows as billed in the billing results table. The second task's UPDATE billing_queue SET status = 'processed' runs on rows that were just updated by the first task — but both Stripe charges have already been submitted.

-- UNSAFE: Snowflake Task definition with overlapping execution allowed
-- A slow billing run + ALLOW_OVERLAPPING_EXECUTION = TRUE produces two
-- concurrent Snowpark stored procedure executions reading the same pending queue

CREATE OR REPLACE TASK billing.charge_customers_task
  WAREHOUSE = BILLING_WH
  SCHEDULE = 'USING CRON 0 2 * * * America/New_York'
  ALLOW_OVERLAPPING_EXECUTION = TRUE  -- UNSAFE: enables concurrent billing runs
AS
  CALL billing.charge_pending_customers();

The most common scenario: a monthly billing run at the end of the month processes more customers than usual — enterprise accounts added during the month, reactivations, annual plan renewals. The billing Task, normally completing in 8 minutes, takes 22 minutes on the first day of the billing cycle. The next scheduled 15-minute interval fires at minute 15 while the first is still running. Both instances read the 3,400 customers in the pending queue. The second instance finishes faster (the Snowpark environment is warm from the first), and 3,400 customers receive two charges.

# SAFE: Snowpark Python stored procedure with advisory lock + idempotency keys
# Uses a billing_run_locks table to prevent concurrent billing executions
# even when ALLOW_OVERLAPPING_EXECUTION = TRUE at the Task level
#
# billing/charge_pending_customers.py (Snowpark stored procedure)

import hashlib
import datetime
import stripe

VAULT_KEY = "vk_live_..."  # per-billing-period vault key from Keybrake
PROXY_BASE = "https://proxy.keybrake.com/stripe"

def charge_pending_customers(session) -> str:
    billing_period = datetime.datetime.utcnow().strftime("%Y-%m")
    lock_id = f"billing-{billing_period}"

    # Acquire advisory lock: INSERT fails if lock row already exists
    # for this billing period, preventing concurrent execution
    try:
        session.sql(f"""
            INSERT INTO billing.billing_run_locks (lock_id, acquired_at, task_run_id)
            VALUES ('{lock_id}', CURRENT_TIMESTAMP(), CURRENT_TASK_GRAPH_RUN_GROUP_ID())
        """).collect()
    except Exception:
        # Lock already held — another task run is active for this billing period
        return f"skipped: lock held for {billing_period}"

    try:
        stripe.api_key  = VAULT_KEY
        stripe.api_base = PROXY_BASE

        pending_df = session.sql("""
            SELECT q.customer_id, q.amount_cents, q.stripe_customer_id,
                   q.billing_period
            FROM billing.billing_queue q
            LEFT JOIN billing.billing_results r
              ON r.customer_id = q.customer_id
             AND r.billing_period = q.billing_period
            WHERE q.status = 'pending'
              AND r.charge_id IS NULL  -- pre-flight: skip if already charged
        """).to_pandas()

        charged = 0
        for _, row in pending_df.iterrows():
            raw_key = f"{row['CUSTOMER_ID']}:{row['AMOUNT_CENTS']}:{row['BILLING_PERIOD']}:snowflake-billing"
            idempotency_key = hashlib.sha256(raw_key.encode()).hexdigest()[:32]

            try:
                charge = stripe.Charge.create(
                    amount=int(row["AMOUNT_CENTS"]),
                    currency="usd",
                    customer=row["STRIPE_CUSTOMER_ID"],
                    description=f"Billing for {row['BILLING_PERIOD']}",
                    idempotency_key=idempotency_key
                )
                # Write result immediately — survives concurrent + partial runs
                session.sql(f"""
                    INSERT INTO billing.billing_results
                      (customer_id, billing_period, charge_id, charged_at)
                    VALUES ('{row["CUSTOMER_ID"]}', '{row["BILLING_PERIOD"]}',
                            '{charge.id}', CURRENT_TIMESTAMP())
                """).collect()
                charged += 1
            except Exception as e:
                session.sql(f"""
                    INSERT INTO billing.billing_errors
                      (customer_id, billing_period, error_msg, errored_at)
                    VALUES ('{row["CUSTOMER_ID"]}', '{row["BILLING_PERIOD"]}',
                            '{str(e)[:500]}', CURRENT_TIMESTAMP())
                """).collect()

        return f"charged {charged} customers for {billing_period}"

    finally:
        # Always release the lock — even on error
        session.sql(f"""
            DELETE FROM billing.billing_run_locks WHERE lock_id = '{lock_id}'
        """).collect()

The advisory lock uses a billing_run_locks table with a unique constraint on lock_id. The first task instance inserts the lock row successfully. The second instance's INSERT fails with a unique constraint violation, and the stored procedure returns immediately without calling Stripe. The idempotency key provides a second guard within Stripe's 24-hour deduplication window.

Failure mode 2: Snowflake serverless compute suspension terminates a mid-run billing stored procedure

Snowflake Tasks can run on serverless compute clusters instead of virtual warehouses. Serverless Tasks automatically provision and scale compute, suspending when idle to reduce cost. The suspension model introduces a failure mode specific to long-running billing stored procedures.

When a serverless compute cluster is executing a Snowflake Task stored procedure and the cluster's auto-suspend window is shorter than the stored procedure's execution time, Snowflake may terminate the procedure mid-execution. This is uncommon in simple SQL Tasks but occurs in Snowpark Python stored procedures that make serial external API calls — 500 customers at 150ms per Stripe API round-trip takes 75 seconds, longer than a serverless cluster's minimum suspension window for some configurations. The task is marked as FAILED in Task History. Snowflake's Error Notification integration fires if configured. The next scheduled run starts, reads the same billing_queue WHERE status = 'pending' set — because the terminated procedure did not commit a clean UPDATE before termination — and re-processes every customer from the beginning.

-- What the task history looks like after serverless suspension terminates the run
--
-- SELECT *
--   FROM TABLE(information_schema.task_history(
--     SCHEDULED_TIME_RANGE_START => DATEADD('hour', -2, CURRENT_TIMESTAMP()),
--     TASK_NAME => 'CHARGE_CUSTOMERS_TASK'
--   ))
--   ORDER BY SCHEDULED_TIME DESC;
--
-- TASK_NAME              | STATE  | ERROR_MESSAGE
-- charge_customers_task  | FAILED | Execution timed out after 60 seconds
-- charge_customers_task  | SUCCEEDED | ...    <-- previous run

-- The FAILED run called Stripe for customers 1-387 before termination.
-- The next SUCCEEDED run re-calls Stripe for customers 1-387 again
-- because the billing_queue table still shows them as status = 'pending'
-- (the terminated procedure never committed the UPDATE to 'processed').

The root issue is not the serverless suspension itself — it is that the billing stored procedure updates the queue status after the billing loop, in a single UPDATE billing_queue SET status = 'processed' WHERE ... at the end. When the procedure is terminated mid-loop, neither the per-row inserts into billing_results nor the queue status update has happened for any rows, so the entire customer set appears unbilled to the next run.

The fix is the same row-by-row persistence pattern used in the overlapping execution fix: write a billing result row immediately after each successful Stripe charge, inside the loop, before moving to the next customer. The pre-flight JOIN between billing_queue and billing_results then correctly skips customers charged by a terminated run.

-- Create the billing_results table with a unique constraint on (customer_id, billing_period)
-- so that concurrent or repeated inserts for the same customer-period are rejected

CREATE TABLE IF NOT EXISTS billing.billing_results (
  customer_id    VARCHAR(255) NOT NULL,
  billing_period VARCHAR(7)   NOT NULL,   -- 'YYYY-MM'
  charge_id      VARCHAR(255) NOT NULL,
  charged_at     TIMESTAMP_NTZ NOT NULL DEFAULT CURRENT_TIMESTAMP(),
  CONSTRAINT uq_billing_results UNIQUE (customer_id, billing_period)
);

-- The pre-flight query in the stored procedure then becomes:
--
--   SELECT q.*
--   FROM billing.billing_queue q
--   LEFT JOIN billing.billing_results r
--     ON r.customer_id = q.customer_id
--    AND r.billing_period = q.billing_period
--   WHERE q.billing_period = CURRENT_BILLING_PERIOD
--     AND r.charge_id IS NULL   -- NULL join = not yet in results table
--
-- A task run that terminated after charging customers 1-387 wrote 387 rows
-- into billing_results before termination. The next run's pre-flight JOIN
-- returns only the 113 customers with no billing_results row — re-running
-- the termination-safe Snowpark code charges only those 113.

Set the serverless Task's USER_TASK_TIMEOUT_MS generously — billing stored procedures should have a timeout long enough to complete the full customer set. But the row-by-row write pattern means a timeout-terminated run leaves a correct partial record rather than an all-or-nothing failure: the next run only charges the remainder.

-- Snowflake Task definition with generous timeout and serverless compute
-- The stored procedure's row-by-row write pattern handles partial runs correctly

CREATE OR REPLACE TASK billing.charge_customers_task
  USER_TASK_MANAGED_INITIAL_WAREHOUSE_SIZE = 'XSMALL'  -- serverless compute
  SCHEDULE = 'USING CRON 0 2 1 * * America/New_York'   -- monthly at 2 AM
  ALLOW_OVERLAPPING_EXECUTION = FALSE
  USER_TASK_TIMEOUT_MS = 600000  -- 10 minutes: generous for up to 4,000 customers
  ERROR_INTEGRATION = BILLING_ERROR_NOTIFY  -- fire on failure for human visibility
AS
  CALL billing.charge_pending_customers();

Failure mode 3: Task DAG root task retry re-triggers already-successful child billing tasks

Snowflake Task DAGs allow a root task to trigger one or more child tasks after it completes successfully. A common billing architecture uses this pattern: a root task builds or refreshes the billing queue (aggregating usage data from raw tables, computing amounts, writing rows into billing_queue), and then triggers child tasks — one per billing tier or geographic region — each of which processes a subset of the billing queue and calls Stripe for its customer segment.

The failure mode occurs when a child task fails and the team's incident response is to re-trigger the root task. The root task runs successfully, rebuilding the billing queue — which now includes customers from both the failed child task's segment and the successful child task's segment, because the queue rebuild is based on usage data and not on whether Stripe was already called. The successful child task's segment customers appear in the rebuilt queue as unbilled (their queue rows were marked as processed, but the queue was rebuilt from scratch). When the root task triggers all children, the child task that previously succeeded re-processes its customer segment, calling Stripe for customers it already charged.

-- UNSAFE Task DAG: root task rebuilds the billing queue from scratch on every run.
-- If child_task_region_us succeeds but child_task_region_eu fails,
-- re-triggering root_task causes child_task_region_us to re-charge US customers.

CREATE OR REPLACE TASK billing.root_task
  WAREHOUSE = BILLING_WH
  SCHEDULE = 'USING CRON 0 2 1 * * UTC'
AS
  -- Rebuilds billing queue from usage data — does NOT check if already billed
  INSERT OVERWRITE INTO billing.billing_queue
  SELECT customer_id, SUM(usage_units) * unit_price AS amount_cents,
         stripe_customer_id, TO_CHAR(CURRENT_DATE(), 'YYYY-MM') AS billing_period
  FROM analytics.customer_usage
  WHERE usage_month = TO_CHAR(CURRENT_DATE(), 'YYYY-MM')
  GROUP BY 1, 3, 4;

CREATE OR REPLACE TASK billing.child_task_region_us
  WAREHOUSE = BILLING_WH
  AFTER billing.root_task
AS
  CALL billing.charge_pending_customers_by_region('US');

CREATE OR REPLACE TASK billing.child_task_region_eu
  WAREHOUSE = BILLING_WH
  AFTER billing.root_task
AS
  CALL billing.charge_pending_customers_by_region('EU');

The correct incident response when a child task fails is to re-trigger the failed child task directly — not the root task. But in practice, teams trigger the root task because it is the entry point they know, or because they want to ensure the billing queue reflects the latest usage data. The result is a double-charge for customers in every segment that previously succeeded.

-- SAFE: Root task uses MERGE instead of INSERT OVERWRITE
-- Only inserts new queue rows for customers not already in billing_results
-- Re-triggering the root task does not re-add already-billed customers

CREATE OR REPLACE TASK billing.root_task
  WAREHOUSE = BILLING_WH
  SCHEDULE = 'USING CRON 0 2 1 * * UTC'
AS
  -- MERGE: only insert queue rows for customers with no billing_results entry
  MERGE INTO billing.billing_queue AS q
  USING (
    SELECT u.customer_id,
           SUM(u.usage_units) * p.unit_price AS amount_cents,
           u.stripe_customer_id,
           TO_CHAR(CURRENT_DATE(), 'YYYY-MM') AS billing_period
    FROM analytics.customer_usage u
    JOIN billing.pricing p ON p.customer_id = u.customer_id
    LEFT JOIN billing.billing_results r
           ON r.customer_id = u.customer_id
          AND r.billing_period = TO_CHAR(CURRENT_DATE(), 'YYYY-MM')
    WHERE u.usage_month = TO_CHAR(CURRENT_DATE(), 'YYYY-MM')
      AND r.charge_id IS NULL  -- exclude already-billed customers
    GROUP BY 1, 3, 4, p.unit_price
  ) AS src
  ON q.customer_id = src.customer_id
 AND q.billing_period = src.billing_period
  WHEN NOT MATCHED THEN INSERT (
    customer_id, amount_cents, stripe_customer_id, billing_period, status
  ) VALUES (
    src.customer_id, src.amount_cents, src.stripe_customer_id,
    src.billing_period, 'pending'
  );

With MERGE instead of INSERT OVERWRITE, re-triggering the root task is safe: the MERGE only inserts queue rows for customers not yet in billing_results. Customers successfully charged by a prior child task run have rows in billing_results and are excluded from the MERGE. The successful child task re-runs but finds no pending queue rows for its segment and exits cleanly. Only the failed child task's customers — who were never inserted into billing_results — appear in the rebuilt queue.

The two-layer governance pattern for Snowflake Tasks

All three failure modes share the same structural fix, applied at two layers that together provide defense in depth:

Layer 1 — Content-hash idempotency key. Every Stripe API call in the Snowpark stored procedure uses an idempotency key derived from stable business inputs: customer ID, amount in cents, and billing period. The key is a SHA-256 hash of those concatenated values, truncated to 32 characters. Within Stripe's 24-hour deduplication window, a second call with the same idempotency key returns the original charge object rather than creating a new charge. This is a last-resort guard that catches double-charges the pre-flight check misses — for example, if two task instances read the queue in the same second before either has written a billing result row.

import hashlib

def make_idempotency_key(customer_id: str, amount_cents: int, billing_period: str) -> str:
    raw = f"{customer_id}:{amount_cents}:{billing_period}:snowflake-billing"
    return hashlib.sha256(raw.encode()).hexdigest()[:32]

# Usage in the billing loop:
charge = stripe.Charge.create(
    amount=amount_cents,
    currency="usd",
    customer=stripe_customer_id,
    description=f"Billing for {billing_period}",
    idempotency_key=make_idempotency_key(customer_id, amount_cents, billing_period)
)

Layer 2 — Per-task-run vault key with a spend cap. Instead of using a long-lived Stripe live key stored in Snowflake Secrets, the billing stored procedure fetches a vault key from a spend-cap proxy — Keybrake in this architecture — at the start of each billing run. The vault key is scoped to a daily dollar cap equal to approximately 110% of the expected billing run total (computed from the billing queue count and average customer charge). If a double-charge scenario bypasses Layer 1, the proxy rejects Stripe charges that would push the run past the cap. Every request is logged in an audit trail queryable from the Keybrake dashboard.

The vault key is issued per billing run, not per month. A run that fails and is retried gets a fresh vault key — with a fresh cap reset to the remaining uncharged customer set's total. This means a billing retry that processes 113 remaining customers (from the 500-customer total) is capped at 113 customers' worth of charges, not the full 500-customer total, even if someone mistakenly re-runs the stored procedure against the full queue.

Failure mode	Primary fix	Idempotency key guard	Vault key cap guard
`ALLOW_OVERLAPPING_EXECUTION` concurrent runs	Advisory lock table; second instance exits on lock contention	Same key for same customer-period across instances	Cap blocks second instance's total if it gets past the lock
Serverless suspension + terminated mid-run	Row-by-row `billing_results` write inside loop; pre-flight JOIN on next run	Same key for same customer-period across runs	Cap sized to remaining customers, not full set
Task DAG root retry re-triggers successful children	MERGE instead of INSERT OVERWRITE; excludes already-billed customers from queue	Same key for same customer-period if child re-runs with full queue	Child finds empty queue or cap rejects excess

Snowflake audit gaps vs. Stripe billing audit

Snowflake Task History records task run start time, end time, state (SUCCEEDED, FAILED, SKIPPED), and error message. It does not record what external API calls were made during the task run, how many Stripe charges were created, or the total dollar amount charged to customers. A task run that shows SUCCEEDED in Snowflake Task History does not guarantee that every customer was charged exactly once — it only guarantees that the stored procedure returned without raising an uncaught exception.

Keybrake's audit log fills this gap: every proxied Stripe request is logged with timestamp, vault key, Stripe endpoint, request amount, Stripe request ID, and response status code. You can query the audit log after a billing run to verify the exact set of Stripe charges made during that task's vault key lifetime, compare it against the expected billing queue count, and confirm no customer appears twice. The audit log is queryable via the Keybrake dashboard and exportable as CSV for reconciliation against your Snowflake billing results table.

Connecting Snowflake Secrets to Keybrake vault keys

Snowflake supports external secrets management via the Secrets DDL. You can store the Keybrake API key as a Snowflake secret and reference it in the stored procedure without hardcoding credentials in the procedure body:

-- Create a Snowflake secret for the Keybrake API key
CREATE OR REPLACE SECRET billing.keybrake_api_key
  TYPE = GENERIC_STRING
  SECRET_STRING = 'kb_api_your_keybrake_api_key_here';

-- Grant the BILLING_ROLE access to use the secret
GRANT USAGE ON SECRET billing.keybrake_api_key TO ROLE BILLING_ROLE;

-- In the stored procedure, fetch the vault key at run time:
-- (Snowpark Python can read Snowflake secrets via the session)

import json
import urllib.request

def get_vault_key_for_run(session, billing_period: str, customer_count: int,
                           avg_charge_cents: int) -> str:
    # Read Keybrake API key from Snowflake secret
    kb_api_key = session.sql(
        "SELECT SYSTEM$GET_SECRET_STRING('billing.keybrake_api_key')"
    ).collect()[0][0]

    # Calculate spend cap: customer_count * avg_charge + 10% buffer
    expected_total_cents = customer_count * avg_charge_cents
    spend_cap_usd = round((expected_total_cents / 100) * 1.1, 2)

    # Issue a scoped vault key for this billing run
    payload = json.dumps({
        "vendor":        "stripe",
        "daily_usd_cap": spend_cap_usd,
        "label":         f"snowflake-billing-{billing_period}",
        "expires_in":    "4h"
    }).encode()

    req = urllib.request.Request(
        "https://api.keybrake.com/v1/vault-keys",
        data=payload,
        headers={
            "Authorization": f"Bearer {kb_api_key}",
            "Content-Type":  "application/json"
        }
    )
    resp = json.loads(urllib.request.urlopen(req).read())
    return resp["vault_key"]  # vk_live_... — use as stripe.api_key

FAQ

Does `ALLOW_OVERLAPPING_EXECUTION = FALSE` fully prevent double charges?

No. ALLOW_OVERLAPPING_EXECUTION = FALSE prevents a new scheduled run from starting if the previous run is still executing. It does not prevent a manual task resume via EXECUTE TASK, does not prevent re-triggering a Task DAG from the root, and does not prevent a terminated run's already-submitted Stripe charges from being re-submitted by the next run. The advisory lock in the stored procedure closes those gaps.

Can I use Snowflake's `SYSTEM$TASK_RUNTIME_INFO` as an idempotency key?

You can include the task run ID in the idempotency key but it should not be the only component. SYSTEM$TASK_RUNTIME_INFO('CURRENT_TASK_GRAPH_RUN_GROUP_ID') returns the run group ID, which is unique per task graph execution — but if the stored procedure is called directly via EXECUTE IMMEDIATE or in a different context, the run group ID changes, and Stripe will treat the call as a new charge even if the business inputs are identical. A content-hash key from stable business inputs (customer ID, amount, billing period) is more reliable than a Snowflake-session-specific key.

What happens if the billing_run_locks INSERT deadlocks?

Snowflake does not have table-level row locks in the traditional database sense — DML operations are serialized at the micro-partition level. An INSERT INTO billing_run_locks that conflicts with an existing row raises a unique constraint violation immediately, not a deadlock. The stored procedure catches this exception and exits cleanly. Deadlocks are not a concern in this advisory lock pattern.

How do I handle a Task DAG where the root task needs to update pricing mid-month?

Separate the billing queue build from the pricing update. Run pricing updates in a separate Task that writes to a pricing table. The billing root task's MERGE reads from that pricing table but only inserts queue rows for unbilled customers. Pricing changes mid-month are reflected for customers not yet billed; already-billed customers are unaffected because the MERGE skips them via the billing_results JOIN.

Can I use Snowflake Streams instead of a billing queue table?

Snowflake Streams are append-only change-capture records — they track what rows were inserted, updated, or deleted in a source table since the last consumption. Teams use Streams to trigger billing Tasks when new usage rows appear. The double-charge risk with Streams is that a Task that fails after consuming the Stream offset but before committing billing results will not re-see those Stream rows on the next run (Streams advance their offset on consumption, not on downstream success). This is the opposite problem from the billing queue pattern — you lose charges rather than double-charging. Hybrid approach: consume the Stream to build a persistent billing queue table, then bill from the queue with the row-by-row write pattern described in this post.

Stop giving Snowflake Tasks your unrestricted Stripe key

Keybrake issues scoped vault keys per billing run with spend caps, allowed-endpoint lists, and a full audit log of every Stripe request. When a task runs twice, the second run hits the cap and stops — not your customers' bank accounts.

Failure mode 1: ALLOW_OVERLAPPING_EXECUTION = TRUE lets two billing task instances charge the same customers simultaneously