agent_signal

Free tool · agentsignal-check

Crash-test your agent before your users do.

Crash-test your AI agent for double execution and lost approvals before your users find them.

LangGraph 1.2.15 · agentsignal-check 0.2.0 · Kill after effect

Same approval, same hard kill. Left: charged three times. Right: charged once, because the charge carries an idempotency key.

✘ naive_graphcharge before interrupt()
  1. 01Pause pay node: payments:charge(), then interrupt()
  2. 02Approve one approval resumes the node; it re-runs from the top
  3. 03Charge payments:charge() runs again
  4. 04Kill os._exit(137) right after the effect
  5. 05Resume fresh process resumes from the SQLite checkpoint → charge re-runs

3 charges

payments:charge() executed 3× for one approval (expected 1×)

✔ idem_graphidempotency key
  1. 01Pause approve node: interrupt() for approval
  2. 02Approve one approval; execute node runs
  3. 03Charge payments:charge(idempotency_key=order-1999) → ch_1
  4. 04Kill os._exit(137) right after the effect
  5. 05Resume fresh process resumes; same key → provider returns ch_1

1 charge

PASS · no duplicates across 1 crash point(s)

agentsignal-check 0.2.0 run, 2026-10-15 22:55 MT. Real hard kill of a child process; the payment provider is a test fixture that honours idempotency keys like Stripe.

naive_graph: 3 charges. idem_graph: 1 charge.

Real output · same charge, two graphs

✗ Charge before interrupt() · charged twiceexit 1
$ agentsignal check --graph tests.fixtures.naive_graph:build --effect tests.fixtures.payments:charge --input '{"amount":1999}'[FAIL] baseline: 2 effect call(s) in a clean run; duplicates {'tests.fixtures.payments:charge(1999,)[]': 2}[FAIL] double_execution.kill_resume: crash after effect #1 -> re-executed {'tests.fixtures.payments:charge(1999,)[]': 3}; crash after effect #2 -> re-executed {'tests.fixtures.payments:charge(1999,)[]': 3}[FAIL] double_execution.kill_before_checkpoint: crash before checkpoint #2 -> re-executed {'tests.fixtures.payments:charge(1999,)[]': 3}; crash before checkpoint #3 -> re-executed {'tests.fixtures.payments:charge(1999,)[]': 2}[PASS] approval.lost_on_restart: pending interrupt visible from a fresh process[FAIL] approval.replay: same approval delivered twice -> duplicates {'tests.fixtures.payments:charge(1999,)[]': 3}
✓ Idempotent approval · charged onceexit 0
$ agentsignal check --graph tests.fixtures.idem_graph:build --effect tests.fixtures.payments:charge --input '{"amount":1999}'[PASS] baseline: 1 effect call(s) in a clean run[PASS] double_execution.kill_resume: no duplicates across 1 crash point(s)[PASS] double_execution.kill_before_checkpoint: no duplicates across 4 checkpoint write(s)[PASS] approval.lost_on_restart: pending interrupt visible from a fresh process[PASS] approval.replay: second resume caused no duplicate effects

Install and run

pip install "agentsignal-check[langgraph] @ git+https://github.com/MounirMeziani/agentsignal-check@v0.2.0"

agentsignal check --graph myapp.graph:build --effect myapp.payments:charge \
  --input '{"amount":1999}' --report agentsignal-report.md
View on GitHub →

What it detects

  • baseline

    Faultnone (clean run, auto-approve)

    FAIL meansan effect ran twice with no crash (side effect before interrupt() in the same node)

  • double_execution.kill_resume

    Faulthard kill (os._exit) right after each effect, then resume in a fresh process

    FAIL meansthe effect was re-executed

  • double_execution.kill_before_checkpoint

    Faulthard kill before each checkpoint put(), after put_writes (langgraph#8039)

    FAIL meansthe effect was re-executed on recovery

  • approval.lost_on_restart

    Faultrestart the process while paused

    FAIL meansthe pending approval is gone

  • approval.replay

    Faultdeliver the same approval twice, including against a stale checkpoint

    FAIL meansthe effect ran again

A duplicate means the same effect args returned different results. A retry that returns the same result (for example, a provider idempotency key) passes.

Every FAIL in the report links to a free tested pattern. Want it done for you? $79 Fix

Free starter pack

The approval-flow starter pack

Tested human-in-the-loop approval starters for LangGraph, OpenAI Agents SDK, Vercel AI SDK and Google ADK, plus the weekly Agent Signal newsletter. Free.

Signal Pro

Want every tested pattern, kept current?

Signal Pro includes the Pro-only idempotent-approval templates and the full pack.

See Signal Pro