Foresight
The Fate Map is a tree of every path an agent’s completed runs took: each branch with its success rate and a 95% interval, the forks that kill runs flagged, the places it cannot make up its mind marked. A live run is walked down the same tree. Nothing calls or embeds a model; it is counting, done carefully.
Fate Map · support-agent
180 completed runs · base rate 54% · 2 fatal forks · 6 runs in flight
llm:gpt-4o tool:search_orders llm:gpt-4o tool:issue_refund llm:gpt-4o
54 of 60 succeeded · 90%, 95% CI 79.9–95.3%
llm:gpt-4o tool:search_orders llm:gpt-4o tool:issue_refund×2 llm:gpt-4o
15 of 30 succeeded · 50%, 95% CI 33.2–66.8%
Fatal fork at tool:issue_refund×2: 50% against
90% for its siblings, Fisher p = 5.1 × 10⁻⁵
llm:gpt-4o tool:search_orders×3-5 llm:gpt-4o
5 of 50 succeeded · 10%, 95% CI 4.3–21.4% · 5 abandoned
Fatal fork at tool:search_orders×3-5: 10% against
72% for its siblings, Fisher p = 2.2 × 10⁻¹⁴
llm:gpt-4o tool:lookup_customer llm:gpt-4o tool:escalate_ticket llm:gpt-4o
24 of 40 succeeded · 60%, 95% CI 44.6–73.7% · 4 abandoned
llm:gpt-4o tool:search_orders×3-5
runs that took this path succeeded 5 of 50 times (10.0%, 95% CI 4.3–21.4%); the agent's overall success rate is 54%
10.0% on 50 runs · 95% CI 4.3–21.4%
Predictive fuse, odds 0.7: stop foresight: runs that took this path succeeded 5 of 50 times (≤21.4%)
llm:gpt-4o tool:search_orders llm:gpt-4o tool:issue_refund
runs that took this path succeeded 69 of 90 times (76.7%, 95% CI 66.9–84.2%); the agent's overall success rate is 54%
76.7% on 90 runs · 95% CI 66.9–84.2%
Predictive fuse, odds 0.7: continue
If it calls issue_refund next: stop foresight: runs that took this path and then called tool:issue_refund succeeded 15 of 30 times (≤66.8%)
Best next step from here: llm:gpt-4o
llm:gpt-4o tool:delete_database
no earlier run of this agent took tool:delete_database at this point; the 180 runs that got here went on with tool:search_orders, tool:search_orders×3-5, tool:lookup_customer
Best next step from here: tool:search_orders
| Path | Runs | Succeeded | Failed | Abandoned | Rate | 95% CI | Lift | Flag |
|---|---|---|---|---|---|---|---|---|
llm:gpt-4o | 180 | 98 | 73 | 9 | 54.4% | 47.2–61.6% | 1.00 | hotspot, 1.50 bits |
llm:gpt-4o › tool:search_orders | 90 | 69 | 21 | 0 | 76.7% | 66.9–84.2% | 1.41 | |
llm:gpt-4o › tool:search_orders › llm:gpt-4o | 90 | 69 | 21 | 0 | 76.7% | 66.9–84.2% | 1.41 | |
llm:gpt-4o › tool:search_orders › llm:gpt-4o › tool:issue_refund | 60 | 54 | 6 | 0 | 90.0% | 79.9–95.3% | 1.65 | |
llm:gpt-4o › tool:search_orders › llm:gpt-4o › tool:issue_refund › llm:gpt-4o | 60 | 54 | 6 | 0 | 90.0% | 79.9–95.3% | 1.65 | |
llm:gpt-4o › tool:search_orders › llm:gpt-4o › tool:issue_refund×2 | 30 | 15 | 15 | 0 | 50.0% | 33.2–66.8% | 0.92 | fatal fork, p = 5.1 × 10⁻⁵ |
llm:gpt-4o › tool:search_orders › llm:gpt-4o › tool:issue_refund×2 › llm:gpt-4o | 30 | 15 | 15 | 0 | 50.0% | 33.2–66.8% | 0.92 | |
llm:gpt-4o › tool:search_orders×3-5 | 50 | 5 | 40 | 5 | 10.0% | 4.3–21.4% | 0.18 | fatal fork, p = 2.2 × 10⁻¹⁴ |
llm:gpt-4o › tool:search_orders×3-5 › llm:gpt-4o | 50 | 5 | 40 | 5 | 10.0% | 4.3–21.4% | 0.18 | |
llm:gpt-4o › tool:lookup_customer | 40 | 24 | 12 | 4 | 60.0% | 44.6–73.7% | 1.10 | |
llm:gpt-4o › tool:lookup_customer › llm:gpt-4o | 40 | 24 | 12 | 4 | 60.0% | 44.6–73.7% | 1.10 | |
llm:gpt-4o › tool:lookup_customer › llm:gpt-4o › tool:escalate_ticket | 40 | 24 | 12 | 4 | 60.0% | 44.6–73.7% | 1.10 | |
llm:gpt-4o › tool:lookup_customer › llm:gpt-4o › tool:escalate_ticket › llm:gpt-4o | 40 | 24 | 12 | 4 | 60.0% | 44.6–73.7% | 1.10 |
Nodes with fewer than 3 runs are pruned (none here). Lift is the node’s rate over the agent’s base rate.
Demo corpus
Real output of traza-server 0.28.0 over the engine’s seeded behaviour corpus (226 runs, 1,342 spans), recorded 2026-10-18 by
scripts/demo-numbers.mjs. Synthetic runs with known outcomes, not customer data.
Fatal forks
A branch is a fatal fork when its success rate is at least 25 points below its siblings’ pooled rate, both sides have at least 3 runs, and a two-sided Fisher exact test puts the difference below p = 0.05. The test is what keeps three unlucky runs from being called a fork.
The demo researcher shows both sides of the rule. Its tool:query_db branch succeeded 3 of
10 times against 57% for its siblings — 27 points
worse — and is not flagged: p = 0.27. Its
tool:ask_human branch, 1 of 10, is: p = 0.008.
| Fork | Runs | Rate | Siblings | Gap | Fisher p |
|---|---|---|---|---|---|
tool:search_orders×3-5after llm:gpt-4o | 50 | 10% | 72% of 130 | 62 pts | 2.2 × 10⁻¹⁴ |
tool:issue_refund×2after llm:gpt-4o › tool:search_orders › llm:gpt-4o | 30 | 50% | 90% of 60 | 40 pts | 5.1 × 10⁻⁵ |
Foresight for one run
A run’s path so far is walked down its agent’s map. A completed run is left out of the statistics it is judged by. A live run’s last step is still open — having issued one refund, it may issue a second — so it is compared with every run that repeated that step at least as often.
That is why live-healthy-0 reads 69 of 90,
not the 54 of 60 of the runs that stopped at one refund: its pool is
tool:issue_refund (54 of 60) and tool:issue_refund×2 (15 of 30).
The run took a step no earlier run of its agent took at that point. The answer says where, what it took, and what the runs that got there did instead.
Fewer than 30 earlier runs share the path, the agent has no completed runs in the window, or the map was cut short there by its node bound.
The interval’s upper bound is below 50%: with 95% confidence, runs on this path fail more often than they succeed.
The upper bound is below the agent’s base rate: runs on this path do measurably worse than the agent’s average.
Neither.
| Run | Verdict | What the engine says |
|---|---|---|
live-doomed-05 steps | likely_failure | runs that took this path succeeded 5 of 50 times (10.0%, 95% CI 4.3–21.4%); the agent's overall success rate is 54% |
live-healthy-04 steps | on_track | runs that took this path succeeded 69 of 90 times (76.7%, 95% CI 66.9–84.2%); the agent's overall success rate is 54% |
live-novel-02 steps | novel | no earlier run of this agent took tool:delete_database at this point; the 180 runs that got here went on with tool:search_orders, tool:search_orders×3-5, tool:lookup_customer |
live-early-01 step | on_track | runs that took this path succeeded 98 of 180 times (54.4%, 95% CI 47.2–61.6%); the agent's overall success rate is 54% |
Demo corpus
Real output of traza-server 0.28.0 over the engine’s seeded behaviour corpus (226 runs, 1,342 spans), recorded 2026-10-18 by
scripts/demo-numbers.mjs. Synthetic runs with known outcomes, not customer data.
Predictive fuses
A fuse policy may set min_success_odds. At the pre-call check, when a run’s foresight stands on at
least 30 earlier runs, the run has at least 3 steps, and the
upper bound of the 95% interval is below the odds, the policy’s action applies. When the check
names the next tool, the path is judged with that step appended.
A foresight stop is a prediction, made again at every question: it is not latched, lifts by itself when the run’s path moves on to better odds, and never rides on an ingest acknowledgement. To let a run go on against its history, reset it with a grant of lower odds.
curl -s https://api.trazadb.com/v1/fuses/policies -H "Authorization: Bearer $TRAZA_API_KEY" \
-d '{"name":"support","match":{"service":"support-agent"},"limits":{"min_success_odds":0.7},"action":"stop"}'POST /v1/fuses/check {"run":"live-doomed-0","next":{}}
stop foresight likely_failure
foresight: runs that took this path succeeded 5 of 50 times (≤21.4%)
POST /v1/fuses/check {"run":"live-healthy-0","next":{}}
continue foresight on_track
POST /v1/fuses/check {"run":"live-healthy-0","next":{"tool":"issue_refund"}}
stop foresight on_track
foresight: runs that took this path and then called tool:issue_refund succeeded 15 of 30 times (≤66.8%)
Odds 0.7 under the policy above; engine output from the demo corpus.
Calibration
Each completed run is predicted at several depths from the half of the cohort it is not in, only from what it had done by then, and scored against how it ended. Here depth 1 knows nothing, because every run’s first step is the same. At depth 3 the demo map removes 31% of the base rate’s error. An agent whose outcome does not depend on its path scores near zero at every depth, and its verdicts are then worth no more than its base rate.
Skill by depth
| Depth | Skill | Brier | Base rate’s | Runs |
|---|---|---|---|---|
| 1 | 0.2514 | 0.2514 | 180 | |
| 2 | 0.1998 | 0.2514 | 180 | |
| 3 | 0.1727 | 0.2514 | 180 | |
| 5 | 0.1923 | 0.2365 | 130 |
Skill is 1 − Brier ÷ the base rate’s Brier: the share of the “assume the average” error the map removes, on runs from the half of the cohort it was not built from.
Reliability at depth 3
Each dot is a tenth of the predictions, sized by its runs. On the diagonal, a prediction of about 10% came true about 10% of the time.
On Traza Cloud
Per agent across your organization’s projects: success rate over time, the top fatal forks with example runs to diff, and behaviours that are new this week. Each links into the project’s own Fate Map.
Every live run carries its foresight — verdict, success rate with its interval, and the runs it stands on — read from the run guard’s memory and the cached map, so the grid ranks what is running by how runs like it ended.
A run reaching likely_failure can raise an alert: a push to your phone or an interactive Slack message, with Approve, Kill and Nudge. See Steer.
Stated plainly
session.outcome, a run’s outcome is the status of the span that ended last. Declare outcomes where you can.