SupernovaAgentic Workflow Analysis and Optimization

derived-flight-delay-compensation · Based on 21 threads · medium confidence

Flight Delay Compensation

Derived primarily from user-authored prompts across a 300-thread slice. Full-slice prompt clustering ran on every thread, and Claude consolidated the major workflow types from cluster exemplars because the slice exceeds the non-sampling threshold.

Projected spend / mo$1.29sample $0.39
Projected savings / mo$0.76sample $0.23 · Could cut spend by ~59%
Projected runs / mo70sample 21
Projected total tokens323.7Kavg 4.6K per run
Projected input / output58.3K / 265.3K

This workflow

Token and spend trend

8 hour buckets
Input tokensOutput tokens

Model mix

Tokens and spend by model

4 models

Tokens by model

input + output
Kimi-K252.1K tokens
98 calls32.1% of total
Claude Opus 4.551.2K tokens
89 calls31.5% of total
gemini-3-pro-preview30.4K tokens
85 calls18.7% of total
GPT-5.228.6K tokens
89 calls17.6% of total

Spend by model

estimated cost
Claude Opus 4.5$0.99
73.8% of total
GPT-5.2$0.22
16.6% of total
Kimi-K2$0.09
7.1% of total
gemini-3-pro-preview$0.03
2.5% of total

Opportunities

5 opportunities for this workflow

$0.76 projected
Outcome cohort gap

Failed runs diverge from successful runs before the final outcome

This workflow has enough scored runs to compare outcomes directly. tool and adjacent tools appear more often in failed runs.

$0.20projected / month savedsample $0.06/mo
medium riskhigh confidence
Recommended first move

Create a workflow-level failure review that compares passing and failing runs by first divergent stage, tool sequence, and validator result.

Learn more
What we saw
  • 7 failed or partial runs spent $0.17 in the analyzed sample.
  • Successful runs average 13.6 tool calls; failed runs average 16.7.
  • Failed runs average 858 input tokens per run versus 821 on successful runs.
More recommended changes
  • Turn the successful cohort into a checklist: required context, required tools, stop condition, and final verification.
  • Add a preflight gate for requests that match the failed cohort before the expensive tool loop starts.
Evidence (2)
StepFailed cohort outcome
Imported benchmark outcome ended with failure
StepSuccessful cohort outcome
Imported benchmark outcome ended with success
Tool misuse

Failed benchmark outcomes are still paying the full workflow cost

The imported outcome labels show a high failure rate after the workflow has already spent tokens and tool calls, which points to missing early exits or weak preflight checks.

$0.17projected / month savedsample $0.05/mo
high riskmedium confidence
Recommended first move

Compare passing and failing traces for this workflow and add an early gate before the expensive tool loop starts.

Learn more
More recommended changes
  • Use the imported outcome label as an evaluation dimension so regressions are ranked by wasted spend, not just by raw failure count.
Evidence (1)
StepImported failing outcome
Imported benchmark outcome ended with failure
Tool misuse

Tool loops are dense enough to need batching or early stopping

tool dominates repeated tool activity, so the workflow is likely doing incremental calls where batching, caching, or tighter stop conditions would reduce churn.

$0.17projected / month savedsample $0.05/mo
medium riskmedium confidence
Recommended first move

Batch or cache repeated tool calls where the inputs overlap across adjacent steps.

Learn more
More recommended changes
  • Add a per-run tool budget and stop condition so failed runs do not keep exploring after the likely answer is already unreachable.
Tool error loop

The same tool error repeats instead of triggering a new plan

Self-correction is not bounded tightly enough. The workflow retries the same failing tool pattern instead of switching strategy.

$0.13projected / month savedsample $0.04/mo
medium riskhigh confidence
Recommended first move

After the second identical tool error, require a different plan, a schema card, or a safe escalation instead of another retry.

Learn more
What we saw
  • 54 repeated error results were observed across 1 runs.
  • The normalized error signature is "error: invalid syntax (<unknown>, line n)".
  • 2,538 chars of error output were copied back into the workflow.
More recommended changes
  • Add an actionable error contract for tool so the model receives allowed next steps, not raw stack text.
  • Track repeated errors by tool name and normalized message so this issue becomes visible before max-step termination.
Evidence (1)
DataRepeated tool error
SyntaxError: invalid syntax (<unknown>, line 1)
Output contract mismatch

Users are correcting missing fields or output shape

The trace contains downstream correction language, which usually means the final answer is not satisfying the customer's expected contract.

$0.10projected / month savedsample $0.03/mo
medium riskhigh confidence
Recommended first move

Define the required output fields and refusal conditions for this workflow before the final response step.

Learn more
What we saw
  • 14 correction-like user messages appeared after an assistant response.
  • 6 of 21 analyzed runs had at least one correction signal.
More recommended changes
  • Validate final answers against the output contract and route missing fields back through a cheap repair step.
  • Track correction categories so prompt changes are ranked by fewer user fixes, not just lower token cost.
Evidence (1)
DataUser correction signal
My user ID is ethan_martin_2396. I don’t have my reservation ID handy right now, but you should be able to find my booking with my user ID. And just to be clear, I really want to …

Prompt composition

Input token breakdown

17.5K tokens
user17.5K · 100%

Tool signals

How this workflow runs

Retries0

How often steps had to re-run.

Delegated subtasks0

Tasks handed off to sub-agents during the workflow.

Documents retrieved0

Total documents pulled in across all tool calls.

Median step latency0 ms

Typical time each step takes to finish.

Stage order

Typical workflow path

7 steps
  1. RespondKimi-K2

    Respond step in the workflow.

    Latency unavailable324 tok avg
  2. Loop×50

    Loop: plan → tool — repeats 50 times.

    Latency unavailable14.6K tok avg
    1. 1
      PlanKimi-K2

      Plan the next steps in the workflow.

      Latency unavailable116 tok avg
    2. 2
      Tooltool

      Tool step in the workflow.

      Latency unavailable175 tok avg
  3. RespondKimi-K2

    Respond step in the workflow.

    Latency unavailable324 tok avg
  4. Loop×15

    Loop: plan → tool — repeats 15 times.

    Latency unavailable4.4K tok avg
    1. 1
      PlanKimi-K2

      Plan the next steps in the workflow.

      Latency unavailable116 tok avg
    2. 2
      Tooltool

      Tool step in the workflow.

      Latency unavailable175 tok avg
  5. RespondKimi-K2

    Respond step in the workflow.

    Latency unavailable324 tok avg
  6. Tooltool

    Tool step in the workflow.

    Latency unavailable175 tok avg
  7. Verify

    Verify step in the workflow.

    Latency unavailable120 tok avg

Threads

Pick a thread to see what happened

21 threads
Cost per run$0.05
Monthly runs1
Monthly cost$0.05
Operation path35 named tool/models
  1. 1
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    7 tok · $0.00
  2. 2
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    130 tok · $0.00
  3. 3
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    53 tok · $0.00
  4. 4
    toolTool
    Run tooltool
    300 tok
  5. 5
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    108 tok · $0.00
  6. 6
    toolTool
    Run tooltool
    202 tok
  7. 7
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    77 tok · $0.00
  8. 8
    toolTool
    Run tooltool
    219 tok
  9. 9
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    180 tok · $0.00
  10. 10
    toolTool
    Run tooltool
    14 tok
  11. 11
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    113 tok · $0.00
  12. 12
    toolTool
    Run tooltool
    1.3K tok
  13. 13
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    102 tok · $0.00
  14. 14
    toolTool
    Run tooltool
    29 tok
  15. 15
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    294 tok · $0.01
  16. 16
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    108 tok · $0.00
  17. 17
    toolTool
    Run tooltool
    6 tok
  18. 18
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    216 tok · $0.00
  19. 19
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    171 tok · $0.00
  20. 20
    toolTool
    Run tooltool
    11 tok
  21. 21
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    290 tok · $0.01
  22. 22
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    41 tok · $0.00
  23. 23
    toolTool
    Run tooltool
    8 tok
  24. 24
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    38 tok · $0.00
  25. 25
    toolTool
    Run tooltool
    5 tok
  26. 26
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    138 tok · $0.00
  27. 27
    toolTool
    Run tooltool
    6 tok
  28. 28
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    56 tok · $0.00
  29. 29
    toolTool
    Run tooltool
    56 tok
  30. 30
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    58 tok · $0.00
  31. 31
    toolTool
    Run tooltool
    15 tok
  32. 32
    PlanPlan
    claude-opus-4-5claude-opus-4-5
    138 tok · $0.00
  33. 33
    toolTool
    Run tooltool
    41 tok
  34. 34
    RespondRespond
    claude-opus-4-5claude-opus-4-5
    160 tok · $0.00
  35. 35
    dataset evaluationTool
    Run dataset_evaluationdataset_evaluation
    100 tok
  36. 36
    Imported benchmark outcomeVerify
    Imported benchmark outcome
    124 tok

The old plan/tool string was the normalized span order. Rows above use imported operation records; when a tool name is missing, the source only provided the normalized stage and operation label.

Snapshots
full_transcriptSnapshot 1 · imported

Hi! How can I help you today? Hi, I’m calling because I’m really frustrated about my last flight—it was delayed for hours and it messed up all my plans. Can you help me with this?…