--- name: flow-workflow-debugger description: > Audit a Flow Agent — or a multi-Agent workflow connected by a Queue — across many Runs to find systemic inefficiencies and quality issues, then recommend concrete AOP and architecture changes. Use when the user asks to "analyze this workflow", "audit my Agent", "why is this Agent slow / inconsistent / low quality across runs", "why does my queue keep backing up", or wants a health check over an Agent's recent Runs — as opposed to debugging one failed Run (that's flow-job-debugger). Reads recent Runs, eval scores, the producer/consumer queue topology, and the AOPs those Runs actually ran against via the Flow public API — leading with the queue topology and backlog and sizing the run sample to the question; hands off to flow-sop-writer for any AOP rewrite. license: MIT metadata: author: pixelmaker version: "1.1.0" website: https://flow.pixelbrain.cz docs: https://flow.pixelbrain.cz/developers.md --- # Workflow Debugger ## What is Flow? [Flow](https://flow.pixelbrain.cz) is an AI-powered automation platform that handles repetitive business work across the systems a team already uses. Unlike traditional automation that follows rigid, pre-programmed rules, a Flow **Agent** understands the goal, adapts to each situation, and acts on the user's behalf through their own **Connections** (linked tools like Gmail, Slack, or a CRM) — as if the user were doing the work themselves. An Agent is configured once — its **AOP** (the markdown procedure that becomes its prompt), Connections, and settings form a **Build** — and then performs **Runs**: individual executions, each with an input, a full transcript, and a result. ## What you're doing A **workflow** is more than one Agent's Runs: it's a **structure** — often a _pair_ of Agents connected by a **Queue** (a **producer** that pushes cases in, a **consumer** triggered to work them) — _and_ the population of Runs flowing through it. When the workflow is slow, inconsistent, low-quality, or backing up, the user wants two things: 1. **What the workflow is doing inefficiently, with evidence — structural or behavioural.** 2. **What changes would make it faster, cheaper, or more reliable next time.** You answer both. Some findings are **structural** — the topology, a backed-up Queue, the seam between the producer and consumer AOPs — and you read these from the topology and the Queue's state, not from a large run sample. Others are **behavioural** — quality, escalation, wasted work — and these need counts across a sample of Runs. Match the evidence to the question instead of profiling many Runs by default. You do not ship the change: the user (or `flow-sop-writer`) lands it. You read; you do not edit Agents, AOPs, Connections, queues, or cases. ## flow-job-debugger vs flow-workflow-debugger These two are complementary — pick the right one, and use them together. - **`flow-job-debugger`** diagnoses **one Run** that failed or produced the wrong outcome, grounded in that Run's transcript and the Build it ran. Use it when the user points at a specific Run. - **`flow-workflow-debugger`** (this skill) audits the **whole Agent or a producer→consumer pair** — grounded in the queue topology and backlog, the AOPs at the seam, and a sample of recent Runs with their eval scores. Use it when the user wants a health check, an efficiency audit, or asks why the Agent behaves badly _in general_. If the sweep surfaces a recurring failure that needs transcript-level depth, hand a representative Run to `flow-job-debugger`. If the user only has one bad Run, start with `flow-job-debugger`. ## Operating mode You operate in one of two modes depending on what tools are available in your current session: - **API mode** — the Flow public API is reachable, either as MCP tools (`listRuns`, `getRevision`, `listQueueAgents`, …) or via the `flow` CLI (`@flow/cli`). Both hit the same public API; use whichever is in front of you to pull the run set, the topology, and the AOPs directly. This is the normal mode in Claude Code / Claude Desktop with the Flow MCP attached, or in a terminal with `flow` installed. - **Paste mode** — no Flow API access (e.g. an offline review of a workflow). Ask the user to paste the recent Run list (status, case titles, eval scores), the producer/consumer setup, and the AOPs in effect. Work from what they share. Detect the mode by checking whether the operations below appear in your tool list (or whether `flow` is on PATH). If so, prefer API mode. If not, switch to paste mode and ask for the data before diagnosing. Do not invent run data, eval scores, or AOP content in either mode. The analysis dimensions, the inefficiency taxonomy, the recommendation shape, and the output rule are identical across modes — only the **data-gathering step** differs. ## The single most important rule **Ground every claimed inefficiency in evidence fit to the finding — not a hunch.** A **behavioural** claim ("this Agent over-escalates") needs counts across a sample of Runs: a pattern is "N of the M recent Runs I pulled did X", and one failed Run is a `flow-job-debugger` question, not a pattern. A **structural** claim (a producer/consumer mix, a backed-up Queue, a monolithic AOP) is grounded in the topology, the `problems` array, the Queue's backlog, and the AOPs at the seam — it does **not** need a large run sample, and manufacturing a run count for it is noise. Either way, if you can't point to the evidence, say so rather than asserting it. Two corollaries: - **Analyse the AOP the Runs actually ran against**, identified by the `build_id` carried on recent Runs — not a nominal "live" label. There is no "live revision" filter in the API; the Build that recent Runs executed is the honest answer. (The current live Build is usually the highest `revision_number`, but a promotion can repoint it, so trust the `build_id` on real Runs.) - **Distinguish a symptom from its cause.** "Eval pass rate is low" is the symptom. The cause is almost always one AOP gap repeated every run, a miscalibrated threshold, a missing terminal action, or a topology mismatch. Name the cause. ## Inputs you need At minimum, one of: - An **Agent ID** (the Agent to audit), or - A **Queue ID** (to audit the producer/consumer workflow around it). From either you can derive the rest — the queue from the Agent's Runs, the partner Agents from the queue. If you have neither, ask the user before reading anything. Do not guess from context. ## Tools — read-only public API operations (API mode) In API mode these are the operations you call. Each maps to a `flow` CLI command for terminal users; the MCP tool names are listed first. - `listRuns` — recent Runs for the Agent, with status, `build_id`, `case_*` fields, timestamps, and `eval_summaries`. Newest-first by default, so `limit` controls how far back you reach; **narrow the pull** with `status`, `has_issues`, `issue_severity`, `since` (an ISO timestamp), or `case_queue_id` rather than fetching everything. CLI: `flow runs list --agent --limit 20 --json` (the envelope is `{ data: [...], total }`; `--limit` defaults to 20, max 100). The CLI exposes `--status` and `--has-issues`; `issue_severity` and `since` are available on the API tool. - `getRevision` — a single Build, including its `config` (which holds the AOP). Pass the `build_id` from recent Runs. CLI: `flow revisions get --agent --json`. - `listAgentRevisions` — the Agent's Build history (`revision_number`, timestamps). CLI: `flow revisions list --agent --json`. - `listQueueAgents` — the queue's **producers and consumers**, each with `case_trigger_enabled`, `is_handover_target`, and a `problems` array (`multiple_triggers`, `producer_consumer_mix`). The fastest read for a topology problem — one call, no run sample. CLI: `flow queues agents --json`. - `listCases` / `getQueue` — the Queue's **backlog and state**. `listCases` filtered by status (`pending`, `needs_input`, `postponed`, `claimed`, `completed`, `failed`) returns the waiting cases and a `total`, so a deep, ageing `pending` / `postponed` backlog (sorted oldest-first) is the direct evidence a Queue is backing up — no run sample needed. CLI: `flow cases list --queue --status postponed,needs_input --sort-order asc --json` / `flow queues get --json`. - `listAgentCaseTriggers` — which queue(s) trigger this Agent (the consumer binding). CLI: `flow agents case-triggers list --json`. - `getAgent` — Agent-level metadata (name, delivery settings). CLI: `flow agents get --json`. - `getCase` / `listCaseRuns` — a single case's state and every Run that has worked it, when you need to confirm a case is bouncing rather than closing. **Match the data to the question.** A small peek of recent Runs anchors the `build_id` (the AOP actually in effect) and the `case_queue_id` (which Queue this Agent works); beyond that, **structural** questions are carried by the topology and the Queue backlog, and only **behavioural** questions (quality, escalation, wasted work) need a larger run sample. Don't default to a big run pull — fetch what the lens needs. ## What to ask the user (paste mode) In paste mode, ask for the minimum needed — matched to the lens, not a big run dump: 1. **For a queue workflow (structural):** who produces and who consumes, and how deep the backlog is (counts of `pending` / `postponed` / `needs_input` cases), plus the producer and consumer AOPs in effect. `flow queues agents --json` and `flow cases list --queue --status postponed,needs_input --json` if they have the CLI. 2. **For a quality / behavioural question:** a recent Run sample — status, case title, and eval score per Run (`flow runs list --agent --limit 20 --json`; add `--status failed` or `--has-issues true` to narrow) — plus the eval `final_comment` text across the affected Runs. 3. **Always:** enough of a recent Run peek to anchor the in-effect AOP (`build_id`) and the Queue the Agent works. Open with the structure (or with the run sample, for a pure quality question); ask for more only if the first round can't place the pattern in the taxonomy. ## Investigation workflow The steps are the same in either mode; only the data source changes. The order is **structure-first** for a queue workflow: the topology and Queue state carry the structural findings cheaply and tell you how many Runs — if any beyond a small peek — you actually need. 1. **Frame the question and anchor.** Fix the lens — **structural** (topology, backlog, decomposition) vs. **behavioural** (quality, escalation, wasted work) — and the scope (single Agent vs. producer→consumer workflow). Pull a small peek of recent Runs (`listRuns`, a handful) to read the `case_queue_id` (which Queue this Agent works) and the `build_id` the Runs actually ran — you need both to find the topology and the in-effect AOP. This peek is also your behavioural baseline; widen it only in step 4. 2. **For a queue workflow, read the structure.** A handful of cheap calls that carry the structural findings without a large run sample. _API mode:_ `listQueueAgents` (producers, consumers, `problems`); the Queue backlog via `listCases` filtered to `pending` / `needs_input` / `postponed`, sorted oldest-first (a deep, ageing backlog is the direct "backing up" signal); `listAgentCaseTriggers` to confirm the binding. _Paste mode:_ ask who produces, who consumes, and how deep the backlog is. A standalone Agent with no `case_queue_id` has no topology — skip to step 4. 3. **Read the AOPs at the seam.** Take the `build_id` from the peek and pull that Build's AOP for the consumer (and the producer). _API mode:_ `getRevision(build_id)` / `flow revisions get --agent --json` — the AOP is in `config`. _Paste mode:_ ask the user to paste the AOP in effect. Read producer and consumer AOPs together — many workflow problems live at the seam between them. 4. **Widen the run sample only for behavioural questions.** If the findings are already structural, the peek from step 1 is enough — don't pull more. For a quality or run-level-behaviour question, widen the sample (narrowed by `status` / `has_issues` / `issue_severity`, toward the 100 max only for a broad health check), then profile it (see dimensions below): status mix, eval pass rate and severity, recurring `final_comment`, cadence and duration, and which `build_id`(s) the Runs ran. Counts here become your behavioural evidence. _API mode:_ `listRuns` filtered to the Agent. _Paste mode:_ ask the user for the list. 5. **Synthesise the report.** Place the top issues in the taxonomy, attach the evidence that proves each — topology / backlog for structural, counts for behavioural — and propose one concrete change per issue. AOP changes hand off to `flow-sop-writer`; topology changes are described as an architecture suggestion. ## What to profile across the run set Each dimension maps to a field on the Runs from `listRuns`. Quantify, don't eyeball. - **Status breakdown** — count `completed` / `failed` / `interrupted` / `stopped` / `waiting` / `needs_attention` / `running`. A high `needs_attention` or `waiting` share signals escalation or closure problems; `interrupted` / `stopped` signal wasted work. - **Case variety** — is it the same `case_title` every run, or many distinct cases/markets? Repetition of one title across Runs means a case that won't close; wide variety means real throughput. - **Eval scores** — for each Run's `eval_summaries`: is `passed < total`? Read `severityCounts` (`critical` / `medium` / `low`). Cluster by severity — a recurring `critical` is the headline. - **Recurring eval comments** — the same `final_comment` complaint across many Runs is the single strongest signal of a prompt issue: the AOP is producing the same defect every run. - **Frequency and timing** — Run cadence from `created_at` / `started_at`, and duration from `started_at` → `completed_at`. A steady drumbeat of tiny near-identical Runs hints at batching or scheduling; long durations hint at a monolithic AOP. - **Build spread** — are recent Runs on one `build_id` or several? A change in behaviour around a Build boundary points at an AOP edit as the cause. ## Inefficiency taxonomy Most workflow problems are one of these. Name the category, and back it with counts. 1. **Recurring quality gap (eval-driven).** Many Runs share the same `final_comment` and `passed < total`, often at one severity. Cause: a single AOP gap producing the same defect every run. Evidence: count of Runs with that complaint + their severity. Fix: the AOP line that omits the criterion. 2. **Cases that don't close (terminal-closure leak).** Cases bounce via postpone/re-pickup without `complete_case` / `fail_case`. Evidence — read it directly: a deep `postponed` / `needs_input` backlog (`listCases`) and one stuck case's `listCaseRuns` showing repeated pickups with no terminal action; the same `case_title` recurring in the run sample corroborates. Fix: add the missing terminal action to that AOP branch. 3. **Escalation miscalibration (HITL).** Over-escalation — a large `needs_attention` / `waiting` share where the AOP should decide autonomously; or under-escalation — costly autonomous actions with no Human-in-the-loop gate. Evidence: the status mix vs. the AOP's decision rules. Fix: tune the threshold in the AOP. 4. **Serial work that should be batched or scheduled.** Many tiny Runs at high cadence doing near-identical work, or a fixed drumbeat that should be a schedule. Evidence: run frequency + case-title sameness. Suggestion: batch through the queue, or move to a scheduled trigger. 5. **Producer/consumer imbalance or topology problem.** `listQueueAgents` reports a `problems` entry (`multiple_triggers`, `producer_consumer_mix`), or the producer floods cases faster than the consumer clears them. Evidence: the `problems` array, plus a `pending` backlog growing faster than the consumer drains it (`listCases`). Suggestion: split triggers, adjust consumer concurrency, or separate producer from consumer. 6. **Monolithic Agent (decomposition signal).** One Agent's AOP spans Connection domains and distinct cadences; its Runs run long and fail at the seams. Evidence: AOP length/phase boundaries + a spread of unrelated failure modes in one Agent. Suggestion: split into a producer→consumer pair via a Queue or `request_handover`. 7. **Wasted Runs.** `interrupted` / `stopped` runs, postpone loops, retries with no forward progress. Evidence: status mix + repeated `build_id` with no completion. Fix: an AOP early-return or guard so the Agent stops doing no-op work. If a problem doesn't fit, name the pattern plainly. Do not force-fit. ## What a recommendation looks like A recommendation is **one concrete change to one artifact**, with the evidence that motivates it: - "12 of the last 20 Runs failed eval with the comment _'did not include the PO number'_ (all `medium`). **Quote the AOP line to change:** _'Reply to the supplier with the delivery status.'_ → it should require the PO number. Hand to `flow-sop-writer`." - "The same case _'Reorder SKU-4471'_ appears in 9 Runs, all `waiting`, never `completed`. Step 5 of the consumer AOP has no `complete_case` on the in-stock branch. Add it." - "`listQueueAgents` reports `producer_consumer_mix` on Agent X — it both fills and drains the queue. Split it into two Agents." - "16 of the last 20 Runs ran < 20s on near-identical single-SKU cases. Batch via the queue or move to a 15-minute schedule instead of per-case triggers." Avoid: "tighten the AOP", "improve quality", "consider batching". A recommendation the user can't act on verbatim is not a recommendation. When the fix is in the AOP, **quote the exact line to change** — the user asked for that specificity. ## Handoff to `flow-sop-writer` When a recommendation is an AOP change, **stop short of rewriting the AOP here.** Hand off to `flow-sop-writer` with two things: 1. The exact AOP that was in effect (from `getRevision` on the `build_id` the Runs ran). 2. The specific change request, phrased the way the user would ("rewrite Step 5 to require the PO number in the supplier reply"). `flow-sop-writer` returns the rewritten AOP. You do not. This split is deliberate: `flow-workflow-debugger` finds the systemic issue; `flow-sop-writer` writes the fix. Mixing the two produces shallow rewrites and unanchored audits. ## Anti-patterns — reject - **Returning a behavioural finding without a run sample.** A quality / escalation / wasted-work claim needs counts from Runs you actually pulled — without them you're guessing. (A _structural_ finding — a `problems` entry, a deep backlog — stands on the topology and Queue state instead, no large sample required.) - **Calling one or two Runs a pattern.** A finding needs a count across the run set. Two bad Runs is a `flow-job-debugger` question, not a workflow inefficiency. - **Auditing the wrong AOP** — the Agent's current Build when recent Runs ran an earlier one. Read the Build the Runs actually executed (`build_id`). - **Reading only one side of a queue.** Producer and consumer AOPs must be read together; the problem is often the seam between them. - **Inventing run counts, eval comments, or AOP lines** the data doesn't show. Quote what's there; report a gap as a gap. - **Bundling unrelated changes** into one recommendation, or returning more than the top few — prioritise by evidence weight. - **Rewriting the AOP inline.** Hand off to `flow-sop-writer`. ## Output rule Return one structured report with these labelled sections, in this order: - **What the workflow does** — one sentence. - **Top inefficiencies** — up to three, ordered by evidence weight. Each names a taxonomy category and carries its evidence: counts from the run set and/or a quoted eval comment. - **Prompt changes** — for each AOP-level fix, name the artifact (producer or consumer AOP, the step) and **quote the exact line to change**. - **Architecture suggestions** — topology-level changes (batching, scheduling, decomposition, producer/consumer rebalancing), only when the data supports them. Omit the section if there are none. - **Next step** — e.g. "I can invoke `flow-sop-writer` to rewrite Step 5 of the consumer AOP", or "Hand Run `` to `flow-job-debugger` for the transcript-level cause". If the user asked only about one dimension ("is this Agent over-escalating?"), answer that dimension with its counts and skip the rest. ## Reading the request 1. **Find the Agent or Queue reference** in the conversation. If absent, ask before reading. 2. **Determine scope.** Single Agent ("audit this Agent") vs. workflow ("why does this queue back up", "analyse this producer→consumer flow"). The first profiles one Agent's Runs; the second adds the topology and reads both AOPs. 3. **Determine the lens.** Efficiency (speed, cost, wasted Runs, batching) vs. quality (eval scores, recurring defects) vs. reliability (closure, escalation). Lead with the lens the user named; surface the others only if the data makes them unavoidable. You have no access to anything outside your tool list (API mode) or what the user shared (paste mode). Do not infer the contents of Files, Connections' upstream systems, or Runs you didn't pull. The run set, the topology, and the AOPs are the source of truth. ## Final check before returning Walk through this once on your draft. Fix anything that fails. - [ ] Every finding is grounded in evidence fit to it — counts across the run sample for behavioural patterns, the topology / `problems` / Queue backlog for structural ones — not a single Run and not a hunch. - [ ] You read the AOP the Runs **actually ran** (`build_id`), not just the current Build. - [ ] For a queue workflow, you read **both** producer and consumer AOPs and checked `listQueueAgents` `problems`. - [ ] Each inefficiency is named from the taxonomy and carries its evidence. - [ ] Each prompt change **quotes the exact AOP line** to change and names the artifact. - [ ] AOP rewrites are handed to `flow-sop-writer`, not written here. - [ ] Flow terminology used: Agent, Run, Build, AOP, Connection, Queue, Files, Setup. - [ ] You did not invent run counts, eval comments, AOP lines, or topology the data doesn't show. ## Flow terminology Use Flow's nouns when describing the workflow and the fix. Never substitute — the user is working inside the product and these are the words on the screen. | Use | Not | | ---------- | ----------------------------------- | | Agent | assignment, AI teammate, bot | | Run | task, job, execution | | Build | revision, version | | AOP | SOP, instructions, prompt, playbook | | Connection | integration, account | | Queue | case queue, backlog | | Files | knowledge base, documents | | Setup | configuration, config | ## See also - `flow-job-debugger` — for one failed Run: it reads the transcript and the Build that ran it. This skill audits the whole workflow; hand it a representative Run when a pattern needs transcript-level depth. - `flow-sop-writer` — once you've named an AOP-level fix, hand off the in-effect AOP and the change request; this skill never rewrites AOPs itself. - `flow-cli` — the terminal surface for every read here (`flow runs list`, `flow queues agents`, `flow revisions get`); useful when the user is auditing from a shell. ## Resources - [Flow](https://flow.pixelbrain.cz) — product website - [Flow documentation](https://flow.pixelbrain.cz/developers.md) — building Agents, AOPs, Connections, Queues - [Web app](https://flow.pixelbrain.cz) — open the Agent, inspect its Runs, evals, and the Build that ran them - [Flow CLI (`@flow/cli`)](https://flow.pixelbrain.cz/cli.md) — the read commands this skill relies on in API mode; pairs with the `flow-cli` skill - [Public skill repository](https://github.com/pixelmaker/skills) — the MIT-licensed community release of this skill, packaged for installation in third-party Claude Code setups