FANOUT-BENCH-RESULTS — one master goal → N sub-agents, swept N=1…1024 on the real kernel

What this measures. fak turntax prices ONE agent; fleetbench (FLEET-SWEEP-RESULTS.md) prices A independent agents that happen to overlap. This sweep prices the topology neither captures and that no public benchmark maps (see experiments/fanout/RESEARCH-BRIEF-fanout-2026-06-17.md): a single master goal decomposed into N sub-agents — the orchestrator-worker / lead-subagent pattern (Anthropic’s lead-researcher → 3–5 subagents, scaled to N=1024). The regime N≥50 on one goal is unbenchmarked in the literature, and shared-prefix KV reuse across N siblings has never been quantified for it.

What is grounded vs modeled (the honest line — same discipline as TURN-TAX §3.2 / FLEET-SWEEP). Two halves, kept strictly apart:

Reproduce: wsl bash -c "cd fak && go run ./cmd/fanbench -profile research -trials 16 -out experiments/fanout/fanbench-research.json -csv experiments/fanout/fanbench-research.csv". Prefix-scale sweep: add -prefixes all for smoke,small,medium,long,big; set -model-config /path/to/config.json or -model-context N so big/max fills the target model context after reserving the private suffix and decode budget. Code: internal/turnbench/fanout.go, runner cmd/fanbench/, tests internal/turnbench/fanout_test.go + cmd/fanbench/main_test.go.

High-signal external references. The result should be read against the shared-prefix serving literature and provider cache economics, not as a claim that FAK invented the prefix lever: vLLM Automatic Prefix Caching (docs, PagedAttention paper), SGLang/RadixAttention (blog, paper), Hydragen shared-prefix attention (paper), Anthropic prompt caching (docs), OpenAI prompt caching (docs), and Anthropic’s orchestrator-worker research-system writeup (engineering note).

§0 — The MEASURED real-run capstone: 1,024 real agents on one goal (fanrun)

Everything in §1–§5 below prices a synthetic call stream: the cross-agent dedup is a real k.Syscall event, but the “agents” are call sequences, and the headline 72.8× is a modeled projection. fanrun (cmd/fanrun, engine internal/bench/fanrun.go) closes that gap — it actually runs N real agent sessions decomposing one shared goal and wall-clocks them. Each sub-agent is a genuine internal/agent.RunArm loop through a real kernel.New("localtools") with the vDSO fast path on and real tool dispatch across the syscall boundary; they all gather the same research goal (look up the account → fetch the refund policy → search flights → convert the price), the read-only orchestrator-worker sub-agent role. Every number here is a wall-clock, a real kernel counter, or exact geometry — the artifact carries zero modeled fields (a test gate enforces it).

Measured on a CPU-only box (no GPU, no model weights, no API key — a deterministic offline planner drives each sub-agent so the run is reproducible and needs nothing). Checked-in artifact experiments/fanout/fanrun.json:

N (sub-agents) tasks completed tool errors wall-clock (serial) agents/s (serial) cross-agent vDSO hits vdso_fills prefix tokens elided = (N−1)·P
1 1 0 0.6 ms 1,708 0 3 0
4 4 0 1.7 ms 2,393 9 3 6,144
16 16 0 5.3 ms 3,038 45 3 30,720
64 64 0 25.0 ms 2,561 189 3 129,024
256 256 0 88.3 ms 2,899 765 3 522,240
1024 1024 0 364 ms 2,814 3,069 3 2,095,104

Read three things off the capstone:

  1. 1,024 real agents complete one goal end-to-end in 364 ms on a laptop with no GPU. Every sub-agent finishes the research gather (tasks_completed = N, tool_errors = 0) — this is the capability itself, not a projection of it. The whole N=1…1024 sweep wall-clocks in under 2 s.
  2. vdso_fills is FLAT at 3 for every N — the cross-agent-vs-warm-per-agent-cache thesis, made concrete. Sub-agent 0 fills 3 cache entries for the shared goal sources; all of sub-agents 1…1023 reuse them. A warm per-agent cache would fill 3·N (3,072 fills at N=1024); cross-agent dedup fills 3. The cross-agent vDSO hit count (the sibling-only delta over the N=1 baseline) grows ~linearly to 3,069 at N=1024.
  3. The prefill the kernel never redoes is exact (N−1)·P — 2,095,104 tokens at N=1024. With -model-dir this is additionally wall-clocked as a reuse-vs-no-reuse prefill race (the cmd/fleetserve methodology): the per-clone prefill speedup measured on a tiny CPU model is 3.4× (N=4) / 8.2× (N=16) / 11.7× (N=64), climbing with N as the geometry predicts.

The honest fence (why this is not a parallel-speedup claim). fanrun is serial by construction: the kernel’s fast-path world-version is process-global, so the N sub-agents must share one world epoch — which is exactly what makes the cross-agent dedup real and reproducible — and they run one after another. agents_per_sec_serial is N ÷ Σt, not a parallel rate, and fanrun deliberately does not reproduce §3’s modeled 72.8×. The measured win is prefill elision + real cross-agent dedup, not parallelism. The read-only research role is faithful to the lead-researcher → subagent pattern (sub-agents gather, the lead folds): a sub-agent that wrote would bump the process-global world and flush every sibling’s warmed read, the honest invalidation tension §4’s write-goal profile reports. The no-share profile gives exactly 0 cross-uplift at every N (the anti-inflation control, internal/bench.TestFanrunNoShareZeroUplift).

Still open (the real-MODEL rung, #982). fanrun’s sub-agents are driven by a deterministic offline planner, so the wall-clock is the kernel’s orchestration + dedup cost, not live decode. A real-model fan-out (live token generation per sub-agent) across the N grid is reachable on the lab GPU fleet and is the next rung; it does not change the measured orchestration/dedup numbers above, which stand on their own.

Reproduce (deterministic counter+geometry — no model, runs on any agent host): go run ./cmd/fanrun -agent-max 1024 -grid log -prefix 2048 -out experiments/fanout/fanrun.json. Add the prefill-elision wall-clock with a small model: go run ./cmd/fanrun -agents 1,4,16,64 -reps 3 -model-dir <dir>. Witness: go test ./internal/bench ./cmd/fanrun.

§1 — The headline surface (research-goal profile, 16 seeded trials, sub-turns=4)

research-goal: sub-agents mostly read the goal’s shared sources (p_shared=0.55, goal_pool=8), a 4-read master plan warms the catalog, no writes. Medians over 16 trials (experiments/fanout/fanbench-research.csv):

N (sub-agents) calls = plan+N·T shared_saved isolated_saved cross_uplift prefix_tokens_saved token_mult naive→reuse tax_clawed_back parallel_speedup
1 8 2 2 0 0 1.07→1.26 0% (loss) 1.0
4 20 10 8 +2 6,144 4.3→2.5 54.8% 3.0
16 68 46 34 +12 30,720 17.1→7.4 60.4% 11.0
64 260 191 134 +58 129,024 68.6→27.1 61.4% 29.0
256 1028 785 536 +251 522,240 274→106 61.7% 57.7
1024 4100 3155 2152 +1005 2,095,104 1098→420 61.7% 72.8

Read four things off this surface:

  1. The fan-out structure deletes turns the same sub-agents run apart cannot. cross_uplift (the MEASURED shared − isolated path-swap) grows ~linearly with N: at N=1024 the interleaved fan-out deletes 3,155 of 4,100 calls (77%) in one world epoch, of which +1,005 is the cross-agent bonus the same 1,024 sub-agents run solo (2,152 deleted) could not get. N=1 is exactly 0 — a lone worker has no sibling to share with (the budget-controlled single-agent control; partner of the no-share profile that is 0 at every N).

  2. The prefix-cache lever claws back ~62% of the multi-agent token tax — and saturates there. tax_clawed_back rises from 54.8% (N=4) to a 61.7% plateau by N≈256. This is the modeled prefix-cache economics: the master-goal prefix is written to cache once (1.25×) and read cheap by the other N−1 sub-agents (0.1×) instead of materialized N times — 1.25 + (N−1)·0.1 vs naive N·1.0. The analytic asymptote is 1 − 0.9P/(P+S+D+fold) ≈ 0.618; the measured curve lands on it.

  3. Prefix reuse is (N−1)·P prefill the kernel never redoes. At N=1024 that is 2,095,104 prefill tokens the SHARED arm does not recompute — the exact geometry of NewBatchFromPrefix (prefill once + N bit-identical clones).

  4. Fanning out to ONE sub-agent is a NET LOSS. At N=1 token_mult is 1.07 (naive) / 1.26 (reuse) and net_tokens_saved = −512: the orchestration fold plus the cache-write overhead cost more than just doing the goal yourself. This is the literature’s “a single agent beats trivial multi-agent,” surfaced honestly rather than hidden — the levers only pay once there are siblings to amortize across.

The D-001 acceptance grid (N=1 / 100 / 500 / 1000) — host-computed, reproducible (#255)

The §1 surface above uses the geometric N-ladder (the right spacing to read a curve spanning three orders of magnitude). The D-001 acceptance criteria (#255) name the literal scale points N=100, N=500, N=1000 and ask for coordination overhead vs a baseline next to the cross-agent reuse uplift. The dedicated scale harness (internal/bench/fanscale.go) prices exactly those points against the N=1 single-agent baseline and is wired to a reproducible command path. Medians over 16 seeded trials (research-goal, P=2048, sub-turns=4, seed 1, fak 0.34.0, go1.26.3); checked-in artifact experiments/fanout/fanscale-d001.json:

N (sub-agents) calls coord overhead (turns vs N=1) coord overhead ÷ baseline parallel_speedup cross_uplift (measured) prefix_tokens_saved = (N−1)·P tax_clawed_back (modeled)
1 8 0 0× 1.0 0 0 0%
100 404 4 0.67× 40.6 +93 202,752 61.6%
500 2,004 24 4.0× 67.5 +495 1,021,952 61.7%
1000 4,004 49 8.2× 73.7 +990 2,045,952 61.7%

Reproduce (deterministic, in-process kernel arithmetic — no model call, runs on the agent host): go run ./cmd/fanbench --scale --grid canonical --prefix 2048 --trials 16 --seed 1 --out experiments/fanout/fanscale-d001.json.

Read three things off the acceptance grid:

  1. Coordination overhead vs the N=1 baseline grows with the join, not the work. The single-agent critical path is 6 turns; the fold/synchronous-join tax adds +4 turns at N=100, +24 at N=500, +49 at N=1000 — an 8.2× depth multiple over baseline by N=1000. The lead must wait on the slowest of N sub-agents and then fold N results, so added width buys parallel throughput (speedup 1.0→73.7) but pays a critical-path tax that keeps rising — the coordination overhead the acceptance criterion names, priced against one agent so a “fan-out win” is always budget-controlled.
  2. Cross-agent reuse uplift scales ~linearly with N (the MEASURED SHARED − ISOLATED sibling dedup): +93 / +495 / +990 calls the interleaved fan-out deletes that the same sub-agents run apart cannot. N=1 is exactly 0 (a lone worker has no sibling to share with — the budget-controlled control).
  3. Prefix reuse is exact (N−1)·P geometry, not a model — 202,752 / 1,021,952 / 2,045,952 prefill tokens the SHARED arm never recomputes, and the modeled prompt-cache clawback sits on its ~61.7% plateau at every acceptance point (P fixed).

What this grid does NOT do (the honest gate that keeps #255 open). It is the host-computed reuse + coordination geometry, exactly the measured/modeled split of §1 — not the live-model wall-clock. The structural cross-framework comparison (which orchestrator re-sends what, and the token multiplier that implies) is given in the next subsection; what stays deferred is the live-model wall-clock half. The remaining D-001 acceptance — benchmark fak against LangGraph / AutoGen / CrewAI at these N with a live model (end-to-end wall-clock) — is a DeferredRun (deferred_run field in the artifact): it needs a bench node with the three frameworks installed and a served model, which the agent host lacks. Running it here would starve the agent seat and skew the number, so it is named as deferred rather than fabricated (the BENCHMARK-AUTHORITY rule). That cross-framework task-success axis overlaps #429 (the §5 controlled litmus) and the live-model seam #106.

Cross-framework coordination-overhead model — the token multiplier each orchestrator imposes (structural, not wall-clock) (#255)

The D-001 scope names two cross-framework asks — “Compare against LangGraph / AutoGen / CrewAI” and “Document token multiplier.” The live-model wall-clock half of that is the DeferredRun above. This subsection ships the half that is host-runnable: a structural comparison of the coordination-token cost each orchestrator imposes by construction, derived from each framework’s documented multi-agent topology — not a measured wall-clock, and not a head-to-head throughput claim. It is the same honest discipline as the committed CrewAI manager-worker model (examples/crewai-crew/, 4.93×): deterministic token arithmetic from a stated geometry, governed by BENCHMARK-AUTHORITY.md, and it sits inside the already-declared [SIMULATED] reuse claim in CLAIMS.md (the win is reuse-vs-reprefill over a stack that re-sends the shared prefix per coordination step — the common framework default — NOT a win over a tuned shared-prefix engine).

The one structural fact all three share. A multi-agent orchestrator advances by re-invoking a model endpoint, and on every coordination step it re-sends a stable, growing shared prefix — the goal/brief plus the accumulated history. None of the three implements cross-call KV-prefix reuse in its own coordination layer: the prefix is re-materialized each step unless the serving backend deduplicates it (vLLM Automatic Prefix Caching, SGLang/RadixAttention — both cited at the top of this doc) or the provider offers prompt caching. fak supplies that reuse kernel-side, independent of backend — the measured (N−1)·P prefill elision and the ~61.7% clawback plateau in the grid above.

Token accounting at one stated geometry — shared brief B = 2,000 tokens, per-step unique instruction u = 300 tokens, S = 12 coordination steps (a 3-worker crew over ~4 rounds, the geometry the CrewAI example already uses). “Naive” re-prefills the shared prefix each step; “reuse” prefills it once and (for the append-only histories) incrementally caches the growing prefix so each step adds only its new u tokens:

Framework (multi-agent mode) Documented coordination topology — what is re-sent per step naive prefill with prefix reuse structural token multiplier
CrewAI — hierarchical manager re-sends the shared brief on every delegation (docs) S·B + S·u = 27,600 B + S·u = 5,600 4.93×
AutoGen — GroupChat every agent re-reads the full, append-only transcript each turn — brief + growing history (conversation patterns, paper, growing speaker-selection history) S·B + u·S(S−1)/2 = 43,800 B + S·u = 5,600 7.82×
LangGraph — StateGraph each node re-receives the accumulated state; the default add_messages reducer is append-only, never deleting (state ref, state management) S·B + u·S(S−1)/2 = 43,800 B + S·u = 5,600 7.82×

Read three things off it:

  1. The token multiplier is the count of coordination steps on the shared prefix. Without reuse, the brief (and, for AutoGen/LangGraph, the append-only transcript) is re-prefilled every step — S× on the brief, super-linear once the transcript grows. With prefix reuse it collapses toward 1×. This is the “token multiplier” the scope asks to document, generalized across the three frameworks.
  2. AutoGen and LangGraph pay more than CrewAI because their history is broadcast/accumulated, not just re-briefed. The full transcript every agent re-reads each turn grows the prefill ~S² (u·S(S−1)/2), which is why AutoGen’s own docs report a 10-round/4-agent chat landing at 40k–80k tokens — the model here lands at 43,800 for S=12, inside that documented band (a consistency cross-check, not an asserted measurement).
  3. The frameworks are not the lever; the prefix cache is. All three reach the “with reuse” column only through a prefix-caching backend; fak is the one that provides it in the kernel regardless of provider, so the reuse is the measured geometry above rather than a backend the operator must hope is enabled.

Still deferred (unchanged): the live-model end-to-end wall-clock with the three frameworks installed — token and latency, real model — remains the DeferredRun named above (bench-node-gated). This subsection prices the structural coordination tax the frameworks impose by design; it does not assert a live throughput number.

Prefix-scale impact (new, fixed N=256)

The longer-prefix pscale artifacts keep the fan-out topology fixed (N=256, sub_turns=4, trials=8, seed 1592652270) and vary the shared master-goal prefix. This isolates the model-side question: as the stable prompt grows from smoke-size to near/full long-context, how much more of the naive N× prefix tax can reuse remove?

(pscale rows re-run with 8 trials + fixed seed 1592652270; the +247 here is within ±3 calls of the §1 headline’s +251 for the same N=256/P=2048 shape — trial-count variance, not a different result.)

shared prefix P cross_uplift p50 prefix_tokens_saved tax_clawed_back net modeled value/run
1,024 +247 261,120 47.0% $0.98
2,048 +247 522,240 61.7% $1.69
8,192 +247 2,088,960 80.7% $5.91
32,768 +247 8,355,840 87.4% $22.81
131,072 +247 33,423,360 89.3% $90.42
262,144 +247 66,846,720 89.6% $180.57
524,288 +247 133,693,440 89.7% $360.86
1,048,576 +247 267,386,880 89.8% $721.44

The impact is clean:

  1. Tool-result dedup is independent of prefix size. cross_uplift stays +247 because the same stochastic tool-call workload is being replayed; longer prompts do not manufacture more real kernel hits.
  2. The prefix lever scales almost linearly in dollars and prefill work avoided. prefix_tokens_saved = (N−1)·P, so doubling P doubles the avoided redundant prefix materialization.
  3. The tax-clawed-back curve approaches the provider-cache ceiling. At 1M tokens the modeled curve is 89.8%, close to the 90% cached-read discount assumed by the default Anthropic-style price model; the remaining gap is cache-write overhead plus suffix, decode, and fold tokens.
  4. This is a strategy result, not a real 1M-token wall-clock result. The pscale row prices a byte-identical shared prefix under a transparent cache model; it does not yet prove that a selected model/backend can actually hold and prefill a 1M-token context within memory or latency targets. That validation is tracked in #104 / #431.

Measured-path probe — real wall-clock vs modeled economics (#431)

The rows above are modeled. To keep that token economics strictly apart from what a real model can actually serve, tools/fanout_longctx_probe.py interrogates the three candidate long-context paths on the host and, for each target P, either measures a real prefill or records the structured CEILING that stopped it — it never extrapolates the modeled curve into a wall-clock number. It sizes the fp16 KV cache from real qwen25-7b geometry (mirrored from internal/turnbench/longcontext.go: 28 layers × 4 KV heads × 128 head-dim) so the memory ceiling is a quantity, not a guess. Artifacts: experiments/fanout/pscale/longctx-measure-p262144.json, longctx-measure-p524288.json, longctx-measure-p1048576.json, and the rolled-up longctx-measure.csv.

On the reference host that produced the checked-in artifacts (win32, 256 GB RAM, no dedicated GPU VRAM) every target P is SKIPPED_NO_LONGCTX_PATH:

candidate path reason recorded ceiling
fak in-kernel CONTEXT_CEILING selected checkpoint max_position_embeddings=512 ≪ 262144
llama.cpp MODEL_UNAVAILABLE llama-cli/llama-bench present, but no GGUF with context ≥ P on host
vLLM / SGLang SERVER_UNAVAILABLE no prefix-caching server reachable (127.0.0.1:8000 / :30000)

Two honest facts fall out, neither visible in the modeled table:

  1. The blocker on this host is the missing long-context checkpoint, not memory. The KV cache is 14.0 GiB at P=262144, 28.0 GiB at P=524288, and 56.0 GiB at 1048576 — all of which fit this host’s 256 GB RAM (kv_fits_host_ram: true). A real 256K–1M validation is gated on a checkpoint and a backend that admit the context length, not on this host’s memory.
  2. The skip is recorded, never extrapolated. ttft_ms / prefill_ms stay null for a skipped path; the modeled dollars and tokens-saved live only in the pscale table above. Re-run the probe on a host with a qualifying path — an in-kernel checkpoint whose context ≥ P, a long-context GGUF for llama.cpp, or a running vLLM/SGLang prefix-cache server — and the corresponding row flips to measured with the wall-clock fields filled. The probe embeds only stable host facts (no wall-clock timestamp), so a re-run on the same host reproduces the artifact byte-for-byte.

Alignment with real model / live FAK runs

This fanbench result is aligned with the rest of the shipped evidence, but it sits on a different axis:

§2 — The over-claim guardrail (what this is NOT)

The prefix-reuse / tax_clawed_back numbers are a reuse-vs-no-reuse / vs-stateless ablation, exactly like cmd/fleetserve’s — the win over a consumer that re-sends the master-goal prefix per sub-agent (the common framework default: every sub-agent ships the full system+goal prompt; or a stateless API consumer that does not use prompt caching). It is NOT a head-to-head throughput win over a tuned shared-prefix serving engine. SGLang/RadixAttention, vLLM automatic-prefix-caching, and llama.cpp under kv_unified+seq_cp also prefill a byte-identical prefix once — they occupy the same lever. fanbench measures how much of the naive N× prefix redundancy the reuse removes, priced at documented Anthropic prompt-cache multiples (read 0.1× / write 1.25×); it does not claim to beat those engines. The cross_uplift is a fak-vs-fak SHARED-vs-ISOLATED ablation (the fan-out’s benefit over running the sub-agents apart), not a vs-competitor claim.

Three axes, kept strictly apart — fanbench geometry vs cache pricing vs tuned-engine wall-clock (#430)

The over-claim guardrail above is now backed by a checked-in peer artifact (experiments/fanout/peer-baselines.json) so the three things this result mixes are never blended into one number. Reference cell N=256:

axis what it is value (N=256) is it a vs-engine claim?
A — fak-vs-fak reuse geometry (MEASURED) the SHARED−ISOLATED path-swap + exact prefill-tokens the kernel never redoes cross_uplift = +251 calls; (N−1)·P = 522,240 prefix tokens saved (P=2048) No — fak’s fan-out vs the same sub-agents run apart
B — provider prompt-cache pricing (MODELED) the naive-N× tax clawed back at documented cache multiples tax_clawed_back = 61.7% at Anthropic read 0.1× / write 1.25× No — vs a stateless consumer that re-sends the prefix per sub-agent
C — tuned engine wall-clock (MEASURED, peer artifact) what an engine that already shares the byte-identical prefix does in wall-clock llama.cpp kv_unified+seq_cp: 17.2 vs fak 5.2 agents/s @ C=32, P=1024, D=32 — llama.cpp ahead, preliminary; SGLang RadixAttention: 86.7% agents hit-rate (fak matches), 4.87–5.84× live prefill speedup, SGLang published up to 6.4× throughput / 3.7× latency (GPU regime); vLLM APC: not run on this host (SERVER_UNAVAILABLE) Yes — and it does not favour fanbench

The honest reading of column C: the tuned engines occupy the same prefix-reuse lever. On the one CPU shared-prefix point measured head-to-head, llama.cpp is ahead of fak (17.2 vs 5.2 agents/s, flagged preliminary in LLAMACPP-HEADTOHEAD-RESULTS.md §Axis 4); against SGLang/RadixAttention fak matches the cache-hit-rate axis (RADIXATTENTION-RESULTS.md) but does not out-throughput a GPU serving engine; vLLM’s Automatic Prefix Caching was not runnable on this no-GPU host and is recorded as a ceiling, never extrapolated. So columns A and B (where fanbench’s numbers live) are kept apart from column C (where the engines win or tie) — exactly so no reader mistakes the reuse-vs-reprefill geometry for a vs-engine throughput win. Full provenance, the per-C llama.cpp sweep, and the host ceiling that gates a same-host fresh run are in ../../experiments/fanout/peer-baselines.json.

§3 — The saturation knee (latency, not quality)

parallel_speedup = total_work_turns / critical_path_turns rises then saturates: 3.0 (N=4) → 29 (N=64) → 57.7 (N=256) → 72.8 (N=1024). The N sub-agents run in parallel, so the wave costs one sub-agent’s turns; but the fold grows with N (fold_turns = ⌈N·fold_tokens / fold_budget⌉ — the synchronous-join coordination tax, “the lead waits on the slowest of N then synthesizes N results”). Past N≈256 the fold dominates the critical path and added width buys little wall-clock — the unmapped high-N knee, made visible.

This is a latency/throughput saturation. The quality inversion the literature reports (realized accuracy peaks then declines with an imperfect verifier; “more agents ≠ better”) is a task-success phenomenon, on a different axis from this latency knee — fanbench’s cost sweep measures kernel cost, not goal correctness (no ground-truth sub-results). That quality axis is now its own controlled-litmus lane in §5 (#429: coverage@N / realized@N / matched-budget control with ground-truth atoms), which shows the realized-accuracy inversion the latency knee here cannot. See also the research brief §5–6 for the published anchors and the remaining seams (MAST failure tagging, hierarchical depth>1, parallel-vs-sequential at matched budget).

§4 — Profiles & determinism

A fixed (profile, N, sub-turns, trials, seed) yields the identical surface (TestFanoutDeterministic). The model-backed witness (TestPrefixReuseFanoutWitness) asserts all N clones are bit-identical to an independent full prefill — cross-agent prefix reuse provably never changes a sub-agent’s result.

Prefix options are now first-class in the artifact: CSV rows include prefix_tokens, JSON cells include prefix_tokens, and JSON sweeps include prefix_grid. -prefix keeps the old single-P run. -prefixes smoke,small,medium,long,big expands to 1K/2K/8K/32K plus a near-full-context prefix; -prefixes all is the same range. The big/max preset is computed as model_context - suffix - max(sub_turns)*decode, using -model-config (max_position_embeddings / model_max_length) when provided, otherwise -model-context (default 131,072).

§5 — Task-quality litmus: does fan-out improve the ANSWER? (#429)

Everything above prices kernel cost geometry — reuse, dedup, prefix economics, latency. It is silent on the question a reader actually cares about: does fanning wider make the answer better? The research brief (§8) lists “real task success / coverage@N / realized@N” as an explicit out-of-scope seam precisely because cost geometry and task quality are different axes — and blending them is the most common multi-agent benchmark error. This section keeps them apart on purpose.

tools/fanout_taskquality.py runs a controlled litmus task suite over the same flat fan-out: one master goal whose complete solution is a known ground-truth set of G=12 sub-result atoms. N sub-agents (4 sub-turns each, homogeneous pool) draw atoms, an imperfect verifier/fold selects from their outputs, and a fraction are fed an adversarial tool result. Each task row is joined to the real fanbench cost cell for the same N (coverage/realized next to the MEASURED cross_uplift / tax_clawed_back / parallel_speedup), so the artifact literally connects a litmus task run to the fanbench-like N grid. Artifact: experiments/fanout/taskquality-litmus.json (+ .csv), 64 seeded trials, medians. Reproduce / gate (byte-identical re-run, same discipline as the pscale probe): python tools/fanout_taskquality.py --check.

HONEST FRAME — this is [SIMULATED], NOT a real-model run. The task outcomes are a transparent knobbed model grounded in published anchors — homogeneous pools saturate ~4 agents (Agent Forest, arXiv:2402.05120); imperfect-verifier realized accuracy peaks then declines, compute-optimal K≤5 (arXiv:2411.17501); step-repetition is the single most frequent MAST failure mode at 15.7% (arXiv:2503.13657); naive MAS sits ~33–59% correct. The injection arms are grounded in the real quarantine evidence in LIVE-RESULTS.md (Gemini / local Qwen: poisoned tool output reaches the naive baseline; fak quarantines it). The cost columns are joined verbatim from the measured fanbench-research.csv. This lane proves the cost/quality separation; it does not prove real-model quality.

Medians over 64 trials (experiments/fanout/taskquality-litmus.csv):

N coverage@N realized@N verifier_success duplicate_work inj_contained fak/naive matched-budget single-agent coverage (joined) tax_clawed_back (joined) parallel_speedup
1 0.083 0.083 1.00 0.00 1.00 / — 0.250 0% 1.0
4 0.250 0.167 1.00 0.25 1.00 / 0.00 0.500 54.8% 3.0
16 0.583 0.500 1.00 0.53 1.00 / 0.00 0.833 60.4% 11.0
64 0.917 0.667 1.00 0.82 1.00 / 0.00 1.000 61.4% 29.0
256 1.000 0.583 0.875 0.95 1.00 / 0.00 1.000 61.7% 57.7

Read five things — and note that none of them is “wider fan-out makes a better answer”:

  1. Coverage@N grows, then SATURATES. Distinct ground-truth atoms reached climbs log-linearly (0.08 → 1.0 by N≈64) and then can rise no further — the homogeneous-pool knee the literature reports at ~4 agents, made visible against the cost curve.
  2. Realized@N PEAKS then DECLINES. The verifier/fold is imperfect: by N=64 it surfaces 66.7% of atoms, but by N=256 the decoy pile from failed sub-agents crowds correct atoms out of the finite fold, dropping realized to 58.3% and verifier precision to 0.875. More agents past the knee make the delivered answer worse, not better — exactly the inversion §3’s latency knee could not show.
  3. Duplicate work explodes. 25% of competent outputs re-derive a covered atom at N=4, 95% at N=256 — the step-repetition failure mode. Wider fan-out buys mostly redundant work.
  4. Injection containment is the real, decisive split. Under the fak arm every injection-exposed sub-agent is quarantined (1.00); under the naive arm none is (0.00 at N≥4), and the lost work drags naive-arm coverage below fak’s — the litmus echo of the real weak-model rescue in LIVE-RESULTS.md. (At N=1–2 the naive cell is —/vacuous: too few trials draw an injection to be meaningful.)
  5. The matched-budget single-agent control matches or BEATS the fan-out at every N. Given the fan-out’s total call budget as one sequential trajectory (no cross-agent redundancy, no parallelism), the lone agent’s coverage is higher at every width — 0.50 vs 0.25 at N=4, 0.83 vs 0.58 at N=16. This is the research brief’s budget-controlled design law: at equal compute, fanning out does not win on task quality.

The one-line honest reading. Fan-out’s wins in §1–§3 are real and they are cost/latency/reuse wins — the joined columns show 62% of the token tax clawed back and a 57× parallel speedup at N=256. They are not quality wins: at matched budget a single agent covers as much or more, realized accuracy inverts past the verifier knee, and most of the added width is duplicate work. The kernel makes wide fan-out cheaper to run; it does not make the answer better. The only quality axis where fak changes the outcome here is injection containment, and that is a safety-floor result, not a fan-out-width result.

What is still open (the real-model half). This is a controlled litmus, not a live model sweep. A real fak agent fan-out across the N grid with real ground-truth tasks (the seam #429 also names) needs a model/host budget this no-GPU node does not have; the single live A/B that exists today (LIVE-RESULTS.md) is single-task, single-agent. The litmus pins the metric definitions and the cost/quality separation; flipping it to a measured real-model row is the open work.

§6 — Tracked limitations and next steps

The limitations are now explicit GitHub issues rather than hidden caveats: