Researcher measuring agent behaviour — ADOPT

Needs the run to be reconstructable more than they need it to be fast, because an experiment you cannot explain afterwards is not a result. Tolerance is a few days – they will build tooling, but only if it answers questions their own logging cannot. Throughput matters to them only as an experiment-turnaround cost.

• Next best option overall — hand-rolled logging around an agent SDK, plus an LLM tracing product for the model calls
• Tolerance — 24 engineer-hours of unfamiliarity before this buyer walks away. Every cell’s load is measured against it.
• Switch bar — +0.50: the w_eff one axis must clear before switching is rational.
• Verdict — ADOPT. run-integrity clears the switch bar (+0.50)

facet	weight	w_eff	verdict	β	load	vs
Run integrity	0.10	+1.00	CRUISE	+1.00	+0.33	an agent loop that trusts its own completion report
Observability	0.40	+0.80	CRUISE	+0.67	+0.00	OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel)
Session longevity	0.05	+0.65	CRUISE	+0.76	+0.33	no context management – run until the window fills
Token economy	0.15	+0.63	CRUISE	+0.75	+0.33	tuned warm per-agent KV cache
Raw speed	0.15	-0.60	DRAG	-0.25	+0.33	a raw SGLang endpoint, unmediated
Injection control	0.10	?	NEVER MEASURED	–	–	–
Portability	0.05	?	NEVER MEASURED	–	–	–

Run integrity — +1.00 (CRUISE)

When you are not watching, does the run end in a state you can trust and resume?

• fak 1 — MODELED, tools/loopgate verified-vs-naive-loop.report.json: governed arm cuts false-done 0.50 -> 0.00
• next best 0.5 — an agent loop that trusts its own completion report, the ungoverned arm of the same report: false-done 0.50
• ceiling 1 (definitional)
• β +1.000 → w +3.800
• adoption 8 h fak vs 0 h alternative, of 24 h → load +0.33 → τ +0.347
• w_net +3.454 → capped at CRUISE (MODELED evidence) → w_eff +0.999
• cost basis POLICY.md 1,726 w + docs/governed-agent-quickstart.md 1,361 w
• Read A false completion in a research sweep is worse than a crash: it silently contaminates the result set rather than failing loudly.
• Fence Authored corpus, not field measurement; capped at CRUISE by the MODELED provenance rule. A researcher is exactly the person who would notice.

Observability — +0.80 (CRUISE)

After the fact, from one artifact, can you reconstruct why it did what it did?

• fak 0.8 — OBSERVED, the session journal records the consequential decision classes – model traffic, cache reuse, compaction, tool-policy verdicts, recovery – post-hoc with no live attach, every DENY stamped with a consequence class. Scored 4 of 5, not 5 of 5: all five classes are RECORDED, but not in one artifact – the session journal, docs/nightrun/cache-savings.jsonl, and the /metrics split are separate surfaces, so a full reconstruction means joining across them. Everything is captured; single-artifact reconstruction is not.
• next best 0.4 — OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel), OTel GenAI conventions instrument the model call and partially the cache; no span exists for a compaction event, a tool-policy verdict, or a recovery decision. 2 of 5 classes
• ceiling 1 (definitional)
• β +0.667 → w +0.805
• adoption 8 h fak vs 8 h alternative, of 24 h → load +0.00 → τ +0.000
• w_net +0.805 → capped at RELATIVISTIC (OBSERVED evidence) → w_eff +0.805
• cost basis fak: the journal is on by default; the cost is learning what the record contains – docs/fak/tutorial.md 4,252 w; alternative: hand-rolled logging plus a tracing product, which costs this buyer roughly the same to stand up
• Read The cleanest cell on the board: the two options cost the same to adopt, so load is exactly zero and the score is pure axis gap. It is also this buyer’s highest-weighted facet by a distance (0.40). An experiment you cannot reconstruct afterwards is not a result, and three of the five decision classes are simply absent from the alternative.
• Fence A capability decomposition, not a benchmark – nobody has scored both artifacts by running a fixed set of post-hoc questions against them. Capped at RELATIVISTIC by the OBSERVED provenance rule; a finer decomposition than five classes would likely pull fak below 0.8 as well.

Session longevity — +0.65 (CRUISE)

How long can one session keep going before context forces you to start over?

• fak 0.24 — OBSERVED, compaction_budget=48000 in docs/nightrun/cache-savings.jsonl against a 200,000-token window
• next best 1 — no context management – run until the window fills, definitional: an unmanaged client holds the whole history and then stops
• ceiling 0 (definitional)
• β +0.760 → w +0.996
• adoption 8 h fak vs 0 h alternative, of 24 h → load +0.33 → τ +0.347
• w_net +0.650 → capped at RELATIVISTIC (OBSERVED evidence) → w_eff +0.650
• cost basis compaction is on by default once the guard is running
• Fence Scores the enforced resident budget, not work preserved across a fire – and for a researcher, a compaction that silently changes what the agent knows is a confound, not a feature. That interaction is unmeasured.

Token economy — +0.63 (CRUISE)

For the same delivered work, how much of the input-token bill does it remove?

• fak 0.983 — MEASURED, BENCHMARK-AUTHORITY.md headline row, headline-qwen-50x5.json: 1 unit of prefill work against naive re-prefill’s 60.3, eliminating 0.983
• next best 0.932 — tuned warm per-agent KV cache, the tuned arm of the same artifact: 4.1 units of 60.3, eliminating 0.932
• ceiling 1 (definitional)
• β +0.750 → w +0.973
• adoption 8 h fak vs 0 h alternative, of 24 h → load +0.33 → τ +0.347
• w_net +0.626
• cost basis docs/serve-config.md 3,286 w – configuring a shared endpoint for repeated experimental runs
• Read For this buyer the token win is an experiment-turnaround cost, not a bill: the same sweep finishes sooner. That is why it is weighted 0.15 and not 0.25.
• Fence Measured on Qwen2.5-1.5B Q8_0, 50 turns x 5 agents. A researcher sweeping one agent over many seeds gets less of this than a 5-agent measurement implies – though repeated runs over a shared prefix is exactly the shape radix reuse is good at.

Raw speed — -0.60 (DRAG)

How fast do tokens actually come out, against the hardware’s physical limit?

• fak 0.75 — MEASURED, BENCHMARK-AUTHORITY.md gateway row, compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64
• next best 1 — a raw SGLang endpoint, unmediated, the raw-SGLang arm of the same comparison
• ceiling 1 (physical)
• β -0.250 → w -0.255 · mode pure_tax
• adoption 8 h fak vs 0 h alternative, of 24 h → load +0.33 → τ +0.347
• w_net -0.602
• cost basis same endpoint configuration as the token-economy cell
• Read Worst measured point, and the one that hurts most here: research sweeps run at low concurrency, which is where the gateway tax is largest. The convergence to ~3% at saturation does NOT rescue this buyer, because they rarely saturate.

Injection control — never measured (weight 0.10)

the formal-isolation tier costs more than this buyer’s whole tolerance, so it is not their next-best option; the honest alternative is their own ad-hoc filtering, which nobody has measured.

Next: add an unguarded-harness arm to the AgentDojo run so the comparison is against real practice

Portability — never measured (weight 0.05)

no committed comparison of what an SDK-plus-logging setup locks in versus what fak locks in.

Next: score both on the same swap list with a witness per swap point