Regulated buyer who must show their work — ADOPT
Finance, health, defence, or anyone whose agent output ends up in front of an auditor. Their binding constraint is not cost or speed, it is being able to prove, months later, what the system was told and why it did what it did – and to show that hostile input could not have steered it. They will absorb six engineer-weeks without blinking. That deep tolerance cuts both ways: it is why they can adopt fak, and it is why fak’s ‘it is only a wrapper’ advantage is worth much less to them than to anyone else.
- Next best option overall — a container-isolated agent behind a reviewed policy, with tracing retained for audit and a human in the loop on effectful actions
- Tolerance — 240 engineer-hours of unfamiliarity before this buyer walks away. Every cell’s
loadis measured against it. - Switch bar —
+0.50: thew_effone axis must clear before switching is rational. - Verdict — ADOPT. run-integrity clears the switch bar (+0.50)
| facet | weight | w_eff | verdict | β | load | vs |
|---|---|---|---|---|---|---|
| Run integrity | 0.10 | +1.00 |
CRUISE | +1.00 |
+0.20 |
an agent loop that trusts its own completion report |
| Session longevity | 0.05 | +0.83 |
CRUISE | +0.76 |
+0.17 |
no context management – run until the window fills |
| Token economy | 0.05 | +0.80 |
CRUISE | +0.75 |
+0.17 |
tuned warm per-agent KV cache |
| Observability | 0.30 | +0.72 |
CRUISE | +0.67 |
+0.08 |
OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel) |
| Injection control | 0.35 | +0.17 |
DRIFT | +0.00 |
-0.17 |
formal-isolation defenses (CaMeL, MELON) |
| Raw speed | 0.05 | -0.42 |
DRAG | -0.25 |
+0.17 |
a raw SGLang endpoint, unmediated |
| Steady-state ops | 0.05 | -0.72 |
DRAG | -0.50 |
+0.17 |
a raw SGLang endpoint, unmediated |
| Portability | 0.05 | ? | NEVER MEASURED | – | – | – |
Run integrity — +1.00 (CRUISE)
When you are not watching, does the run end in a state you can trust and resume?
- fak
1— MODELED, tools/loopgateverified-vs-naive-loop.report.json: governed arm cuts false-done 0.50 -> 0.00 - next best
0.5— an agent loop that trusts its own completion report, the ungoverned arm of the same report - ceiling
1(definitional) - β
+1.000→w+3.800 - adoption 48 h fak vs 0 h alternative, of 240 h → load
+0.20→τ+0.203 - w_net
+3.597→ capped at CRUISE (MODELED evidence) →w_eff+0.999 - cost basis POLICY.md 1,726 w + docs/governed-agent-quickstart.md 1,361 w, plus review and sign-off on the policy itself
- Fence Authored corpus, not field measurement; capped at CRUISE by the MODELED provenance rule. An auditor would ask for the corpus, and the honest answer is that it was constructed.
Session longevity — +0.83 (CRUISE)
How long can one session keep going before context forces you to start over?
- fak
0.24— OBSERVED,compaction_budget=48000indocs/nightrun/cache-savings.jsonlagainst a 200,000-token window - next best
1— no context management – run until the window fills, definitional: an unmanaged client holds the whole history and then stops - ceiling
0(definitional) - β
+0.760→w+0.996 - adoption 40 h fak vs 0 h alternative, of 240 h → load
+0.17→τ+0.168 - w_net
+0.828→ capped at RELATIVISTIC (OBSERVED evidence) →w_eff+0.828 - cost basis compaction is on by default once the guard is running
- Fence Scores the enforced resident budget, not work preserved across a compaction fire – which for an audited buyer is the question that actually matters and is unmeasured.
Token economy — +0.80 (CRUISE)
For the same delivered work, how much of the input-token bill does it remove?
- fak
0.983— MEASURED, BENCHMARK-AUTHORITY.md headline row,headline-qwen-50x5.json: fak does 1 unit of prefill work where naive re-prefill does 60.3, eliminating 1 - 1/60.3 = 0.983 - next best
0.932— tuned warm per-agent KV cache, the tuned arm of the same artifact: 4.1 units of 60.3, eliminating 0.932 - ceiling
1(definitional) - β
+0.750→w+0.973 - adoption 40 h fak vs 0 h alternative, of 240 h → load
+0.17→τ+0.168 - w_net
+0.805 - cost basis docs/serve-config.md 3,286 w + AGENTS.md 10,166 w, plus a change-control cycle this buyer applies to any new component
- Fence Measured on Qwen2.5-1.5B Q8_0 on a single host. This buyer runs something far larger under change control; the mechanism transfers in principle, the ratio is not measured at that scale.
Observability — +0.72 (CRUISE)
After the fact, from one artifact, can you reconstruct why it did what it did?
- fak
0.8— OBSERVED, the session journal records the consequential decision classes – model traffic, cache reuse, compaction, tool-policy verdicts, recovery – post-hoc with no live attach, every DENY stamped with a consequence class. Scored 4 of 5, not 5 of 5: all five classes are RECORDED, but not in one artifact – the session journal,docs/nightrun/cache-savings.jsonl, and the /metrics split are separate surfaces, so a full reconstruction means joining across them. Everything is captured; single-artifact reconstruction is not. - next best
0.4— OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel), OTel GenAI conventions instrument the model call and partially the cache; no span exists for a compaction, a tool verdict, or a recovery decision. 2 of 5 classes - ceiling
1(definitional) - β
+0.667→w+0.805 - adoption 40 h fak vs 20 h alternative, of 240 h → load
+0.08→τ+0.084 - w_net
+0.721→ capped at RELATIVISTIC (OBSERVED evidence) →w_eff+0.721 - cost basis fak: the journal is on by default; the cost is retention policy and evidence review; alternative: tracing stack plus retention, which this buyer would deploy regardless
- Read This is the cell that decides the segment. It is the only facet where fak is not merely better but answers questions the alternative structurally cannot – and it is the exact question an audit asks. Note the shape: fak costs MORE to adopt here than the tracing stack (40h vs 20h), and still wins by a distance, because the axis gap is that large.
- Fence Capability decomposition, not a benchmark; capped at RELATIVISTIC by the OBSERVED provenance rule. Nobody has run a fixed set of post-hoc audit questions against both artifacts, which is the experiment this cell is standing in for.
Injection control — +0.17 (DRIFT)
Can hostile text inside the data the agent reads make it act for the attacker?
- fak
0— MEASURED, BENCHMARK-AUTHORITY.md AgentDojo row: full-stack ASR 0/38 = 0.000, versus 29/38 = 0.763 detection-only - next best
0— formal-isolation defenses (CaMeL, MELON), CaMeL ASR 0, MELON 0.0-2.4% – the tier is already at the definitional floor - ceiling
0(definitional) - β
+0.000→w+0.000· modeparity_at_ceiling - adoption 40 h fak vs 80 h alternative, of 240 h → load
-0.17→τ-0.168 - w_net
+0.168 - cost basis fak: POLICY.md 1,726 w plus wiring the agent behind the guard, under change control; alternative: restructuring the agent’s data flow so untrusted content cannot reach the control path, under the same change control – two engineer-weeks is conservative
- Read The most counter-intuitive result in the whole scorecard, and it is a direct consequence of this buyer’s deep tolerance. On their highest-weighted facet (0.35) fak scores near the bottom of positive, because it TIES the state of the art on the number and its only edge – being a wrapper rather than a rewrite – saves 40 hours against a 240-hour budget they were happy to spend anyway. Deep pockets and patience are exactly what neutralize fak’s main advantage. A buyer who can afford the rewrite should evaluate CaMeL on equal terms.
- Fence Benign and under-attack utility on the same suite is NEEDS_KEY. 0/38 bounds harm; it does not show the agent still does its job, and that is the question an auditor asks second.
Raw speed — -0.42 (DRAG)
How fast do tokens actually come out, against the hardware’s physical limit?
- fak
0.75— MEASURED, BENCHMARK-AUTHORITY.md gateway row,compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64 - next best
1— a raw SGLang endpoint, unmediated, the raw-SGLang arm of the same comparison - ceiling
1(physical) - β
-0.250→w-0.255· modepure_tax - adoption 40 h fak vs 0 h alternative, of 240 h → load
+0.17→τ+0.168 - w_net
-0.424 - cost basis same serve configuration as the token-economy cell
- Read Reported at the worst measured point; the same row records convergence to ~3% at saturation. For this buyer the throughput tax is close to irrelevant – it is weighted 0.05 precisely because an auditor never asks about tokens per second.
Steady-state ops — -0.72 (DRAG)
After you have adopted it, what does it cost every month to keep it running?
- fak
12— MODELED, authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents layered on existing infrastructure, with its own upgrade and config-drift burden - next best
8— a raw SGLang endpoint, unmediated, authored estimate: an isolated agent behind a reviewed policy already carries a real recurring bill - ceiling
0(definitional) - β
-0.500→w-0.549 - adoption 40 h fak vs 0 h alternative, of 240 h → load
+0.17→τ+0.168 - w_net
-0.718→ capped at CRUISE (MODELED evidence) →w_eff-0.718 - cost basis same serve configuration as the other cells in this segment
- Fence Both endpoints are authored estimates grounded in the measured surface. The ratio is the claim, not the hours.
Portability — never measured (weight 0.05)
no committed comparison of vendor lock-in against a container-plus-policy incumbent; the model x backend grid is not what this buyer means by portability.
Next: enumerate the exit path from each and mark which steps have a witness