Dents — every cell where fak loses
16 of 45 scored cells are net-negative. A scorecard with no dents would not be measuring anything.
Two distinct failure modes, and the distinction is the whole point: β < 0 means fak is worse on the axis itself — no amount of easier onboarding fixes that. β > 0 with w_net < 0 means the gain is real but the adoption surface eats it — that one is fixed by cutting the surface, not by shipping more features.
On this board all 16 dents are the first kind. Not one negative cell is a case of a real gain being eaten by the adoption surface — every one of them is fak being genuinely behind on the axis. That is the more expensive kind to fix: trimming verbs or shortening docs would not move a single one of them.
| w_eff | use case × facet | verdict | cause | vs |
|---|---|---|---|---|
-4.35 |
local-first × raw-speed | REGRESSIVE | worse on the axis | llama.cpp serving GGUF locally |
-4.06 |
solo-max × steady-state-ops | REGRESSIVE | worse on the axis | bare claude, as installed |
-1.52 |
local-first × steady-state-ops | REGRESSIVE | worse on the axis | llama.cpp serving GGUF locally |
-1.20 |
local-first × portability | REGRESSIVE | worse on the axis | llama.cpp serving GGUF locally |
-1.06 |
framework-builder × steady-state-ops | REGRESSIVE | worse on the axis | a raw SGLang endpoint, unmediated |
-0.86 |
platform-team × steady-state-ops | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.72 |
regulated × steady-state-ops | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.71 |
framework-builder × portability | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.68 |
fleet-operator × steady-state-ops | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.63 |
solo-max × token-economy | DRAG | worse on the axis | bare claude, as installed |
-0.60 |
researcher × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.56 |
platform-team × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.51 |
fleet-operator × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.51 |
framework-builder × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.42 |
regulated × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
-0.28 |
solo-max × raw-speed | DRAG | worse on the axis | a raw SGLang endpoint, unmediated |
local-first × raw-speed (-4.35)
- β
-1.064— fak0.391vs next-best0.705(llama.cpp serving GGUF locally), ceiling1 - load
+0.50— 6 h fak vs 0 h alternative, against a 12 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md CPU-parity row,
model-ladder/qwen25-1.5b-q8-cpu-parity-m3pro.json: fak decode 38.1 tok/s on the M3 Pro; against the ~97.4 tok/s memory-bandwidth roofline (150 GB/s / 1.54 GB Q8_0 resident) that is 0.391 of physics - THE BLOCKING CELL, and the single most important result on this board. fak is not merely slower than llama.cpp – it sits at 39% of the physical roofline where llama.cpp sits at 71%, so beta is NEGATIVE and no amount of easier onboarding can fix it. It is this buyer’s highest-weighted facet (0.35), so the whole segment comes back BLOCKED. The correct advice is unambiguous: do not adopt fak as your inference engine. If you want fak’s management layer on a local box, point it AT llama.cpp rather than replacing it – and note that doing so moves you to the gateway shape, where the ceiling becomes llama.cpp itself and fak can only take a cut.
- Fence: Single-stream CPU decode of a 1.5B Q8_0 model on one host. It does not measure the accelerated paths, batch serving, or larger models – but it is the shape this buyer actually runs, which is why it is scored here.
solo-max × steady-state-ops (-4.06)
- β
-1.500— fak2.5vs next-best1(bareclaude, as installed), ceiling0 - load
+0.25— 0.5 h fak vs 0 h alternative, against a 2 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level
fakverbs, 34 operating .md files at repo root, ~2.0M words across root+docs, plus its own upgrade cadence, its own config file (dos.toml), and its own failure modes layered on the agent’s - The worst cell on the board for this buyer, and the most important one. Because the ceiling is 0 hours/month, the denominator is -N: any layer costing more than double the incumbent’s monthly attention falls straight to the display floor. Adding a management layer to a setup that already works more than doubles a solo developer’s recurring burden, and no token or safety win on this board is currently measured well enough to pay for it.
- Fence: Both endpoints are authored estimates grounded in the measured surface, not timing studies. The RATIO is the claim, not the absolute hours: a 100-verb, 2M-word layer on top of a working single-binary CLI is not a wash, whatever the true hours are.
local-first × steady-state-ops (-1.52)
- β
-0.750— fak3.5vs next-best2(llama.cpp serving GGUF locally), ceiling0 - load
+0.50— 6 h fak vs 0 h alternative, against a 12 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents, plus build-tag and backend churn on a single box
- Doubling the recurring burden of a single-box setup drops straight to the display floor, by the ceiling’s construction. For a hobbyist that is the correct verdict rather than an artifact.
- Fence: Authored estimates grounded in the measured surface. The ratio is the claim.
local-first × portability (-1.20)
- β
-0.571— fak0.4286vs next-best1(llama.cpp serving GGUF locally), ceiling1 - load
+0.50— 6 h fak vs 0 h alternative, against a 12 h tolerance - provenance MEASURED —
docs/HARDWARE-MATRIX.mdsupport-maturity matrix (GENERATED byfak support-maturity-scorecard --write-doc, freshness-gated in CI): of 14 model families, 6 are SUPPORTED on the accelerated backends (cuda/metal/vulkan) and 8 are FENCED there – 6/14 = 0.4286 for a buyer running on Metal - pure_tax, because the incumbent IS the ceiling: adopting fak can only narrow what you can run. The slice matters – across the whole grid it is 32/56, but this buyer runs on an accelerator, where it is 6 of 14 families. The other 8 are honestly FENCED (the accelerated path refuses rather than returning a wrong answer), which is good engineering and bad portability simultaneously.
docs/HARDWARE-MATRIX.mdgrades the first thing A; this cell scores the second. - Fence: BENCHMARK-AUTHORITY.md:106 still reports 19/56 for this instrument. That row is a stale snapshot; the generated, CI-freshness-gated doc says 32/56, and the generated doc is authoritative. The authority row needs updating.
framework-builder × steady-state-ops (-1.06)
- β
-0.667— fak10vs next-best6(a raw SGLang endpoint, unmediated), ceiling0 - load
+0.25— 30 h fak vs 0 h alternative, against a 120 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents, and a builder tracks the dependency’s releases and behaviour changes on behalf of every one of their own users
- Doubling the recurring bill lands on the display floor by the ceiling’s construction. For a builder the burden is worse than the number suggests, because they absorb their dependency’s churn on behalf of every downstream user rather than just themselves.
- Fence: Authored estimates grounded in the measured surface. The ratio is the claim.
platform-team × steady-state-ops (-0.86)
- β
-0.500— fak12vs next-best8(a raw SGLang endpoint, unmediated), ceiling0 - load
+0.30— 12 h fak vs 0 h alternative, against a 40 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level
fakverbs and 34 root operating documents, layered on the proxy and endpoint this team already runs – the layer’s own upgrades, config drift, and failure modes are additive, not substitutive - Half again on top of an existing operational bill. Materially better than the solo-developer case, because a team already carrying infrastructure absorbs a proportionally smaller increment – the same absolute burden lands very differently depending on what you were already carrying.
- Fence: Both endpoints are authored estimates grounded in the measured surface, not timing studies. The ratio is the claim.
regulated × steady-state-ops (-0.72)
- β
-0.500— fak12vs next-best8(a raw SGLang endpoint, unmediated), ceiling0 - load
+0.17— 40 h fak vs 0 h alternative, against a 240 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents layered on existing infrastructure, with its own upgrade and config-drift burden
- Fence: Both endpoints are authored estimates grounded in the measured surface. The ratio is the claim, not the hours.
framework-builder × portability (-0.71)
- β
-0.429— fak0.5714vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.25— 30 h fak vs 0 h alternative, against a 120 h tolerance - provenance MEASURED —
docs/HARDWARE-MATRIX.mdsupport-maturity matrix (GENERATED byfak support-maturity-scorecard --write-doc, CI freshness-gated): 32 of 56 cells SUPPORTED across 14 model families x 4 backends – 24 FENCED, 0 PROOF-PATH-ONLY, 0 UNDEFINED - This buyer weights portability higher than anyone else (0.20) because a narrow model matrix becomes a ceiling on THEIR product, permanently. Across the whole grid it is 32/56; the shape matters more than the fraction – all 14 families run on CPU, but only 6 of 14 on any accelerator, so a product whose users have GPUs inherits the narrower number.
- Fence: A FENCED cell refuses honestly rather than returning a wrong answer, which is good engineering. This facet scores portability only;
docs/HARDWARE-MATRIX.mdscores the honesty and grades it A. Also note BENCHMARK-AUTHORITY.md:106 still reports 19/56 for this instrument – a stale snapshot; the generated doc is authoritative.
fleet-operator × steady-state-ops (-0.68)
- β
-0.400— fak14vs next-best10(a raw SGLang endpoint, unmediated), ceiling0 - load
+0.25— 20 h fak vs 0 h alternative, against a 80 h tolerance - provenance MODELED — authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents layered on an existing fleet, with its own upgrades, lease semantics, and failure modes
- The smallest ops penalty on the board in proportional terms, for the same reason this segment scores best overall: an operator already carrying a fleet absorbs the increment better than anyone else.
- Fence: Authored estimates grounded in the measured surface. The ratio is the claim.
solo-max × token-economy (-0.63)
- β
-0.359— fak0.641vs next-best1(bareclaude, as installed), ceiling1 - load
+0.25— 0.5 h fak vs 0 h alternative, against a 2 h tolerance - provenance MEASURED — cross-agent ablation
experiments/ablate/cross-agent-pong-opus.json, K=5 per arm: output tokens 0.98x, turns 1.00x (the same work delivered), total ingested 1.56x – +28,986 tokens. Work-per-token relative to the bare agent = 1/1.56 = 0.641 - pure_tax because the bare agent is the reference point, not a rung below the ceiling: the absolute elimination fraction cancels in the ratio, so only the 1.56x matters. On the ONE committed head-to-head against what this buyer actually uses, fak costs MORE tokens, not fewer. The compaction row cannot rescue this cell: BENCHMARK-AUTHORITY.md:66 fences shed tokens to a proportional blend and forbids ratioing them against cache_read, so no like-for-like elimination fraction is derivable from it.
- Fence: K=5 per arm on ONE tiny tool-light task. The guard’s fixed preamble dominates a short session and would amortize over a long one – which is precisely the long-session comparison this segment’s session-longevity cell records as never run. Read this as ‘unamortized overhead is real and measured’, not as ‘fak costs 1.56x forever’.
researcher × raw-speed (-0.60)
- β
-0.250— fak0.75vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.33— 8 h fak vs 0 h alternative, against a 24 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md gateway row,
compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64 - Worst measured point, and the one that hurts most here: research sweeps run at low concurrency, which is where the gateway tax is largest. The convergence to ~3% at saturation does NOT rescue this buyer, because they rarely saturate.
platform-team × raw-speed (-0.56)
- β
-0.250— fak0.75vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.30— 12 h fak vs 0 h alternative, against a 40 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md gateway row,
experiments/qwen36/gpu-server-r4-20260622/compare.json: fak-gateway 1085.6 tok/s vs raw-SGLang 1451.6 at C=64 = 0.75x - pure_tax is the correct shape: when fak fronts SGLang, SGLang IS the ceiling for that deployment, so mediation can only take a cut. Reported at the WORST measured point (peak throughput, C=64). The same row records the tax converging to ~3% at saturation (C=128, 0.97x), so a team running hot pays far less than this cell says – the conservative number is used because a buyer should plan against the worst measured point, not the best.
fleet-operator × raw-speed (-0.51)
- β
-0.250— fak0.75vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.25— 20 h fak vs 0 h alternative, against a 80 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md gateway row,
experiments/qwen36/gpu-server-r4-20260622/compare.json: fak-gateway 1085.6 tok/s vs raw-SGLang 1451.6 at C=64 = 0.75x - Quoted at the worst measured point. This buyer in particular should read the convergence: the same row records the tax falling to ~3% at saturation (C=128, 0.97x), and a fleet operator runs saturated by definition. The conservative number is scored; the realistic one is better.
framework-builder × raw-speed (-0.51)
- β
-0.250— fak0.75vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.25— 30 h fak vs 0 h alternative, against a 120 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md gateway row,
compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64 - Worst measured point; converges to ~3% at saturation. A builder inherits whichever end of that range their users actually run at, which they do not control.
regulated × raw-speed (-0.42)
- β
-0.250— fak0.75vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.17— 40 h fak vs 0 h alternative, against a 240 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md gateway row,
compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64 - Reported at the worst measured point; the same row records convergence to ~3% at saturation. For this buyer the throughput tax is close to irrelevant – it is weighted 0.05 precisely because an auditor never asks about tokens per second.
solo-max × raw-speed (-0.28)
- β
-0.029— fak0.971vs next-best1(a raw SGLang endpoint, unmediated), ceiling1 - load
+0.25— 0.5 h fak vs 0 h alternative, against a 2 h tolerance - provenance MEASURED — BENCHMARK-AUTHORITY.md Gemma-4-31B serving row: end-to-end p50 1.03x versus the unmediated endpoint, i.e. 3% slower; expressed as a speed fraction, 1/1.03 = 0.971
- A ~3% end-to-end tax at the median. Small, and small is the correct answer here – mediation is cheap when the model call dominates.
- Fence: PROXY, and scope-mismatched on purpose: measured on a self-hosted Gemma-4-31B endpoint, not on the hosted-provider path this buyer actually uses. It is the closest committed measurement of fak’s end-to-end mediation cost. The equivalent measurement against a hosted provider has not been run. It is quoted here because it costs fak points rather than winning them; a flattering out-of-scope number would not be admissible.