fak vs Alternatives — Infrastructure Comparison

fak is an agent-kernel KV cache layer that adds cross-worker and cross-session prefix sharing on top of what a tuned single-instance prefix-caching engine (vLLM Automatic Prefix Caching, SGLang RadixAttention, llama.cpp, or provider prompt caching) already does. Within one serving instance, those engines prefill a shared prefix once and are roughly at parity with fak; fak’s incremental win is sharing that prefix across separate workers and sessions plus addressable mid-run eviction and a default-deny safety floor. Against that tuned SOTA, the modeled cross-worker delta is about 1.1-1.2x at 4 workers on a 2k prefix, rising toward the agent count as the shared-prefix fraction grows. The eye-catching 20-24x figures in this page are only versus a naive re-prefill-every-turn loop that no serving stack ships — a floor, never the SOTA comparison.

Date: 2026-06-20 Status: ✅ Complete with Quantitative Analysis

---

Executive Summary

Approach	Multi-Agent	Cross-Worker	Cross-Session	Prefix reuse (vs a tuned engine)	When It Wins
Server-Side Only (Anthropic/OpenAI)	❌	❌	❌	Intra-session prompt caching	Single-agent, single-session
Per-Session Frameworks (vLLM APC, SGLang, llama.cpp)	❌	❌	Limited	Per-instance prefix-once — ≈ matches fak within one instance	Single-agent, multi-turn
fak Fused	✅	✅	✅	+ cross-worker/session sharing: ~1.1–1.2× at 4 workers (2k prefix), rising toward the agent count as the shared-prefix fraction grows	Multi-agent fleets, shared context

Bottom line. The realistic SOTA here is a tuned prefix-caching engine (vLLM Automatic Prefix Caching, SGLang RadixAttention, provider prompt caching, llama.cpp seq_cp) — it already prefills a shared prefix once per instance and batches decode, so on raw within-instance work it is ~parity with fak. fak’s incremental infra win over that SOTA is cross-worker / cross-session prefix sharing — ~1.1–1.2× at 4 workers on a small 2k prefix, climbing toward the agent count as the shared-prefix fraction grows — plus addressable mid-run eviction and a default-deny safety floor those engines structurally don’t offer. The eye-catching 20–24× is only versus a naive re-prefill-every-turn loop — a worst case no serving stack ships, and never the SOTA comparison.

---

1. Server-Side Only (What Providers Do)

What It Is

Anthropic, OpenAI, and other frontier providers implement KV cache caching within your session only:

• First request: They cache what you send
• Next request: If you send the same prefix, they check their cache (via a hash)
• If hit: Skip processing, return cached result
• Charge: ZERO for cached tokens

Limitations

Limitation	Impact
Per-session only	Each session starts from scratch
No cross-worker sharing	Multiple agents can’t share cached context
No cross-session persistence	Cache evaporates when session ends
Append-only, no eviction	Can’t remove stale data from cache

When It Works

• ✅ Single-agent conversations
• ✅ Multi-turn within one session
• ✅ Large contexts (5K+ tokens)

When It Doesn’t

• ❌ Multi-agent fleets (each agent has its own cache)
• ❌ Shared problem statements across workers
• ❌ Cross-session reuse (cache disappears when session ends)

Quantitative Impact

For a 20-turn session with 5K shared prefix:

Approach	Tokens Prefilled	Cost (Claude @ $3/M)
Provider cache (session)	5K (first turn) + 600×19	$0.04
Provider cache (across sessions)	5K×20 (no sharing)	$0.30

The gap: Provider caching saves within sessions but not across sessions or workers.

---

2. Other Client-Side Approaches

Per-Session Caching (What Most Frameworks Do)

vLLM Automatic Prefix Caching

What it does:

• Caches KV states per serving instance
• Shared across requests within the same instance
• RadixAttention-style prefix matching

Limitations:

• ❌ Single-tenant only — Each serving instance has its own cache
• ❌ No cross-worker sharing — Workers in different instances can’t share
• ❌ Eviction pressure — Cache fills up, older prefixes dropped

Quantitative comparison (from SWE-bench smoke test):

Workers	Naive (A)	Per-Agent KV (B)	fak Fused (C)	B/C Ratio
1	1.04M tokens	52.9K tokens	52.9K tokens	1.00x
2	2.09M tokens	105.8K tokens	93.3K tokens	1.13x
4	4.17M tokens	211.6K tokens	174.1K tokens	1.22x

Interpretation: Per-agent KV gives ~1.2x benefit at 4 workers. The remaining gap is cross-worker reuse — exactly what fak provides.

SGLang/RadixAttention

What it does:

• Open-source RadixAttention implementation
• OBSERVED external-engine 86.7% cache hit rate on agent workloads
• 7.50× token speedup vs naive re-prefill

Measured against fak (from benchmark authority):

Metric	SGLang	fak	Notes
Cache hit rate	86.7%	Same regime	fak matches SGLang’s hit rate
Token speedup	7.50×	Same	Same underlying mechanism
Cross-worker reuse	0%	1.22x	fak adds what SGLang misses

Key finding: SGLang is excellent at within-instance reuse but doesn’t solve cross-worker reuse.

llama.cpp

What it does:

• Local inference engine
• Per-session KV persistence
• No sharing across sessions

Limitations:

• ❌ Each session is isolated
• ❌ No multi-agent coordination
• ❌ No cross-session prefix sharing

---

3. fak’s Differentiator

The Three Arm Comparison

Arm	What It Does	Prefix Handling	Decode
A — Naive	Re-send everything every turn	Re-prefills whole context (O(T²))	Serial
B — Per-Agent KV	Each agent caches its own state	Once per agent	Serial
C — fak Fused	Shared prefix across all workers	Once total	Batched

The Value Stack Concept

What makes fak different:

1. Multi-session aggregation — Context isn’t just cached; it’s aggregated across sessions
2. Cross-worker prefix sharing — All workers share ONE cache for common parts
3. Session persistence — KV cache reuse across turns and sessions
4. Addressable eviction — Can remove stale data from cache

Why This Matters for Fleet Operations

Scenario: 100 Agents, 100 GitHub Issues

Without fak:

Agent 1: Caches system prompt + tools + issue #1 (5,500 tokens)
Agent 2: Caches system prompt + tools + issue #2 (5,500 tokens, duplicate!)
Agent 3: Caches system prompt + tools + issue #3 (5,500 tokens, duplicate!)
...
Agent 100: Caches system prompt + tools + issue #100 (5,500 tokens, duplicate!)

Total cached: 550,000 tokens (mostly duplicates)

With fak:

Shared Cache: System prompt + tools (5,000 tokens, ONE TIME)
Each Agent: Adds only its issue statement (500 tokens each)

Total cached: 5,000 + 100×500 = 55,000 tokens (10x less)

The savings: 90% less cached data, 90% less prefill work.

---

4. Quantitative Comparison

Smoke Test Results (SWE-bench, 5 instances)

Workers	A (Naive)	B (Per-Agent KV)	C (fak Fused)	A/C	B/C
1	1.04M tokens	52.9K tokens	52.9K tokens	19.7x	1.00x
2	2.09M tokens	105.8K tokens	93.3K tokens	22.4x	1.13x
4	4.17M tokens	211.6K tokens	174.1K tokens	24.0x	1.22x

Interpreting the Ratios

• A/C (Net Work-Elimination): fak reduces 95%+ of prefill work vs naive re-prefill-every-turn
• B/C (Cross-Worker Reuse): Shared prefix gives 1.22x benefit at 4 workers (the value stack)
• A/B (Turn-Tax): 19.7x — re-prefill vs KV persistence, worker-independent

Which of these is the SOTA comparison? Only B/C. A/C and A/B are measured against the naive re-prefill loop — a worst case no serving stack ships. A tuned prefix-caching engine (vLLM APC, SGLang RadixAttention, provider prompt caching) eliminates the same turn-tax fak does, so against that SOTA fak’s incremental infra win is the cross-worker / cross-session B/C reuse (1.1–1.2× at 4 workers on a 2k prefix, larger as the shared-prefix fraction grows), not the 20–24× floor.

Cost Comparison (Claude 4.5 Opus at $3/M input)

Approach	Input Tokens	Cost
Naive (4 workers)	4.17M	$12.51
Per-Agent KV	211.6K	$0.63
fak Fused	174.1K	$0.52

Per benchmark run (vs the realistic SOTA): against a warm per-agent KV cache fak saves $0.11 (the cross-worker shared-prefix delta at 4 workers). The $11.99 “vs naive” figure is vs the re-prefill-every-turn floor a tuned engine already eliminates — not a SOTA comparison.

At scale (500 instances): the cross-worker delta grows with the shared-prefix fraction and agent count (see the B/C trend), not the naive multiple.

---

5. When fak Wins (And When It Doesn’t)

fak Wins When:

Scenario	Why fak Wins
Multi-agent fleets	Each agent reuses the same cached prefix
High-turn conversations	Each turn hits the cache (95%+ tokens cached)
Large shared context	5K+ tokens of system prompts, tools, problem statements
Fleet operations	Cross-worker reuse (1.13-1.22x) multiplies with agent count
Fan-out patterns	One master goal → N sub-agents (N=1024 measured)

fak Doesn’t Help When:

Scenario	Why
Single-turn requests	No reuse possible
Zero shared context	Everything is unique
Tiny contexts	Caching overhead > benefit

When fak Provides the MOST Value

Pattern	vs tuned warm-cache SOTA (the honest number)	(vs naive floor — not SOTA)
Multi-agent + high-turn (50×5 agents, 50 turns each)	~4.1×	60.3×
Fan-out (N=1024 sub-agents)	shared-prefix reuse; ~parity throughput vs a batched engine	72.8× parallel-vs-serial (a fleet metric, not a SOTA win)
Fleet-scale (100+ agents)	1.13–1.22× cross-worker reuse (rises with shared-prefix fraction)	—

---

6. Summary — Comparison Table

Feature	Server Only	Per-Session (vLLM/SGLang)	fak Fused
Single agent	✅	✅	✅
Multi-agent	❌	❌	✅
Cross-worker sharing	❌	❌	✅
Cross-session persistence	❌	❌	✅
Shared prefix	Per-session	Per-instance	Global
Addressable eviction	❌	Limited	✅
Prefix-once vs naive re-prefill (floor; a tuned engine matches fak)	1×	~7.5×+	~7.5×+
Cross-worker reuse (the real delta vs a tuned engine)	0%	0%	1.13–1.22×
Fan-out support	❌	❌	✅ (N=1024 measured)
Safety floor	❌	❌	✅ (quarantine, deny)

What This Means in $

Example: WebBench-style web agent fleet (100 agents, 20 turns each)

Approach	Prefill Tokens	Cost (Claude @ $3/M)
Server-side cache only	10M×100 agents	$3,000
Per-session (vLLM)	2M×100 agents	$600
fak Fused	500K×100 agents	$150

Savings: measured against the realistic SOTA — a tuned per-session prefix-caching engine (vLLM) — fak saves $450 on one benchmark run from cross-worker/session sharing. (Against server-side-only caching it is $2,850; there is no naive re-prefill row here — that floor would be larger still and is not the SOTA comparison.)

---

7. Why This Is Infrastructure, Not Magic

This isn’t a new algorithm. The building blocks are well-established:

• Prompt/KV prefix caching — Provider APIs, vLLM, SGLang
• Content-addressed storage — Git, CAS systems
• Capability-based security — OS capability systems

What fak does:

1. Integrates these mechanisms at the syscall boundary
2. Shares across workers — not just per-session
3. Aggregates across sessions — persistent value stack
4. Provides safety floor — quarantine, deny-as-value
5. Measures and proves the savings — deterministic benchmarks

The point: Most frameworks solve caching within one agent/session. fak solves it across agents, sessions, and workers — exactly what fleet-scale operations need.

---

8. Reproduce These Numbers

# SWE-bench smoke test (5 instances)
fak swebench describe --difficulty testdata/swebench_smoke.json

# WebBench value stack analysis
fak webbench describe --dataset testdata/webbench/sample-tasks.jsonl

# Full comparison with markdown report
fak webbench compare --dataset <tasks.jsonl> --md report.md

# Fan-out benchmark (N=1024)
go run ./cmd/fanbench -profile research -trials 16 \
  -out experiments/fanout/fanbench-research.json

# Session value-stack (50×5 agents)
FAK_WORKERS=6 go run ./cmd/sessionbench -hf <qwen2.5-1.5b> -lean \
  -turns 50 -agents 5 -prefix 2048 -decode 32 -result 64 \
  -out experiments/session/headline-qwen-50x5.json

---

Sources

• SOTA Comparison: SOTA-COMPARISON.md — SWE-bench Verified results
• WebBench Baselines: docs/webbench-baselines.md — Frontier web agent benchmarks
• Session Value Stack: SESSION-VALUE-STACK-ONEPAGER.md — 60.3× vs naive
• Fan-out Results: FANOUT-BENCH-RESULTS.md — N=1024 sub-agents
• Prefill Explained: docs/prefill-elimination-explained.md — Non-technical explanation
• Disaggregated Memory: DISAGGREGATED-AGENT-MEMORY.md — Strategic positioning

---

Last updated: 2026-06-20