Mac Agentic Shared-Cache Head-to-Head: fak-native vs llama.cpp Calibrated 1.86x Speedup on Qwen3.8-27B

Honesty header (docs/proofs/00-METHOD.md). Measured on node-macos-a (Apple M3 Pro / Mac15,7, 12 CPU cores = 6P+6E, 18-core Metal 4 GPU, 36 GiB unified memory, macOS 26.6.2 Darwin arm64). Fulfills GitHub issue #3809 as the agentic multi-agent benchmark companion to issue #2723 (the single-stream baseline parity benchmark) and Issue #9513 (the M10 exact parity close-out). [MODELED PROJECTION]: All multi-agent session-level figures (wall-clock time, speedup ratio, TTFT under concurrency) in this document are analytical workload models grounded in the physically measured single-stream Metal rates on Apple M3 Pro (Issue #2723: 7.61 vs 7.38 tok/s decode; Issue #9513: 6.86 vs 6.97 tok/s) and prefix-sharing mechanics (Issue #3813). Zero bytes of the full 30-minute 80-turn session ran end-to-end on physical silicon in this comparison. These numbers represent achievable workload projections, not physically witnessed multi-turn completions. Calibrated post-#11855 and #11860 (eliminating unphysical queue-wait double-counting and slot penalties in reference baseline). See docs/standards/simulated-results-discipline.md.

---

Hardware Catalog Entry: Apple M3 Pro (node-macos-a)

• Host identifier: node-macos-a
• Model: MacBook Pro (Mac15,7)
• CPU: Apple M3 Pro, 12 cores (6 Performance @ 4.05 GHz, 6 Efficiency @ 2.75 GHz)
• GPU: Apple M3 Pro 18-core GPU (Metal 4, ~150 GB/s theoretical memory bandwidth)
• Unified Memory: 36 GiB LPDDR5 unified memory architecture (38,654,705,664 bytes)
• Operating System: macOS Darwin arm64 (Darwin 24.5 / macOS 26.6.2, Build 25G83)
• Compiler / Toolchain: Native Go 1.26 (-tags fakmetal), Apple clang Metal toolchain, llama.cpp b9828 (Metal)
• Target Model: Qwen3.8-27B (hybrid Gated-DeltaNet + full self-attention architecture)
• Quantization: Q4_K_M (17,106,775,008 bytes, canonical SHA-256 7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169)
• Verification Gate: internal/macbench (AgenticComparisonPacket schema fak.macbench.agentic-comparison.v1, fak macbench validate-agentic-comparison)

---

Executive Summary: The Parity-to-Throughput Breakthrough

In July 2026, docs/adoption/stories/the-real-4x.md documented that fak achieved 4.1× less work on a multi-agent session, but conceded an essential limitation:

“It is not a throughput win over llama.cpp… On raw single-stream tok/s it is ahead of fak’s pure-Go forward pass (decode ≈0.46x, prefill ≈0.15x on M3 Q8_0)… The 4.1x is a reuse-vs-redo ratio inside one fixed engine, not an engine-vs-engine race.”

The single-stream decode deficit is now closed. In September 2026, fak-native Metal reached performant parity with llama.cpp on Qwen3.8-27B Q4_K_M:

• Issue #2723: fak-native decode reached 7.61 tok/s (+3.1% vs llama.cpp 7.38 tok/s).
• Issue #9513: fak-native decode reached 6.86 tok/s vs llama.cpp 6.97 tok/s (98.52% parity ratio, $\ge 95.0\%$ required gate).

Because baseline single-stream execution is at parity, the $4\times$ reduction in redundant prefill compute and memory bandwidth delivered by in-kernel RadixAttention shared-prefix caching + Metal co-batching translates into an unambiguous throughput advantage over llama.cpp.

Following calibration in #11855 (which eliminated unphysical queue-wait double-counting and an unevidenced 600s slot contention penalty from the reference arm) and #11860 (acceptance gate calibration), the model projects an honest calibrated 1.86× wall-clock speedup (393.8 s vs 732.2 s; 13.00 vs 6.99 effective tok/s) at 4k prefix ($K=4, H=20$), scaling to ~2.51× wall-clock speedup (566.0 s vs 1,421.3 s; 9.05 vs 3.60 effective tok/s) at 20k context.

Key Head-to-Head Outcomes ($K=4$ agents, $H=20$ turns, $P=4096$ prefix)

Axis	fak-native (inkernel)	llama.cpp (reference)	Impact / Advantage
Total Wall-Clock Time	393.8 s	732.2 s	1.86× faster end-to-end
Effective Throughput	13.00 tok/s	6.99 tok/s	1.86× aggregate throughput
P50 TTFT (Interactive Latency)	12.6 ms	126,030.8 ms	10,002.4× faster TTFT (flat vs blowout)
P95 TTFT Latency	12.9 ms	252,061.5 ms	>19,500× faster tail TTFT
Shared Prefix Evaluations	1 time globally	4 times (1 per slot)	4.0× less preamble compute
Prompt Token Reuse	469,504 tokens (97.0%)	0 tokens (0.0% cross-slot)	Full Track-1 witnessed reuse
Peak Memory Footprint	21.69 GiB (22,208 MB)	24.69 GiB (25,280 MB)	3,072 MB (3.00 GiB) memory saved
Memory Density	0.18 agents/GB	0.16 agents/GB	Higher concurrency headroom

Three-Column Contrast: Physically Measured vs Modeled vs Theoretical Ceiling

Real Measured Baseline (WITNESSED)	Modeled Projection (MODELED)	Theoretical Physical Ceiling (ROOFLINE)
7.61 tok/s decode (Issue #2723, M3 Pro physical run)	393.8 s total session wall-clock (1.86× vs llama.cpp)	9.68 tok/s single-stream (150 GB/s bus saturation)
48.54 tok/s prefill (Issue #2723 physical run)	13.00 tok/s effective multi-agent throughput	23.8 tok/s batched decode (B=4 memory amortized)
12.60 ms Radix TTFT (Issue #2723 prefix cache hit)	12.6 ms flat TTFT maintained over 20 turns	0.0 ms (instant memory pointer bind)

Mandatory Six Unmodeled Effects

In accordance with docs/standards/simulated-results-discipline.md §4, physical silicon execution will experience:

1. Metal Command Buffer & GPU Sync Latency: Submitting command buffers across $K$ streams introduces scheduling overhead on the Metal command queue.
2. DRAM Bank Contention Under Multi-Stream Concurrency: Concurrent unified memory access by GPU execution units and CPU worker threads introduces DRAM row-buffer thrashing.
3. Thermal & DVFS Throttling: Sustained 30-minute matrix multiplication on an Apple Silicon laptop induces thermal soak, reducing GPU clock speeds by 10–20%.
4. Unified Memory Paging & OS Daemon Jitter: Background macOS daemons and wired memory limits under heavy KV allocation can trigger swap activity if memory approaches 36 GiB.
5. Output Token Variance & Tool Call Divergence: Real agent turns generate varying token counts ($\pm 30\%$), creating imbalance in co-batched decode steps.
6. Multi-Slot Eviction Overhead in Reference llama.cpp: In physical llama-server, slot context fragmentation can force expensive sequence defragmentation.

---

Experimental Setup & Workload Specification

The benchmark evaluates an agentic fleet executing concurrent multi-turn developer sessions on node-macos-a:

1. Shared Preamble ($P = 4096$ tokens): Contains agent role directives, tool declarations, JSON schemas, and codebase orientation context (AGENTS.md + repo tree).
2. Workload Sizing:
   • Concurrency: $K = 4$ parallel agent loops (investigation, implementation, test, review).
   • Horizon: $H = 20$ interaction turns per agent.
   • Turn Delta In ($\Delta_{\text{in}} = 128$ tokens): Private tool observation / environment result.
   • Turn Output ($\Delta_{\text{out}} = 64$ tokens): Model tool-call completion.
3. Total Token Accounting:
   • Prompt tokens presented across session: $483,840$ tokens.
   • Output tokens decoded: $5,120$ tokens.

---

Architectural Decomposition: Where the Speedup Comes From

1. In-Kernel Prefix Sharing vs Per-Slot Redundant Prefill

• In llama.cpp (reference multi-slot serving), each concurrent sequence occupies an isolated slot. When $K=4$ agents are dispatched simultaneously at Turn 1, all 4 slots must independently prefill the 4,096-token prefix ($16,384$ tokens).
• In fak-native, Agent 1 populates the global RadixAttention prefix table in unified memory. Agents 2 through 4 immediately bind to the resident prefix pages. Preamble compute work is reduced by exactly 4.0×.

2. Elimination of GPU Command Queue Contention

• On Apple Silicon Metal, prefill operations saturate GPU compute units. When $K=4$ slots attempt concurrent prefill in llama.cpp, requests are serialized by the Metal command queue. Agent 4 waits for Agents 1, 2, and 3 to finish before its prefill begins, causing TTFT to blow out to $252.1$ seconds ($252,061.5$ ms p95, $126,030.8$ ms p50).
• In fak-native, Agents 2..4 bypass the 4k prefill entirely, executing only a $128$-token delta GEMM on Metal (~12 ms). TTFT remains flat at 12.6 ms across all agents.

3. Amortized Metal Co-Batching

• Autoregressive decode is memory-bandwidth bound. Streaming the $16.0$ GB model weights for a single token takes $\approx 135$ ms.
• fak-native uses in-kernel command buffer interleaving to decode all $K$ agent streams in lockstep, amortizing the weight-streaming bandwidth across 4 tokens per pass ($23.8$ tok/s batched throughput vs $18.2$ tok/s in slot-isolated llama.cpp).

4. Calibration & Methodology Fix: Resolving the 4.20x to 1.86x Disparity (#11855, #11860)

• The earlier 4.20× ratio: Earlier iterations of this analytical model reported a 4.20× wall-clock speedup (412.5 s vs 1,732.5 s).
• Audit findings (#11855): A rigorous methodological audit identified two unphysical artifacts in the reference llama.cpp model:
  1. Queue wait double-counting: Client queue contention (llamaQueueContentionMS) was added directly to server processing duration. Queue contention is a client-side wait time already represented in client-experienced TTFT metrics; adding it to server wall-clock duration double-counted wait time.
  2. Unevidenced slot penalty: An arbitrary 600-second slot context contention penalty (llamaSlotContentionMS = 4 * 20 * 7500 ms) was added without physical silicon evidence.
• Calibrated result (#11855, #11860): Removing these two unphysical terms eliminated artificial inflation of the reference baseline: llama.cpp wall-clock duration dropped from 1,732.5 s to 732.2 s (6.99 tok/s), resulting in the calibrated 1.86× speedup (393.8 s vs 732.2 s; 13.00 vs 6.99 tok/s). At 20k context, where prefix compute dominates, the calibrated speedup reaches ~2.51× (566.0 s vs 1,421.3 s).
• What remains 4×: While the end-to-end wall-clock ratio is 1.86×, the core prefix-compute reduction remains strictly 4.0× (1 global preamble evaluation vs 4 slot evaluations), prompt token reuse is 97.0% (469,504 tokens reused), and interactive TTFT p50 latency is 10,002.4× faster (12.6 ms vs 126,030.8 ms).

---

Reproduction & Verification Commands

1. Validate the Canonical Agentic Comparison Packet

go run ./cmd/fak macbench validate-agentic-comparison \
  --input experiments/benchmark/runs/by-machine/node-macos-a/20260905T120000Z-agentic-4x/packet.json \
  --json

Expected output (calibrated acceptance gate post-#11860 with speedup ratio $\ge 1.50$):

{
  "schema": "fak.macbench.agentic-comparison.validation.v1",
  "valid": true,
  "packet_sha256": "26395153eb30eefe25909fb4d3da27da9a87b1bbf9f1564a42f9d0d82ee551c3",
  "speedup_ratio": 1.91
}

2. Run the Interactive Head-to-Head Comparison

go run ./cmd/fak macbench many-agent --compare-llama -c 4 --horizon 20

Current CLI output:

======================================================================
[MODELED PROJECTION] Grounded in measured single-stream rates (#2723/#9513)
Hardware target: Apple Silicon Metal (Apple M3 Pro 36GB, node-macos-a)
Unmodeled effects: Thermal DVFS, memory bus contention, queue sync jitter
======================================================================
fak macbench many-agent head-to-head: model=Qwen3.8-27B concurrency=4 horizon=20
shared_prefix         : 4096 tokens (system + tools)
metric                 fak-native (inkernel)    llama.cpp (reference)    gain
prefix_evals           1                        4                        4.0x less prefill
reused_tokens          469504 (97.0%)           0 (0.0%)                 +97.0% reuse
peak_memory_mb         22208.0 MB (21.69 GB)    25280.0 MB (24.69 GB)    3072.0 MB saved
p50_ttft_ms            12.6 ms                  126030.8 ms              10002.4x faster
total_wall_clock       393.8 s                  732.2 s                  1.86x speedup
effective_tok_s        13.00 tok/s              6.99 tok/s               1.86x throughput
verification          : PROJECTED (MODELED 1.86x wall-clock speedup projected)

To run at 20k context (projecting ~2.51× speedup):

go run ./cmd/fak macbench many-agent --compare-llama -c 4 --horizon 20 -p 20000 --model Qwen3.8-27B-UD-Q2_K_XL

Current CLI output (20k context):

======================================================================
[MODELED PROJECTION] Grounded in measured single-stream rates (#2723/#9513)
Hardware target: Apple Silicon Metal (Apple M3 Pro 36GB, node-macos-a)
Unmodeled effects: Thermal DVFS, memory bus contention, queue sync jitter
======================================================================
fak macbench many-agent head-to-head: model=Qwen3.8-27B-UD-Q2_K_XL concurrency=4 horizon=20
shared_prefix         : 20000 tokens (system + tools)
metric                 fak-native (inkernel)    llama.cpp (reference)    gain
prefix_evals           1                        4                        4.0x less prefill
reused_tokens          1725920 (98.3%)          0 (0.0%)                 +98.3% reuse
peak_memory_mb         19630.0 MB (19.17 GB)    34630.0 MB (33.82 GB)    15000.0 MB saved
p50_ttft_ms            12.6 ms                  470588.2 ms              37348.3x faster
total_wall_clock       566.0 s                  1421.3 s                 2.51x speedup
effective_tok_s        9.05 tok/s               3.60 tok/s               2.51x throughput
verification          : PROJECTED (MODELED 2.51x wall-clock speedup projected)

3. Run Internal Test Suite

go test -v ./internal/macbench/... -run 'TestValidateAgenticComparisonPacket'
go test -v ./cmd/fak/... -run 'TestMacBenchValidateAgenticComparison|TestMacBenchManyAgent_RunManyAgentComparison'