GLM-5.2 fak-kernel cache value, on a solved ticket — runbook

What this is. The executable path for epic #1010: serve GLM-5.2 from fak’s own CUDA forward pass on our sm_80 8-GPU datacenter server box, drive the Claude harness against it over a real, already-solved SWE-bench Verified instance, and observe the cache value — the repeated system+tools+repo prefix fak’s RadixAttention serves from cached KV when reuse is available, reported as aggregate prefill the kernel did not redo. That reused-token count, reported as WITNESSED, is this runbook’s headline datum.

Status: the observation seam is SHIPPED and tested; the live GLM-5.2 number is the box residual. fak swebench cache-witness (commit 52dfea0d, child #1011) reads the cache value off a live gateway. The number itself comes from a run on the box (child #1012). Nothing here invents a tok/s or a reuse figure.

1. Why a solved ticket and the cache value, not a throughput race

GLM-5.2 in fak’s kernel decodes at ~0.03–0.17 tok/s under --cpu-offload-experts (the #996 / #971 host expert-GEMM wall) — too slow to generate a full patch in reasonable wall-clock today. So the runnable proof is not “GLM-5.2 writes the whole patch fast.” It is:

  1. Take a real, already-solved instance (gold patch + gold test known), so correctness is checkable from evidence rather than dependent on a slow full generation.
  2. Drive it through the Claude harness wired to the GLM-5.2 fak-kernel gateway.
  3. Observe the cache value — the lever the goal names. This routes around the throughput wall (#996), not through it: aggregate KV-prefix reuse during the solved-ticket run proves the in-kernel cache-value lever end-to-end even if the full patch is not generated.

2. The data observation, in DOS terms (two numbers, two trust classes)

The codebase draws the provenance line the conflation scorecard requires. fak swebench cache-witness folds it into one record:

Field Metric Provenance Meaning
kv_prefix.reused_tokens fak_gateway_kv_prefix_reused_tokens_total WITNESSED fak’s OWN cache: the RadixAttention prefix match the kernel did not re-prefill. fak authored it.
kv_prefix.prompt_tokens fak_gateway_kv_prefix_prompt_tokens_total WITNESSED the prefill-token denominator of the realized cache-hit.
kv_prefix.{frozen,partial,cold}_turns ..._turns_by_regime_total WITNESSED the cliff distribution from the live cacheobs.FrozenFloor / cacheobs.ColdCeil thresholds; frozen is the append-only regime the value comes from.
provider_cache_read_tokens fak_gateway_inference_cached_prompt_tokens_total OBSERVED the upstream provider’s cache_read, relayed verbatim. 0 on the pure in-kernel path (no provider). Never proof fak preserved anything.

The record never sums the two — they are distinct caches over distinct paths. CacheBit() reports honestly whether fak’s own cache engaged in the aggregate run/window (reused_tokens > 0) versus an all-cold run; the /metrics family does not attribute reuse to a specific solved-ticket turn.

The warmup tax is a THIRD axis — never read a cold first turn as cache behavior. cache_bit_scope is aggregate-run-kv-prefix-reuse: cache_bit=true means the KV-prefix cache engaged somewhere in the window (the warm sub-second turns), not that the cold first turn was cache-accelerated. On the GLM-5.2 cache-value run the cold first turns (~511s / ~501s, per #3053) carry cache_bit=true alongside the warm turns purely as that scope artifact. Their cost is one-time backend warmup — weight load + CUDA graph capture + DeepGEMM/JIT compile — a different axis from KV-prefix reuse entirely. The ablation proves it: a fixed 11-token say pong prefix (nothing meaningful to reuse) still pays the cold tax (experiments/agent-live/gcp-glm-night2-20260706T133856Z/). So when reading cache-witness.json, treat a multi-hundred-second first turn as warmup-dominated, never as a cache-value datum.

Axis Where it lives Provenance Never do
KV-prefix reuse kv_prefix.reused_tokens (fak_gateway_kv_prefix_reused_tokens_total) WITNESSED attribute a cold-turn latency to it
Warmup tax (boot → warmup-first-token) serve-side fak_serving_time_to_first_token_seconds (OBSERVED latency) OBSERVED sum it into reused_tokens, or read cache_bit as covering it

The fence holds the same way the WITNESSED/OBSERVED split does: the warmup tax is a serve-side OBSERVED latency reading, never derived from or summed into the WITNESSED reused_tokens.

Remaining code work (tracked in #3053): emit the warmup tax as its own first-class time_to_first_ready field on cachewitness.Record, sourced from the serve fak_serving_time_to_first_token_seconds histogram, so the emitted packet — not just this runbook — carries the de-conflation. That change lands in internal/cachewitness (code lane), out of this docs note’s scope.

3. Serve GLM-5.2 from the pure kernel (on the box)

fak serve \
  --gguf /mnt/.../GLM-5.2-UD-Q4_K_M-00001-of-00011.gguf \
  --engine inkernel --backend cuda --cpu-offload-experts \
  --context-budget-tokens 8192 \
  --addr 127.0.0.1:8080 --model glm-5.2

(--cpu-offload-experts puts the MoE experts on host RAM; --context-budget-tokens 8192 keeps the KV plan off GLM-5.2’s 1M-context default that otherwise trips FitTooBig.)

4. Drive the Claude harness over the solved ticket, then read the cache value

Drive the fak coding agent (the harness) against the gateway on the solved instance:

fak swebench run --agent fleet \
  --gateway 127.0.0.1:8080 --model glm-5.2 \
  --filter smoke --difficulty testdata/swebench_smoke.json \
  --output run-glm52-cache

Then fold the cache value the run realized:

# direct, if the box is HTTP-reachable:
fak swebench cache-witness --gateway 127.0.0.1:8080 --out run-glm52-cache/cache-witness.json
# or, when the box is reachable only over the lab bridge, capture and relay /metrics:
#   curl -s localhost:8080/metrics > metrics.txt   (on the box)
#   fak swebench cache-witness --metrics-file metrics.txt --out cache-witness.json

cache-witness.json is the raw evidence: kv_prefix.reused_tokens (WITNESSED) beside provider_cache_read_tokens (OBSERVED), with the human summary stating whether the cache BIT and at what reuse fraction.

5. The honest result fence

6. Provenance