The performance-parity RSI loop

Issue: #733 · Epic: #637 (throughput parity over the shared spine) · Track B (#306).

The dispatch fleet now runs through the kernel by default (0302eed): every unattended dev turn flows over fak serve’s /v1/messages passthrough. That makes the fleet a continuous, non-self-authored performance signal — the right fitness function for an RSI propose→witness→keep/revert loop over gateway/forward changes. This doc defines that signal and wires it onto the loop the repo already ships, so a gateway/perf change is kept only when it measurably helps the real workload, not a synthetic micro-bench.

This is the rsi rung of fak’s loop ladder (engineering-is-building-loops) pointed at one concrete payload. It does not introduce a new loop engine — it reuses docs/rsi-loop.md’s rsiloop.Harness seam and the guard-hop fitness harness from #734. The pieces exist independently; this is the contract that binds them.

What’s already shipped (and what is not)

Piece Where State
RSI loop engine — propose/measure/keep-or-revert, worktree-isolated off main, non-forgeable keep-bit, breaker, journal, vs-main track gate internal/rsiloop + cmd/rsiloop, docs/rsi-loop.md shipped (demo payload: DefaultCacheSize / LRU hit-rate)
Guard-hop fitness harness — overhead + prompt-cache-preservation row with a --check honesty gate tools/guard_hop_bench.py, docs/benchmarks/GUARD-HOP-OVERHEAD-PENDING.md harness shipped; overhead arm PROJECTED; TTFB + cache arms PENDING (hardware-gated)
Loop-health metric — closure_rate / regression_rate tools/issue_closure_audit.py, #382 closure_rate shipped; regression_rate is an honest placeholder pending a live verdict-journal
Loop-verdict telemetry — each keep/revert mirrored to the DOS audit journal cmd/rsiloop -dos-observe (dosobserve.go, #588) shipped (observe-only; never re-gates the keep-bit)
The live signal source — per-request latency, in-flight, status mix, cache_read_input_tokens fak serve /v1/messages; the FAK Dogfood Slow Requests dashboard; fak guard’s exit summary (provider cache — N prompt tokens served from cache) emitting on the live fleet (DOGFOOD-CLAUDE.md, docs/fak/always-on-dogfood-server.md)

The one thing not present is a measured win kept by a live run. That arm is hardware-gated — see The honest gate at the bottom. Everything above is the machinery; this doc is the wiring; the live keep is the deferred step.

1. The fitness signal (the witness reads this, not its own work)

The loop optimizes two quantities, both read from the live fleet’s /v1/messages telemetry over a window of W real turns (not a synthetic bench). Both are non-self- authored: the loop proposes the change, but the numbers come from the workload, not from the author of the change.

A. Guard-hop TTFB delta — ttfb_delta_ms = p50(guarded) − p50(direct).

B. cache_read preservation rate — cache_preservation = cache_read(guarded) / cache_read(direct), summed over the window.

The combined keep predicate (what Measure() returns to the keep-bit):

KEEP  iff  suite_green
      AND  truth_clean                         (worktree git status clean)
      AND  cache_preservation >= 1.0           (the cache floor — non-negotiable)
      AND  ttfb_delta_ms < baseline_ttfb_delta (a STRICT improvement, like rsiloop)

The >= 1.0 floor and the strict-improvement rule mean a candidate that “looks fine” (green suite, clean tree) is still REVERTED unless it measurably lowered the hop cost without spending the cache — the same load-bearing discipline as rsiloop’s cycle-3 case (rsi-loop.md), here keyed on a real fleet metric.

Window and stability

2. Wiring it as an rsiloop.Harness payload

docs/rsi-loop.md already states the extension contract: “A real optimization plugs in by supplying its own Harness: a Candidates() that proposes real changes, and a Measure() that applies each in a worktree and returns the measured KPI + suite-green + truth-clean. The keep-bit, the breaker, the journal, and the vs-main discipline are reused unchanged.” The perf-parity payload is exactly that:

Cycle part rsiloop.Harness seam Perf-parity payload
1. Propose Candidates() one gateway/forward optimization per candidate (e.g. reuse the forward buffer; avoid a header re-parse; widen a flush window)
2. Verify-correct Measure().SuiteGreen go build + go vet + the internal/gateway suite in the worktree (the byte-for-byte cache_control forwarding tests already live here)
3. Measure-faster Measure().Metric + .TruthClean the fitness signal of §1: run guard_hop_bench.py measure against the worktree’s fak serve over the window W; .Metric = −ttfb_delta_ms gated by cache_preservation >= 1.0; .TruthClean = worktree git status clean
4. Keep-or-revert shipgate.Evaluate + shipgate.Gate unchanged: the non-forgeable keep-bit + the K-consecutive-non-keep breaker

Because the metric is −ttfb_delta_ms (so “larger is better”, matching rsiloop’s monotone-up convention) and the cache floor is folded into the returned metric (a sub-1.0 preservation forces a non-improving value), the existing keep-bit needs no change — the payload supplies the witness, the loop supplies the discipline.

# the closed perf-parity loop (once a live gateway + cache-reporting provider exist):
go run ./cmd/rsiloop -mode improve -repo . -baseline-ref main \
  -journal /tmp/perf-rsi.jsonl -dos-observe        # propose→witness→keep/revert, recurse

# ongoing benchmark against latest main (append one point; exit 3 on regression):
go run ./cmd/rsiloop -mode track -repo . -baseline-ref main -journal /tmp/perf-rsi.jsonl

The Harness impl that yields gateway candidates + drives guard_hop_bench.py measure is the payload to land next (a Go seam in internal/rsiloop / a cmd/ driver; the tools lane, not this docs lane). This doc is the contract it implements; it is deliberately not the implementation, so the spec lands before — and outlives — any one candidate.

3. Recording loop-health (the loop is itself checkable)

The perf loop must be auditable the same way it audits candidates. Two already-shipped rungs cover it:

python tools/issue_closure_audit.py --json     # closure_rate over the audited slice
python tools/guard_hop_bench.py describe --json # the PROJECTED+PENDING fitness row

The track-mode gate is the regression backstop: .github/workflows/ci.yml runs rsiloop -mode track on every push and fails the build on a strict KPI drop. When the perf payload lands, its committed baseline floor extends this gate — a gateway change that silently regresses the hop now blocks the trunk, not just the dashboard.

The honest gate (what is NOT done here)

This doc lands the definition and the wiring — issue task 1 (“define a fitness signal from the live fleet”) as a concrete, checkable contract, plus the runbook for tasks 2–3. It does not land a measured win, because that arm is hardware-gated and cannot be honestly produced without:

  1. a live fak serve gateway and a matched direct mock on one box (guard_hop_bench.py measure --gateway-url … --direct-url …),
  2. a cache-reporting provider so cache_read_input_tokens is non-zero on both arms (the prompt-cache-preservation arm is PENDING per GUARD-HOP-OVERHEAD-PENDING), and
  3. the live dogfood fleet emitting the window of real /v1/messages turns the witness reads.

Per the BENCHMARK-AUTHORITY honesty rules, no measured TTFB or cache number is asserted here — the harness’s --check gate refuses any row that smuggles a number into a PENDING arm. The smallest next step to close #733 fully: implement the gateway-candidate rsiloop.Harness (the tools/internal payload), stand up the gateway+mock+provider, run -mode improve -dos-observe, and fold the first KEPT measured TTFB reduction into BENCHMARK-AUTHORITY.md — tombstoning the PENDING row.

See also