vCache — A Virtual API Cache over Providers We Don’t Control

Date: 2026-06-24 Status: design note plus shipped off-path slices. M5 Governor is implemented in internal/vcachegov and surfaced through fak vcache status|prove|prove-telemetry; the live provider control loop remains issue-tracked by M1-M3 (#716-#718). A live Claude Code prefix probe is recorded at experiments/agent-live/vcache-claude-prefix-probe-2026-06-25.jsonl and proves 13,141.5 token-equivalents saved (4.73%) with first positive request 4. The Codex/OpenAI #727 probe is proven from replayable Codex CLI token-count telemetry at experiments/agent-live/vcache-codex-token-count-proof-2026-06-25.jsonl, which proves 9,147,340.8 token-equivalents saved (85.98%) over 68 events. The OpenAI-compatible verifier path accepts raw Responses/Chat usage cached_tokens, Codex CLI token_count rows, and codex exec --json usage rows; the raw OpenAI API probe remains optional and skips without OPENAI_API_KEY. Relation to tree: active control plane proposed on top of internal/cachemeta (tier-1, passive) and generalizing the single-session prefix-coherence shaper (cachemeta.ShapeGLMTurnSegmentWitnessed + the internal/agent/glm_coherence.go message↔segment bridge) to a multi-request, multi-provider regime.

Audit provenance (2026-06-24). Every cachemeta symbol named below was grounded against internal/cachemeta at HEAD, and the §5/§10 numbers were independently re-derived and checked against current public provider docs. Three corrections from that pass are folded in: (1) there is no CacheLane type in the tree — the single-session work it referred to is the cachemeta coherence layer + glm_coherence.go shaper, named precisely throughout; (2) the Anthropic minimum cacheable prefix is per-model and currently 1024 tokens for Opus 4.8 / Sonnet 4.6 / Sonnet 4.5 (4096 belongs to Haiku 4.5), so the old “4096 (Opus)” rows are corrected; (3) RecommendLayout is advisory-only today (it returns a recommendation, it does not apply it) — “enforce it” is therefore net-new vCache work, not a tweak to existing behavior. The economics (break-even, Zipf s≈1.74, chain-horizon ceilings, the 300× recall loss) all reproduced to the decimal.

0. The one-sentence idea

We cannot tell a provider “cache this.” So we make our caching unit (a cachemeta entry — call it a vBlock) map onto whatever caching the provider will do on its own, by controlling the order, shape, and timing of the requests we send — and we keep a manifest that lets us rebuild and reorder a series of requests to recall any unit on demand.

vCache is virtual memory for a remote KV cache we can observe but not address. Our virtual page is a vBlock; the physical frame is a warm prefix sitting in some provider backend’s KV cache; the page table is the vCache manifest; page-in is replaying a prefix chain; the MMU is the request scheduler that arranges requests so the provider serves the prefix from its cache instead of re-prefilling it.

1. Why this is even a problem (the asymmetry)

A local engine (SGLang/vLLM/llama.cpp) lets cachemeta name an exact KV span (KVManifest, ExactSpanTarget) and evict it precisely. Providers give us none of that addressability:

Capability Local engine Anthropic OpenAI Gemini DeepSeek
“Cache this span” command yes (write KV) yes — cache_control breakpoints no yes — CachedContent w/ handle no
Query “is X warm?” yes no (only cache_read_input_tokens after a call) no (cached_tokens after) partial (handle exists) no (after)
Evict X yes no (TTL only) no yes (delete handle) no
Pin / TTL control yes partial — 5m or 1h no (auto idle) yes (set TTL) no
Address by content yes (digest) no — by byte-prefix only by byte-prefix by handle by byte-prefix

The whole field splits into two regimes, and vCache must serve both behind one abstraction:

The unifying observation: even the explicit providers are, underneath, doing prefix caching. cache_control is just a hint about where to cut the prefix. So if vCache can drive the implicit regime by shaping prefixes, the explicit regime is the same machine with an extra, cheaper lever. Assemble a correct prefix chain and default caching works for free — that is the goal’s closing line, and it’s the load-bearing insight.

2. The vBlock and the manifest (what we map)

A vBlock is the unit of cacheable work. In the goal’s example: 10,000 units × ~10 tokens. In practice a vBlock is a cachemeta.Entry whose identity already carries every axis that must match for provider reuse:

vBlock identity  (re-using cachemeta.ProviderCache / KVManifest axes)
  ├─ content digest         (SourceDigest / SpanDigest)
  ├─ ModelID                (cache is model-scoped — switching model = cold)
  ├─ TokenizerID            (tokenizer drift ⇒ different tokens ⇒ different prefix)
  ├─ SerializerID           (deterministic serialization hash — see §6)
  ├─ Vary axes              (Endpoint, ReasoningMode — SILENT cache-breakers, already in provider.go)
  └─ position in a prefix DAG  (NEW — the recall plan)

The manifest is the page table. For each vBlock it records:

  1. The canonical prefix path — the exact ordered token sequence (or chain of parent vBlocks) whose concatenation reconstructs this unit’s request prefix, byte-for-byte.
  2. Provider binding — which provider/model/endpoint/shard-affinity-key this unit was last warmed against.
  3. Warmth belief — last-warmed timestamp, observed cached_tokens, estimated TTL, a decaying probability that it is still resident (§7). This is a cachemeta.Lifecycle record pointed at TierProvider.

The manifest is the only durable state vCache owns. Like cachemeta, it stores no payloads — it stores how to make the provider reproduce them.

The prefix DAG

Provider prefix caching is linear: the cache is on the literal token prefix, so cacheable work forms chains (or trees) where each node shares a common prefix with its parent.

ANCHOR  P0  (shared system prompt + retrieved docs + few-shot,  ≥ Mₘᵢₙ tokens)
  │
  ├── u1   (P0 ++ unit-1 query, 10 tok)
  ├── u2   (P0 ++ unit-2 query)         ← siblings: all share P0, differ in the 10-tok tail
  ├── …
  └── u_k

CHAIN form (when units genuinely depend on each other):
  P0 → P0+u1 → P0+u1+u2 → P0+u1+u2+u3 → …
  (cache prefix grows; recall of u_k needs u1..u_{k-1} warm)

Two shapes, two economics:

Design rule: model the anchor, not the 10-token unit, as the cacheable thing. A lone 10-token unit can never be cached (it’s ~100× below the minimum cacheable prefix — §4). Units are cheap suffixes that ride a warm anchor.

3. Architecture — the active loop above cachemeta

cachemeta is passive: FromProviderCache records a hit after the fact (AdmissionDefer, “provider_cache_telemetry”); prefix_stability.go lints a recorded session offline; lifecycle.go models state given known transitions. None of it causes a cache to warm.

vCache is the closed loop that turns those passive primitives into control:

                          ┌───────────────────────────── vCache control plane (NEW, tier ≥2) ──────────────────────────────┐
   work units ───────────►│  Canonicalizer ──► Planner ──► Scheduler ──► Warmer ──► [PROVIDER] ──► Telemetry ──► Governor   │
   (vBlocks)              │     (§6)            (§2 DAG)     (§9)         (§5)                       fold        (§7 belief)  │
                          │       │                │           │           │                         │            │         │
                          │       └── uses ────────┴───────────┴───────────┴─────────── reads ───────┘            │         │
                          └──────────────────────────────────┬──────────────────────────────────────────────────┘         │
                                                              ▼                                                             │
   ┌──────────────────────────── cachemeta (tier-1, passive, EXISTING) ─────────────────────────────────────────┐         │
   │  prefix_stability.Diverge/RecommendLayout │ ProviderCache/FromProviderCache │ Lifecycle/TierTTL │ KVManifest │◄────────┘
   └────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
vCache component Job Built on (cachemeta)
Canonicalizer Make the cacheable prefix byte-stable; hoist volatile content to the tail Diverge, LintPrefixLayout, RecommendLayout, SerializerID
Planner Build/maintain the prefix DAG; choose anchors; decide star vs chain KVManifest identity axes
Warmth model Probabilistic belief “is this prefix resident on the shard I’ll hit?” Lifecycle + TierTTL aimed at TierProvider
Warmer Issue the warming request (the primitive — §5) (new; provider adapter)
Scheduler Topological recall order, send-one-then-fan, rate-limit budget AccessRatePerSec for ranking
Governor TTL policy: pin (heartbeat) vs lazy-rebuild vs evict Advance/Touch/MoveTo
Telemetry fold After every real call, update belief from cached_tokens FromProviderCache, SavingsSplit

Critically, vCache never lets a provider hit become a trust claim — it inherits ProviderCacheVerdict (“cost_latency_only”). A warm prefix is performance material, never proof a result may be re-served. (provider_guard.go’s double-count guard stays in force: provider savings are a distinct metric from local reuse.)

4. The warming primitive — how you “cache this” without a cache verb

4a. Explicit regime: max_tokens: 0 pre-warm

Anthropic gives the clean primitive the goal gropes toward (“individual requests with token max 1”). A max_tokens: 0 request runs prefill — writing the KV cache at your cache_control breakpoint — and returns immediately: content: [], stop_reason: "max_tokens", 0 output tokens billed, normal cache-write charge on cache_creation_input_tokens. It is strictly better than the old max_tokens: 1 decode-a-token trick (no token to discard, intent unambiguous).

Rejected combinations to encode as guards: stream: true, thinking enabled, output_config.format, tool_choice of tool/any, and inside Batches. Breakpoint goes on the last block shared with the real request (the anchor), never on the placeholder user turn.

4b. Implicit regime: the decode-1 prefill

No verb exists, so we manufacture a prefill: send the anchor as the prompt with max_tokens: 1 (the floor). The provider prefills the whole prompt to produce one token — and that prefill is what populates its prefix cache. We throw the one token away. This is the goal’s “token max 1 for decode.”

Caveats that become design rules (hardened by the red-team panel, §11):

4c. Once a correct prefix chain exists, default caching just works

The most important consequence: after the Canonicalizer + Planner have produced a byte-stable, correctly-ordered anchor, you often don’t need any dedicated warm at all. The first natural request warms the anchor; every subsequent sibling reads it. Dedicated warming is then only for (a) latency (cut first-request TTFT before a known burst) or (b) heartbeat (keep alive across an idle gap longer than the TTL). That is the whole point of “assemble a correct prefix chain and default caching can work too.”

5. The quantitative core (the “what does N have to equal?” questions)

First-pass derivations below; an independent derivation panel (workflow vcache-design-foundations) cross-checks every number — reconciliation in §10.

Provider constants used (Anthropic confirmed; others approximate):

  read mult r write mult w min prefix Mₘᵢₙ block B TTL T
Anthropic 5m 0.1× 1.25× 1024 (Opus 4.8 / Sonnet 4.6 / Sonnet 4.5) · 4096 (Haiku 4.5) breakpoint 5 min
Anthropic 1h 0.1× 2.0× same breakpoint 60 min
OpenAI ~0.5× ~1.0× (auto) ~1024 128 ~5–10 min (≤1 h hard)

Correction (audit): the minimum cacheable prefix is published per model, not per family. As of 2026-06-24 Opus 4.8, Sonnet 4.6, and Sonnet 4.5 are all 1024; the 4096 tier is Haiku 4.5, not Opus. The earlier “4096 (Opus) / 2048 (Sonnet 4.6)” was wrong and over-stated the uncacheable-prefix wall by ~4× for the flagship models. Re-probe per model on each bump (§7) — these are version-dependent.

5.1 Break-even: when is a dedicated warm worth it?

A dedicated warm is an extra request you’d otherwise not send. Let the warm cost c_w·P and each later read cost r·P; the no-warm alternative is k misses at 1·P each.

vCache gate: only spend a dedicated warm on a prefix whose expected reuse before TTL exceeds break-even (≥3 for auto/OpenAI, ≥2 for Anthropic-discount). Otherwise let the first natural request warm it.

5.2 Coverage: “maybe 7 units is enough for 85%”

Model prefix popularity as Zipf(s) over V distinct prefixes. Volume covered by warming the top-N is the generalized-harmonic ratio:

coverage(N) = H(N,s) / H(V,s), H(n,s) = Σ_{i=1..n} i^(−s)

Backing out the goal’s intuition — solve H(7,s)/H(V,s) = 0.85:

s (concentration) top-7 coverage (V≈10³) interpretation
1.0 (classic Zipf) ~0.35 flat-ish → 7 anchors nowhere near enough
1.5 ~0.72 concentrated
~1.74 ~0.85 the goal’s “7 → 85%” world (panel-exact: s*=1.74, ≈V-independent over 10³–5×10³)
2.0 ~0.92 very concentrated

So “7 units ≈ 85%” presumes a steep workload (s ≈ 1.8) — a few dominant anchors (one system prompt, a handful of retrieved-doc bundles). That is realistic for agent fleets but not universal. The actionable law:

Measure s (the concentration) before enabling vCache cost/latency warming. Rank vBlocks by frequency × size × reuse-density and warm only the head — the cache working set. On a flat workload (s ≈ 1) the head is huge and vCache barely helps; on a steep one (s ≳ 1.5) a tiny pinned set captures most of the volume. This is AccessRatePerSec-driven ranking applied to the provider tier.

5.3 Chain horizon: tiny units and the minimum-prefix wall

With unit size u = 10 tokens and provider minimum cacheable prefix Mₘᵢₙ:

cache-horizon C₀ = ⌈Mₘᵢₙ / u⌉ units before any caching is possible.

Mₘᵢₙ C₀ (units of 10 tok) meaning
1024 (Opus 4.8 / Sonnet 4.6 / 4.5) 103 first ~103 chained units are uncacheable — the common case
2048 205 (no current Anthropic model; OpenAI-family / generic)
4096 (Haiku 4.5) 410  

With block quantization B = 128 (OpenAI), the cache then grows in B/u = 12.8-unit steps and the un-cached tail can be up to B/u ≈ 12 units. Consequence and rule:

A 10-token unit cannot be cached alone — ever. It must ride a shared anchor of ≥ Mₘᵢₙ tokens, or be aggregated with peers into a ≥ Mₘᵢₙ super-block, and chains should be cut on B-aligned boundaries so the tail isn’t stranded. The goal’s “n=7” only makes sense at the anchor granularity, not the 10-token-unit granularity.

5.4 TTL heartbeat vs lazy rebuild

Keeping a prefix warm across window W costs ⌈W/T⌉ heartbeat prefills (P each). Lazy rebuild pays the recall cost R per miss; misses ≈ arrivals after an idle gap > T. Let λ = arrival rate for the prefix.

Governor policy: hot (λT≥1) → ride natural traffic; valuable-but-sparse with expensive recall → heartbeat-pin the tip; everything else → lazy rebuild; cold → evict from manifest. The 1h-TTL lever trades a 2× write for 12× fewer heartbeats — prefer it for bursty traffic with long idle gaps. (§10 reframes the pin/lazy cutoff precisely: on pure dollars lazy weakly dominates; pin only when latency value L or rate-limit shadow price μ tip it — λT > ln((W5+μ+L)/L) ≈ 1.)

5.5 Rate-limit feasibility: how big a warm set is sustainable

Tier gives R RPM and X TPM; real traffic uses R_real, X_real. Each warm of prefix P costs 1 request and P input tokens.

warms/min = min(R − R_real, (X − X_real)/P) sustainable warm-set = warms/min × T(min) RPM/TPM crossover at P* = (X − X_real)/(R − R_real)

Worked example, half-utilized tier R=4000, X=400000, so R_avail=2000, X_avail=200000, P*=100 tokens. Any realistic anchor (P ≥ 1024) is therefore TPM-bound:

anchor P warms/min sustainable warm anchors (×5 min)
4,096 ~48 ~244
16,384 ~12 ~61
65,536 ~3 ~15

Rule: budget warms inside headroom, prioritize by the §5.2 ranking, and degrade by warming fewer anchors (lower hit rate), never by 429-ing. “Reasonable numbers” = on a mid-tier key you can keep a few hundred 4k anchors hot, or a couple dozen 64k contexts — pick anchor size accordingly.

6. Canonicalization — the load-bearing discipline

Everything fails if the cacheable prefix is not byte-stable. prefix_stability.go already gives the offline linter; vCache makes it a pre-flight gate on every manifest entry before it is counted as warm economics.

Silent invalidators to canonicalize away (the prefix must be deterministic to the byte):

The dangerous failure the Canonicalizer exists to prevent: manifest says HIT, provider says MISS. You pay full price and your savings accounting believes you won. The defense is verify-then-account: after every real call, fold cache_read_input_tokens; if a believed-warm entry reads 0, mark the economic belief broken and diff the rendered prefix bytes to find the invalidator (Diverge / FirstDivergeTokenOffset localizes it — ProviderCache.FirstDivergeAt is the field they populate).

7. The warmth belief model — control without observability

On implicit providers you cannot ask “is X warm?”; you only learn warmth after spending a real request (cached_tokens). vCache therefore runs an open-loop estimator with feedback, not a known-state machine. It reuses cachemeta.Lifecycle but reinterprets the transitions as beliefs:

Calibration by probing. Provider TTL/min-prefix/discount are version-dependent and undocumented for some providers. vCache probes: send a known anchor, wait, re-send, read cached_tokens, and fit T, Mₘᵢₙ, and r per (provider, model, endpoint). The result feeds providerTTLMillis and the §5 constants instead of hard-coding them.

8. Recall — rebuild and reorder a series of requests

To recall unit K from a chain, vCache reconstructs the request from the manifest and replays the prefix chain in dependency order so the provider serves P0..K-1 from cache and freshly prefills only the last u tokens. The hazards (red-team §11) and the protocol:

Recall is therefore a topological-order scheduling problem over the DAG, gated by the §5.4 (pin vs rebuild) and §10.1 (break-even) economics, with the send-one-then-fan barrier between every fan level.

9. Cross-shard routing & multi-tenancy (the leakiest assumption)

Providers run many backends; the KV cache is per-server. A prefix warm on shard A is cold on shard B, and your next request may route to B. Mitigations and limits:

10. Reconciliation with the derivation panel

Five independent derivations (workflow vcache-design-foundations, Opus 4.8 × 10 agents, 850k tokens) cross-check §5. Result: §5.1, §5.3, §5.5 confirmed to the decimal; §5.2 sharpened; §5.4 reframed.

§5 law verdict correction / sharpening
5.1 break-even ✅ confirmed k > w/(1−r) → 5 min k≥2, 1 h k≥3 (= Anthropic’s own N>(w−r)/(1−r)). Dedicated warm k>1/(1−r). New: on auto-cache providers the first real request warms for free (no write line-item), so dedicated-warm break-even is k*=0 and savings are P·k·(1−r) from k=1 — never spend a decode-1 warm on OpenAI/DeepSeek; just order requests so the first real one warms.
5.2 coverage ✏️ sharpened “7→85%” implies s*=1.74 exactly (I estimated ~1.8), and it’s ≈V-independent over 10³–5×10³. At s=1, top-7-of-1000 = 34.6%; at s=0.8, 19.7%. To cover 90%: s=1.5 needs N=51 (0.5% of V=10⁴) but s=0.8 needs N=6381 (64% of V) — a 125× swing from skew alone. New corollary: when s≤1 the scheme is structurally defeated; the right move is not to warm the tail but to manufacture skew — canonicalize/aggregate so popularity concentrates onto fewer anchors. vCache can create the distribution it exploits.
5.3 chain horizon ✅ confirmed (labels corrected) C₀=⌈M/u⌉ = 103 / 205 / 410 units (u=10, M=1024/2048/4096). The arithmetic is exact; the model→M binding was wrong and is fixed above — the common Anthropic case is M=1024 → C₀=103 (Opus 4.8 / Sonnet 4.6 / 4.5), and 4096→410 is Haiku 4.5, not Opus. Tail ≤ B−1 = 127 tokens (~13 units) stranded at any instant; eligibility is a cliff at C₀ (0% → 99.4%) then →99.9%. Append-only is mandatory (any mid-chain byte change re-writes all downstream).
5.4 TTL 🔁 reframed My “pin iff λR > P/T” is the expensive-recall special case. Canonical result: on pure cache dollars, lazy rebuild weakly dominates for all λ (gap = W5·e^{−λT}·P → 0 only as λT→∞). Pinning is justified only by latency value L (cold-TTFT off the critical path) or rate-limit shadow price μ: pin iff λT > ln((W5+μ+L)/L) ≈ 0.81 → ~1 req/TTL (L=1, μ=0), rising to ~1.2–1.5 under rate pressure. And 1 h opt-in is 7.5× cheaper to hold warm than 5 min+heartbeat (2·P vs 15·P per hour) for W≥10 min prefixes reused ≥3×/h.
5.5 rate-limit ✅ confirmed N* = min(R−R_real, (X−X_real)/P)·T; crossover P* = (X−X_real)/(R−R_real) ≈ 100 tok on a 4000/400k tier → every realistic anchor is TPM-bound. P=4k → 500 anchors (5 min) / 6000 (1 h); P=64k → 31 / 375. 1 h TTL buys 12× the set for the same headroom. Below P*, you’re RPM-starved → pack tiny units behind one shared prefix.

Net: the goal’s instinct is right at the anchor granularity — a steep workload (s≈1.74) lets ~7 anchors cover ~85%, each anchor warmed once and read cheaply — but every number lives or dies on measuring s and the per-model minimum, and on billing at the uncached price until a hit is telemetry-confirmed (§11).

11. Failure modes & design rules (adversarial)

Five skeptics each tried to break one mechanism. The findings collapse into four cross-cutting laws plus one headline correction to the goal’s own recall idea.

11.0 The headline correction — recall-by-rebuild is the wrong tool for tiny-unit recall

The goal imagines recalling a unit by “rebuilding and reordering a series of requests.” The economics say: for a single ~10-token unit pulled from a long cold/warm chain, this almost always loses.

Replaying a 30 k-token prefix to rebuild one 10-token unit costs 0.1×·30 000 = 3000 token-equivalents of read to avoid 1.0×·10 = 10 token-equivalents of fresh prefill — a 300× loss. Recall-by-rebuild is net-negative whenever prefix_tokens·r > unit_tokens, i.e. essentially always for the 10-token target.

So the cost gate refuses almost every single-unit chain rebuild — and that is correct. Recall-by-rebuild only wins in the opposite regime: a large warm prefix amortized across many sibling units recalled together while the entry is still hot — i.e. the star/anchor + fan-out shape, not the deep-chain + single-unit-recall shape. The reorder/rebuild machinery is real but gated off by default and reserved for amortized fan-out. The 80% win is §13-M2 (anchors), not chains.

11.1 Law A — Warmth is a belief, never a boolean; correctness never depends on it

The lethal failure is not a cache miss (cheap, visible) — it’s “manifest says HIT, provider says MISS.” A datetime.now() in the prefix makes every recall a cache write; the manifest still reports the unit warm, so you book 90% savings while paying 1250%.

11.2 Law B — Fingerprint the wire bytes, scoped by the full identity tuple

The whole scheme rests on byte-identical prefixes, and the manifest must key on what’s actually sent, not the logical object.

11.3 Law C — The warming primitive has an irreducible race and an observability gap

11.4 Law D — On implicit providers you run open-loop; calibrate, don’t assume; and routing is a roulette

11.5 Residual risk (honest floor)

Even with every rule: on auto-cache providers vCache cannot achieve closed-loop control — the only state channel is one rounded integer (cached_tokens) reported after a paid request, against a distributed, multi-tenant, nondeterministic plant whose policy the provider changes silently. So a non-zero, time-varying false-warm miss rate is structural, indistinguishable from shard-miss / eviction / below-min without per-call reconciliation; calibration is least accurate exactly under high load (when caching matters most); and invisible regret from false-cold beliefs (savings offered but never claimed) is unmeasurable by construction. The cross-tenant timing channel is a provider property vCache can refuse to amplify but cannot close. Design posture: be correct and cheaper-in-expectation, never dependent on a hit. Keep the cold path always correct and always affordable; the moment warmth gates correctness or hard-commits an SLO, one eviction turns a probabilistic hint into a confident lie.

12. What’s new vs what exists (the build boundary)

Reuse unchanged: KVManifest/Entry identity; prefix_stability.{Diverge,LintPrefixLayout,RecommendLayout,AnalyzeStability}; ProviderCache/FromProviderCache/ProviderCacheVerdict; lifecycle.{Lifecycle,TierTTL,Advance,Touch}; SavingsSplit double-count guard; AccessPolicy access control.

Net-new (the vCache plane, tier ≥2):

  1. Manifest with a prefix DAG — the recall plan (parent chain per vBlock), not just identity.
  2. Warmer — the max_tokens:0 / decode-1 request issuer + per-provider capability adapter (explicit vs implicit).
  3. Warmth belief estimator — Lifecycle at TierProvider driven by cached_tokens feedback + TTL decay + probing calibration.
  4. Scheduler — rate-limit warm-budgeting, topological recall ordering, send-one-then-fan barrier.
  5. Governor — pin/lazy/evict policy from §5.4 + §5.5.
  6. Canonicalizer as a gate — enforce RecommendLayout, not just report it.

Generalizes the single-session coherence shaper: today’s cachemeta.ShapeGLMTurnSegmentWitnessed + glm_coherence.go (SegmentsFromMessages / ApplyBreakToMessages / GLMCoherenceShaper) is the single-session, intra-request prefix-break placer — it shapes one turn’s messages so a stale span breaks the provider cache exactly where it must. vCache is the multi-request, multi-provider, TTL-governed superset; the existing SegmentKind classifier (SegStable / SegToolSchema / SegVolatile / SegToolResult / SegMessage / SegSealed) and the cache-stability gate become vCache’s per-anchor canonical form.

Prior art in the tree & related issues

vCache is the active control plane over a passive substrate that mostly already exists. It does not duplicate these — it consumes them:

13. Phased plan

  1. M1 — Observe & calibrate. Probe each provider for T, Mₘᵢₙ, r; build the warmth-belief estimator on top of FromProviderCache. No warming yet — just predict hits and measure prediction error.
  2. M2 — Star anchors. Canonicalizer-gated anchors + first-natural-request warming + sibling fan-out. The 80% win, lowest risk. Enforce RecommendLayout.
  3. M3 — Dedicated warming. max_tokens:0 (explicit) / decode-1 (implicit) under the §10.1 break-even gate; send-one-then-fan.
  4. M4 — Chains & recall (gated OFF by default — see §11.0). Prefix DAG, topological replay, 20-block intermediate breakpoints, cost-gated rebuild. The cost gate refuses single-unit chain rebuilds (≈always net-negative for 10-token units); enable rebuild only for a large warm prefix amortized across many siblings recalled together while hot. For most of the imagined “rebuild/reorder” workload the correct action is fresh prefill, not rebuild.
  5. M5 — Governor. Heartbeat-pin vs lazy-rebuild vs evict; rate-limit warm-budget scheduler; affinity-key routing.

14. Open questions