The SOTA prior-art matrix

Read this before you optimize a kernel. Almost every contraction fak performs — a quantized GEMM, a fused attention, a KV-cache reuse, a MoE dispatch — has a production reference that already solved it: llama.cpp’s GGML kernels, Marlin, CUTLASS, FlashInfer, vLLM PagedAttention, SGLang RadixAttention, a named paper. The default answer to “should I write this from scratch?” is no — study the reference, decide borrow / bind / stay-minimal, and hold the result to an oracle. Re-deriving known art badly is the failure mode this matrix exists to stop.

This is the inward engineering counterpart of docs/industry-scorecard/: the scorecard positions fak against the field (an outward competitive map); this matrix maps each operation fak’s kernel actually performs to the production stack worth learning from, the route to relate to it, and the oracle that proves a fak version correct.

Why this is load-bearing, not a note

Prior art was always being researched here — tools/idea_scout.py files adjacent work as issues, the docs/notes/RESEARCH-* corpus is deep, and this matrix began as a single dated note (RESEARCH-backend-sota-matrix-2026-06-26.md). The gap was that the research was inert: nothing on the kernel-commit path forced an agent to consult it, so an agent could (and did) reach for “implement the Mac Q6_K fused MLP from scratch” without first reading the Metal kernel llama.cpp already ships.

The fix is to make the prior-art map a maintained datum that three surfaces read:

Surface	What it does	Source
`fak sota <op\\|file>`	Agent-facing lookup. Run it before writing a kernel: prints the SOTA reference, route, oracle, and the link to read.	`internal/sotamatrix`
`PRIOR_ART` pre-commit gate	When a commit touches a kernel file, prints the matching op’s SOTA reference and suggests a `Prior-art:` trailer. Advisory — never blocks.	`internal/hooks/gate_priorart.go`
`tools/sota_coverage_scorecard.py`	Cross-checks the matrix against the tree: every kernel file is covered by a row, every row’s fak-path file exists, every row carries a primary link + oracle. Folds gaps into one `sota_debt` integer.	`tools/sota_coverage_scorecard.py`

The single source of truth is the flat literal in internal/sotamatrix/sotamatrix.go — the same in-binary-registry discipline as internal/benchcatalog. The gate, the command, and the scorecard all read it; none keeps a rival copy. Adding a kernel operation means adding one row there.

The process (what to actually do)

Before you write or optimize a kernel, run fak sota <operation> (or fak sota <the-file-you're-about-to-edit>). Read the PrimaryLink. Decide your route.
Route honestly. stay-minimal (the bit-exact contract is fak’s value, not raw speed), bind (use the production library/format directly), or borrow (adapt the reference technique) — and borrow a kernel only after a witness for the current path exists, so the choice is evidence-based rather than a premature bet.
Prove it against the oracle the row names (almost always cpuref f32 with a cosine floor, or bit-identity). A kernel with no oracle is not done.
Stamp the commit with a Prior-art: trailer naming what you consulted (e.g. Prior-art: Marlin fused dequant-MMA (IST-DASLab/marlin); cosine ≥ 0.995 vs HF AWQ). This silences the advisory gate and leaves a durable record of the reference for the next person.

The matrix

The rows below are rendered from internal/sotamatrix; run fak sota list for the live table and fak sota <slug> for one operation’s full detail (route note, oracle, papers).

Operation	fak path	SOTA to learn	Route	Oracle
Dense f32 GEMM	`internal/compute/cpuref.go` + `model/parallel.go`	cuBLAS / CUTLASS	stay-minimal	`cpuref` bit-identity
Dense fp16/Q8/Q4_K device GEMM	`internal/compute/cuda.go` + `cuda_kernels.cu`	cuBLAS; llama.cpp	bind	`cpuref`, argmax + cosine ≥ floor
AWQ 4-bit GEMV/GEMM	`internal/compute/cuda.go:1101` + `cuda_kernels.cu`	Marlin; AutoAWQ	borrow	`cpuref` + HF AWQ (cosine ≥ 0.995)
GPTQ resident (CPU)	`internal/model/gptq.go`	AutoGPTQ / Marlin	bind	HF GPTQ dequant / `cpuref`
EXL2 loader	`internal/model/exl2.go`	ExLlamaV2	stay-minimal	ExLlamaV2 reference
GGUF quant-at-load (Q4/5/6_K)	`internal/ggufload/`, `internal/model/quant_q4k.go`	llama.cpp GGUF	bind	llama.cpp 2-token parity
CPU K-quant SIMD dequant-GEMM	`internal/model/quant_kquant.go`, `quant_amd64_kquant.go`, `quant_arm64_q6k.go`	llama.cpp ggml-quants	borrow	`cpuref` dequant (max\|Δ\|=0)
MoE expert dispatch	`internal/model/moe.go`, `glm_dsa.go`	DeepEP / TensorRT-LLM	borrow	dense reference / HF
Collective communication (all-reduce / process-group)	`internal/compute/cuda_nccl.cu`, `cuda_nccl_pg.cu`, `collective.go`	NVIDIA NCCL; NVSHMEM; MSCCL++	bind	host reduce — argmax + cosine (Approx)
Fused attention (MHA/GQA/MQA)	`internal/compute/cuda_kernels.cu` `k_flash_attention`	FlashInfer / FlashAttention	stay-minimal	`cpuref` (cosine ≥ 0.999)
GLM sparse attention (DSA)	`internal/compute/dsa.go`	TensorRT-LLM custom sparse	stay-minimal	`cpuref` (cosine ≥ 0.999)
KV cache (paging / prefix reuse)	`internal/model/kv.go`, `internal/radixkv`	vLLM PagedAttention; SGLang RadixAttention	stay-minimal	bit-identity
Metal Q4_K / Q6_K GEMM	`internal/metalgemm/`, `internal/model/metal_q4k*.go`	llama.cpp Metal / MLX	borrow	`cpuref` (GEMV cosine 1.000000)

The milestone ladders — where fak is on each capability’s SOTA arc

The matrix answers a tactical question: “before writing THIS kernel, what reference already solved the contraction?” It cannot answer the strategic one an agent kept answering from memory: “for a whole capability — attention, batching, quantization — what are the obvious SOTA baseline milestones, in order, and which rung is fak actually on?” That is what the milestone ladders add (internal/sotamatrix/ladder.go, rendered by fak sota milestones [axis]).

Each rung is external prior art — a named, dated, citable technique that moved the baseline, never a fak self-claim. The only fak-facing field is FakRung: the highest rung fak has reached, pinned conservatively to what the tree-verified matrix Op already witnesses (or the serving position, or -1 when fak does not implement the axis). So the ladder tells you both the landscape and the next milestone to target.

Axis	fak rung today	Next milestone
attention	FlashAttention-1 (own fused online-softmax, cosine ≥ 0.999)	FlashAttention-2 (work partitioning) → FA-3 FP8 → FlashInfer paging
batching	Dynamic, padding-aware batching (`internal/gateway/batchsched.go`)	Continuous / in-flight batching (Orca) — the batcher’s own deferred rung
quantization	INT4 PTQ (AWQ / GPTQ / gguf k-quants)	Fused INT4 Marlin kernel — the matrix’s recorded gap
kv-cache	PagedAttention + RadixAttention prefix reuse on an exact f32 cache	KV quantization / host offload
speculative-decoding	not implemented (exact-decode runner)	Draft-model speculative decoding (external landscape)

$ fak sota milestones attention
  => L1  FlashAttention-1 (2022)   IO-aware tiling + online softmax; no N×N materialization
  -> L2  FlashAttention-2 (2023)   better warp partitioning — ~2× FA-1
     L3  FlashAttention-3 (2024)   Hopper warp-specialization + FP8
     L4  FlashInfer (2024)         paged / variable-length + block-sparse

Run fak sota milestones for the overview across all axes, fak sota milestones <axis> for one ladder rung-by-rung, and fak sota <slug> on a mapped op (e.g. fused-attention) prints a pointer to its ladder. Adding a rung means adding a real, citable milestone — do not invent a rung to flatter fak’s position; the OpSlug cross-reference is unit-tested against the matrix so a ladder cannot claim a fak rung the tree does not back.

The rule the matrix encodes

Borrow a kernel only after the witness for the current path exists. The reference is inspiration and a correctness target — read it first, route deliberately, prove against the oracle, and record what you read. Throughput is a real program (the kernel-optimization program is never “done”), but it is pursued on top of the known art, not in ignorance of it.