Choose a model-execution backend

Audience: operators deciding where model inference runs behind fak. fak remains the policy, routing, cache, and audit layer in every choice; the backend supplies model tokens.

Current default: front an existing OpenAI-compatible model server, local or remote. That is the maintained operating path for production model serving and lets fak stay independent of the server and its silicon. Next action: choose an engine from the serving-engine table and use that row’s wiring route.

Choose by operating envelope

Choose	When it fits	Current support boundary	Continue with
Remote or separately managed server	You already use a hosted endpoint, vLLM, SGLang, Ollama, llm-d, llama.cpp, LM Studio, or another OpenAI-compatible server. This is the default operator mode.	The provider or engine owns inference and accelerator support. fak fronts its public API; support is scoped to the documented wire and configuration, not the server’s internals.	Serving engines, or clouds and hosted providers for a provider-native wire.
Local CPU reference	You need deterministic correctness work, offline development, a portable baseline, or a small-model proof without accelerator requirements.	cpu-ref is the shipped correctness floor for fak’s in-kernel engine. It is a reference path, not the production-throughput default.	In-kernel reference engine.
In-kernel GPU backend	You are evaluating fak’s forward pass directly on supported CUDA or Vulkan hardware and can build and run on the matching device.	CUDA and Vulkan are build-tagged Approx backends with hardware-specific witnesses. A device-family result does not establish support for every model, shape, driver, or host.	Hardware portability and build tags, then require a non-reference run in the model benchmark.
External GPU serving engine	You want production GPU throughput while keeping inference in a dedicated serving stack.	GPU lifecycle, kernels, capacity, and model coverage belong to the selected engine; fak’s support claim remains the integration wire.	Select vLLM, SGLang, llm-d, or another compatible server in serving engines.

“Remote” describes where the model server runs; “CPU” and “GPU” describe where its inference executes. They are not mutually exclusive: a remote server can use either CPU or GPU. When an operator only needs to choose a fak integration, choose the server/wire first. Choose an in-kernel silicon backend only when the in-kernel engine itself is the subject of the run.

Mode and generation

This page describes the current (gen/now) backend-selection route. It covers two execution modes:

1. Gateway mode: fak fronts a local or remote server over a documented provider wire. This is the operator default.
2. In-kernel backend-evaluation mode: the model benchmark invokes the forward pass through internal/compute.Backend; cpu-ref is the reference implementation, while CUDA and Vulkan are opt-in, build-tagged registrations. The linked hardware-portability page gives the build and -require-non-reference commands. A generic in-kernel run does not select one of those accelerator registrations: the optimized legacy prefill/batch implementation remains that runtime’s default.

Use gateway mode for normal operation. Choose in-kernel backend-evaluation mode only when the HAL path and its parity witness are the subject of the run.

Support and lifecycle labels

Backend support is evidence-scoped rather than inferred from a device or vendor name:

• Shipped wire means fak currently implements the named API integration.
• Reference means the in-kernel backend is the exact correctness floor.
• Approx means the backend passed the hardware page’s named non-bit-exact parity checks (for example, argmax agreement and logit cosine); it does not establish exact logits or coverage beyond the witnessed model, shape, build, and device.
• Partial means the linked support row lists working and missing methods, shapes, or capabilities. not-yet means no current witness supports the path.

The maintained support row and its linked witness are the status authority. Add a backend to this route only after its wire or compute registration and matching witness ship. Relabel it Partial or not-yet when that witness no longer covers current code; remove it from this current route when the maintained support row names a replacement generation and links the superseding route. For the detailed accelerator fields and conformance vocabulary, see supported silicon backends. For machine-specific measurements, use the hardware matrix; those measurements are evidence for the listed machine and build, not a blanket support promise.

Decision recap

• For normal operation, front a maintained local or remote model server.
• For portable correctness and offline proofs, use cpu-ref.
• For direct in-kernel accelerator evaluation, build the matching CUDA or Vulkan backend and require a non-reference witness.
• For production GPU serving, choose a dedicated GPU engine and connect fak over its supported wire.