fak Server Troubleshooting

Common startup failures, port conflicts, and resource issues when running fak serve or the in-kernel model engine.

For operators running fak serve who hit a startup or runtime error: match your error message to a symptom below, then run the diagnosis command and apply a fix. Assumes you already have fak installed and a model (or --base-url) to point it at — if not, start with the server quickstart.

Port Conflicts
Memory Issues
GPU/CUDA Issues
Model Loading Failures
Policy and Configuration Issues
Startup Failures
Debugging Tools

Port Conflicts

Symptom: “bind: Only one usage of each socket address”

Example error:

listen tcp 127.0.0.1:8080: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

Diagnosis:

# Check what's using the port (Windows)
netstat -ano | findstr :8080

# Check what's using the port (Linux/macOS)
lsof -i :8080

Solutions:

Kill the conflicting process:
- Windows: taskkill /PID <pid> /F
- Linux/macOS: kill -9 <pid>
Use a different port:
```
fak serve --addr 127.0.0.1:8081
```
Check for multiple fak instances:
```
Get-Process fak
```

Memory Issues

Symptom: Out of memory during model load

Example errors:

cannot allocate memory
alloc.*failed
Process termination with OOM

Common causes:

Model too large for available RAM:
- Qwen3.6-27B requires ~26 GB RSS with KV cache
- SmolLM2-135M requires ~500 MB
- Qwen2.5-0.5B requires ~2 GB
Context window too large:
- Larger context windows require more KV cache memory
- Reduce context length or use a smaller model

Solutions:

Check available memory:

# Windows
Get-ComputerInfo | select CsTotalPhysicalMemory, CsFreePhysicalMemory

# Linux
free -h

Use a smaller model:

# Instead of 27B
fak serve --gguf models/qwen2.5-0.5b-q8.gguf --tokenizer ~/.cache/fak-models/tokenizers/qwen2.5

# Or SmolLM2-135M
fak serve --gguf internal/model/.cache/smollm2-135m

Reduce concurrent sessions:
- Each session maintains its own KV cache
- Process concurrent requests sequentially or use fewer agents

Check for memory leaks:

# Monitor memory usage
watch -n 1 'ps aux | grep fak'

Symptom: WSLL / FSL OOM during tests

Issue: Model tests may intermittently OOM on the 538MB weights.f32 test data.

Solution: Run weight-backed tests in isolation:

.\fak\test.ps1 ./internal/model -run TestWeight

GPU/CUDA Issues

Symptom: CUDA initialization failures

Example errors:

compute: cuda device allocation failed
cudaGetLastError() returned non-zero
CUDA driver/library not found

Diagnosis:

Check NVIDIA GPU availability:
```
nvidia-smi
```
Check CUDA toolkit:
```
nvcc --version
```

Verify WSL2 GPU passthrough (Windows):

# In WSL
ls /usr/lib/wsl/lib/libcuda.so

Solutions:

Install CUDA toolkit (no sudo required):

# From fak/
bash internal/compute/setup_cuda_wsl.sh

Build with CUDA support:
```
bash internal/compute/build_cuda.sh
```

Use CPU backend instead:

fak serve --engine inkernel
# Or explicitly
fak serve --engine cpu-ref

Symptom: Vulkan device allocation failed

Example error:

fak-vulkan: device-local alloc(X bytes) failed VkResult=...

Diagnosis:

# Check Vulkan support
vulkaninfo

Solutions:

Check GPU driver is up to date
Verify Vulkan runtime is installed
Try CPU backend: fak serve --engine cpu-ref

Model Loading Failures

Symptom: GGUF file not found or invalid

Example errors:

open models/qwen.gguf: no such file or directory
invalid GGUF magic
unsupported GGUF version

Diagnosis:

# Verify file exists and is readable
ls -lh models/qwen.gguf
file models/qwen.gguf

Solutions:

Download model using provided script:

# From repo root
python fak/scripts/fetch_model.ps1

Use correct model path:

# Relative to current directory
fak serve --gguf ./models/qwen.gguf

# Absolute path
fak serve --gguf /full/path/to/model.gguf

Verify GGUF format:
- Use llama.cpp tools to inspect/convert
- Ensure model architecture is supported (Llama, Qwen, etc.)

Symptom: GGUF embeds no usable BPE tokenizer (rare; SPM-only checkpoints)

fak serve --gguf X (no --base-url) serves real in-kernel chat using the tokenizer embedded in the GGUF — no separate --tokenizer is needed for the common case (Qwen, Gemma, Phi, and other byte-level BPE models). Only a checkpoint that embeds no usable BPE tokenizer (e.g. an SPM-only model) falls back to the offline mock planner, with this stderr note:

fak serve: --gguf set without --tokenizer and no embedded BPE tokenizer (...);
  /v1/chat/completions will use the offline mock planner. Pass --tokenizer <dir|file> for real chat.

Solution: point --tokenizer at a tokenizer.json (or its directory) for that model:

fak serve --gguf models/qwen.gguf --tokenizer ~/.cache/fak-models/tokenizers/qwen3.6

Symptom: FAK_Q4K model load fails

Example error:

q4k-direct-load failed

Diagnosis:

FAK_Q4K path is for direct Q4_K matmul tensors
Requires compatible model (Qwen3.6-27B q4_k_m)

Solutions:

Verify model compatibility:
```
# Check if model is Qwen3.6-27B q4_k_m
```

Use default Q8 path:

unset FAK_Q4K
fak serve --gguf models/qwen.gguf

Policy and Configuration Issues

Symptom: Policy validation failure

Example error:

fak policy: <policy-file>: validation error

Diagnosis:

# Validate policy before using
fak policy --check policy.json

Solutions:

Dump default policy for reference:

fak policy --dump > default-policy.json

Check policy syntax:
- Verify JSON is valid
- Check tool names match registered tools
- Ensure reason classes are from closed vocabulary
Use built-in policy:
```
fak serve  # Uses DefaultPolicy
```

Symptom: API key not configured

Example error:

fak serve: env OPENAI_API_KEY is empty

Solutions:

Set API key:

$env:OPENAI_API_KEY="sk-..."
fak serve --base-url https://api.openai.com/v1 --api-key-env OPENAI_API_KEY

Use offline mode (no API key):

fak serve  # Uses mock planner with no --base-url

Startup Failures

Symptom: Gateway fails to start

Example error:

fak serve: gateway.New: ...

Common causes:

Invalid engine ID:

# Check available engines
fak run --trace testdata/tau2/smoke.json --engine invalid

Invalid invalidation granularity:

# Must be: global | namespace | resource
fak serve --invalidation global  # correct
fak serve --invalidation invalid  # fails

Engine cache misconfiguration:

# --engine-cache-base-url required when --engine-cache-engine is set
fak serve --engine-cache-engine sglang --engine-cache-base-url http://localhost:10000

Symptom: Model load hangs or takes very long

Diagnosis:

Check model size and I/O speed:

# Large models (27B) can take 30+ seconds to load

Monitor progress:
- Metrics endpoint shows load phases: GET /metrics
- Look for fak_model_load_phase_duration_seconds

Solutions:

Use smaller model for testing:

fak serve --gguf internal/model/.cache/smollm2-135m

Pre-load weights:
- Gateway eager-loads by default
- First request is fast

Debugging Tools

Health check endpoint

curl http://localhost:8080/healthz

Returns HTTP 200 when gateway is ready.

Metrics endpoint

curl http://localhost:8080/metrics

Key metrics for troubleshooting:

fak_gateway_time_to_ready_seconds - Total startup time
fak_gateway_startup_phase_duration_seconds - Per-phase boot cost
fak_model_load_duration_seconds - Model load time
fak_model_load_bytes - Bytes loaded

Verbose logging

# Enable debug logging
FAK_LOG=debug fak serve

Test kernel in isolation

# Test adjudication without model
fak run --trace testdata/tau2/smoke.json

# Test with mock planner
fak serve  # No --base-url = offline mode

Check registered engines

# View available engines
fak run --trace testdata/tau2/smoke.json --engine ?

Quick Reference: Common Commands

# Minimal server (no model, offline mode)
fak serve

# With local GGUF model
fak serve --gguf models/qwen.gguf --tokenizer ~/.cache/fak-models/tokenizers/qwen

# Proxy to external model
fak serve --base-url https://api.openai.com/v1 --api-key-env OPENAI_API_KEY

# With custom policy
fak serve --policy policy.json

# Check policy before using
fak policy --check policy.json

# Verify model load
fak serve --gguf models/qwen.gguf --policy-check

Additional Resources

Getting Started - Install and basic usage
GPU Support - CUDA and Vulkan setup
README - Project overview
Architecture - System design

Next: once the server is up, Observability explains the /metrics and log surfaces this guide leans on for diagnosis.

Still stuck?

Check the logs: fak serve writes to stderr by default
Verify prerequisites: Go 1.26+, sufficient RAM, compatible model
Try minimal config first: fak serve (no model, offline)
Check GitHub issues: https://github.com/anthony-chaudhary/fak/issues

fak server troubleshooting: ports, memory, GPU

Diagnose and fix common fak serve failures, covering port conflicts, out-of-memory loads, GPU and CUDA errors, model loading, and policy issues.

fak Server Troubleshooting

Table of Contents

Port Conflicts

Symptom: “bind: Only one usage of each socket address”

Memory Issues

Symptom: Out of memory during model load

Symptom: WSLL / FSL OOM during tests

GPU/CUDA Issues

Symptom: CUDA initialization failures

Symptom: Vulkan device allocation failed

Model Loading Failures

Symptom: GGUF file not found or invalid

Symptom: GGUF embeds no usable BPE tokenizer (rare; SPM-only checkpoints)

Symptom: FAK_Q4K model load fails

Policy and Configuration Issues

Symptom: Policy validation failure

Symptom: API key not configured

Startup Failures

Symptom: Gateway fails to start

Symptom: Model load hangs or takes very long

Debugging Tools

Health check endpoint

Metrics endpoint

Verbose logging

Test kernel in isolation

Check registered engines

Quick Reference: Common Commands

Additional Resources

Still stuck?