Does GPU Unlock Speed Up Ollama and llama.cpp?
Why quantized GGUF models barely change after gpu-unlock, where local LLMs do get faster, and which setup sees the 25–55% prefill gain.
Mostly not. Reported gains for llama.cpp and Ollama with quantized GGUF models are 0–15%, mostly at long context. The big numbers come from FP16/BF16 workloads.
Why quantized models barely move
The unlock doubles one specific path: FP16, BF16 and TF32 tensor math that accumulates into FP32. Two things keep most Ollama setups off that path:
- Token generation is memory bound. Each new token reads the whole model from VRAM. The tensor cores wait on memory, so making them faster changes nothing. The upstream table lists LLM decode as "no change".
- Quantized kernels use other instructions. INT8 and INT4 paths are not capped and do not change. One owner test of a quantized mixture-of-experts model on an RTX 4090 measured 0.983× at 8K prefill.
Where local LLMs do get faster
| Workload | Reported speedup | | ---------------------------------------------------------------- | ---------------- | | Prefill with FP16/BF16 or AWQ/GPTQ (vLLM, PyTorch, Transformers) | +25–55% | | Time to first token | 25–30% lower | | FlashAttention / SDPA kernels | +50–60% | | LoRA and full fine-tuning in BF16 | +25–50% | | llama.cpp / Ollama, quantized GGUF | 0–15% |
So the people who notice are those processing long prompts (RAG, coding agents, document summarizing) in vLLM or PyTorch, and anyone fine-tuning in BF16.
Should you bother?
If you only chat with a quantized model in Ollama, probably not. If you run long-context prefill, batch inference, fine-tuning or image generation on an RTX 30 card, it is one of the larger free speedups available. Check your card first, then follow the install steps.