Which AI models can my GPU run?

Pick your graphics card, system RAM and context length. See which open models run fully on the GPU, which run slowly by spilling into RAM, and which won't run.

Runs on GPURuns slowly (uses RAM)Won't run
ModelQ4Q5Q8FP16

Numbers are estimates: weights + KV cache at your context length + about 0.6 GB for the runtime, for llama.cpp/Ollama-style GGUF models. Real use varies a little by app and settings. Q4 is the usual sweet spot for home GPUs.

Questions people ask

What do Q4, Q5, Q8 and FP16 mean?

They are quantisation levels: how many bits store each of the model's weights. FP16 is the full-size original. Q8 is about half that size with almost no quality loss. Q5 and Q4 shrink it further. Q4 (Q4_K_M) is the usual choice for home GPUs: about a quarter of the FP16 size and still close in quality for chat and coding.

What does "Runs slowly (uses RAM)" mean?

The model is bigger than your graphics card's memory (VRAM), so Ollama or llama.cpp keeps part of it in normal system RAM and runs that part on the CPU. It works, but replies come much slower, often 3 to 10 times slower than a model that fits fully on the GPU.

How accurate are these numbers?

They are estimates: model weights, plus the KV cache for your context length, plus about 0.6 GB for the runtime. We checked them against real runs on our own RTX 3060 12 GB: for Qwen3.5 9B the estimate was within 1% of the measured memory. Models we have run ourselves are marked "Tested", with the real memory use and speed. Real use still changes a little with the app, its version and settings.

Which context length should I pick?

Context is how much text the model can see at once: your chat so far plus any document you paste. 4K tokens (roughly 3,000 words) is Ollama's usual default and fine for chat. Pick 16K or 32K only if you paste long documents or code, because a longer context needs more memory.

Does this work for laptop GPUs and AMD or Intel cards?

Yes. The memory needed is the same on any GPU, so the fit result holds. Speed differs: NVIDIA cards have the widest support, AMD works well through ROCm or Vulkan on most recent Radeon cards, and Intel Arc works through Vulkan or SYCL builds. Laptop GPUs have less VRAM than the desktop card with the same name, so check the GB in brackets.

How do I actually run a model?

Install Ollama from ollama.com, open a terminal and paste the command shown in the result box above, for example "ollama run qwen3.5:9b". It downloads the model once (usually at Q4) and then you can chat with it offline.

Spotted a wrong number? Tell us and we'll fix it.