Quick Answer: RTX 4080 offers 16GB VRAM and starts around $1496.19. It delivers approximately 162 tokens/sec on ibm-granite/granite-3.3-2b-instruct. It typically draws 320W under load.
RTX 4080 balances throughput and efficiency. It crushes 8B–13B models, handles most 70B work with clever quantization, and stays manageable in terms of power and thermals.
Buy directly on Amazon with fast shipping and reliable customer service.
💡 Not ready to buy? Try cloud GPUs first
Test RTX 4080 performance in the cloud before investing in hardware. Pay by the hour with no commitment.
| Model | Quantization | Tokens/sec | VRAM used |
|---|---|---|---|
| ibm-granite/granite-3.3-2b-instruct | Q4 | 161.91 tok/sEstimated Auto-generated benchmark | 1GB |
| deepseek-ai/deepseek-coder-1.3b-instruct | Q4 | 161.62 tok/sEstimated Auto-generated benchmark | 2GB |
| meta-llama/Llama-3.2-1B-Instruct | Q4 | 160.52 tok/sEstimated Auto-generated benchmark | 1GB |
| Qwen/Qwen2.5-3B-Instruct | Q4 | 160.21 tok/sEstimated Auto-generated benchmark | 2GB |
| nari-labs/Dia2-2B | Q4 | 159.83 tok/sEstimated Auto-generated benchmark | 2GB |
| deepseek-ai/DeepSeek-OCR | Q4 | 158.59 tok/sEstimated Auto-generated benchmark | 2GB |
| google/gemma-2b | Q4 | 157.92 tok/sEstimated Auto-generated benchmark | 1GB |
| apple/OpenELM-1_1B-Instruct | Q4 | 157.43 tok/sEstimated Auto-generated benchmark | 1GB |
| TinyLlama/TinyLlama-1.1B-Chat-v1.0 | Q4 | 154.93 tok/sEstimated Auto-generated benchmark | 1GB |
| tencent/HunyuanOCR | Q4 | 153.42 tok/sEstimated Auto-generated benchmark | 1GB |
| unsloth/Llama-3.2-1B-Instruct | Q4 | 153.14 tok/sEstimated Auto-generated benchmark | 1GB |
| google/gemma-2-2b-it | Q4 | 152.89 tok/sEstimated Auto-generated benchmark | 1GB |
| bigcode/starcoder2-3b | Q4 | 152.86 tok/sEstimated Auto-generated benchmark | 2GB |
| meta-llama/Llama-Guard-3-1B | Q4 | 152.48 tok/sEstimated Auto-generated benchmark | 1GB |
| LiquidAI/LFM2-1.2B | Q4 | 149.85 tok/sEstimated Auto-generated benchmark | 1GB |
| Qwen/Qwen2.5-3B | Q4 | 148.23 tok/sEstimated Auto-generated benchmark | 2GB |
| google/gemma-3-1b-it | Q4 | 148.12 tok/sEstimated Auto-generated benchmark | 1GB |
| google-bert/bert-base-uncased | Q4 | 145.00 tok/sEstimated Auto-generated benchmark | 1GB |
| google-t5/t5-3b | Q4 | 145.00 tok/sEstimated Auto-generated benchmark | 2GB |
| ibm-research/PowerMoE-3b | Q4 | 144.00 tok/sEstimated Auto-generated benchmark | 2GB |
| google/embeddinggemma-300m | Q4 | 142.71 tok/sEstimated Auto-generated benchmark | 1GB |
| meta-llama/Llama-3.2-3B | Q4 | 142.19 tok/sEstimated Auto-generated benchmark | 2GB |
| WeiboAI/VibeThinker-1.5B | Q4 | 142.16 tok/sEstimated Auto-generated benchmark | 1GB |
| meta-llama/Llama-3.2-3B-Instruct | Q4 | 141.28 tok/sEstimated Auto-generated benchmark | 2GB |
| meta-llama/Llama-3.2-1B | Q4 | 141.20 tok/sEstimated Auto-generated benchmark | 1GB |
| allenai/OLMo-2-0425-1B | Q4 | 140.71 tok/sEstimated Auto-generated benchmark | 1GB |
| inference-net/Schematron-3B | Q4 | 139.36 tok/sEstimated Auto-generated benchmark | 2GB |
| context-labs/meta-llama-Llama-3.2-3B-Instruct-FP16 | Q4 | 136.30 tok/sEstimated Auto-generated benchmark | 2GB |
| dicta-il/dictalm2.0-instruct | Q4 | 135.00 tok/sEstimated Auto-generated benchmark | 4GB |
| deepseek-ai/DeepSeek-R1 | Q4 | 134.84 tok/sEstimated Auto-generated benchmark | 4GB |
| unsloth/gemma-3-1b-it | Q4 | 134.70 tok/sEstimated Auto-generated benchmark | 1GB |
| Qwen/Qwen3-4B-Instruct-2507 | Q4 | 134.65 tok/sEstimated Auto-generated benchmark | 2GB |
| HuggingFaceTB/SmolLM-135M | Q4 | 134.54 tok/sEstimated Auto-generated benchmark | 4GB |
| facebook/sam3 | Q4 | 134.53 tok/sEstimated Auto-generated benchmark | 1GB |
| Qwen/Qwen2.5-Coder-7B-Instruct | Q4 | 134.12 tok/sEstimated Auto-generated benchmark | 4GB |
| openai-community/gpt2-medium | Q4 | 133.69 tok/sEstimated Auto-generated benchmark | 4GB |
| skt/kogpt2-base-v2 | Q4 | 133.30 tok/sEstimated Auto-generated benchmark | 4GB |
| deepseek-ai/DeepSeek-V3 | Q4 | 132.96 tok/sEstimated Auto-generated benchmark | 4GB |
| deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct | Q4 | 132.72 tok/sEstimated Auto-generated benchmark | 4GB |
| unsloth/Llama-3.2-3B-Instruct | Q4 | 132.64 tok/sEstimated Auto-generated benchmark | 2GB |
| Gensyn/Qwen2.5-0.5B-Instruct | Q4 | 132.49 tok/sEstimated Auto-generated benchmark | 3GB |
| Qwen/Qwen2.5-1.5B | Q4 | 132.49 tok/sEstimated Auto-generated benchmark | 3GB |
| microsoft/DialoGPT-medium | Q4 | 132.18 tok/sEstimated Auto-generated benchmark | 4GB |
| numind/NuExtract-1.5 | Q4 | 131.85 tok/sEstimated Auto-generated benchmark | 4GB |
| kaitchup/Phi-3-mini-4k-instruct-gptq-4bit | Q4 | 131.81 tok/sEstimated Auto-generated benchmark | 2GB |
| Qwen/Qwen2-0.5B | Q4 | 131.72 tok/sEstimated Auto-generated benchmark | 3GB |
| lmstudio-community/DeepSeek-R1-0528-Qwen3-8B-MLX-4bit | Q4 | 131.45 tok/sEstimated Auto-generated benchmark | 4GB |
| allenai/Olmo-3-7B-Think | Q4 | 131.45 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/Phi-4-multimodal-instruct | Q4 | 131.42 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/Phi-3.5-vision-instruct | Q4 | 130.87 tok/sEstimated Auto-generated benchmark | 4GB |
Note: Performance estimates are calculated. Real results may vary. Methodology · Submit real data
| Model | Quantization | Verdict | Estimated speed | VRAM needed |
|---|---|---|---|---|
| lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit | Q8 | Not supported | 45.02 tok/sEstimated | 31GB (have 16GB) |
| Qwen/Qwen2.5-14B-Instruct | Q4 | Fits comfortably | 88.01 tok/sEstimated | 8GB (have 16GB) |
| lmstudio-community/Qwen3-4B-Thinking-2507-MLX-8bit | Q8 | Fits comfortably | 78.95 tok/sEstimated | 4GB (have 16GB) |
| openai-community/gpt2 | Q4 | Fits comfortably | 112.87 tok/sEstimated | 4GB (have 16GB) |
| openai-community/gpt2 | Q8 | Fits comfortably | 92.74 tok/sEstimated | 7GB (have 16GB) |
| openai-community/gpt2 | FP16 | Fits (tight) | 42.95 tok/sEstimated | 15GB (have 16GB) |
| Qwen/Qwen2.5-7B-Instruct | Q4 | Fits comfortably | 122.24 tok/sEstimated | 4GB (have 16GB) |
| Qwen/Qwen2.5-7B-Instruct | Q8 | Fits comfortably | 92.91 tok/sEstimated | 7GB (have 16GB) |
| Qwen/Qwen2.5-7B-Instruct | FP16 | Fits (tight) | 45.14 tok/sEstimated | 15GB (have 16GB) |
| Qwen/Qwen3-0.6B | Q4 | Fits comfortably | 126.54 tok/sEstimated | 3GB (have 16GB) |
| Qwen/Qwen3-0.6B | Q8 | Fits comfortably | 78.07 tok/sEstimated | 6GB (have 16GB) |
| Qwen/Qwen3-0.6B | FP16 | Fits comfortably | 50.52 tok/sEstimated | 13GB (have 16GB) |
| Gensyn/Qwen2.5-0.5B-Instruct | Q4 | Fits comfortably | 132.49 tok/sEstimated | 3GB (have 16GB) |
| Gensyn/Qwen2.5-0.5B-Instruct | Q8 | Fits comfortably | 81.36 tok/sEstimated | 5GB (have 16GB) |
| Gensyn/Qwen2.5-0.5B-Instruct | FP16 | Fits comfortably | 42.69 tok/sEstimated | 11GB (have 16GB) |
| meta-llama/Llama-3.1-8B-Instruct | Q4 | Fits comfortably | 124.44 tok/sEstimated | 4GB (have 16GB) |
| meta-llama/Llama-3.1-8B-Instruct | Q8 | Fits comfortably | 92.63 tok/sEstimated | 9GB (have 16GB) |
| meta-llama/Llama-3.1-8B-Instruct | FP16 | Not supported | 44.90 tok/sEstimated | 17GB (have 16GB) |
| dphn/dolphin-2.9.1-yi-1.5-34b | FP16 | Not supported | 17.42 tok/sEstimated | 70GB (have 16GB) |
| openai/gpt-oss-20b | Q4 | Fits comfortably | 61.23 tok/sEstimated | 10GB (have 16GB) |
| openai/gpt-oss-20b | Q8 | Not supported | 44.02 tok/sEstimated | 20GB (have 16GB) |
| google/gemma-3-1b-it | Q8 | Fits comfortably | 104.58 tok/sEstimated | 1GB (have 16GB) |
| facebook/opt-125m | Q8 | Fits comfortably | 85.63 tok/sEstimated | 7GB (have 16GB) |
| TinyLlama/TinyLlama-1.1B-Chat-v1.0 | Q8 | Fits comfortably | 102.34 tok/sEstimated | 1GB (have 16GB) |
| TinyLlama/TinyLlama-1.1B-Chat-v1.0 | FP16 | Fits comfortably | 51.25 tok/sEstimated | 2GB (have 16GB) |
| trl-internal-testing/tiny-Qwen2ForCausalLM-2.5 | Q4 | Fits comfortably | 112.91 tok/sEstimated | 4GB (have 16GB) |
| Qwen/Qwen3-4B-Instruct-2507 | Q8 | Fits comfortably | 85.28 tok/sEstimated | 4GB (have 16GB) |
| Qwen/Qwen3-4B-Instruct-2507 | FP16 | Fits comfortably | 49.89 tok/sEstimated | 9GB (have 16GB) |
| meta-llama/Llama-3.2-1B-Instruct | Q4 | Fits comfortably | 160.52 tok/sEstimated | 1GB (have 16GB) |
| meta-llama/Llama-3.2-1B-Instruct | Q8 | Fits comfortably | 97.93 tok/sEstimated | 1GB (have 16GB) |
| openai/gpt-oss-120b | Q4 | Not supported | 24.09 tok/sEstimated | 59GB (have 16GB) |
| openai/gpt-oss-120b | Q8 | Not supported | 18.52 tok/sEstimated | 117GB (have 16GB) |
| openai/gpt-oss-120b | FP16 | Not supported | 9.25 tok/sEstimated | 235GB (have 16GB) |
| Qwen/Qwen2.5-3B-Instruct | Q4 | Fits comfortably | 160.21 tok/sEstimated | 2GB (have 16GB) |
| Qwen/Qwen2.5-3B-Instruct | Q8 | Fits comfortably | 93.22 tok/sEstimated | 3GB (have 16GB) |
| Qwen/Qwen3-32B | Q4 | Fits (tight) | 44.69 tok/sEstimated | 16GB (have 16GB) |
| Qwen/Qwen3-32B | Q8 | Not supported | 27.21 tok/sEstimated | 33GB (have 16GB) |
| Qwen/Qwen3-32B | FP16 | Not supported | 16.12 tok/sEstimated | 66GB (have 16GB) |
| Qwen/Qwen3-Next-80B-A3B-Instruct | Q4 | Not supported | 23.12 tok/sEstimated | 39GB (have 16GB) |
| Qwen/Qwen3-Next-80B-A3B-Instruct | Q8 | Not supported | 16.67 tok/sEstimated | 78GB (have 16GB) |
| Qwen/Qwen3-Next-80B-A3B-Instruct | FP16 | Not supported | 9.13 tok/sEstimated | 156GB (have 16GB) |
| microsoft/Phi-3-mini-4k-instruct | FP16 | Fits (tight) | 51.12 tok/sEstimated | 15GB (have 16GB) |
| openai-community/gpt2-large | Q4 | Fits comfortably | 122.31 tok/sEstimated | 4GB (have 16GB) |
| openai-community/gpt2-large | Q8 | Fits comfortably | 92.66 tok/sEstimated | 7GB (have 16GB) |
| openai-community/gpt2-large | FP16 | Fits (tight) | 44.97 tok/sEstimated | 15GB (have 16GB) |
| Qwen/Qwen3-1.7B | Q4 | Fits comfortably | 114.77 tok/sEstimated | 4GB (have 16GB) |
| Qwen/Qwen3-1.7B | Q8 | Fits comfortably | 83.15 tok/sEstimated | 7GB (have 16GB) |
| Qwen/Qwen3-1.7B | FP16 | Fits (tight) | 46.29 tok/sEstimated | 15GB (have 16GB) |
| Qwen/Qwen3-4B | Q4 | Fits comfortably | 128.39 tok/sEstimated | 2GB (have 16GB) |
| unsloth/Llama-3.2-1B-Instruct | Q8 | Fits comfortably | 110.31 tok/sEstimated | 1GB (have 16GB) |
Note: Performance estimates are calculated. Real results may vary. Methodology · Submit real data
Data-backed answers pulled from community benchmarks, manufacturer specs, and live pricing.
Umbrella’s CUDA build runs the 16 GB chat preset for Llama 3.3 70B at roughly 10 tokens/sec on a stock RTX 4080—around 20× faster than older GGUF pipelines on the same card.
Source: Reddit – /r/LocalLLaMA (m7daipg)
Yes. One builder logged Llama 3.3 70B Q3_s at ~15 tok/s on Windows with Ollama, then jumped to ~30 tok/s after switching to Linux with ExLlama and performance-tuned CUDA kernels.
Source: Reddit – /r/LocalLLaMA (mi1gu0s)
RTX 4080 carries a 320 W board power rating, ships with 16 GB of GDDR6X, and uses the 16-pin 12VHPWR connector. NVIDIA recommends at least a 750 W PSU.
Source: TechPowerUp – RTX 4080 Specs
Only with heavy offloading. Users experimenting with DDR6 system memory and PCIe offload confirm that 70B models can run, but bandwidth limits keep throughput well below 24 GB cards.
Source: Reddit – /r/LocalLLaMA (m76rp0l)
Price snapshot from Nov 2025: Amazon at $1,199 in stock.
Source: Supabase price tracker snapshot – 2025-11-03
Explore how RTX 4090 stacks up for local inference workloads.
Explore how RTX 4070 Ti stacks up for local inference workloads.
Explore how RTX 4070 stacks up for local inference workloads.
Explore how RTX 3080 stacks up for local inference workloads.
Explore how RX 7900 XTX stacks up for local inference workloads.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.