This GPU offers reliable throughput for local AI workloads. Pair it with the right model quantization to hit your desired tokens/sec, and monitor prices below to catch the best deal.
Quick Answer: NVIDIA H200 SXM 141GB has 141GB VRAM, enough for models up to roughly 352B parameters at 4-bit quantization. It draws 700W under load.
With 141GB VRAM, NVIDIA H200 SXM 141GB can run models up to approximately 352B parameters using 4-bit quantization. That covers most popular models, including 70B-class ones at 4-bit.
Consider H100 or MI300X — Maximum VRAM for enterprise workloads.
Buy directly on Amazon with fast shipping and reliable customer service.
Essential accessories to pair with NVIDIA H200 SXM 141GB
Accessories Total
Typical prices — check Amazon for current pricing
💡 Not ready to buy? Try cloud GPUs first
Test NVIDIA H200 SXM 141GB performance in the cloud before investing in hardware. Pay by the hour with no commitment.
Showing 12 of 80 rows. Speeds are calculated estimates, not measurements — search for your model to jump straight to it.
| Model | Size | Quantization | Tokens/sec | VRAM used |
|---|---|---|---|---|
| Deepseek AI Deepseek Coder 1.3B Instruct | 1.3B | Q4 | ~1035 tok/sEstimated | 1GB |
| Deepseek AI Deepseek R1 Distill Qwen 1.5B | 1.5B | Q4 | ~1035 tok/sEstimated | 1GB |
| Deepseek AI Deepseek Ocr 2 | Unknown | Q4 | ~860 tok/sEstimated | 2GB |
| Deepseek AI Deepseek Ocr | Unknown | Q4 | ~860 tok/sEstimated | 2GB |
| Lmstudio Community Deepseek R1 0528 Qwen3 8B Mlx 8bit | 8B | Q4 | ~860 tok/sEstimated | 4GB |
| Lmstudio Community Deepseek R1 0528 Qwen3 8B Mlx 4bit | 8B | Q4 | ~860 tok/sEstimated | 4GB |
| Deepseek AI Deepseek R1 Distill Qwen 7B | 7B | Q4 | ~860 tok/sEstimated | 4GB |
| Nineninesix Kani Tts 2 En | Unknown | Q4 | ~825 tok/sEstimated | 1GB |
| Qwen Qwen3 Tts 12hz 1 7B Customvoice | 7B | Q4 | ~825 tok/sEstimated | 1GB |
| Zai Org Glm Ocr | Unknown | Q4 | ~825 tok/sEstimated | 1GB |
| Qwen Qwen3 Asr 1 7B | 7B | Q4 | ~825 tok/sEstimated | 2GB |
| Nari Labs Dia2 2B | 2B | Q4 | ~825 tok/sEstimated | 1GB |
Showing 12 of 240 rows.
| Model | Size | Quantization | Verdict | Estimated speed | VRAM needed |
|---|---|---|---|---|---|
| 01 AI Yi 1 5 34B Chat | 34B | Q4 | Fits comfortably | ~240 tok/sEstimated | 18GB (have 141GB) |
| 01 AI Yi 1 5 34B Chat | 34B | Q8 | Fits comfortably | ~170 tok/sEstimated | 35GB (have 141GB) |
| 01 AI Yi 1 5 34B Chat | 34B | FP16 | Fits comfortably | ~92 tok/sEstimated | 69GB (have 141GB) |
| AI Forever Rugpt 3.5 13B | 13B | Q4 | Fits comfortably | ~515 tok/sEstimated | 7GB (have 141GB) |
| AI Forever Rugpt 3.5 13B | 13B | Q8 | Fits comfortably | ~360 tok/sEstimated | 13GB (have 141GB) |
| AI Forever Rugpt 3.5 13B | 13B | FP16 | Fits comfortably | ~195 tok/sEstimated | 26GB (have 141GB) |
| AI Mo Kimina Prover 72B | 72B | Q4 | Fits comfortably | ~140 tok/sEstimated | 37GB (have 141GB) |
| AI Mo Kimina Prover 72B | 72B | Q8 | Fits comfortably | ~97 tok/sEstimated | 73GB (have 141GB) |
| AI Mo Kimina Prover 72B | 72B | FP16 | Not supported | ~52 tok/sEstimated | 146GB (have 141GB) |
| Alibaba Nlp Gte Qwen2 1.5B Instruct | 1.5B | Q4 | Fits comfortably | ~825 tok/sEstimated | 1GB (have 141GB) |
| Alibaba Nlp Gte Qwen2 1.5B Instruct | 1.5B | Q8 | Fits comfortably | ~580 tok/sEstimated | 2GB (have 141GB) |
| Alibaba Nlp Gte Qwen2 1.5B Instruct | 1.5B | FP16 | Fits comfortably | ~315 tok/sEstimated | 4GB (have 141GB) |
Note: Performance estimates are calculated. Real results may vary. Methodology · Submit real data
Explore how RTX 4090 stacks up for local inference workloads.
Explore how RTX 4080 stacks up for local inference workloads.
Explore how RTX 4070 Ti stacks up for local inference workloads.
Explore how RTX 3090 stacks up for local inference workloads.
Explore how RX 7900 XTX stacks up for local inference workloads.