Frequently asked questions

What is a "cluster"?

A cluster is multiple servers wired together so tightly — usually with a fast dedicated network called NVLink — that they act like one much bigger computer instead of many separate ones. You need one when a single model is too big to fit in the memory of any one server on its own.

Real datacenter clustering (like our NVL72 racks) uses purpose-built, low-latency interconnects so GPUs can share memory almost as if it were local. A pile of desktop GPUs connected over ordinary networking cannot do this the same way — it can run separate copies of small models in parallel, but it can't pool its memory to run one giant model the way a real cluster can.

What is GPU memory (VRAM), and why does it matter more than anything else?

A model's "weights" — the billions of numbers that encode what it learned — have to sit entirely in the GPU's own fast memory while it's running. If they don't fit, the model simply cannot run on that hardware, no matter how fast the GPU otherwise is. That's why we size every recommendation around GPU memory first, not raw speed.

Rule of thumb: a model needs roughly 2 GB of GPU memory per billion parameters (running in the standard FP16/BF16 format), plus about 20% more on top for the "working memory" — the KV cache — it uses while generating a response.

Why do watts matter if I'm not paying the electric bill directly?

Power draw is really a proxy for three things you do care about: your monthly hosting cost, how much cooling and electrical capacity your server room (or colocation provider) needs, and how much of a commitment the hardware is. A single B200 GPU alone draws 1,000 watts — as much as running a hair dryer continuously, 24 hours a day.

Why does a mixture-of-experts model need memory for parameters it isn't using?

In a mixture-of-experts model, each token only activates a small slice of the model's total parameters — but which slice gets used changes token by token, and can be any of them. So all the parameters have to be loaded into memory and ready to go, even though only a fraction compute at once. That's why we always size hardware to a model's total parameter count, not its smaller "active" count.

Why don't your prices for H100/H200/B200 match a public price list?

NVIDIA doesn't publish list prices for its datacenter GPUs and systems — they're sold through OEM partners and negotiated per deal. The prices on this site are realistic, sourced estimates based on current broker and market pricing, clearly noted on each product page, rather than invented numbers.