Renting an H100 in 2026 costs anywhere from about 1.90 to nearly 7 dollars per GPU-hour depending on where you rent it, and a B200 runs roughly 4 to 14 dollars per hour for the exact same silicon. The spread between the cheapest marketplace and the most expensive hyperscaler is 3x or more, which means picking the right provider matters more than almost any optimization you can do inside your training script. Here is where the market sits, what the tiers actually trade off, and which class of GPU fits which job.
The market in one table
Prices below are representative on-demand rates per GPU-hour as of mid-2026. GPU pricing moves constantly, sometimes weekly, so treat these as a snapshot of the relative order rather than gospel. For live per-model API pricing, our rankings stay current.
| Provider | H100 (approx) | B200 (approx) | Style |
|---|---|---|---|
| Vast.ai | from ~1.90/hr | varies by host | Peer-to-peer marketplace |
| RunPod | ~2.00 to 2.70/hr | ~5.90/hr | Self-serve cloud, per-minute billing |
| Lambda | ~3.00/hr | ~5.00 to 5.30/hr | Fixed pricing, ML-focused |
| Modal | ~4.30/hr equivalent | available | Serverless, per-second billing |
| CoreWeave | ~6.16/hr (8-GPU nodes) | higher, node-based | Enterprise clusters |
| AWS | ~6.88/hr (p5) | ~14/hr (p6) | Hyperscaler |
A few things jump out. The same H100 costs more than 3x as much on AWS as on Vast.ai. CoreWeave sells H100s as full 8-GPU HGX nodes, so the entry price is a node, not a card. And the B200, despite being the newer Blackwell part, is already renting for under 6 dollars per hour at the ML-native clouds while hyperscalers charge more than double that.
What the tiers actually trade off
The price spread is not irrational. You are paying for different things.
- Marketplaces (Vast.ai, RunPod Community Cloud): cheapest raw compute. Hosts are third parties, so availability, network speed, and reliability vary machine to machine. Fine for experiments and fault-tolerant training. Not where you put customer data with strict compliance needs.
- ML-native clouds (RunPod Secure Cloud, Lambda): datacenter hardware, predictable pricing, fast provisioning, per-minute or per-second billing. This is the sweet spot for most startups doing fine-tuning or self-hosted inference.
- Serverless (Modal, RunPod Serverless): highest per-hour sticker price, but you pay only for active seconds and scale to zero when idle. For bursty inference, the break-even against an always-on instance sits around 30 percent utilization. Below that, serverless usually wins despite the higher rate.
- Enterprise and hyperscalers (CoreWeave, AWS, GCP, Azure): you pay for compliance, integration with the rest of your cloud, big interconnected clusters, and account teams. If you have hyperscaler credits from a program like AWS Activate, the sticker price matters less; see our AWS Activate guide.
Spot vs on-demand
Spot (or interruptible) instances are the biggest single lever on GPU cost. Discounts typically run 50 to 80 percent off on-demand, in exchange for the provider being able to reclaim the machine when a higher-paying customer shows up.
The rule of thumb:
- Use spot for: batch training with checkpointing, data preprocessing, hyperparameter sweeps, offline batch inference, anything that can resume from a saved state.
- Use on-demand for: live inference endpoints, long training runs without checkpointing (fix that first), demos, anything a user is waiting on.
If your training loop checkpoints every few minutes, an interruption costs you a few minutes of compute. At a 60 to 70 percent discount, spot is almost always worth the small amount of resume logic.
Do you even need an H100?
The most common overspend I see is renting datacenter GPUs for jobs that fit on consumer cards. On marketplaces and community clouds, an RTX 4090 rents for well under a dollar per hour, and newer consumer cards are not far behind. A100 80GB cards sit around 1.50 per hour on the cheaper clouds.
Rough fit guide:
| Workload | Sensible GPU |
|---|---|
| LoRA fine-tune of a 7B to 13B model | RTX 4090 or 5090, single card |
| Full fine-tune of a 7B model | A100 80GB or H100 |
| Fine-tune of a 70B model | Multi-GPU H100 node |
| Pretraining or frontier-scale work | H100 or B200 clusters, reserved |
| Inference for a small model | Consumer card or serverless |
| High-throughput inference, big models | H100, or B200 for the best tokens per dollar |
The B200 case is specific: it delivers roughly 2 to 3x the training and inference throughput of an H100. If a B200 costs 5 to 6 dollars per hour and finishes the job in less than half the time of a 3 dollar H100, it is cheaper per unit of work. Always compare cost per job, not cost per hour.
When renting a GPU is the wrong answer entirely
If your actual goal is tokens out of an open-weight model, a raw GPU rental is often not the cheapest path. Serverless inference APIs bill per token, amortize the hardware across thousands of customers, and remove all ops work. We ran the math in self-hosting vs using an API, and the short version is that below sustained high utilization, the API wins. See also the cheapest way to run LLMs for the full decision tree.
Rent GPUs when you need to fine-tune, run custom architectures, control weights and data locality, or when your utilization is genuinely high enough that per-token pricing loses.
How to pay less for the GPUs you do rent
- Start with credits. Modal includes 30 dollars per month of compute on its starter plan, several GPU clouds give signup credits, and hyperscaler startup programs can cover serious training budgets. We keep the dated list in the catalog, and the compute-specific options are in where to get free GPU compute and startup cloud credits.
- Match the billing model to the workload. Per-second serverless for bursty inference, per-minute on-demand for interactive work, spot for batch.
- Checkpoint everything, then move training to spot.
- Benchmark one hour before committing to a hundred. Marketplace hosts vary; a cheap listing with slow disk or throttled bandwidth can cost more in wall-clock time than a pricier one.
- Recheck prices monthly. H100 rates fell hard through 2025 and B200 supply is still ramping, so today's premium card is next quarter's commodity.
Bottom line
In 2026, H100s rent for roughly 2 to 3 dollars per hour at ML-native clouds and marketplaces, and 6 to 7 dollars at hyperscalers. B200s start around 5 to 6 dollars at the specialists and can be the cheaper option per unit of work despite the higher rate. Use spot with checkpointing for training, serverless for bursty inference, and consumer cards for anything that fits in 24GB. And before you rent anything, spend the credits: between monthly serverless allowances, signup grants, and startup cloud programs, a lot of early GPU spend can be someone else's money. Browse the current compute offers in the catalog, or create an account to track the ones you have claimed.
Related reading: Where to Get Free GPU Compute, Startup Cloud Credits, Self-Host vs API