The Nvidia H100 is Nvidia’s Hopper-generation data-center GPU, announced on March 22, 2022, and built to train and run large AI models. The SXM version of the Nvidia H100 carries 80 GB of HBM3 memory at 3.35 TB/s and draws up to 700 W; the PCIe-based H100 NVL carries 94 GB at 3.9 TB/s and draws 350–400 W. Nvidia rates the SXM part at 3,958 teraFLOPS of FP8 Tensor Core throughput, a figure it lists with sparsity; the dense figure is half that, 1,979 teraFLOPS.
Updated November 3, 2026: specs re-checked against Nvidia’s H100 product page; added dense throughput, power figures for an 8-GPU DGX H100 system, and the September 2022 production date.
On this page
What the H100 is
The H100 is the first GPU on Nvidia’s Hopper architecture, named after the computer scientist Grace Hopper. According to Nvidia’s launch release, it has 80 billion transistors and is made on a TSMC 4N process customized for Nvidia.
Nvidia’s architecture write-up gives the layout: the full GH100 chip has 144 streaming multiprocessors (SMs) and a 60 MB L2 cache; the H100 SXM5 enables 132 SMs and the H100 PCIe enables 114, each with 50 MB of L2. The die measures 814 mm².
Two features define Hopper for AI work:
- Transformer Engine. Nvidia describes it as hardware and software that “intelligently manages and dynamically chooses between FP8 and 16-bit calculations.” FP8 halves the bits per value compared with FP16, so more math fits through the same silicon and memory.
- Fourth-generation NVLink. The SXM version connects to other GPUs at 900 GB/s, and Nvidia’s NVLink Switch System can join up to 256 GPUs.
H100 release date
| Date | What happened | Nvidia source |
|---|---|---|
| March 22, 2022 | H100 and the Hopper architecture announced at GTC; Nvidia says availability starts in the third quarter of 2022 | Launch release |
| September 20, 2022 | H100 “in full production”; partners plan to roll out the first H100 products and services in October; DGX H100 systems open for orders | Full-production release |
In the September release Nvidia said H100 systems from server makers were “expected to ship in the coming weeks,” with more than 50 server models on the market by the end of 2022.
H100 versions: SXM, PCIe and NVL
“H100” covers more than one product, and the numbers differ between them.
- H100 SXM is a module that mounts on Nvidia’s HGX H100 boards with four or eight GPUs and in Nvidia’s own DGX H100 system. It has the highest power budget (up to 700 W, configurable) and the full 900 GB/s NVLink. Our H100 SXM record in the AI chips catalog carries the same vendor figures.
- H100 PCIe was the original card version. Nvidia’s architecture post lists it with 80 GB of HBM2e memory at more than 2 TB/s and a 350 W TDP.
- H100 NVL is a dual-slot, air-cooled PCIe card with 94 GB of HBM3 at 3,938 GB/s and a 350–400 W configurable TDP. Pairs of cards can be bridged with NVLink at 600 GB/s, which is how Nvidia arrives at the 188 GB of HBM3 it quotes for an NVL pair. Nvidia’s product brief calls it “the most optimized platform for LLM Inferences,” and the card ships with a five-year Nvidia AI Enterprise subscription.
H100 specs at a glance
Figures below are from Nvidia’s H100 product page, checked November 3, 2026. Tensor Core rows are the numbers Nvidia lists with sparsity; dense throughput is in the next section.
| Spec | H100 SXM | H100 NVL |
|---|---|---|
| FP64 | 34 TFLOPS | 30 TFLOPS |
| FP64 Tensor Core | 67 TFLOPS | 60 TFLOPS |
| TF32 Tensor Core (sparse) | 989 TFLOPS | 835 TFLOPS |
| BF16 / FP16 Tensor Core (sparse) | 1,979 TFLOPS | 1,671 TFLOPS |
| FP8 Tensor Core (sparse) | 3,958 TFLOPS | 3,341 TFLOPS |
| INT8 Tensor Core (sparse) | 3,958 TOPS | 3,341 TOPS |
| GPU memory | 80 GB | 94 GB |
| Memory bandwidth | 3.35 TB/s | 3.9 TB/s |
| Max TDP | Up to 700 W | 350–400 W |
| Multi-Instance GPU | Up to 7 × 10 GB | Up to 7 × 12 GB |
| Interconnect | NVLink 900 GB/s, PCIe Gen5 128 GB/s | NVLink 600 GB/s, PCIe Gen5 128 GB/s |
| Server options | HGX H100 (4 or 8 GPUs), DGX H100 (8 GPUs) | Partner systems (1–8 GPUs) |
The H100 NVL has more memory and more bandwidth than the SXM part but lower compute, because it runs at roughly half the power.
Dense vs sparse: the H100 FLOPS most workloads see
Nvidia’s starred Tensor Core numbers assume structured sparsity: a model whose weights have been pruned into a pattern the Tensor Cores can skip. Nvidia’s Hopper architecture post says the sparsity feature doubles “the performance of standard Tensor Core operations,” so the dense rate is half the listed figure. Most models are trained and served dense, which makes the dense column the fairer number to compare with other chips. The dense values are our arithmetic from Nvidia’s listed figures, rounded to the nearest whole number.
| Precision | H100 SXM, sparse (Nvidia) | H100 SXM, dense | H100 NVL, sparse (Nvidia) | H100 NVL, dense |
|---|---|---|---|---|
| TF32 | 989 TFLOPS | ~495 TFLOPS | 835 TFLOPS | ~418 TFLOPS |
| BF16 / FP16 | 1,979 TFLOPS | ~990 TFLOPS | 1,671 TFLOPS | ~836 TFLOPS |
| FP8 | 3,958 TFLOPS | 1,979 TFLOPS | 3,341 TFLOPS | ~1,671 TFLOPS |
| INT8 | 3,958 TOPS | 1,979 TOPS | 3,341 TOPS | ~1,671 TOPS |
Per watt of maximum TDP, the dense FP8 rate works out to about 2.8 TFLOPS per watt for the SXM part (1,979 ÷ 700 W) and about 4.2 TFLOPS per watt for the NVL card at its 400 W ceiling (1,671 ÷ 400 W). These are ratios of two vendor maximums, not measured efficiency: a GPU rarely sustains its peak FLOPS, and actual draw depends on the workload and the configured power limit.
H100 power consumption, per GPU and per server
A single H100 SXM is rated at up to 700 W; the H100 NVL at 350–400 W. Those are GPU-only limits. A server adds CPUs, memory, networking, storage and fans, and Nvidia publishes that total for its own 8-GPU system.
| Configuration | Power figure | Basis |
|---|---|---|
| One H100 SXM | Up to 700 W | Nvidia H100 product page |
| One H100 NVL | 350–400 W | Nvidia H100 product page |
| Eight H100 SXM GPUs (TDP only) | 5.6 kW | Our sum: 8 × 700 W |
| DGX H100 system, 8 GPUs | 10.2 kW max | Nvidia DGX H100/H200 user guide (six 3.3 kW power supplies, 4+2 redundancy) |
On those numbers the eight GPUs account for at most about 55% of the DGX H100’s 10.2 kW maximum. As an upper bound on energy, one H100 SXM held at 700 W around the clock would use about 6,100 kWh in a year (0.7 kW × 8,760 hours); a DGX H100 at its 10.2 kW maximum would use about 89,000 kWh. Real figures are lower, because neither runs at its ceiling all year. How these numbers add up at campus scale is covered in AI data center power consumption.
What the H100 is used for
The H100 was designed for three kinds of work, and Nvidia’s own claims are organized the same way:
- Training large language models. Nvidia claims “up to 4X faster training over the prior generation for GPT-3 (175B) models,” the prior generation being the A100.
- Inference. Nvidia claims the H100 can “accelerate inference by up to 30X” on the largest models compared with A100, crediting the Transformer Engine and FP8.
- High-performance computing. Nvidia states that the H100 “triples the floating-point operations per second (FLOPS) of double-precision Tensor Cores” and claims up to 7X higher performance for HPC applications. Its DPX instructions, which speed up dynamic-programming algorithms used in genomics and route planning, are rated by Nvidia at 7X over A100.
Beyond raw speed, two features shape how clouds sell the H100. Multi-Instance GPU (MIG) splits one card into up to seven isolated instances, which lets a provider rent a fraction of a GPU. Confidential computing, which Nvidia calls “a built-in security feature of the NVIDIA Hopper architecture,” keeps data and models encrypted while in use. Current hourly rental rates from cloud providers are tracked on our GPU prices page, and our H100 rental price explainer compares on-demand, spot and reserved rates.
Where the H100 sits next to H200 and B200
H200 uses the same Hopper architecture and the same compute ratings: Nvidia lists the H200 SXM at 3,958 teraFLOPS FP8 with sparsity, identical to the H100 SXM. The change is memory. Nvidia calls the H200 “the first GPU to offer 141 gigabytes (GB) of HBM3e memory at 4.8 terabytes per second,” which it describes as “nearly double the capacity” of the H100 with 1.4X more memory bandwidth. In an eight-GPU DGX system that is 1,128 GB of GPU memory for H200 against 640 GB for H100, according to Nvidia’s DGX user guide. For inference on large models, where the GPU spends much of its time waiting on memory, that difference shows up directly in throughput.
B200 is the next architecture, Blackwell. Nvidia’s Blackwell GPU has 208 billion transistors across two dies joined by a 10 TB/s chip-to-chip link, and adds FP4 support through a second-generation Transformer Engine. Nvidia publishes B200 figures mainly at the system level: an HGX B200 board with eight GPUs is rated at 72 petaFLOPS of FP8 with sparsity, and the DGX B200 system carries 1,440 GB of HBM3e in total. Nvidia claims the DGX B200 delivers “3X the training performance and 15X the inference performance” of previous-generation systems.
Side-by-side figures for every Nvidia and AMD data-center accelerator are in our AI chips catalog.





