80 GB HBM3 memory
Budget memory for model weights, KV cache and runtime state. Training also requires space for activations, gradients and optimizer state; parameter count alone does not determine the required GPU count.
Hopper architecture with 80 GB of HBM3 memory per GPU. The H100 SXM reference configuration supports large language model inference, fine-tuning and distributed training.
Check the console or contact sales for prices and available instance configurations.
H100 combines fourth-generation Tensor Cores, the Hopper Transformer Engine and fourth-generation NVLink. Select software and topology to match the model, precision and communication requirements.
Budget memory for model weights, KV cache and runtime state. Training also requires space for activations, gradients and optimizer state; parameter count alone does not determine the required GPU count.
Hopper Transformer Engine supports FP8 alongside higher-precision computation. FP8 requires compatible frameworks, kernels and numerical validation; it is not applied automatically to every model.
H100 SXM provides 900 GB/s of aggregate bidirectional NVLink bandwidth per GPU. Actual GPU connectivity depends on the server platform, NVSwitch configuration and allocated GPUs.
The hardware can partition one GPU into up to seven isolated GPU instances. MIG availability, partition size and access mode depend on the actual rental configuration.
From model serving to distributed training, select resources for the workload's memory, compute precision and GPU communication needs.
Run models with Hopper-compatible FP8 kernels and Transformer Engine. Validate numerical accuracy and fallback precision; H100 does not provide native FP4 Tensor Core computation.
Budget the 80 GB HBM3 memory for weights, KV cache and runtime buffers. Context length, batch size and concurrent requests determine how much memory remains for serving.
Include activations, gradients and optimizer state alongside model weights. Plan checkpoint storage and I/O, and check whether the training state needs to be distributed across GPUs.
Evaluate the parallel strategy against the actual NVLink topology and its 900 GB/s aggregate bidirectional bandwidth per GPU. Cross-node training also needs separately confirmed network interfaces, switching and host resources.
These values describe the H100 SXM 80 GB GPU, not H100 PCIe or H100 NVL. Theoretical Tensor Core peaks do not represent measured model throughput or the CPU, memory, storage and network of a complete instance.
| Specification | Value per GPU |
|---|---|
| Architecture | HopperHopper GPU architecture. |
| GPU memory | 80 GB HBM3Stores model weights, KV cache and runtime state. |
| Memory bandwidth | 3.35 TB/sData transfer bandwidth between the GPU and local HBM3 memory. |
| FP8 Tensor compute · Dense | 1,979 TFLOPSTheoretical peak for dense Tensor Core operations. |
| FP8 Tensor compute · Sparse | 3,958 TFLOPSTheoretical peak under the applicable structured-sparsity conditions. |
| GPU interconnect | 900 GB/s · NVLink 4Aggregate bidirectional GPU-to-GPU bandwidth within compatible HGX systems. |
Before renting, confirm CPU, system memory, storage, networking, deployment region, pricing, and delivery arrangements.
SOC 2 Type II audited
ISO/IEC 27001 certified
GDPR compliant
Compare rental options, explore GPU specifications, then discuss configuration and delivery with our team.
Check the console or contact sales for prices and available instance configurations.
Check the console or contact sales for prices and available instance configurations.
Distinguish GPU specifications, system platforms and the actual delivered configuration.
No. This page uses the 80 GB HBM3 SXM reference. H100 PCIe has different specifications; H100 NVL uses 94 GB per GPU and a different interconnect arrangement. Confirm the exact SKU rather than combining specifications from these variants.
Hopper supports FP8 Tensor Core computation through compatible software and Transformer Engine. H100 does not provide Blackwell's native FP4 Tensor Core path. Choose precision only after checking model accuracy, framework support and available kernels.
No. It is the aggregate bidirectional NVLink bandwidth per H100 SXM GPU. Communication between servers depends on the installed network interfaces, switching fabric and cluster topology, which must be confirmed separately.
Provide the GPU quantity, preferred region, start date and rental duration, plus any CPU, system memory, storage and network requirements. Review the specific instance, GPU topology, available capacity and quote in the console or with sales.
View configurations, or submit your GPU quantity, region and rental term for a quote.