Nvidia DGX H100 User Manual

NVIDIA DGX H100 User Guide
1.1. Hardware Overview
1.1.1. DGX H100 Models and Component Descriptions
There are two models of the NVIDIA DGX H100 system: the NVIDIA DGX H100 640GB system and the
NVIDIA DGX H100 320GB system.
Table 1: Table 1. Component Description
Component Description
GPU 8 x NVIDIA H100 GPUs that provide 640 GB total GPU memory
CPU 2 x Intel Xeon 8480C PCIe Gen5 CPU with 56 cores 2.0/2.9/3.8
GHz (base/all core turbo/Max turbo)
NVSwitch 4 x 4th generation NVLink that provide 900 GB/s GPU-to-GPU
bandwidth
Storage (OS) 2 x 1.92 TB NVMe M.2 SSD (ea) in RAID 1 array
Storage (Data Cache) 8 x 3.84 TB NVMe U.2 SED (ea) in RAID 0 array
Network (Cluster) card 4 x OSFP ports for 8 NVIDIA ConnectX-7 Single Port InniBand
cars that each provide the following speeds:
InniBand (default): Up to 400Gbps
Ethernet: 400GbE, 200GbE, 100GbE, 50GbE, 40GbE, 25GbE, and
10GbE
Network (Storage) card 2 x NVIDIA ConnectX-7 Dual Port Ethernet cards that each pro-
vide the following speeds:
Ethernet (default): 400GbE, 200GbE, 100GbE, 50GbE, 40GbE,
25GbE, and 10GbE
InniBand: Up to 400Gbps
System Memory (DIMM) 2 TB per 32 DIMMs
BMC (out-of-band system man-
agement)
1 GbE RJ45 interface
Supports Redsh, IPMI, SNMP, KVM, and Web user interface
In-band system management 50 GbE in slot 3 and 10 GbE RJ45 interface
Power Supply 6 x 3.3 kW
2 Chapter 1. Introduction to the NVIDIA DGX H100 System

Nvidia DGX H100 Specifications

General

GPU	8x NVIDIA H100 Tensor Core GPUs
GPU Memory	640 GB HBM3 (80GB per GPU)
System Memory	2 TB DDR5
Form Factor	6U rackmount
Storage	30TB NVMe SSD
Power Supply	10kW
Interconnect	NVLink 4.0