Nvidia DGX Station A100 User Manual

Getting Started with DGX Station A100
DGX Station A100 DU-10189-001 _v5.0.2|13
3 185.55 185.32 184.86 1589.52 15.71
4 16.26 16.28 16.16 15.69 139.43
P2P=Disabled Latency Matrix (us)
GPU 0 1 2 3 4
0 3.53 21.60 22.22 21.38 12.46
1 21.61 2.62 21.55 21.65 12.34
2 21.57 21.54 2.61 21.55 12.40
3 21.57 21.54 21.58 2.51 13.00
4 13.93 12.41 21.42 21.58 1.14
CPU 0 1 2 3 4
0 4.26 11.81 13.11 12.00 11.80
1 11.98 4.11 11.85 12.19 11.89
2 12.07 11.72 4.19 11.82 12.49
3 12.14 11.51 11.85 4.13 12.04
4 12.21 11.83 12.11 11.78 4.02
P2P=Enabled Latency (P2P Writes) Matrix (us)
GPU 0 1 2 3 4
0 3.79 3.34 3.34 3.37 13.85
1 2.53 2.62 2.54 2.52 12.36
2 2.55 2.55 2.61 2.56 12.34
3 2.58 2.51 2.51 2.53 14.39
4 19.77 12.32 14.75 21.60 1.13
CPU 0 1 2 3 4
0 4.27 3.63 3.65 3.59 13.15
1 3.62 4.22 3.61 3.62 11.96
2 3.81 3.71 4.35 3.73 12.15
3 3.64 3.61 3.61 4.22 12.06
4 12.32 11.92 13.30 12.03 4.05
NOTE: The CUDA Samples are not meant for performance measurements. Results may vary
when GPU Boost is enabled.
The example above shows the peer-to-peer bandwidth and latency test across all five GPUs,
including the DGX Display GPU. The application also shows that there is no peer-to-peer
connectivity between any GPU and GPU 4. This indicates that GPU 4 should not be used for
high-performance workloads.
Run the example one more time by using the CUDA_VISIBLE_DEVICES variable, which limits
the number of GPUs that the application can see.
Note: All GPUs can communicate with all other peer devices.
lab@ro-dvt-058-80gb:/usr/local/cuda-11.2/samples/bin/x86_64/linux/release$
CUDA_VISIBLE_DEVICES=0,1,2,3 ./p2pBandwidthLatencyTest
[P2P (Peer-to-Peer) GPU Bandwidth Latency Test]
Device: 0, Graphics Device, pciBusID: 1, pciDeviceID: 0, pciDomainID:0
Device: 1, Graphics Device, pciBusID: 47, pciDeviceID: 0, pciDomainID:0
Device: 2, Graphics Device, pciBusID: 81, pciDeviceID: 0, pciDomainID:0
Device: 3, Graphics Device, pciBusID: c2, pciDeviceID: 0, pciDomainID:0
Device=0 CAN Access Peer Device=1
Device=0 CAN Access Peer Device=2
Device=0 CAN Access Peer Device=3
Device=1 CAN Access Peer Device=0
Device=1 CAN Access Peer Device=2
Device=1 CAN Access Peer Device=3
Device=2 CAN Access Peer Device=0
Device=2 CAN Access Peer Device=1
Device=2 CAN Access Peer Device=3
Device=3 CAN Access Peer Device=0
Device=3 CAN Access Peer Device=1
Device=3 CAN Access Peer Device=2

Nvidia DGX Station A100 Specifications

General

Brand	Nvidia
Model	DGX Station A100
Category	Desktop
Language	English

Summary

Introduction to the NVIDIA DGX Station A100

Registering Your DGX Station A100

Instructions for registering your DGX Station A100 for support.

DGX Station A100 Hardware Summary

Getting Started with DGX Station A100

Connecting and Powering on the DGX Station A100

Step-by-step guide to connect and power on the DGX Station A100.

Using DGX Station A100 as a Server Without a Monitor

Configuration for operating the DGX Station A100 without a display.

Running Workloads on Systems with Mixed Types of GPUs

Methods for running workloads utilizing mixed GPU types effectively.

Running with Docker Containers

Guide to running workloads within Docker containers on DGX.

Running on Bare Metal

Instructions for running applications directly on the system hardware.

Using Multi-Instance GPUs

How to utilize Multi-Instance GPU (MIG) on NVIDIA A100 GPUs.

Completing the Initial Ubuntu OS Configuration

Steps to finalize the initial setup of the Ubuntu OS.

Using the BMC

Understanding the BMC Controls

Overview of the primary controls available in the BMC dashboard.

Configuring a Static IP Address for the BMC

Steps to assign a static IP address to the BMC.

Configuring a BMC Static IP Address Using ipmitool

Using ipmitool to set a static IP address for the BMC via command line.

Configuring a BMC Static IP Address Using the System BIOS

Setting a static IP for the BMC through the system BIOS.

Logging into the BMC

Procedure for accessing the BMC via a web browser.

Changing Your Default BMC Password

Instructions to change the default BMC password for security.

Logging in After Entering an Incorrect Password

Information on recovering access after multiple failed login attempts.

Common BMC Tasks

Configuring the BMC Login Credentials

Using the Remote Control

Setting Up Active Directory or LDAP/E-Directory

Uploading or Generating SSL Certificates

Generating the SSL Certificate

Uploading the SSL Certificate

Enable MIG Mode in DGX Station A100

Managing Self-Encrypting Drives on DGX Station A100

Overview

Installing the nv-disk-encrypt Package

Steps to install the nv-disk-encrypt software package.

Initializing the System for Drive Encryption

Procedure to initialize DGX system drives for encryption.

Enabling Drive Locking

How to enable automatic drive locking after initialization.

Initialization Examples

Example 1: Passing in the JSON File

Determining the Drives Can be Managed as Self Encrypting

Creating the Drive/Password Mapping JSON File

Example 2: Generating Random Passwords

Example 3: Specifying Passwords One at a Time When Prompted

Disabling Drive Locking

Erasing Your Data

Procedure for securely erasing data and reconfiguring RAID.

Configuring Trusted Computing

Enabling the TPM

Steps to enable the Trusted Platform Module (TPM) in BIOS.

Clearing the TPM

Changing Disk Passwords, Adding Disks, or Replacing Disks

Recovering a Lost Key

Unpacking and Repacking the DGX Station A100

Unpacking the DGX Station A100

Repacking the DGX Station A100 for Shipment

Security

Changing Your BMC Credentials

Procedure to change BMC username and password for security.

Plugging in the Network Cable

Safety

General Precautions

General safety guidelines for operating the DGX Station A100.

Electrical Precautions

Safety measures related to power cables and electrical connections.