EasyManua.ls Logo

Mellanox Technologies ConnectX-6 User Manual

Mellanox Technologies ConnectX-6
64 pages
To Next Page IconTo Next Page
To Next Page IconTo Next Page
Page #1 background imageLoading...
Page #1 background image
www.mellanox.com
Mellanox Technologies Confidential
Mellanox ConnectX®-6 VPI Adapter Cards User Manual for Dell
EMC PowerEdge Servers
OPNs: Y1T43, 7TKND, 1GK7G, CY7GD
Rev 1.1

Table of Contents

Question and Answer IconNeed help?

Do you have a question about the Mellanox Technologies ConnectX-6 and is the answer not in the manual?

Mellanox Technologies ConnectX-6 Specifications

General IconGeneral
Product TypeNetwork Adapter
ManufacturerMellanox Technologies
ModelConnectX-6
InterfacePCIe 4.0 x16
Maximum Throughput200 Gbps
Data Link ProtocolEthernet, InfiniBand
RoCE (RDMA over Converged Ethernet)Yes
iWARP SupportYes
VirtualizationSR-IOV
Operating Temperature0°C to 55°C (Operating)
Power Consumption25 W
Ports1 or 2 ports (depending on model)

Summary

About This Manual

ConnectX-6 Part Numbers

Table of ordering part numbers for ConnectX-6 InfiniBand/VPI adapter cards.

Intended Audience

Manual is for installers and users, assumes familiarity with InfiniBand and Ethernet.

Technical Support

Dell Support site URL provided.

Firmware Updates

Information on obtaining software, firmware, and knowledge database.

1 Introduction

1.1 Product Overview

User guide for ConnectX-6 VPI adapter cards, detailing features, performance, and applications.

1.2 ConnectX-6 SNAPI (Single-slot) Configuration

Explains SNAPI configuration for dual-socket servers using Mellanox Socket Direct technology.

1.2.1 ConnectX-6 Dual-slot Socket Direct Cards 2x PCIe x16

Details ConnectX-6 Socket Direct cards for 200Gb/s connectivity on PCIe Gen 3.0 servers.

1.2.2 ConnectX-6 SNAPI (Single-slot) PCIe 2 x8 in a Row

Describes ConnectX-6 SNAPI configuration using two PCIe x8 lanes for direct CPU connection.

1.3 Features and Benefits

PCI Express Features

Lists PCIe interface features and configurations supported by ConnectX-6 cards.

InfiniBand/Ethernet Capabilities

Details InfiniBand and Ethernet standards compliance, speeds, and protocols.

Key Technologies and Features

Explains features like RDMA, RoCE, PeerDirect, Overlay Networks, and Manageability.

Supported Operating Systems

Lists supported OS/Distributions and notes limitations for certain OPNs.

Connectivity and Manageability

Covers interoperability, cable support, and manageability via BMC.

2 Interfaces

InfiniBand Interface Details

States InfiniBand interface compliance and QSFP56 connector usage.

Ethernet Interface Details

Describes Ethernet interface compliance and ESD protection for copper cables.

PCI Express Interface Details

Details PCIe Gen 3.0/4.0 support, master/slave operation, and features.

LED Link Status Indicators

Explains the bi-color I/O LED for link status indications.

Heat Sink Interface

Describes the heatsink attachment and thermal shutdown safety mechanism.

SMBus Interface for Management

Details SMBus interface for manageability through BMC.

Voltage Regulator Function

Explains how voltage regulators on the card receive power from the PCIe edge connector.

3 Hardware Installation

General Hardware Installation Safety

Provides general installation safety warnings and precautions for electronic equipment.

Installation Procedure Steps

Outlines the steps for installing ConnectX-6 adapter cards.

System Requirements Overview

Details hardware, airflow, software requirements, safety, and pre-installation checklist.

Installation Safety Precautions

Lists safety precautions to avoid injury and system damage during installation.

Pre-Installation Checklist

Provides a checklist of steps and precautions before installing the card.

Adapter Card Installation Instructions

R740 Server Installation

Detailed steps for installing ConnectX-6 cards in Dell R740 servers.

C6400 Server Installation

Detailed steps for installing ConnectX-6 cards in Dell C6400 servers.

C4140 Server Installation

Detailed steps for installing ConnectX-6 cards in Dell C4140 servers.

HDR100 Card Installation

Detailed steps for installing the HDR100 ConnectX-6 card.

Cables and Modules Management

Cable Installation Procedures

Instructions for inserting and removing cables, including connector orientation and latching.

Identifying the Adapter Card

Identifying Card in Linux

Steps to identify the Mellanox card in Linux using lspci command.

Identifying Card in Windows

Steps to identify the Mellanox card in Windows Device Manager using Hardware Ids.

4 Driver Installation

Linux Driver Installation

Guide to installing Mellanox OFED for Linux, including prerequisites and download.

4.1.1 Prerequisites

Lists requirements for installing Mellanox OFED for Linux on a server.

4.1.2 Downloading Mellanox OFED

Instructions for downloading the Mellanox OFED package from the Mellanox website.

4.1.3 Installing Mellanox OFED

Details the mlnxofedinstall script and its functions, including firmware upgrades.

4.1.3.1 Installation Script

Describes the mlnxofedinstall script and its capabilities.

4.1.3.2 Installation Procedure

Step-by-step guide for logging in and running the installation.

4.1.3.3 Installation Results

Example output after successful installation and firmware update.

4.1.3.4 Installation Logs

Explains where installation log files are saved.

4.1.3.5 Openibd Script

Information on the openibd script for managing network interfaces.

4.1.3.6 Driver Load on Boot Control

How to control automatic loading of Mellanox drivers upon system boot.

4.1.3.7 MLNX_OFED Install Return Codes

Lists and explains return codes from the mlnxofedinstall script.

4.1.3.8 Uninstalling MLNX_OFED

Instructions for uninstalling the Mellanox OFED package.

4.1.4 MLNX_OFED Installation via YUM

Guides for installing MLNX_OFED packages using YUM.

4.1.4.1 Setting up YUM Repository

Steps to set up the YUM repository for MLNX_OFED package.

4.1.4.2 Installing MLNX_OFED with YUM

Procedures for installing MLNX_OFED packages via YUM.

4.1.5 MLNX_OFED Installation via apt-get

Guides for installing MLNX_OFED packages using apt-get.

4.1.5.1 Setting up apt-get Repository

Steps to set up the apt-get repository for MLNX_OFED on Debian/Ubuntu.

4.1.5.2 Installing MLNX_OFED with apt-get

Procedures for installing MLNX_OFED packages via apt-get.

4.1.6 Firmware Update After Installation

Methods for updating adapter firmware after driver installation.

4.1.6.1 Updating Firmware Online

Steps to update firmware online using the mlxfwmanager command.

4.1.6.2 Updating Firmware Manually

Information on manually updating firmware via OEM download page.

4.1.6.3 Automatic Firmware Update on Boot

How to enable automatic firmware updates upon system boot.

4.1.7 UEFI Secure Boot Support

Information on supporting Secure Boot with Mellanox drivers.

4.1.7.1 Enrolling Mellanox Public Key

Steps to enroll Mellanox's public key for Secure Boot.

4.1.7.2 Removing Kernel Module Signatures

How to remove signatures from kernel modules.

4.1.8 Performance Tuning Guide

Guide to modifying adapter configurations for performance tuning.

Windows Driver Installation

Guide to downloading and installing Mellanox WinOF-2 driver for Windows.

4.2.1 Software Requirements

Lists software requirements for installing Mellanox WinOF-2 for Windows.

4.2.2 Downloading WinOF-2 Driver

Instructions for downloading the Mellanox WinOF-2 driver.

4.2.3 Installing WinOF-2 Driver

Covers attended and unattended installation procedures for WinOF-2 driver.

4.2.3.1 WinOF-2 Attended Installation

Step-by-step guide for performing an attended installation of the WinOF-2 driver.

4.2.3.2 WinOF-2 Unattended Installation

Steps for performing an unattended installation of the WinOF-2 driver.

4.2.3.3 WinOF-2 Installation Results

How to verify successful installation via Device Manager.

4.2.4 Uninstalling WinOF-2 Driver

Procedures for uninstalling the WinOF-2 driver.

4.2.4.1 WinOF-2 Attended Uninstallation

Steps for attended uninstallation of WinOF-2 driver.

4.2.4.2 WinOF-2 Unattended Uninstallation

Steps for unattended uninstallation of WinOF-2 driver.

4.2.5 Extracting WinOF-2 Files

Steps to extract driver and tool files without running the installer.

4.2.6 WinOF-2 Firmware Upgrade

Information on automatic firmware updates during WinOF-2 installation.

Adapter Firmware Update

Using mlxup for Firmware Updates

Guide to using mlxup for querying and updating adapter firmware versions.

Troubleshooting Guide

General Troubleshooting Steps

Common issues and solutions for adapter detection, operation, and link status.

Linux Troubleshooting Procedures

Commands and steps for troubleshooting Mellanox adapters in Linux environments.

Windows Troubleshooting Procedures

Steps for troubleshooting Mellanox adapters in Windows environments.

Product Specifications

Y1T43 & 7TKND Specs

Detailed specifications for Y1T43 and 7TKND adapter cards.

1GK7G & CY7GD Specs

Detailed specifications for 1GK7G and CY7GD adapter cards.

Mechanical Drawings & Dimensions

Mechanical drawings and dimensions for adapter cards and brackets.

ConnectX-6 Adapter Card Dimensions

Mechanical drawings and dimensions for the ConnectX-6 adapter card.

Auxiliary PCIe Card Dimensions

Mechanical drawings and dimensions for the Auxiliary PCIe Connection Card.

ConnectX-6 Bracket Dimensions

Mechanical drawings and dimensions for ConnectX-6 adapter brackets.

SNAPI PCIe Express Pinouts

SNAPI Pinout Definitions

Pin definitions for special PCIe pins required for SNAPI capability.

Adapter Card MAC Address Location

Adapter Label Information

Shows examples of adapter card labels with Part Number, DP/N, GUID, and S/N.

Related product manuals