GB200 requires an optical module

The NVIDIA GB200 uses 800G and 1.6T OSFP optical modules for high-speed inter-cabinet connections, complementing internal copper NVLink interconnects.Overview of Optical Module UsageIn the GB200 NVL72 rack-scale system, internal communication within a cabinet relies primarily on copper interconnects such as 800G/1600G DAC and ACC cables, which provide low-latency, high-bandwidth connections for up to 72 Blackwell GPUs and 36 Grace CPUs . For inter-cabinet or cluster-level connections, the system employs 800G and 1.6T OSFP optical modules, enabling scalable high-speed networking across multiple racks .800G Optical ModulesApplication: Used extensively in large GB200 clusters, such as a 576-GPU deployment, requiring tens of thousands of 800G DR8 optical modules .Function: Facilitate high-bandwidth interconnects for AI workloads, supporting NVLink and InfiniBand networks.Manufacturers: Eoptolink and InnoLight dominate the supply, with Eoptolink supplying over 60% of 800G modules for NVIDIA GB200 servers .1.6T Optical ModulesApplication: Deployed in dual-layer and three-layer InfiniBand networks for GB200 NVL72 systems, with a GPU-to-transceiver ratio of 1:2 or 1:3 depending on network topology .Function: Provide ultra-high-speed interconnects for inter-rack communication, supporting generative AI and large language model workloads.Technology: Utilize 200G PAM4 signaling across 8 parallel channels, achieving 1.6Tbps total transmission per module .Integration with GB200 ArchitectureInternal NVLink Network: Within a single GB200 cabinet, NVLink 5.0 provides up to 1.8TB/s bidirectional bandwidth per GPU, connected via copper DAC/ACC cables .External Cluster Connectivity: Optical modules extend the network beyond a single cabinet, enabling clusters of up to 576 GPUs with high-speed, low-latency interconnects .Rack-Scale Design: Each NVL72 rack includes compute trays, NVLink switch trays, and TOR switches, with optical modules interfacing at the top-of-rack or inter-rack level to maintain scalable AI data center performance .SummaryThe GB200 system balances copper and optical interconnects: copper DAC/ACC cables handle short-range, high-bandwidth internal connections, while 800G and 1.6T OSFP optical modules provide scalable, high-speed inter-cabinet connectivity. This combination ensures low latency, high throughput, and energy-efficient performance for AI workloads and large-scale GPU clusters .