英伟达推出MetroX-3 提升长距离InfiniBand系统带宽至400G

英伟达(Nvidia)表示,MetroX-3长距离系统的推出后,用户现在可以将他们的400Gbps InfiniBand网络扩展到更远的地方,因此Quantum-2交换机的范围可提升到25英里或40千米。

英伟达(Nvidia)表示,MetroX-3长距离系统的推出后,用户现在可以将他们的400Gbps InfiniBand网络扩展到更远的地方,因此Quantum-2交换机的范围可提升到25英里或40千米。

英伟达推出MetroX-3 提升长距离InfiniBand系统带宽至400G

Nvidia sees two major use cases for the tech: the first is high-speed workload migration between physically disparate computing centers, while the other involves pooling those resources to tackle larger problems.


While it's been possible to interconnect two datacenters over InfiniBand in the past using the MetroX-2 platform acquired from Mellanox, the appliance was limited to a pair of 100Gbps uplinks. By comparison, the third iteration of Nvidia's MetroX platform adds support for a pair of 100Gbps dense wave division multiplex (DWDM) modules. The technology allows for substantially higher bandwidths by muxing multiple 100Gbps signals onto a single fiber.

Nvidia从Mellanox收购的MetroX-2平台之前也可以通过InfiniBand将两个数据中心互连,但MetroX-2设备仅限于两个100Gbps的上行链路。而相比之下,Nvidia MetroX 3平台增加了对两个100Gbps密集波分复用(DWDM)模块的支持。MetroX 3技术将多个100Gbps信号复用到单一光纤上,可以大幅提高带宽。

"This really allows your extended campus clusters and your core datacenter to behave as a single-unit — a single datacenter," Dion Harris, head of datacenter product marketing at Nvidia, said during a press briefing.

Nvidia数据中心产品营销主管Dion Harris在新闻发布会上表示,“这确实可以让你的扩展园区集群和你的核心数据中心成为一体,成为一个单一的数据中心。”

The approach isn't without compromise. MetroX-3 is clearly an ecosystem play. It's designed to work with Nvidia's InfiniBand ecosystem of Quantum-2 switches, ConnectX-7 NICs, and/or BlueField data processing units (DPUs). That means if you're already using something like HPE's Ethernet-based Slingshot interconnects or even Nvidia's own Spectrum switches, MetroX-3 isn't for you.


Assuming you do live within Nvidia's walled garden, or want to expand an existing InfiniBand environment to a new location, there are also performance concessions that need to be taken into account. While DWDM allows for massive aggregate bandwidths over a single fiber, the technology is limited to relatively short runs in the neighborhood of 40 to 80 kilometers. As is typical with optics, what you gain in distance you give up on effective bandwidth and vice versa.


The move to DWDM does present a cost advantage. According to Dell'Oro analyst Jimmy Yu, the more you be can pack onto a single fiber strand, the less you need to spend on fiber leases to achieve a given amount of bandwidth.

迁移到DWDM确实会带来成本优势。根据Dell'Oro公司分析师Jimmy Yu的说法,在一根光纤上装的东西越多,为实现一定数量的带宽所需的光纤租赁费用就越少。

And because DWDM is already widely employed by major telcos AT&T and Lumen Technology, Nvidia says customers can now tap into existing fiber infrastructures, rather than needing dedicated fiber connectivity.

而且,由于主要的电信公司AT&T和Lumen Technology已经广泛采用DWDM,Nvidia称客户现在可以利用现有的光纤基础设施,而不需要专门的光纤连接。

### Cutting through the noise


MetroX-3 is part of a broader suite of hardware and software announced by Nvidia at the Supercomputing event this week aimed at addressing the growing volume of streaming data at the edge.


"By creating more high-fidelity research and instrumentation, that means that you're going to have to have a much more efficient way of capturing, analyzing and processing that data," Harris said. When "you're producing 50 to 1,000 times more data, how much do you keep? How much do you move back to the core? How much do you analyze."


In addition to connecting datacenters over InfiniBand, Nvidia is positioning MetroX-3, along with its Quantum-series switches and BlueField DPUs, as a means to extend InfiniBand networks to lab environments where the bulk of data is being generated. By doing so, the company says customers can use its Holoscan HPC framework running on IGX, DGX or HGX platforms at the edge to sift out meaningful data from the noise before funneling that refined dataset back to the core datacenter.

Nvidia除了用MetroX-3通过InfiniBand连接数据中心外,还将MetroX-3以及旗下的Quantum系列交换机和BlueField DPU定位为利用InfiniBand网络扩展那些产生大量数据实验室环境的一种手段。Nvidia表示,这样做以后,客户可以利用其在边缘的IGX、DGX或HGX平台上运行的Holoscan HPC框架,从噪音中筛选出有意义的数据,然后再将这些精炼过的数据集输送到核心数据中心。

Initially launched alongside Nvidia's IGX robotics and edge compute platform this fall, Holoscan AI inference was for medical imaging. However, the platform has since been repurposed for use on a variety of streaming data formats including non-image formats. Holoscan has also been reworked to support C++ and Python APIs, which Nvidia says researchers can use to develop custom data pipelines around their workflows. ®