Understanding AI Ethernet: From Switch Buffers to Adaptive Routing and SuperNIC Intelligence

TLDR: AI Ethernet performance depends on coordinated buffering, adaptive routing, congestion control, and SuperNIC transport capabilities—not one switch specification.

Exploring packet forwarding, congestion control, out-of-order delivery, and network reliability in modern GPU fabrics.

A conversation that made me rethink Ethernet

I recently had the opportunity to speak with Gilad Shainer, Senior Vice President of Networking at NVIDIA, and Alex Shapiro, Senior Manager at NVIDIA, about the evolution of Ethernet for AI infrastructure. We covered switch buffering, adaptive routing, packet reordering, SuperNIC capabilities, and how the network shapes GPU communication performance.

One question stayed with me afterward:

What if the best way to handle congestion isn't to build a larger queue, but to avoid creating that queue in the first place?

As a network engineer, I'm used to evaluating switches on port speed, ASIC capacity, radix, forwarding performance, and buffer architecture. AI networking adds another dimension: how effectively the network keeps thousands of GPUs communicating in lockstep.

Inspired by our conversation, I spent time working through NVIDIA's publicly documented Spectrum-X architecture, along with the newer Multipath Reliable Connection (MRC) transport. Here's what I learned.

1. Four packets enter an AI network

Imagine a GPU training cluster connected through a leaf-spine Ethernet fabric. A source GPU sends four packets to another GPU: P1 → P2 → P3 → P4. The leaf switch has four equal-cost paths through four spine switches.

In a conventional flow-based ECMP design, a hash of the packet headers decides which path a flow uses. Packets in the same flow follow the same path, which preserves ordering. That is an important and useful property.

Now imagine two large RDMA flows hash onto the same spine. That path congests while the other three still have bandwidth to spare, and our packets wait in the congested egress queue. The fabric has capacity, but static ECMP gives these flows no way to use it.

AI training makes this worse than typical data center traffic. It produces a small number of very large, long-lived flows with little entropy for the hash to work with, so collisions are both more likely and more costly.

Can the fabric make better use of available paths without breaking the receiving transport's expectations?

2. Understanding switch buffers

Switch buffers temporarily hold packets when incoming traffic exceeds an outgoing port's capacity.

For example, if 800 Gbps of traffic arrives toward a 400G egress port for 100 microseconds, approximately 5 MB of excess traffic needs to be buffered to avoid packet loss, assuming traffic is not reduced or redirected.

Quick note: Deep vs. shallow buffers

Deep-buffer switches provide more memory to absorb larger bursts, while shallow-buffer switches have less buffering capacity. Both are valid designs depending on workload requirements and congestion-management mechanisms.

What if another eligible path has available bandwidth? Could the fabric use that capacity instead of allowing unnecessary queue buildup?

This brings us to adaptive routing—and the challenge of packet ordering.

3. The packet-ordering problem

Back to our four packets, sent as P1 → P2 → P3 → P4. Suppose the leaf switch spreads them across all four spines: P1 through Spine 1, P2 through Spine 2, and so on. Each path has a different queueing delay, so the destination might receive P3 → P1 → P4 → P2.

Everything arrived, just not in order. That is packet reordering, and reordering is not the same as loss.

The trouble is how transports react to it. Depending on the transport implementation, out-of-order delivery can trigger unnecessary recovery or retransmission. That wasted bandwidth is exactly why conventional load balancing keeps each flow on one path.

But what if the receiving endpoint were built to handle reordering efficiently? That is where the switch-and-SuperNIC relationship comes in.

4. NVIDIA Spectrum-X: when the switch and SuperNIC cooperate

NVIDIA Spectrum-X is an Ethernet platform that pairs Spectrum switches with NVIDIA SuperNICs (BlueField-3 and, in newer deployments, ConnectX-8). NVIDIA now refers to its native transport model on this platform as Spectrum-X Ethernet Adaptive RDMA.

One of its documented capabilities is fine-grained adaptive routing. Instead of pinning every packet in a flow to one path, supported Spectrum-X configurations make per-packet forwarding decisions based on local congestion. If one eligible spine-facing port has a longer queue, the switch picks another eligible port with a shorter one. NVIDIA describes this as a hardware approximation of a Join-Shortest-Queue algorithm.

Different paths, of course, mean packets can arrive out of order. That is where the SuperNIC matters. In NVIDIA's documented architecture, the switch performs the adaptive routing, and the receiving SuperNIC handles out-of-order RoCE delivery: it places each packet's data directly into the correct location in memory as it arrives, regardless of order, without treating reordering alone as packet loss.

Conceptual packet journey

1. Source GPU hands data to the source SuperNIC
2. Spectrum leaf switch receives the packets
3. Adaptive routing sprays them across eligible spine paths by queue depth
4. Destination Spectrum leaf switch forwards them toward the receiver
5. Destination SuperNIC handles out-of-order RoCE and places data in memory
6. Destination GPU consumes the complete message

The key engineering principle is coordination between the network and its endpoints. The switch gains freedom to choose paths because the receiving transport is designed to tolerate the reordering that results.

The important idea is not just what the switch can do, but how the switch and SuperNIC work together.

5. Why adaptive routing needs congestion control

Now consider a different pattern. Four source GPUs each send 400 Gbps toward a receiver whose interface accepts only 400 Gbps. Offered load is 1.6 Tbps; receive capacity is 400 Gbps.

However many spine paths exist, all that traffic converges on the same final link. No path-selection algorithm can make a destination receive four times its physical capacity. This is incast congestion, and adaptive routing alone cannot solve it. The senders must slow down.

The role of congestion control

This is why Spectrum-X pairs adaptive routing with congestion control. NVIDIA describes a telemetry-based approach: Spectrum switches feed in-band congestion information back toward the endpoints, and the sending SuperNICs combine it with signals such as ECN marks and RTT measurements to adjust their injection rate.

The distinction is worth stating plainly:

• Adaptive routing asks: which eligible path should this packet take?
• Congestion control asks: how fast should the sender inject packets into the network?

The two complement each other. Adaptive routing avoids localized hotspots through path choice. When no alternate path can relieve the bottleneck, sender rate reduction is the only fix. Buffers absorb the burst while that feedback takes effect, and depending on configuration, Ethernet flow control and transport recovery provide further protection.

The goal is not to eliminate congestion, which is impossible, but to manage it so workload performance stays predictable.

6. Three hardware control loops

NVIDIA's published Spectrum-X material describes three hardware-accelerated control loops that operate at different scopes.

Loop	Where it runs	What it decides	Problem it addresses
1. In-switch adaptive routing	Spectrum switch	Which eligible egress port each packet takes, by local queue depth	Hash collisions and local hotspots
2. Targeted congestion control	Sending SuperNIC, fed by switch telemetry	How fast the sender injects traffic	Sustained bottlenecks such as synchronized incast
3. SuperNIC plane load balancing	Sending SuperNIC	Which independent network plane carries the traffic	Imbalance across planes in multiplane fabrics

The switch manages local path selection, congestion control manages the sending rate, and in supported multiplane designs the SuperNIC spreads traffic across independent planes using local queue state and end-to-end congestion information. Each loop handles what the others cannot, which is the real value of coordinating network and endpoint behavior.

7. What about buffer-to-buffer credits?

NVIDIA InfiniBand uses link-level, credit-based flow control. The sender tracks how much buffer space the next hop has free and transmits only within those credits, so the link is lossless by construction.

Spectrum-X Ethernet takes a different approach. It does not implement InfiniBand-style credits as its standard mechanism. Instead, it combines adaptive routing, telemetry- and ECN-driven congestion control, standard Ethernet flow control where configured, and transport recovery. MRC, covered in Section 10, takes yet another approach and is designed to operate without PFC or a lossless fabric.

These mechanisms solve different problems:

Mechanism	Primary purpose
Switch buffering	Temporarily absorb traffic bursts
Credit-based flow control	Regulate transmission by downstream buffer availability
Adaptive routing	Select less-congested eligible paths
Telemetry/ECN-based congestion control	Signal congestion so senders adjust their rates
PFC	Pause selected Ethernet priorities hop by hop
Out-of-order handling	Process packets that arrive in a different order
Retransmission	Recover missing data

This is why AI Ethernet performance depends on far more than buffer capacity alone.

8. When the link is UP but the AI job slows down

Imagine a training job running across thousands of GPUs. Everything has been normal, then NCCL bandwidth starts dropping. You check the network: interfaces are UP, BGP sessions are established, and there is no obvious outage. Yet communication phases keep taking longer.

One likely culprit is a fail-slow condition. An optical link can degrade at the physical layer without ever going DOWN.

The failure usually unfolds in stages. Forward error correction (FEC) first corrects rising bit errors silently, so the pre-FEC bit error rate climbs while traffic looks healthy. Once errors exceed what FEC can correct, frames fail their CRC and are dropped. The transport then retransmits, and effective throughput falls. Watching pre-FEC BER trends gives operators an early warning before the job ever notices.

This is where telemetry becomes essential. NVIDIA provides telemetry across Spectrum switches, SuperNICs, and its networking software, letting operators correlate physical errors, congestion, retransmissions, and communication performance. NVIDIA has publicly described a case on its Israel-1 supercomputer where degraded bandwidth was traced through network telemetry to symbol errors on a spine-switch port.

Network availability and network performance are not the same thing. A link being UP does not guarantee the workload is getting the bandwidth it needs.

9. Connecting network telemetry to NCCL performance

NCCL collectives such as AllReduce, AllGather, and ReduceScatter coordinate data movement among GPUs. If a single rank or host suffers network degradation, the whole collective slows down with it. That makes workload-level observability essential, and tools such as NVIDIA's NCCL Inspector help engineers see communication behavior from the application side.

When troubleshooting a slow collective, these are the questions I would want answered:

• Is the slowdown affecting every rank, or only a subset?
• Did RoCE retransmissions increase?
• Are any links reporting rising pre-FEC BER or post-FEC errors?
• Is one network plane more congested than the others?
• Are ECN marks, CNPs, or PFC pauses increasing?
• Did the degradation line up with a maintenance event?

These signals connect the application's symptoms to the underlying network condition. Detection and remediation are separate capabilities, though. Finding a degraded path does not guarantee transparent recovery; that depends on the transport, the fabric configuration, and the nature of the failure.

10. An important evolution: Multipath Reliable Connection (MRC)

MRC is a multipath RDMA transport technology described publicly by NVIDIA, Microsoft, and OpenAI. NVIDIA has discussed its use with Spectrum-X Ethernet in large AI infrastructure.

MRC is designed to let a single reliable RDMA connection use multiple network paths. Its documented capabilities include:

• Multipath load balancing within a single connection, steering around congested paths during a running job
• Multiplane support, splitting one high-speed NIC's bandwidth across several independent Ethernet planes
• Selective retransmission, resending only what was actually lost
• Fast failure bypass, rerouting around failed paths in supported hardware
• No dependency on PFC or lossless Ethernet behavior

This is a different layer of the problem from Section 4. Switch-level adaptive routing decides where eligible packets are forwarded. Transport-level reliability ensures missing data is recovered correctly. MRC moves multipath intelligence and path control into the transport itself, governed largely by the host.

It should not be confused with switch-level Spectrum-X adaptive routing; the technologies operate at different layers.

That MRC is open, multi-vendor, and backed by hyperscale operators is itself a signal. Ethernet for AI is evolving across the whole communication stack, and NVIDIA has publicly described MRC deployments alongside Spectrum-X.

11. What would I validate in a real AI fabric?

Understanding the architecture is only the beginning. The next step is validating how it behaves under realistic traffic. These are the scenarios I would prioritize:

Scenario	How to test it	What to measure
Path imbalance	Large RDMA flows contending for one path while alternates have spare capacity	Throughput, per-port queue occupancy
Packet reordering	Exercise fine-grained adaptive routing toward a supported SuperNIC	Data integrity, out-of-order counters, retransmissions
Synchronized incast	Simultaneous senders toward one constrained receiver	Congestion feedback, rate adjustment, queue growth
Microbursts	Short high-rate bursts	Buffer behavior, tail latency
Fail-slow optics	Controlled physical-layer impairment in a lab	Pre-FEC BER, telemetry alerts, retransmissions, effective bandwidth
NCCL communication	Representative AllReduce and AllGather workloads	Collective bandwidth correlated with network counters
Path and plane failures	Take part of the fabric down mid-job	Traffic redistribution, recovery time, workload impact
Transport comparison	Same workloads under Adaptive RDMA and MRC, where both are supported	Throughput, tail latency, failure recovery

All of these answer one question: does the network keep GPU communication predictable when traffic gets hard or infrastructure starts to degrade?

Final thoughts: AI Ethernet is a systems problem

My biggest takeaway from the discussion with Gilad and Alex was the importance of seeing the complete communication system. Switch buffers matter. So do forwarding algorithms, SuperNIC capabilities, congestion control, packet ordering, telemetry, and transport recovery. Each addresses a different part of the problem, and their interactions determine how the fabric behaves.

What makes Spectrum-X interesting to study is how its switches and SuperNICs coordinate these capabilities. The evolution of transport technologies such as MRC further illustrates this systems-level approach.

High-performance AI networking is not about a single feature or specification. It is about how the complete system manages traffic, congestion, reliability, and application performance together.

That's the perspective I hope to carry into future AI fabric design and validation work.

Acknowledgment

I would like to express my sincere appreciation to Gilad Shainer, Senior Vice President of Networking at NVIDIA, and Alex Shapiro, Senior Manager at NVIDIA, for taking the time to share their insights and perspectives on the evolution of Ethernet for AI infrastructure.

A special thank you to Alex for discovering LosslessNetwork.com and reaching out, which led to this valuable conversation.

It was a pleasure discussing AI networking, adaptive routing, and SuperNIC technologies with both of you. Thank you so much for your time, thoughtful insights, and engaging discussion. I truly appreciate the opportunity to learn from your perspectives.

This article reflects my independent technical understanding and research based on publicly available information. The views expressed are my own and do not represent an official NVIDIA position or endorsement.

References and further reading

1. NVIDIA — Giga-Scale AI and the Ethernet Evolution
2. NVIDIA — Turbocharging AI Workloads with Spectrum-X
3. NVIDIA — Next-Generation AI Factory Telemetry
4. NVIDIA — Spectrum-X Ethernet and MRC
5. HPCwire — NVIDIA Highlights OpenAI, Microsoft Deployments of Spectrum-X Ethernet with MRC
6. SiliconANGLE — Nvidia's MRC: When 'just Ethernet' isn't enough for gigascale AI
7. Converge Digest — MRC Redesigns Ethernet for GPU AI Clusters
8. HPCwire — Networking in the AI Age: Gilad Shainer Interview
9. AI Factory Networking Discussion Featuring Gilad Shainer