Broadcom Thor Ultra: What Eight-Plane RDMA Tells Us About the Future of AI Ethernet
TLDR: Broadcom's Thor Ultra 800GbE NIC shows where AI Ethernet is heading: packet-level multipathing across as many as eight planes, out-of-order placement directly into accelerator memory, selective retransmission, and programmable congestion control. The important shift is that the NIC is becoming an active participant in fabric utilization rather than treating the network as a single ordered path.
Why this caught my attention
For years, a common mental model for RDMA over Ethernet has been straightforward: a flow is hashed onto a path, the network tries to keep queues under control, and the endpoints expect the transport to behave predictably.
That model becomes increasingly difficult as AI clusters grow to thousands or tens of thousands of accelerators. A collective can create many simultaneous elephant flows, and a perfectly healthy fabric can still perform poorly if a few flows collide on the same links while capacity sits idle elsewhere.
Broadcom's Thor Ultra is interesting because it moves several of those problems into the NIC itself.
At Hot Chips 2026, Broadcom disclosed more detail about the architecture behind its 800GbE Thor Ultra NIC. The enhanced RDMA design supports packet-level multipathing across up to eight planes, out-of-order data placement, selective retransmission, and programmable congestion control.
That is a much bigger architectural change than simply moving from 400G to 800G.
The traditional problem: one elephant flow, one ECMP path
Consider a simple leaf-spine fabric:
GPU A
|
NIC
|
Leaf 1
| | | |
S1 S2 S3 S4
| | | |
Leaf 2
|
NIC
|
GPU B
With conventional flow-based ECMP, a long-lived RDMA flow is normally hashed onto one path:
GPU A -> Leaf 1 -> Spine 2 -> Leaf 2 -> GPU B
Another large flow might hash onto the same spine even while Spine 1, Spine 3, and Spine 4 have available capacity.
This is the classic elephant-flow collision problem.
Increasing link speed helps, but it does not eliminate the underlying load-balancing problem.
Thor Ultra changes the model
Thor Ultra's enhanced RDMA architecture allows traffic belonging to the same communication relationship to use multiple network paths at packet granularity.
Conceptually:
+-> Plane 1 -+
+-> Plane 2 -+
GPU -> Thor Ultra NIC ---+-> Plane 3 -+--> Thor Ultra NIC -> GPU
+-> Plane 4 -+
| ... |
+-> Plane 8 -+
Instead of asking the network to find one good path for an elephant flow, the endpoint can distribute packets across several paths.
This can make much better use of a wide AI fabric.
But it immediately creates another problem:
Packets will not necessarily arrive in order.
Out-of-order placement is the key
Traditional transports often associate out-of-order delivery with expensive reordering or retransmission behavior.
For AI workloads, that can waste both network capacity and accelerator time.
Thor Ultra carries placement information that allows received RDMA data to be placed into the correct destination memory location even when packets arrive through different paths and in a different order.
A simplified example:
Sender
Packet 1 -> Plane 1 ----\
Packet 2 -> Plane 5 -----\
Packet 3 -> Plane 2 ------> Receiver
Packet 4 -> Plane 7 -----/
Arrival order:
3, 1, 4, 2
Memory placement:
[1][2][3][4]
The important distinction is that the network no longer has to preserve packet ordering just to make RDMA efficient.
That gives the fabric significantly more freedom to exploit parallel paths.
Selective retransmission matters just as much
Multipathing increases the importance of efficient loss recovery.
Imagine packets 1 through 100 are transmitted and packet 47 is lost.
A coarse recovery mechanism can cause a large amount of already-delivered data to be transmitted again.
Thor Ultra's enhanced RDMA design uses selective ACK/NACK mechanisms so that recovery can focus on missing data rather than replaying an unnecessarily large sequence.
Conceptually:
Received:
1 2 3 ... 45 46 -- 48 49 ... 100
Missing:
47
Retransmit:
47
At 800Gb/s, inefficient recovery becomes expensive very quickly. Selective retransmission is therefore not merely a reliability feature; it is a fabric-efficiency feature.
Congestion control is becoming programmable
Thor Ultra also supports programmable receiver- and sender-based congestion-control mechanisms.
Broadcom describes a receiver credit-based approach alongside support for network signals including ECN, packet trimming, and Congestion Signaling (CSIG).
This is another important evolution.
The traditional RoCE troubleshooting model often looks like:
Queue builds
|
ECN marking
|
CNP
|
Sender reduces rate
Modern AI Ethernet is moving toward something richer:
Fabric telemetry
|
+------------+------------+
| | |
ECN CSIG trimming
| | |
+------------+------------+
|
NIC congestion
algorithm
|
per-path traffic decision
The NIC is becoming part of the network control loop.
Eight planes are more interesting than 800G
The headline specification is an 800GbE NIC with PCIe Gen6 x16.
For network architects, however, the more interesting feature may be support for up to eight-plane multipathing.
A large AI fabric can be built as several independent planes:
GPU Server
|
Thor Ultra
|
+------+------+------+------+------+------+------+------+
| P1 | P2 | P3 | P4 | P5 | P6 | P7 | P8 |
+------+------+------+------+------+------+------+------+
| | | | | | | |
independent network paths / failure domains
This creates several useful properties:
- more aggregate path diversity;
- smaller failure domains;
- better utilization of available links;
- the ability to route around congestion or failures;
- less dependence on any single network path behaving perfectly.
The architectural direction is important: make the endpoints smarter and give them many paths.
What happens to PFC?
This does not mean that every existing RoCE deployment should immediately disable PFC.
Thor Ultra supports RoCEv2 as well as advanced RDMA capabilities, and production behavior depends on the actual end-to-end fabric, switch features, transport configuration, and workload.
But the broader direction of Ultra Ethernet is clear: reduce the assumptions inherited from traditional ordered, loss-sensitive RDMA and make Ethernet more tolerant of real large-scale network behavior.
That means relying more heavily on mechanisms such as:
multipathing
+ out-of-order placement
+ selective recovery
+ richer congestion signals
+ smarter endpoints
rather than trying to make a massive Ethernet network behave like a perfectly lossless wire.
What the performance numbers show
Broadcom's Hot Chips results provide an early indication that these mechanisms are not purely architectural ideas.
Reported RDMA-write throughput reached approximately 781 Gb/s unidirectionally and 1.558 Tb/s aggregate bidirectionally.
In a two-node collective test with eight GPUs per node and sixteen 400G links, Broadcom showed AllReduce around 383.9 GB/s and ReduceScatter around 380.2 GB/s against a stated 400 GB/s ceiling.
Those are vendor-presented early results rather than an independent large-cluster benchmark, so I would not use them to predict production performance at thousands of GPUs.
What they do demonstrate is that packet multipathing and out-of-order placement can coexist with very high RDMA and collective throughput.
What changes for the network engineer
This architecture changes troubleshooting.
In a traditional fabric, an engineer might start with:
GPU utilization
|
NIC counters
|
PFC
|
ECN / CNP
|
switch queues
|
optics
With endpoint-driven multipathing, the mental model becomes closer to:
Collective / GPU
|
GPU <-> NIC locality
|
RDMA transport
|
per-path state
|
packet spraying
|
plane utilization
|
ECN / CSIG / trimming
|
selective recovery
|
switch queues
|
optics
A network can have zero down links and zero obvious packet loss and still deliver poor collective performance because the traffic is not using the available paths efficiently.
That is why visibility into the NIC is becoming just as important as visibility into the switch.
Thor Ultra vs. the broader industry direction
Thor Ultra is one implementation, but the underlying ideas are appearing across several next-generation AI networking efforts.
The common architectural direction is:
Collective library
|
GPU / XPU
|
Smart NIC
|
+-------------+-------------+
| | |
Path 1 Path 2 Path N
| | |
+-------------+-------------+
|
Remote Smart NIC
|
GPU / XPU
The industry is gradually moving intelligence from a network that attempts to hide every imperfection toward endpoints that understand that multiple paths, congestion, reordering, and occasional loss are normal at AI-cluster scale.
That is a fundamental shift.
What I would learn from this
If you work on large-scale network engineering, the useful topics behind this announcement are not the product name itself.
Study these concepts:
- Flow ECMP vs. packet-level multipathing — why elephant-flow collisions waste fabric bandwidth.
- Out-of-order RDMA placement — how the receiver can tolerate different path latencies without forcing the network to preserve ordering.
- Selective retransmission — why retransmitting only missing data matters at 800G and beyond.
- Receiver- and sender-driven congestion control — and how ECN, CSIG, credits, and trimming can feed those algorithms.
- Multiplane topology — how independent planes provide both bandwidth and failure isolation.
- GPU/NIC locality — because a perfect network cannot compensate for a poor PCIe or NUMA data path.
Final thought
The most interesting thing about Thor Ultra is not that Broadcom built another faster NIC.
It is that the definition of a NIC is changing.
Old NIC:
Host -> packets -> network
AI NIC:
GPU
|
NIC
|-- transport
|-- multipathing
|-- placement
|-- loss recovery
|-- congestion control
|-- telemetry
|
AI fabric
At AI scale, the NIC is becoming part of the distributed system that decides how effectively thousands of expensive accelerators communicate.
For network engineers, that means the boundary of the job is moving again — from the switch, deeper into the server and closer to the GPU.