Skip to main content

Load Balancing Playground

Page 4.1 explained the four LB tiers in prose. This page is the hands-on version — pick a method, toggle between web and AI traffic, fail a spine link, and watch what each method actually does on a 4-spine fabric.

How to use it:

  1. Start with ECMP + Web traffic — flows balance across all four spines. The default everyone assumes. ✔
  2. Switch to AI training — same ECMP, same fabric. Watch the polarization. Six of eight elephants land on Spine 0. Throughput collapses.
  3. Try Flowlet DLB + AI traffic — the ASIC notices the load, reassigns flows at flowlet boundaries, all four spines pull load again.
  4. Try Packet Spraying (TELB) — every packet picks its own path. Maximum spread; needs hardware reorder buffer on the receiver side. This is what AWS SRD / Google Falcon / MRC / UEC do.
  5. Fail a spine — watch how each method reacts. ECMP needs BGP to rebuild the group (seconds). DLB sees the queue go infinite and reassigns instantly.

Load Balancing Simulator

Pick a method. Pick a traffic pattern. Watch what happens.

SourceLeafDestLeafSpine 0Spine 1Spine 2Spine 3
Per-spine utilization
Spine 0
6.0 flows
Spine 1
2.0 flows
Spine 2
0.0 flows
Spine 3
0.0 flows
Polarization
3.00×
max ÷ avg load
Effective throughput
50%
of theoretical 4× spine
SLBECMPon AI training

RoCE v2 collapses 4 of the 5 hash fields (same src/dst IPs, same UDP dest port 4791). Six of eight elephants land on Spine 0. The fabric runs at ~30% effective throughput while three spines sit idle.

Inject failure:

What to take away​

ObservationWhat it means
ECMP works for web traffic and fails for AI trafficThe 5-tuple hash needs entropy. RoCE v2 has almost none.
DLB rebalances within ~1 msSwitch silicon sees its own queue depth. Adaptive routing pays for itself.
GLB sees one hop furtherCatches downstream congestion DLB can't see locally.
TELB / packet spraying is the long-term answerPer-packet spread + HW reorder. UEC / MRC / SRD / Falcon.
Failure recovery is a different problemBFD / BGP / IP FRR react to dead paths; they don't balance load.

Once you've played with the simulator, the prose on page 4.1 should read as a deeper explanation of what you just saw.

Next: Cluster Sizing & Cabling → — back to chapter 3. Port counts, optics, cable lengths, and the BOM math that turns the design you've learned into a build plan.