Skip to main content

HPC Networking

Three names you'll hear together — and they get used interchangeably, which is wrong.

  • RDMA is the technique: kernel bypass, direct memory writes between hosts, NIC does the work the CPU used to do.
  • InfiniBand is one wire protocol that implements RDMA. Native, lossless by design, ~99% NVIDIA.
  • RoCE v2 is the other wire protocol — the InfiniBand transport packaged inside standard UDP/IP/Ethernet. The fabric most enterprises (and this curriculum) pick.

Watch the animation first. Then read the three sections that follow for the depth.

What's next​

  • 06. RDMA — the technique. Kernel bypass, verbs, queue pairs, memory regions.
  • 07. InfiniBand — the native RDMA fabric. Credit-based flow control, Subnet Manager, LID/GID addressing.
  • 08. RoCE v2 — IB transport over standard Ethernet. PFC + ECN + DCQCN — the lossless trick.