DESIGN 03 / 05 — KUBERNETES + PHYSICAL NICs + RoCE (HIGH-LEVEL HPC ARCHITECTURE)
Best for: Smaller clusters, simpler networking, teams avoiding SR-IOV complexity [cite: Overview]
4.ML/AI APPLICATIONS
& TRAINING[cite: L7, L12-L13, L15]
📦
[cite: L7]
NCCL in Container Images
Bake NCCL into Docker
No SR-IOV — simpler setup
→
Pod
GPU
GPU
→
🧮
[cite: L12]
Training Code Calls NCCL
Backward pass triggers
all-reduce across GPUs
→
🔍
[cite: L12.5]
NCCL Rank Discovery
Via Kubernetes env vars
RANK, WORLD_SIZE
→
🌐
[cite: L13]
Initialize Comm Topology
Ring or Tree all-reduce
over physical NICs
→
✅
[cite: L15]
Training Job Completes
Pods tear down, GPUs
released, checkpoint saved
⬇
3.KUBERNETES
ORCHESTRATION[cite: L3.5, L5, L9, L10]
☸️
[cite: L5]
Install Kubernetes
kubeadm init
Master + Worker nodes
→
📡
[cite: L3.5]
GPU Resource Tracking
NVIDIA Device Plugin
DaemonSet per node
→
☸️
[cite: L5]
Cluster Control Plane
(Master Node)
Kubelet
Kubelet
→
📋
[cite: L9]
Create Pod Specs
GPU requests + Macvlan
network annotation
→
📌
[cite: L10]
Scheduler Places Pods
Based on GPU availability
from device plugin
⬇
2.NETWORKING LAYER
(RoCE FABRIC)[cite: L4, L6, L11, L13, L14]
🔌
[cite: L4]
Macvlan / IPVLAN CNI
Virtual interfaces on
physical NIC — no SR-IOV
No Multus needed!
→
🔀
[cite: L6]
Ethernet Switches
Leaf-Spine Topology
PFC + ECN + QoS
⟶
🖧
[cite: L11]
Physical NIC Paths to Pods
Macvlan IPs assigned
backed by physical NIC
⟶
🔗
[cite: L13]
NCCL Creates RDMA Queue Pairs
Over physical NICs
No VF overhead
⟶
800 Gbps
800 Gbps
RDMA
⚡ DIRECT RDMA PATH
⬇
1.INFRASTRUCTURE
& BARE METAL[cite: L1-L3, L8]
🐧
[cite: L1]
Install Linux OS
Ubuntu 22.04 LTS
Directly on hardware
→
B300 Servers
→
🎮
[cite: L2]
Install NVIDIA Drivers & CUDA
System-wide CUDA
8 GPUs per server
→
🔌
[cite: L3]
Configure Physical NICs
800G NICs with IPs
NO SR-IOV — simpler!
STORAGE NETWORK
[cite: L8]
🗂 Shared Storage
NFS / Lustre
Object Storage
Checkpoints & Datasets
DESIGN 03 / 05 · KUBERNETES + PHYSICAL NICs + RoCE · HPC RoCE Design Patterns