AI Computing Fabric Dual-Plane Network Architecture

Large Language Model (LLM) training places fundamentally different demands on the network than traditional cloud computing. As a result, conventional data center network architectures are not suitable for LLM training clusters. LLM training traffic is dominated by elephant flows, making ECMP prone to hash polarization and uneven traffic distribution. A dual-plane network architecture effectively mitigates this issue by distributing traffic across independent network planes. In addition, LLM training requires synchronized GPU operations for each training iteration, making the workload highly sensitive to single-point failures. A dual-plane architecture uses dual-homed NIC connections to provide high availability, allowing training to continue even if a Leaf switch or uplink fails.

This document focuses on the design of dual-plane AI GPU backend networks. It presents a standardized deployment solution based on a Clos fabric and a Rail-Optimized architecture, using Asterfusion 400G/800G high-density data center switches as the core networking platform.

This document is intended for solution architects and deployment engineers. Readers are expected to have the following knowledge:

• Familiarity with Asterfusion data center switches
• Basic understanding of EVPN Multi-Homing and RoCE technologies

As artificial intelligence continues to evolve, Large Language Model (LLM) training has introduced new challenges for data center networks. Compared with traditional cloud computing workloads, LLM training has fundamentally different traffic characteristics and networking requirements.

• AI Computing Network Evolution and Challenges
• Explosive scalability: GPU clusters have grown from thousands to tens of thousands of GPUs, with deployment scale continuing to expand.
• Traffic pattern shift: Elephant flows generated by collective communication operations such as All-Reduce and All-Gather dominate network traffic.
• Stringent performance requirements: Ultra-low latency, zero packet loss, and high throughput are essential for efficient AI training.
• Higher reliability requirements: A single point of failure can waste thousands of GPU resources and interrupt long-running training jobs.

Traditional converged data center network designs are prone to congestion and packet loss under high-density AI traffic, which significantly reduces training efficiency. A dual-plane architecture addresses these challenges by providing the following benefits:

• Eliminates hash polarization: Physically isolated forwarding planes remove ECMP hash collisions and improve traffic distribution.
• Improves fault tolerance: Dual-homed NIC connections allow training to continue even if a Leaf switch or uplink fails.
• Ensures training continuity: LLM training relies on synchronized GPU operations during every iteration, making it highly sensitive to single-point failures. A dual-plane architecture provides built-in network redundancy to maintain uninterrupted training.

Leaf switches connected to GPUs with the same index across different servers are grouped into a Rail. Each Rail provides connectivity for GPUs with the same ID through the corresponding Leaf switch. As shown in the figure below, GPUs on each server are numbered from 0 to 7, corresponding to Rail 1 through Rail 8. Intra-rail communication refers to traffic between source and destination GPUs whose corresponding NICs are connected to the same Leaf switch. LLM training uses hybrid parallelism, including data parallelism, tensor parallelism, and pipeline parallelism, to optimize traffic distribution. As a result, most traffic remains within the same server or within the same Rail.

The Rail-Only architecture uses a single-tier network design that physically divides the entire cluster into eight independent Rails. Communication between GPUs on different servers remains within the same Rail, allowing intra-rail traffic to reach its destination in a single hop.

Building on the Rail concept, a Group is defined as a basic deployment unit consisting of multiple Rails, Leaf switches, and GPU servers. As cluster size increases, additional Groups can be added horizontally to scale out the network and support larger GPU clusters.

A computing fabric can be compared to a railway system. Compute nodes act as stations that host GPU resources, while each Rail serves as a dedicated railway connecting GPUs with the same index across different servers. A Group represents a standardized deployment unit that combines multiple Rails and their associated Leaf switches. This modular design enables horizontal scaling while maintaining high-performance communication within each Rail.

As shown above, the Rail-Optimized architecture connects NICs with the same index on every server to the same Leaf switch. This design ensures that inter-server GPU traffic traverses the fewest possible network hops. GPU communication also leverages the local NVSwitch[1]fabric within each server, allowing traffic to reach the destination in a single network hop without traversing multiple switches, thereby minimizing additional latency.

Specifically:

1. Intra-server connectivity: Eight GPUs are connected to an NVSwitch through the NVLink interconnect, enabling low-latency communication within the server and reducing traffic on the scale-out network.
2. Server-to-network connectivity: All servers follow a consistent cabling scheme. NICs are connected to Leaf switches in sequence, for example, NIC1 to Leaf1, NIC2 to Leaf2, and so on.
3. Network fabric connectivity: Leaf and Spine switches are deployed in a full-mesh topology, forming a two-tier Clos fabric.

A dual-plane network architecture is designed to optimize connectivity for large-scale GPU clusters by deploying two identical and independent network planes. Each GPU is associated with two NIC ports, with each port connected to a different network plane, as shown in Figure 3. The two planes share the same topology but have no cross-connections between them.

This design significantly reduces the probability of ECMP hash polarization and improves traffic distribution efficiency. It also enhances network resilience and reliability. If a device or an entire network plane fails, the remaining plane continues to carry traffic, allowing the cluster network to remain operational.

The dual-plane architecture partitions the network into two fully independent forwarding planes (Plane 1 and Plane 2). Each plane contains a complete Clos topology, including Leaf and Spine layers. The two planes are physically isolated with no interconnection, ensuring strict independence of forwarding domains and effective fault isolation.

In a dual-plane network architecture, each NIC is configured with two uplink ports connected to two separate Leaf switches. This forms a dual-homed Leaf design, similar to a stacked dual-Leaf topology.

In traditional data center networks, MC-LAG (Multi-Chassis Link Aggregation Group) is commonly used to implement a dual-Leaf stacking architecture. In such a design, the two Leaf switches are interconnected through a direct link. This interconnect is critical for synchronizing data-plane states such as ARP and MAC entries.

This approach reduces service disruption caused by single Leaf failures in production environments. However, the inter-Leaf link must also handle traffic forwarding during failure scenarios, which consumes significant switch bandwidth.

Asterfusion provides two implementation options for the dual-Leaf design in a dual-plane architecture:

• One relies on NIC-based ARP dual-replication capability
• The other relies on EVPN MH (EVPN Multi-Homing) support on the switch side

The core of this implementation is the removal of the physical interconnection between the two Leaf switches. Instead, coordination is achieved through the combined behavior of NICs and switches to provide a stacking-like high-availability mechanism.

In this design, server NICs operate in bond mode with dual uplinks, connecting to Leaf switches in two separate planes. The two Leaf switches are not physically stacked and do not use dedicated inter-switch synchronization links.

The key challenge is maintaining consistent ARP processing and MAC learning across both Leaf switches without a direct interconnect, ensuring seamless traffic failover.

This is achieved through the following mechanisms:

• NIC-based ARP duplication: The NIC must support sending ARP requests from both uplink ports simultaneously. This ensures that both Leaf switches learn identical MAC and IP bindings from the server.
• Switch-side ARP proxy and ART-to-Host: Leaf switches enable ARP proxy functionality and convert learned ARP entries into /32 host routes (ARP-to-Host Route). These routes are then advertised across the fabric via BGP.

As a result, all switches in different planes can maintain a complete and consistent set of host routes for server NICs.

EVPN MH (EVPN Multi-Homing) is an all-active multi-homing VXLAN gateway solution defined in RFC 7432, using BGP EVPN as the control plane. Similar to MC-LAG, EVPN MH provides high availability for access-layer connectivity, enabling load balancing and fast failover.

Compared with traditional MC-LAG, EVPN MH is implemented based on open standards, which improves interoperability and compatibility. It also removes the need for direct physical interconnects between active devices, making the architecture easier to scale.

In this design, two Leaf switches act as EVPN PE nodes. They establish BGP EVPN sessions over a management network or a dedicated AUX 10G link (non-tenant traffic). Both Leaf switches jointly serve the same GPU server group.

The key mechanism is the Ethernet Segment Identifier (ESI), which identifies the two physical links connecting a server to the Leaf pair. This enables multi-homing of a single Ethernet segment.

The workflow is as follows:

The server NIC sends ARP broadcasts, distributed by bond load balancing to one of the planes. The receiving Leaf switch floods the ARP broadcast over the VXLAN tunnel. The remote Leaf in the other plane receives the flooded traffic and learns the ARP entry. Both Leaf switches then convert the learned ARP entries into /32 host routes (ARP-to-Host Route), and advertise them to the entire fabric via BGP.

ECMP (Equal-Cost Multi-Path) per-flow load balancing is one of the most widely used routing strategies in data center networks. It uses fixed packet fields such as source/destination MAC addresses and the IP five-tuple as hashing inputs. A hash function generates a value that determines the selected path among multiple equal-cost routes. This approach is also referred to as static load balancing, as forwarding decisions are based on packet-level attributes.

However, per-flow load balancing can lead to uneven traffic distribution when flow characteristics are skewed. This issue becomes more severe with elephant flows. A single congested member link may degrade overall performance, causing packet loss and reduced throughput.

Deep learning workloads rely heavily on collective communication operations such as All-Reduce, All-Gather, and Broadcast. These operations generate dense inter-GPU traffic, often reaching multi-terabit per second (Tbps) scale. In addition, collective communication introduces a strong “straggler effect”. Even a single congested path can delay the entire training iteration, amplifying performance degradation. As a result, traditional per-flow ECMP is not suitable for AI computing networks.

To address these limitations, this document introduces three alternative load balancing techniques to replace traditional per-flow ECMP:

• Flowlet-based load balancing
• Intelligent path selection
• Packet spraying

ARS (Adaptive Routing and Switching) is a flowlet-based load balancing mechanism. It leverages ASIC-based hardware ALB (Auto-Load Balancing[2]) capabilities to achieve near per-packet load distribution while minimizing packet reordering.

The mechanism splits flows with identical hash results into multiple flowlets based on time gaps. Each flowlet is treated as an independent scheduling unit. The system continuously monitors link metrics such as bandwidth utilization and queue depth. Flowlets are then dynamically assigned to less congested paths, improving overall network utilization.

Intelligent path selection is classified into dynamic and static modes, depending on the load balancing mechanism.

Dynamic intelligent path selection is a telemetry-driven approach. It evaluates key metrics including bandwidth usage, queue occupancy, and forwarding latency to determine path quality. Bandwidth and queue statistics are collected from ASIC hardware registers with sub-second accuracy. Forwarding latency is measured using INT (In-band Network Telemetry), providing nanosecond-level precision.

Each switch continuously evaluates path quality. This information is propagated through BGP extended attributes. Combined with per-flow dynamic WCMP (Weighted Cost Multipath), traffic is steered to the optimal path. This reduces congestion and improves bandwidth efficiency.

Static intelligent path selection uses predefined policies. Traffic from GPUs to different Leaf downlink interfaces is isolated and steered using Policy-Based Routing (PBR). Traffic is redirected to specific Leaf uplinks and forwarded to designated Spine devices. This achieves 1:1 convergence-based uplink load balancing.

This method tightly binds traffic to physical paths. It is suitable for environments with stable traffic patterns and strict path determinism requirements.

Packet spraying[3] is a per-packet load balancing technique that distributes packets across multiple links to avoid single-path congestion.

Two algorithms are commonly used:

• Random: packets are distributed randomly across available links
• Round Robin: packets are distributed sequentially and evenly across links

While per-packet load balancing can maximize theoretical link utilization, it introduces a key challenge in real deployments. Different link latencies can cause packet reordering at the receiver, which degrades overall performance.

Therefore, packet spraying requires strong hardware support. High-performance NICs must provide packet reordering capabilities at the endpoint to mitigate out-of-order delivery effects.

The diagram above shows a 400G AI computing dual-plane Rail-Optimized network design for a 256-node cluster (2,048 GPUs). The deployment uses 48 CX864E-N switches, including 16 Spine switches and 32 Leaf switches. The network is divided into two independent planes. Each plane contains 16 Leaf switches and 8 Spine switches.

The key design principles are as follows:

• Each GPU is connected to a dedicated 400G NIC. Each NIC exposes two ports, which connect to two separate planes. The first NIC port follows a fixed mapping rule, such as NIC1-Leaf1, NIC2-Leaf2, and so on, and connects to Leaf switches in Plane 1. The second NIC port follows the same mapping rule and connects to Leaf switches in Plane 2.
• Each plane is built as a two-tier Clos architecture. Spine and Leaf switches are fully interconnected within each plane. IPv6 link-local addresses are used to establish unnumbered BGP adjacencies. This enables route exchange for each Rail without requiring IP address planning on Leaf–Spine links.
• The ratio between Leaf downlink capacity and uplink capacity is strictly maintained at 1:1 to ensure a non-blocking design.
• Leaf and Spine switches enable one-click RoCE optimization and load balancing features to build a lossless network.

For building a large-scale 400 Gbps RoCEv2 network, Asterfusion data center switches CX864E-N and CX732Q-N are recommended. The design is based on the ultra-low forwarding latency of both platforms. CX864E-N provides an end-to-end forwarding latency as low as 560 ns, while CX732Q-N can reach 500 ns.

This enables intra-rail communication latency of approximately 600 ns. For inter-rail traffic (Leaf–Spine–Leaf, three-hop forwarding), end-to-end latency is controlled within 2 μs, fully meeting the strict latency requirements of RoCEv2 networks.

In a dual-plane Rail-Optimized design, the number of Leaf switches in a single Group is related to the number of GPUs per server, which defines the number of Rails. Using an NVIDIA DGX H100 GPU server (8 GPUs per node) as an example, one Group requires 16 Leaf switches, corresponding to 8 Rails.

The maximum number of servers per Group is constrained by the Leaf interface configuration. To maintain a 1:1 oversubscription ratio, half of the Leaf interfaces are used for GPU server connectivity, while the other half connect to Spine switches. Therefore, the maximum number of GPU servers per Group is equal to half of the available Leaf interfaces.

The following table summarizes the required node configurations for deploying dual-plane networks with different GPU scales using CX864E-N and CX732Q-N switches.

With the rapid development of artificial intelligence, AI computing networks are undergoing unprecedented transformation. This solution systematically presents the design principles and technical implementation of a dual-plane AI computing network architecture, providing a complete approach for building large-scale GPU cluster networks.

The proposed architecture effectively supports dual-plane network deployments across different AI cluster scales. For detailed configuration examples and implementation cases, please refer to the relevant best practices.