- Improve Token Generation Rate
Token Generation Rate (TGR) increases steadily with the number of concurrent users. When reaching 100 users, TGR improves by 27.5% compared to InfiniBand. - Reduce Inference Latency
By minimizing leaf-to-leaf network forwarding latency, the average inference time per token is reduced by 20.4% compared to InfiniBand. - Congestion Free
INT-driven adaptive routing, packet spray, flowlet-based auto load balancing, and WCMP work together to effectively prevent congestion across the fabric. - Improve GPU & Network Utilization
With the above technologies, the network achieves up to 97% utilization, directly enhancing GPU utilization during parallel computing workloads.
Traffic
During MoE model inference, traffic types such as All-to-All, Prefill-Decode Disaggregation, and Pipeline Parallelism are intelligently distinguished and routed using In-band-Network-Telemetry-driven, topology-aware optimization, which incorporates differentiated QoS priorities and ECMP group selection to achieve a balance between high throughput and low latency — ultimately enhancing network utilization.
Under different concurrent inference request scenarios (ranging from 20 to 100), the inference latency when using Asterfusion’s RoCE switch is consistently lower than that when using an InfiniBand (IB) switch. Specifically, with 50 concurrent requests, the 90th percentile inference latency is reduced by 20.4%.
For 20–100 concurrent inference requests, Asterfusion’s RoCE switch consistently delivers a higher Token Generation Rate (TGR) than InfiniBand (IB) switches. The growth margin widens with more concurrent requests, reaching a 27.5% TGR improvement at 100 requests.
