Table of Contents
Introduction
In modern data center networks and multi-tenant environments, VXLAN (Virtual eXtensible LAN) has become a key technology for providing flexible Layer 2 connectivity. VXLAN is a MAC-in-UDP encapsulation protocol over IP, designed to deliver seamless Layer 2 connectivity over IP networks, allowing workloads to migrate freely across different physical servers and data centers without disrupting the existing Layer 2 network topology. While VXLAN handles unicast traffic straightforwardly, it faces significant challenges when transporting BUM traffic (Broadcast, Unknown Unicast, Multicast), which is precisely why the VXLAN Multicast BUM Traffic Forwarding Method was developed.
Ⅰ. What is BUM Traffic
In a VXLAN/EVPN environment, in addition to regular unicast traffic, there is VXLAN BUM traffic (introduced in IEEE RFC 8365), which stands for Broadcast, Unknown Unicast, and Multicast:
Broadcast: Traffic destined for all hosts in the network.
Unknown Unicast: Unicast traffic whose destination MAC address has not yet been learned.
Multicast: Traffic intended for multiple receivers subscribed to a specific multicast group.
Collectively referred to as BUM traffic, these types share the characteristic of having either an unclear destination or multiple receivers. In traditional Layer 2 networks, such traffic often leads to broadcast flooding, wasting bandwidth and increasing the load on the source device. Particularly in large-scale VXLAN data centers, if uncontrolled, a source VTEP (VXLAN Tunnel Endpoint) may need to replicate BUM traffic individually for each remote VTEP, resulting in high uplink bandwidth usage, uneven latency, and potential network congestion.
Therefore, efficiently and reliably forwarding VXLAN BUM traffic is the primary application scenario for the VXLAN Multicast BUM Traffic Forwarding Method.
Ⅱ. Two VXLAN Multicast BUM Traffic Forwarding Methods
The VXLAN Multicast BUM Traffic Forwarding Method primarily has two implementation approaches: Ingress Replication (IR) and Point-to-MultiPoint multicast transport (P2MP / Multicast Underlay).
Ingress Replication
Ingress Replication (IR) is the simplest method for forwarding VXLAN BUM traffic.
In this model, each VTEP maintains information about which remote VTEPs are interested in which VXLAN segments through the EVPN control plane using Type-2 MAC/IP Advertisements. This allows each VTEP to generate a list of relevant remote VTEPs. In some cases, the list can also be maintained via static configuration.
When the source VTEP needs to send BUM traffic, it consults this ingress replication list. The list contains all remote VTEPs subscribed to the specific VXLAN segment. The source VTEP then encapsulates the original BUM traffic into VXLAN packets and generates a separate packet for each remote VTEP. These packets are sent individually via unicast over the underlay network.
From a bandwidth and efficiency perspective, this approach requires the source VTEP to replicate the traffic itself so that each remote VTEP receives an independent copy. As a result, the uplink bandwidth consumption at the source VTEP increases linearly with the number of remote VTEPs. Therefore, this model is most suitable for low to medium bandwidth BUM traffic scenarios and is a key component of the VXLAN Multicast BUM Traffic Forwarding Method.
As shown in the figure, if a 1 Mbps BUM flow needs to be sent to multiple remote VTEPs—for example, VTEP1 sending BUM traffic to VTEP2, VTEP3, and VTEP4—then the uplink port of VTEP1 would consume 4 Mbps of bandwidth.
P2MP/Multicast Underlay
For high-bandwidth or low-latency scenarios, Ingress Replication (IR) may not be efficient enough. The Point-to-Multipoint (P2MP) model optimizes VXLAN multicast traffic forwarding by leveraging the multicast capabilities of the underlay network. This approach is another key implementation of the VXLAN Multicast BUM Traffic Forwarding Method.
In this approach, the underlay network runs PIM, and all receiving VTEPs join the multicast group advertised by the source VTEP. Downstream switches and routers maintain the multicast state and construct the Shortest Path Tree (SPT) for forwarding.
The source VTEP sends a single VXLAN-encapsulated packet. Using the mapping between the VNI and the underlay multicast address, the BUM traffic is encapsulated into VXLAN, with the destination IP set to the corresponding Multicast Group Address.
Replication is handled entirely by the underlay network. Multicast-enabled routers and switches automatically replicate the VXLAN packet along the multicast tree to each remote VTEP. As a result, the source VTEP consumes uplink bandwidth only once, significantly reducing its load.
As illustrated, if a 1 Mbps BUM flow needs to be sent to 10 remote VTEPs, the uplink at the source VTEP still consumes only 1 Mbps of bandwidth, because replication occurs within the underlay network and does not consume the source VTEP’s uplink bandwidth.
Therefore, the P2MP model is particularly well-suited for high-bandwidth, low-latency scenarios, such as large-scale data centers or media distribution networks, effectively reducing the bandwidth overhead associated with Ingress Replication.
For this model, Asterfusion has already deployed multicast-based BUM traffic forwarding in cooperation with corporations in North America and Japan, enabling more efficient underlay multicast transport. We look forward to sharing more details in an upcoming case study.
Next, the following table compares the two forwarding models:
| Comparison Items | Head-End Replication (Ingress Replication, IR) | Multicast Underlay VxLAN (PIM in the Underlay) |
| Leaf-Spine Node Processing Efficiency | Low. The source VTEP needs to create a unique packet for each interested remote VTEP. | High. The source VTEP only forwards one copy, and the spine node forwards the original VXLAN packet to all interested VTEPs without rewriting. |
| Leaf Node Multicast Bandwidth Consumption | High. A separate copy of the multicast traffic must be created for each interested remote VTEP. | Low. The uplink from the source VTEP only forwards one multicast packet; the spine node handles replication. |
| Delivery Fairness / Latency Sensitivity | Low. Each packet requires header rewriting, causing asynchronous delivery at the source PE. As the number of remote VTEPs increases, fairness and latency variance worsen. | High. All nodes in the path can deliver synchronously. Spine nodes forward in parallel without rewriting VXLAN headers, ensuring all interested VTEPs receive traffic simultaneously. |
| Underlay Network Requirement | No multicast support required. | Requires multicast support in the underlay network. |
It is worth noting that IR does not require deploying additional multicast protocols within the underlay IP network, nor does it need to maintain extra network state. This significantly lowers the overall deployment complexity and is one of the key reasons it is referred to as the “simple approach” in the VXLAN Multicast BUM Traffic Forwarding Method.
Ⅲ. Key Advantages of VXLAN Multicast BUM Traffic Forwarding Method
The VXLAN multicast BUM forwarding method addresses several network challenges：
First, it reduces broadcast flooding by preventing the source VTEP from sending multiple copies to each remote VTEP, saving uplink bandwidth.
Second, it improves bandwidth usage: IR works well for medium- to low-bandwidth traffic, while P2MP is more efficient for high-bandwidth, multi-receiver scenarios.
Third, it ensures reliability and consistent latency. P2MP uses a multicast tree so all remote VTEPs receive traffic at the same time, suitable for live streaming, monitoring, and media distribution.
Fourth, it lowers operational complexity. IR is simple to deploy for small networks, while P2MP leverages existing PIM protocols for automatic replication, reducing manual maintenance.
Ⅳ. Typical Application Secnario
Consider the following scenario: In a broadcast network, high-resolution video streams—such as uncompressed video conforming to SMPTE ST 2110—are typically distributed to multiple receivers, including production studios, monitoring terminals, and other endpoints.
Suppose a source needs to deliver a single 1080p60 uncompressed video stream (approximately 3 Gbps) to 20 receivers. To optimize bandwidth efficiency and maximize network utilization, a point-to-multipoint (P2MP) model can be employed. In this model, the source VTEP encapsulates and sends the stream only once, consuming 3 Gbps of uplink bandwidth. The underlying PIM network constructs a shortest-path tree (SPT) to replicate the traffic, resulting in a total network load of 60 Gbps; however, replication is handled by the network itself, so the source experiences no additional bandwidth pressure. All receivers obtain the video stream simultaneously, ensuring low latency and high synchronization.
Key Technical Highlights are as follows:
1. Endpoint Access: EVPN Multihoming
The sender (HD) and receiver connect to Leaf VTEPs via EVPN multihoming, enabling each endpoint to simultaneously attach to multiple Leafs. This approach provides high redundancy and reliability. The EVPN control plane advertises the mapping between endpoint MAC addresses and VTEPs using BGP Type-2 routes.
2. VNI-to-Multicast Address Mapping and BUM Traffic Handling
Each Leaf VTEP maintains a mapping table between VNI and the underlying multicast IP address. The BUM traffic will lookup the multicast address corresponding to the VNI encapsulate VXLAN UDP packets with the destination IP set to the multicast address.
3. Multicast Routing and Traffic Optimization
The Spine, as the data center backbone, runs PIM-SM for VXLAN-encapsulated multicast traffic, constructs multicast forwarding tables and RR in BGP EVPN network.
4. MPLS over VXLAN Encapsulation for Cross-Data-Center Transport
For cross-data-center transport from DC A to DC B over WAN/MPLS, the Border Leaf encapsulates VXLAN multicast traffic into MPLS, using MPLS labels to differentiate VNIs or tenants. This allows the WAN or public network to forward the traffic based solely on MPLS labels and IP encapsulation, without exposing internal multicast addresses. Upon reaching the destination DC, the Border Leaf decapsulates MPLS back into VXLAN and delivers the original multicast traffic to receivers within the data center.
The relevant key CLI configurations are provided below for reference only. For the complete commands, please refer to our Command Line Reference
- Leaf switch
Configuring the mapping relationship between the VLAN, VNI, and the VNI multicast group address.
sonic(config)# vlan 10
sonic(config-vlan-10)# vni 10
sonic(config-vlan-10)# arp-trap enable
sonic(config)# interface vxlan 0
sonic(config-vxlanif-0)# source 10.1.1.2
sonic(config-vxlanif-0)# vni 10 peer 10.1.1.3
sonic(config-vxlanif-0)# vni 10 mcast-group 224.1.1.10
Configuring BGP EVPN
sonic(config)# router bgp 65101
sonic(config-router)# bgp router-id 10.1.1.2
sonic(config-router)# neighbor 10.1.1.1 remote-as 65100
sonic(config-router)# neighbor 10.1.1.1 description Spine
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 10.1.1.1 activate
sonic(config-router-af)# advertise-all-vni
Configuring PIM route
sonic(config)# interface loopback 0
sonic(config-loif-0)# ip mroute ethernet 49 224.1.1.10 10.1.1.2
- Spine switch
Enable PIM function on Spine switch (interface Ethernet 49 )
sonic(config)# interface ethernet 49
sonic(config-if-49)# ip address 20.1.1.1/30
sonic(config-if-49)# ip pim
sonic(config-if-49)# multicast-enable
BGP EVPN RR configuration
sonic(config)# router bgp 65100
sonic(config-router)# bgp router-id 10.1.1.1
sonic(config-router)# neighbor 10.1.1.2 remote-as 65101
sonic(config-router)# neighbor 10.1.1.2 description Spine_Leaf
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 10.1.1.2 activate
sonic(config-router-af)# neighbor 10.1.1.2 route-reflector-client
sonic(config-router-af)# advertise-all-vni
Ⅴ. Conclusion
The choice of VXLAN multicast forwarding model reflects a natural evolution in network design—from “simple and easy to use” to “efficient and scalable.”
Ingress Replication (IR) is lightweight and easy to deploy, making it suitable for quickly launching services. In contrast, the Multicast Underlay model is more structured and efficient, fitting large-scale data center deployments that evolve over time.
Looking ahead, as EVPN and PIM protocols become further integrated, VXLAN multicast will continue to improve in bandwidth efficiency, traffic synchronization, and scalability, enabling deeper enhancements in cloud data center traffic distribution.
For data center or campus switches that support VXLAN/EVPN and PIM, please refer to the Asterfusion Product Page or contact us.
