MC-LAG Configuration on Asterfusion Enterprise SONiC Distribution Switch
- 1 Purpose
- 2 Model and Version
- 3 MC-LAG introduction
- 4 Asterfusion Enterprise SONiC Distribution switch MC-LAG configuration
- 4.1 Configure MC-LAG
- 4.2 Configure peer-link
- 4.3 Configure the heartbeat detection link
- 4.4 Configure MC-LAG member interfaces
- 4.5 Configure Monitor Link group
- 5 MC-LAG typical configuration case
- 5.1 Requirements
- 5.2 Topology diagram
- 5.3 Testing environment
- 5.4 Preparation before testing
- 5.5 Configuration step
- 6 Conclusion
- Asterfusion CX-N data center switches as the links below
1 Purpose
This article mainly explains the MC-LAG solution and configuration of Asterfusion enterprise SONiC switches.
2 Model and Version
Model:
CX-N series switches
Version:
AsterNOSv3.1
3 MC-LAG introduction
MC-LAG (Multi Chassis Link Aggregation Group) is a mechanism to achieve cross-device link aggregation. By performing cross-device link aggregation between one device and two other devices, ordinary links are retained. It provides the advantages of route aggregation while providing device-level redundancy.
MC-LAG provides a horizontal virtualization technology that virtualizes two physical devices into a single logical device. This virtualized “single device” and its connected uplink or downlink device achieve “one-to-one” link aggregation. As shown below:
The working modes of the device MC-LAG introduced in this article are: control plane active and backup mode, data plane active-active mode, that is:
On the control plane, it is necessary to distinguish between primary equipment and standby equipment, primary links and standby links.
The data plane adopts dual-active mode, and the two devices independently decide to forward the data flow.
4 Asterfusion Enterprise SONiC Distribution switch MC-LAG configuration
Asterfusion switches run the enterprise-level SONiC (AsterNOS) system and ability to flexibly support MC-LAG networking.
4.1 Configure MC-LAG
mclag domain domain-id
# Create an MC-LAG domain and enter the view. Currently, only one domain creation is supported, with a range of 1-4095.
session-timeout timeout
#Timeout period, in seconds, ranges from 3 to 3600, with a default of 15 seconds. The interval for heartbeat detection messages should be less than one-third of the MC-LAG session timeout period. The session timeout period should be a multiple of the heartbeat detection message interval.
Example:
sonic(config)# mclag domain 10
sonic(mclag-domain)# session-timeout 15
4.2 Configure peer-link
vlan vlan-id#Configure a dedicated VLAN in global view, ranging from 1 to 4095.
interface link-aggregation lag-id #Create an aggregation group,1-9999.
mode static #Configure the aggregation mode as static.
commit #Commit configuration.
switchport trunk vlan vlan-id #Specify a dedicated VLAN and add it to the business VLAN.
interface ethernet interface-name #enter interface view.
link-aggregation-group lag-id [port-priority port-priority]
#Add it to the corresponding LAG, with an optional priority setting from 0 to 65535, defaulting to 255.
startup-delay delay
#Configure interface delay, defaulting to 150 seconds, and recommend setting a delay of 145 seconds on the physical interface where the peer-link resides.
mclag domain domain-id #Enter the mclag domain, and the id is the previously configured value.
peer-link link-aggregation name #Specify peer-link.
commit #Commit configuration.
Example:
sonic(config)# vlan 10
sonic(config-vlan-10)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# mode static
sonic(config-lagif-10)# commit
sonic(config-lagif-10)# switchport trunk vlan 10
sonic(config-lagif-10)# exit
sonic(config)# interface ethernet 0/10
sonic(config-if-0/10)# link-aggregation-group 10
sonic(config-if-0/10)# startup-delay 100
sonic(config-if-0/10)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# peer-link link-aggregation 10
sonic(mclag-domain)# commit
4.3 Configure the heartbeat detection link
Explanation: The heartbeat detection link is used to forward MC-LAG control packets. It can be shared with the peer-link or use a separate physical link. When dual-active detection is enabled, the heartbeat detection link must be shared with the peer-link; otherwise, the functionality will be disabled.
Using a separate physical link to configure heartbeat detection
interface ethernet interface-name #Enter interface view.
ip address A.B.C.D/M #Configure interface IP address.
mclag domain domain-id #Enter mclag domain view.
peer-address A.B.C.D #Configure peer IP address for heartbeat detection link.
local-address A.B.C.D #Configure local IP address for heartbeat detection link.
heartbeat-interval interval
#Configure MC-LAG heartbeat detection packet interval in seconds, range 1~60, default is 1 second.
commit #Commit configuration.
Example:
sonic(config)# interface ethernet 0/11
sonic(config-if-0/11)# ip address 10.0.0.11/24
sonic(config-if-0/11)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# peer-address 10.0.0.12
sonic(mclag-domain)# local-address 10.0.0.11
sonic(mclag-domain)# heartbeat-interval 1
sonic(mclag-domain)# commit
Using the peer-link to configure the heartbeat detection link
interface vlan vlan-id #Enter peer-link dedicated vlanif configuration view.
ip address A.B.C.D/M #Configure IP address for vlanif.
mclag domain domain-id #Enter mclag domain view.
peer-address A.B.C.D #Configure peer IP address for heartbeat detection link.
local-address A.B.C.D #Configure local IP address for heartbeat detection link.
commit #Commit configuration.
Example:
sonic(config)# vlan 10
sonic(config-vlan-10)# ip address 10.0.0.11/24
sonic(config- vlan-10)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# peer-address 10.0.0.12
sonic(mclag-domain)# local-address 10.0.0.11
sonic(mclag-domain)# commit
4.4 Configure MC-LAG member interfaces
Explanation: It is recommended to use low-speed interfaces as MC-LAG member interfaces. To enhance system reliability, it is advised to use dynamic aggregation for cross-device aggregation groups and enable short LACP timeout. Both member physical interfaces of the two devices deploying MC-LAG must have identical port IDs for proper aggregation.
vlan vlan-id # Create a business VLAN, range 1 ~ 4094.
interface link-aggregation lag-id
#Enter LAG interface configuration view and create a downstream aggregation group, range 1 ~ 9999.
lacp fast-rate #Enable LACP short timeout.
commit #Commit configuration.
switchport trunk vlan vlan-id #Add business VLAN to trunk.
mclag domain domain-id #Enter mclag domain view.
member lag lag-id #Add MC-LAG member interface.
Example:
sonic(config)# vlan 10
sonic(config-vlan-10)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# lacp fast-rate
sonic(config-lagif-10)# commit
sonic(config-lagif-10)# switchport trunk vlan 10
sonic(config-lagif-10)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# member lag 10
4.5 Configure Monitor Link group
Explanation: It is recommended to configure a Monitor Link group on the primary and secondary devices when deploying MC-LAG. The upstream interface should be configured as ‘uplink’ and the downstream interface as ‘downlink’. Enabling this feature ensures that if the upstream interface goes down, the downstream interface will also go down in response, ensuring rapid convergence in the event of a topology failure.
monitor-link-group group-name [delay-time]
#Create a monitor-link group. delay-time is an optional parameter that indicates the delay time in seconds before the downlink comes up after the uplink comes up. The default value is 0 seconds.
interface ethernet interface-name #Enter interface view.
monitor-link group-name uplink #Configure the interface as uplink.
interface ethernet interface-name #Enter interface view.
monitor-link group-name downlink # Configure the MC-LAG member interface as downlink.
Example:
sonic(config)# monitor-link-group monitor1
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# monitor-link monitor1 uplink
sonic(config-if-0/48)# exit
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# monitor-link monitor1 downlink
5 MC-LAG typical configuration case
5.1 Requirements
A server named server1, running critical applications for users, requires stable and reliable network connectivity. Currently, a network is being set up using 4 Asterfusion Sonic switches. Two leaf devices are configured with MC-LAG to connect to server1. The objective is to test whether server1 will lose network connectivity if one of the links is disconnected during the test. During the testing process, data transmission between server1 and server2 will be conducted.
5.2 Topology diagram
5.3 Testing environment
Hardware
|
Software
|
Management IP
|
Device communication IP
|
5.4 Preparation before testing
Ensure each device is correctly connected according to the topology diagram and verify that the iperf3 testing software is installed correctly.
5.5 Configuration step
Step 1
Configure the IP address and gateway for server1’s bond0 interface.
[admin@Server1~]# sudo vim /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=none
BROADCAST=100.0.10.255
IPADDR=100.0.10.200
NETMASK=255.255.255.0
NETWORK=100.0.10.0
ONBOOT=yes
USERCTL=no
GATEWAY=100.0.10.201
BONDING_OPTS=”miimon=100 mode=4 lacp_rate=fast xmit_hash_policy=layer2+3 fail_over_mac=1″
MASTER=yes
[admin@Server1~]# sudo vim /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=”eth0″
BOOTPROTO=none
ONBOOT=”yes”
MASTER=bond0
SLAVE=yes
HWADDR=”b8:59:9f:42:36:68″
[admin@Server1~]# sudo vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=”eth1″
BOOTPROTO=none
ONBOOT=”yes”
MASTER=bond0
SLAVE=yes
HWADDR=”b8:59:9f:42:36:69″
[admin@Server1~]# sudo systemctl restart network
Step 2
Configure the IP address and gateway for server2’s eth0 interface.
[admin@Server1~]# sudo ifconfig eth0 100.0.20.200/24 up
[admin@Server1~]# sudo route add default gw 100.0.20.201
Step 3
Config Spine switch IP address.
sonic# configure terminal
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# ip address 10.0.10.201/24
sonic(config-if-0/0)# interface ethernet 0/4
sonic(config-if-0/4)# ip address 10.0.11.201/24
sonic(config-if-0/4)# interface ethernet 0/8
sonic(config-if-0/8)# ip address 10.0.12.201/24
sonic(config-if-0/8)# exit
sonic(config)# interface loopback 0
sonic(config-loif-0)# ip address 172.16.0.1/32
Step 4
Configure IP addresses and MC-LAG on three Leaf devices.
Leaf1 configuration
sonic# configure terminal
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# ip address 10.0.10.1/24
sonic(config-if-0/48)# interface loopback 0
sonic(config-loif-0)# ip address 172.16.0.2/32
sonic(config-loif-0)# exit
sonic(config)# vlan 10
sonic(config-vlan-10)# exit
sonic(config)# vlan 30
sonic(config-vlan-30)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# exit
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# speed 10000
sonic(config-if-0/0)# link-aggregation-group 10
sonic(config-if-0/0)# exit
sonic(config)# interface ethernet 0/4
sonic(config-if-0/4)# speed 10000
sonic(config-if-0/4)# switchport trunk vlan 10
sonic(config-if-0/4)# switchport trunk vlan 30
sonic(config-if-0/4)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# switchport trunk vlan 10
sonic(config-lagif-10)# exit
sonic(config)# vlan 10
sonic(config-vlan-10)# ip address 100.0.10.201/24
sonic(config-vlan-10)# exit
sonic(config)# vlan 30
sonic(config-vlan-30)# ip address 11.0.0.11/24
sonic(config-vlan-10)# exit
sonic(config)# monitor-link-group monitor1
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# monitor-link monitor1 uplink
sonic(config-if-0/48)# exit
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# monitor-link monitor1 downlink
sonic(config-if-0/0)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# peer-link ethernet 0/4
sonic(mclag-domain)# local-address 11.0.0.11
sonic(mclag-domain)# peer-address 11.0.0.12
sonic(mclag-domain)# member lag 10
sonic(mclag-domain)# commit
sonic(mclag-domain)# interface vlan 10
sonic(config-vlan-10)# mac-address 00:11:22:33:44:55
sonic(config-vlan-10)# exit
sonic(config)# write
Leaf2 configuration
sonic# configure terminal
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# ip address 10.0.11.1/24
sonic(config-if-0/48)# interface loopback 0
sonic(config-loif-0)# ip address 172.16.0.3/32
sonic(config-loif-0)# exit
sonic(config)# vlan 10
sonic(config-vlan-10)# exit
sonic(config)# vlan 30
sonic(config-vlan-30)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# exit
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# speed 10000
sonic(config-if-0/0)# link-aggregation-group 10
sonic(config-if-0/0)# exit
sonic(config)# interface ethernet 0/4
sonic(config-if-0/4)# speed 10000
sonic(config-if-0/4)# switchport trunk vlan 10
sonic(config-if-0/4)# switchport trunk vlan 30
sonic(config-if-0/4)# exit
sonic(config)# interface link-aggregation 10
sonic(config-lagif-10)# switchport trunk vlan 10
sonic(config-lagif-10)# exit
sonic(config)# vlan 10
sonic(config-vlan-10)# ip address 100.0.10.201/24
sonic(config-vlan-10)# exit
sonic(config)# vlan 30
sonic(config-vlan-30)# ip address 11.0.0.12/24
sonic(config-vlan-10)# exit
sonic(config)# monitor-link-group monitor1
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# monitor-link monitor1 uplink
sonic(config-if-0/48)# exit
sonic(config)# interface ethernet 0/0
sonic(config-if-0/0)# monitor-link monitor1 downlink
sonic(config-if-0/0)# exit
sonic(config)# mclag domain 10
sonic(mclag-domain)# peer-link ethernet 0/4
sonic(mclag-domain)# local-address 11.0.0.12
sonic(mclag-domain)# peer-address 11.0.0.11
sonic(mclag-domain)# member lag 10
sonic(mclag-domain)# commit
sonic(mclag-domain)# interface vlan 10
sonic(config-vlan-10)# mac-address 00:11:22:33:44:55
sonic(config-vlan-10)# exit
sonic(config)# write
Leaf3 configuration
sonic# configure terminal
sonic(config)# interface ethernet 0/48
sonic(config-if-0/48)# ip address 10.0.12.1/24
sonic(config-if-0/48)# interface ethernet 0/0
sonic(config-if-0/0)# speed 10000
sonic(config-if-0/0)# ip address 100.0.20.201/24
sonic(config-if-0/0)# interface loopback 0
sonic(config-loif-0)# ip address 172.16.0.4/32
sonic(config-loif-0)# exit
sonic(config)# write
Step 5
Configure BGP on Spine device.
sonic# configure terminal
sonic(config)# router bgp 65001
sonic(config-router)# bgp router-id 171.16.0.1
sonic(config-router)# no bgp ebgp-requires-policy
sonic(config-router)# neighbor 172.16.0.2 remote-as 65002
sonic(config-router)# neighbor 172.16.0.3 remote-as 65002
sonic(config-router)# neighbor 172.16.0.4 remote-as 65004
sonic(config-router)# address-family l2vpn evpn
sonic(config-router)# neighbor 172.16.0.2 activate
sonic(config-router)# neighbor 172.16.0.3 activate
sonic(config-router)# neighbor 172.16.0.4 activate
sonic(config-router)# advertise-all-vni
sonic(config)# write
sonic(config)# reload
Step 6
Configure BGP on three Leaf devices.
Leaf1 configuration
sonic(config)# router bgp 65002
sonic(config-router)# bgp router-id 171.16.0.2
sonic(config-router)# no bgp ebgp-requires-policy
sonic(config-router)# neighbor 172.16.0.1 remote-as 65001
sonic(config-router)# neighbor 172.16.0.3 remote-as 65002
sonic(config-router)# address-family ipv4 unicast
sonic(config-router-af)# network 172.16.0.2/32
sonic(config-router-af)# exit
sonic(config)# router bgp 65002
sonic(config-router)# address-family l2vpn evpn
sonic(config-router)# neighbor 172.16.0.1 activate
sonic(config-router)# neighbor 172.16.0.3 activate
sonic(config-router)# advertise-all-vni
sonic(config)# write
sonic(config)# reload
Leaf2 configuration
sonic(config)# router bgp 65002
sonic(config-router)# bgp router-id 171.16.0.3
sonic(config-router)# no bgp ebgp-requires-policy
sonic(config-router)# neighbor 172.16.0.1 remote-as 65001
sonic(config-router)# neighbor 172.16.0.2 remote-as 65002
sonic(config-router)# address-family ipv4 unicast
sonic(config-router-af)# network 172.16.0.3/32
sonic(config-router-af)# exit
sonic(config)# router bgp 65002
sonic(config-router)# address-family l2vpn evpn
sonic(config-router)# neighbor 172.16.0.1 activate
sonic(config-router)# neighbor 172.16.0.2 activate
sonic(config-router)# advertise-all-vni
sonic(config)# write
sonic(config)# reload
Leaf3 configuration
sonic(config)# router bgp 65004
sonic(config-router)# bgp router-id 171.16.0.4
sonic(config-router)# no bgp ebgp-requires-policy
sonic(config-router)# neighbor 172.16.0.1 remote-as 65001
sonic(config-router)# address-family ipv4 unicast
sonic(config-router-af)# network 172.16.0.4/32
sonic(config-router-af)# exit
sonic(config)# router bgp 65004
sonic(config-router)# address-family l2vpn evpn
sonic(config-router)# neighbor 172.16.0.1 activate
sonic(config-router)# advertise-all-vni
sonic(config)# write
sonic(config)# reload
Step 7
Use server1 to send traffic to server2, and observe the traffic flow on the spine devices.
[admin@Server2~]# iperf3 -s
[admin@Server1~]# iperf3 -c 100.0.20.200 -l 20k -b 100G -M 9000 -t 1000
Check Spine device traffic
Step 8
Bring down interface 0/48 on leaf1 and observe the traffic flow towards the spine
6 Conclusion
In this case, after the mc-lag link failure, servers were able to communicate normally with each other. Even after the mc-lag link was restored, servers continued to communicate normally. This demonstrates that the mc-lag network composed of Asterfusion Sonic switches can meet users’ requirements for high network stability and reliability.
Asterfusion CX-N data center switches as the links below
If you have more technical questions, feel free to propose a ticket on our
https://help.cloudswit.ch/portal/en/signin