网卡绑定mode 4_DellR740和华为S5720

1、原理解析

1.1 LACP模式链路聚合简介

以太网链路聚合是指将多条以太网物理链路捆绑在一起成为一条逻辑链路，从而实现增加链路带宽的目的。链路聚合分为手工模式和LACP模式。

LACP模式需要有链路聚合控制协议LACP的参与。当需要在两个直连设备间提供一个较大的链路带宽而设备支持LACP协议时，建议使用LACP模式。LACP模式不仅可以实现增加带宽、提高可靠性、负载分担的目的，而且可以提高Eth-Trunk的容错性、提供备份功能。

LACP模式下，部分链路是活动链路，所有活动链路均参与数据转发。如果某条活动链路故障，链路聚合组自动在非活动链路中选择一条链路作为活动链路，参与数据转发的链路数目不变。

1.2 bonding技术

bonding(绑定)是一种linux系统下的网卡绑定技术，可以把服务器上n个物理网卡在系统内部抽象(绑定)成一个逻辑上的网卡，能够提升网络吞吐量、实现网络冗余、负载等功能，有很多优势。

bonding技术是linux系统内核层面实现的，它是一个内核模块(驱动)。使用它需要系统有这个模块, 我们可以modinfo命令查看下这个模块的信息, 一般来说都支持.

1.3 华为交换机和Linux服务器对接采用的方式

服务器网卡绑定模式	交换机对接方式	说明
round-robin	配置手工模式链路聚合	服务器所绑定的网卡被修改成相同的MAC地址，需要交换机通过手工模式链路聚合与之对接。
active-backup	配置对接接口在同一个VLAN	服务器采用双网卡时，一个处于主状态，一个处于从状态，所有数据都通过主状态的端口传输。当主状态端口对应链路出现故障时，数据通过从状态端口传输。因此交换机对应的两个端口建议配置在同一个VLAN。
load balancing	配置手工模式链路聚合	服务器的多网卡基于指定的HASH策略传输数据包，需要交换机配置手工模式链路聚合与之对接。
broadcast	采用两台交换机对接且配置在不同VLAN	服务器的多网卡对于同一份报文会复制两份分别从两个端口传输，建议使用两台交换机，且配置不同VLAN与之对接。
lacp	配置LACP模式链路聚合	服务器的多网卡采用LACP模式绑定，对应交换机应采用LACP模式的链路聚合与之对接。
transmit load balancing	采用两台交换机对接	服务器的多网卡采用适配器传输负载均衡，交换机上不需进行配置。
adaptive load balancing	配置手工模式链路聚合	服务器的多网卡采用适配器适应性负载均衡，需要交换机配置手工模式链路聚合与之对接。

2、S5720交换机配置

2.1 创建Eth-Trunk 15，配置工作模式为LACP

[Sw_hexin]interface Eth-Trunk 15
[Sw_hexin-Eth-Trunk15]bpdu enable
[Sw_hexin-Eth-Trunk15]mode lacp
[Sw_hexin-Eth-Trunk15]quit

2.2 将接口加入Eth-Trunk 15

[Sw_hexin]interface GigabitEthernet 0/0/43
[Sw_hexin-GigabitEthernet0/0/43]eth-trunk 15
[Sw_hexin-GigabitEthernet0/0/43]quit
[Sw_hexin]interface GigabitEthernet 0/0/44
[Sw_hexin-GigabitEthernet0/0/44]eth-trunk 15
[Sw_hexin-GigabitEthernet0/0/44]quit
[Sw_hexin]interface Eth-Trunk 15
[Sw_hexin-Eth-Trunk15] port link-type access
[Sw_hexin-Eth-Trunk15] port default vlan 15
[Sw_hexin-Eth-Trunk15] traffic-policy Security_Policy_15 inbound //加入定义的策略，可以省略

2.3 交换机配置信息

[Sw_hexin]display current-configuration
!Software Version V200R010C00SPC600
#
interface Eth-Trunk15port link-type accessport default vlan 15                     traffic-policy Security_Policy_15 inboundmode lacp
#
interface GigabitEthernet0/0/43eth-trunk 15
#
interface GigabitEthernet0/0/44           eth-trunk 15
#

2.4 验证交换机配置

查看Switch设备的Eth-Trunk信息，查看链路是否协商成功；

[Sw_hexin]display eth-trunk 15
Eth-Trunk15's state information is:
Local:
LAG ID: 15                  WorkingMode: LACP                                 
Preempt Delay: Disabled     Hash arithmetic: According to SIP-XOR-DIP         
System Priority: 32768      System ID: 80b5-7587-c2c0                         
Least Active-linknumber: 1  Max Active-linknumber: 8                          
Operate status: up          Number Of Up Port In Trunk: 2                     
--------------------------------------------------------------------------------
ActorPortName          Status   PortType PortPri PortNo PortKey PortState Weight
GigabitEthernet0/0/43  Selected 1GE      32768   1      3889    10111100  1     
GigabitEthernet0/0/44  Selected 1GE      32768   2      3889    10111100  1     
Partner:
--------------------------------------------------------------------------------
ActorPortName          SysPri   SystemID        PortPri PortNo PortKey PortState
GigabitEthernet0/0/43  65535    dcf4-01e7-e6ce  255     2      9       11111100
GigabitEthernet0/0/44  65535    dcf4-01e7-e6ce  255     1      9       11111100

3、服务器配置bonding

系统：CentOS Linux release 7.7.1908

网卡：ifcfg-em3 、ifcfg-em4

bond0:192.168.15.15

负载模式: mode4（802.3ad 动态链路聚合）

3.1 关闭和停止NetworkManager服务

[root@svn-zt:/root]# systemctl start NetworkManager.service
[root@svn-zt:/root]# systemctl enable NetworkManager.service

3.2 加载bonding模块

[root@svn-zt:/root]# modprobe bonding
查看模块是否被加载
[root@svn-zt:/root]# lsmod | grep bonding
bonding               152979  0

3.3 创建基于bond0接口的配置文件

[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-bond0
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.15.15
GATEWAY=192.168.15.1
NETMASK=255.255.255.0
PEERDNS=yes
ONBOOT=yes
BOOTPROTO=static
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=layer3+4 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=4 lacp_rate=1 arp_interval=0 ad_select=0"##上面的BONDING_OPTS="mode=4 miimon=100" 表示这里配置的工作模式是802.3ad 动态链路聚合, miimon表示监视网络链接的频度 (毫秒), 我们设置的是100毫秒, 根据你的需求也可以指定mode成其它的负载模式。

3.4 修改ifcfg-em3和ifcfg-em4接口的配置文件

[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-em3
TYPE=Ethernet
BOOTPROTO=none
NAME=em3
UUID=5737ffd3-4a8a-46b1-ae2e-603127fdf5a4
DEVICE=em3
ONBOOT=yes
MASTER=bond0
SLAVE=yes
-------------------------------------------------------------------------------------
[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-em4
TYPE=Ethernet
BOOTPROTO=none
NAME=em4
UUID=f0393551-fea5-4ad8-a247-802eae29c2ec
DEVICE=em4
ONBOOT=yes
MASTER=bond0
SLAVE=yes

3.5 编辑/etc/modprobe.d/bond.conf文件

[root@svn-zt:/root]# vim /etc/modprobe.d/bond.conf
alias   bond0   bonding
options bond0   miimon=100      mode=4  lacp_rate=1

3.6 重启网络服务

systemctl restart network
或 直接重启系统
reboot

3.7 查看bond0的接口状态信息

[root@svn-zt:/root]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)Bonding Mode: IEEE 802.3ad Dynamic link aggregation  //绑定模式: 当前是ald模式(mode 4), 也就是802.3ad 动态链路聚合
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0802.3ad info                                       //802.3ad 信息     
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: dc:f4:01:e7:e6:ce
Active Aggregator Info:Aggregator ID: 1Number of ports: 2Actor Key: 9Partner Key: 3889Partner Mac Address: 80:b5:75:87:c2:c0Slave Interface: em3                               
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 4
Permanent HW addr: dc:f4:01:e7:e6:ce
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:system priority: 65535system mac address: dc:f4:01:e7:e6:ceport key: 9port priority: 255port number: 1port state: 63
details partner lacp pdu:system priority: 32768system mac address: 80:b5:75:87:c2:c0oper key: 3889port priority: 32768port number: 2port state: 61Slave Interface: em4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: dc:f4:01:e7:e6:cf
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:system priority: 65535system mac address: dc:f4:01:e7:e6:ceport key: 9port priority: 255port number: 2port state: 63
details partner lacp pdu:system priority: 32768system mac address: 80:b5:75:87:c2:c0oper key: 3889port priority: 32768port number: 1port state: 61

3.8 通过ifconfig命令查看下网络的接口信息

[root@svn-zt:/root]# ifconfig
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500inet 192.168.15.15  netmask 255.255.255.0  broadcast 192.168.15.255inet6 fe80::def4:1ff:fee7:e6ce  prefixlen 64  scopeid 0x20<link>ether dc:f4:01:e7:e6:ce  txqueuelen 1000  (Ethernet)RX packets 17673  bytes 2172638 (2.0 MiB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 2003  bytes 240578 (234.9 KiB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0em1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500ether dc:f4:01:e7:e6:cc  txqueuelen 1000  (Ethernet)RX packets 0  bytes 0 (0.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 0  bytes 0 (0.0 B)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0device interrupt 32  em2: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500ether dc:f4:01:e7:e6:cd  txqueuelen 1000  (Ethernet)RX packets 0  bytes 0 (0.0 B)RX errors 0  dropped 0  overruns 0  frame 0TX packets 0  bytes 0 (0.0 B)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0device interrupt 54  em3: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500ether dc:f4:01:e7:e6:ce  txqueuelen 1000  (Ethernet)RX packets 6352  bytes 806388 (787.4 KiB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 380  bytes 42416 (41.4 KiB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0device interrupt 55  em4: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500ether dc:f4:01:e7:e6:ce  txqueuelen 1000  (Ethernet)RX packets 11321  bytes 1366250 (1.3 MiB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 1623  bytes 198162 (193.5 KiB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0device interrupt 61  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536inet 127.0.0.1  netmask 255.0.0.0inet6 ::1  prefixlen 128  scopeid 0x10<host>loop  txqueuelen 1000  (Local Loopback)RX packets 40  bytes 3320 (3.2 KiB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 40  bytes 3320 (3.2 KiB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0