Hi all,
We are using VPP on two nodes and run VRRP on them. This works fine as long as
we use one VLAN. But if we configure VRs in different VLANs we see strange
behavior: Every 4 seconds, the backup VRRP VR shortly changes to master and
sends an advertisement message to the master and then it changes back to master.
The symptom:
# while true; do date; vppctl show vrrp vr | grep state; echo; sleep 1; done
Fri Feb 18 20:33:15 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:16 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:17 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:18 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Master flags: preempt no accept yes unicast no
Fri Feb 18 20:33:19 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:20 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:21 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
Fri Feb 18 20:33:22 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Master flags: preempt no accept yes unicast no
Fri Feb 18 20:33:23 UTC 2022
state Backup flags: preempt no accept yes unicast no
state Backup flags: preempt no accept yes unicast no
# tcpdump -nve -r /tmp/my_vppExt0_2vrid_2vlan.pcap | grep -B1 "vrid 232"
00:02:25.805841 00:00:5e:00:01:e8 > 01:00:5e:00:00:12, ethertype 802.1Q
(0x8100), length 50: vlan 102, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 0,
offset 0, flags [none], proto VRRP (112), length 32)
172.17.2.2 > 224.0.0.18: vrrp 172.17.2.2 > 224.0.0.18: VRRPv3,
Advertisement, vrid 232, prio 100, intvl 100cs, length 12, addrs: 172.17.2.3
--
00:02:26.190168 00:00:5e:00:01:e8 > 01:00:5e:00:00:12, ethertype 802.1Q
(0x8100), length 60: vlan 102, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 0,
offset 0, flags [none], proto VRRP (112), length 32)
172.17.2.3 > 224.0.0.18: vrrp 172.17.2.3 > 224.0.0.18: VRRPv3,
Advertisement, vrid 232, prio 200, intvl 100cs, length 12, addrs: 172.17.2.3
--
00:02:27.194227 00:00:5e:00:01:e8 > 01:00:5e:00:00:12, ethertype 802.1Q
(0x8100), length 60: vlan 102, p 0, ethertype IPv4, (tos 0x0, ttl 255, id 0,
offset 0, flags [none], proto VRRP (112), length 32)
172.17.2.3 > 224.0.0.18: vrrp 172.17.2.3 > 224.0.0.18: VRRPv3,
Advertisement, vrid 232, prio 200, intvl 100cs, length 12, addrs: 172.17.2.3
The configuration:
BACKUP system:
ip table add 26
ip table add 27
set interface state Ext-0 up
create sub-interfaces Ext-0 101
create sub-interfaces Ext-0 102
set interface state Ext-0.101 up
set interface state Ext-0.102 up
set interface ip table Ext-0.101 26
set interface ip table Ext-0.102 27
set interface ip address Ext-0.101 172.17.1.2/25
set interface ip address Ext-0.102 172.17.2.2/25
vrrp vr add Ext-0.101 vr_id 231 priority 100 no_preempt accept_mode 172.17.1.3
vrrp vr add Ext-0.102 vr_id 232 priority 100 no_preempt accept_mode 172.17.2.3
MASTER system:
ip table add 26
ip table add 27
set interface state Ext-0 up
create sub-interfaces Ext-0 101
create sub-interfaces Ext-0 102
set interface state Ext-0.101 up
set interface state Ext-0.102 up
set interface ip table Ext-0.101 26
set interface ip table Ext-0.102 27
set interface ip address Ext-0.101 172.17.1.1/25
set interface ip address Ext-0.102 172.17.2.1/25
vrrp vr add Ext-0.101 vr_id 231 priority 200 no_preempt accept_mode 172.17.1.3
vrrp vr add Ext-0.102 vr_id 232 priority 200 no_preempt accept_mode 172.17.2.3
Running BACKUP config:
# vppctl show vrrp vr
[0] sw_if_index 2 VR ID 231 IPv4
state Backup flags: preempt no accept yes unicast no
priority: configured 100 adjusted 100
timers: adv interval 100 master adv 100 skew 60 master down 360
virtual MAC 00:00:5e:00:01:e7
addresses 172.17.1.3
peer addresses
tracked interfaces
[1] sw_if_index 3 VR ID 232 IPv4
state Backup flags: preempt no accept yes unicast no
priority: configured 100 adjusted 100
timers: adv interval 100 master adv 100 skew 60 master down 360
virtual MAC 00:00:5e:00:01:e8
addresses 172.17.2.3
peer addresses
tracked interfaces
Running MASTER config:
# vppctl show vrrp vr
[0] sw_if_index 2 VR ID 231 IPv4
state Master flags: preempt no accept yes unicast no
priority: configured 200 adjusted 200
timers: adv interval 100 master adv 100 skew 21 master down 321
virtual MAC 00:00:5e:00:01:e7
addresses 172.17.1.3
peer addresses
tracked interfaces
[1] sw_if_index 3 VR ID 232 IPv4
state Master flags: preempt no accept yes unicast no
priority: configured 200 adjusted 200
timers: adv interval 100 master adv 100 skew 21 master down 321
virtual MAC 00:00:5e:00:01:e8
addresses 172.17.2.3
peer addresses
tracked interfaces
VPP version is the same on both systems:
# vppctl show version verbose
Version: v21.06.0-1~g7d4f93832
Compiled by: suse
Compile host: SUSE
Compile date: 2021-07-05T14:19:31
Compile location: /root/vpp-21.06-release/vpp
Compiler: GCC 7.5.0
Current PID: 11346
The NIC is the same on both systems:
# vppctl show hardware-interface
Name Idx Link Hardware
Ext-0 1 up Ext-0
Link speed: 10 Gbps
RX Queues:
queue thread mode
0 vpp_wk_0 (1) polling
1 vpp_wk_1 (2) polling
2 vpp_wk_2 (3) polling
Ethernet address 78:ac:44:1f:47:60
Intel X710/XL710 Family
carrier up full duplex mtu 9206
flags: admin-up pmd maybe-multiseg subif tx-offload intel-phdr-cksum
rx-ip4-cksum int-supported int-unmaskable
Devargs:
rx: queues 3 (max 192), desc 1024 (min 64 max 4096 align 32)
tx: queues 4 (max 192), desc 1024 (min 64 max 4096 align 32)
pci: device 8086:1572 subsystem 1028:1f9c address 0000:17:00.00 numa 0
max rx packet len: 9728
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip
outer-ipv4-cksum vlan-filter vlan-extend jumbo-frame
scatter keep-crc rss-hash
rx offload active: ipv4-cksum jumbo-frame scatter
tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum
tcp-tso outer-ipv4-cksum qinq-insert vxlan-tnl-tso
gre-tnl-tso ipip-tnl-tso geneve-tnl-tso multi-segs
mbuf-fast-free
tx offload active: udp-cksum tcp-cksum multi-segs
rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other
ipv6-frag
ipv6-tcp ipv6-udp ipv6-sctp ipv6-other l2-payload
rss active: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv6-frag ipv6-tcp
ipv6-udp ipv6-other
tx burst mode: Scalar
rx burst mode: Vector AVX2 Scattered
There is a switch between these interfaces but to me it looks like a bug in VPP
as it is clearly related to the second VLAN configured in VPP. If I swap the
order of configuration (vrrp vr add ...), then it's the other VR which is
toggling every 4 seconds.
Can you help, please?
Thanks, BR/Mechthild
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20880): https://lists.fd.io/g/vpp-dev/message/20880
Mute This Topic: https://lists.fd.io/mt/89242509/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
