Hi all,
we are using VPP on two nodes where we would like to run VRRP. This works fine
if the VRRP VR interface is in fib 0 but if we but the interface into FIB table
1 instead, VRRP is not working correctly anymore. Can you please help?
Our setup:
* 2 nodes with VPP on each node and one DPDK interface (we reduced the
config to isolate the issue) connected to each VPP
* a switch between the nodes which just forwards the traffic, so that it's
like a peer-2-peer connection
The VPP version is (both nodes):
vpp# show version
vpp v21.01.0-6~gf70123b2c built by suse on SUSE at 2021-05-06T12:18:31
vpp# show version verbose
Version: v21.01.0-6~gf70123b2c
Compiled by: suse
Compile host: SUSE
Compile date: 2021-05-06T12:18:31
Compile location: /root/vpp-sp/vpp
Compiler: GCC 7.5.0
Current PID: 6677
The VPP config uses the DPDK interface (both nodes):
vpp# show hardware-interfaces
Name Idx Link Hardware
Ext-0 1 up Ext-0
Link speed: 10 Gbps
Ethernet address e4:43:4b:ed:59:10
Intel X710/XL710 Family
carrier up full duplex mtu 9206
flags: admin-up pmd maybe-multiseg tx-offload intel-phdr-cksum rx-ip4-cksum
Devargs:
rx: queues 1 (max 192), desc 1024 (min 64 max 4096 align 32)
tx: queues 3 (max 192), desc 1024 (min 64 max 4096 align 32)
pci: device 8086:1572 subsystem 1028:1f9c address 0000:17:00.00 numa 0
max rx packet len: 9728
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip
outer-ipv4-cksum vlan-filter vlan-extend jumbo-frame
scatter keep-crc rss-hash
rx offload active: ipv4-cksum jumbo-frame scatter
tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum
tcp-tso outer-ipv4-cksum qinq-insert vxlan-tnl-tso
gre-tnl-tso ipip-tnl-tso geneve-tnl-tso multi-segs
mbuf-fast-free
tx offload active: udp-cksum tcp-cksum multi-segs
rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other
ipv6-frag
ipv6-tcp ipv6-udp ipv6-sctp ipv6-other l2-payload
rss active: none
tx burst mode: Scalar
rx burst mode: Vector AVX2 Scattered
The VRRP configs are (MASTER):
set interface state Ext-0 up
set interface ip address Ext-0 192.168.61.52/25
vrrp vr add Ext-0 vr_id 61 priority 200 no_preempt accept_mode 192.168.61.50
and on the system under test (SUT):
ip table add 1
set interface ip table Ext-0 1
set interface state Ext-0 up
set interface ip address Ext-0 192.168.61.51/25
vrrp vr add Ext-0 vr_id 61 priority 100 no_preempt accept_mode 192.168.61.50
On the MASTER, we started VRRP with:
vrrp proto start Ext-0 vr_id 61
so that it has:
vpp# show vrrp vr
[0] sw_if_index 1 VR ID 61 IPv4
state Master flags: preempt no accept yes unicast no
priority: configured 200 adjusted 200
timers: adv interval 100 master adv 100 skew 21 master down 321
virtual MAC 00:00:5e:00:01:3d
addresses 192.168.61.50
peer addresses
tracked interfaces
On the SUT, we did not yet start VRRP, so we see:
vpp# show vrrp vr
[0] sw_if_index 1 VR ID 61 IPv4
state Initialize flags: preempt no accept yes unicast no
priority: configured 100 adjusted 100
timers: adv interval 100 master adv 0 skew 0 master down 0
virtual MAC 00:00:5e:00:01:3d
addresses 192.168.61.50
peer addresses
tracked interfaces
Here I see already that something is going wrong as the VRRP packets are not
reaching vrrp4-input:
vpp# show errors
Count Node Reason
Severity
5 dpdk-input no error
error
138 ip4-local ip4 source lookup miss
error
(If we configure SUT similar to the MASTER, ie interface in FIB 0, I can see
vrrp4-input at this point)
The trace of dpdk-input gives:
Packet 1
00:00:57:644818: dpdk-input
Ext-0 rx queue 0
buffer 0x9b7ec: current data 0, length 60, buffer-pool 0, ref-count 1,
totlen-nifb 0, trace handle 0x1000000
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 0x26dfb80
packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without
extension headers
RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet
IP4: 00:00:5e:00:01:3d -> 01:00:5e:00:00:12
VRRP: 192.168.61.50 -> 224.0.0.18
tos 0x00, ttl 255, length 32, checksum 0xdd80 dscp CS0 ecn NON_ECN
fragment id 0x0000
00:00:57:644832: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: 00:00:5e:00:01:3d -> 01:00:5e:00:00:12
00:00:57:644840: ip4-input-no-checksum
VRRP: 192.168.61.50 -> 224.0.0.18
tos 0x00, ttl 255, length 32, checksum 0xdd80 dscp CS0 ecn NON_ECN
fragment id 0x0000
00:00:57:644843: ip4-mfib-forward-lookup
fib 1 entry 11
00:00:57:644846: ip4-mfib-forward-rpf
entry 11 itf 1 flags Accept,
00:00:57:644848: ip4-replicate
replicate: 8 via [@1]: dpo-receive
00:00:57:644850: ip4-local
VRRP: 192.168.61.50 -> 224.0.0.18
tos 0x00, ttl 255, length 32, checksum 0xdd80 dscp CS0 ecn NON_ECN
fragment id 0x0000
00:00:57:644854: ip4-drop
VRRP: 192.168.61.50 -> 224.0.0.18
tos 0x00, ttl 255, length 32, checksum 0xdd80 dscp CS0 ecn NON_ECN
fragment id 0x0000
00:00:57:644855: error-drop
rx:Ext-0
00:00:57:644856: drop
ip4-local: ip4 source lookup miss
And the FIB table 1:
vpp# show ip fib table 1
ipv4-VRF:1, fib_index:1, flow hash:[src dst sport dport proto ] epoch:0
flags:none locks:[CLI:2, ]
0.0.0.0/0
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:8 buckets:1 uRPF:7 to:[0:0]]
[0] [@0]: dpo-drop ip4
0.0.0.0/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:8 to:[0:0]]
[0] [@0]: dpo-drop ip4
192.168.61.0/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:14 buckets:1 uRPF:14 to:[0:0]]
[0] [@0]: dpo-drop ip4
192.168.61.0/25
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:13 buckets:1 uRPF:13 to:[0:0]]
[0] [@4]: ipv4-glean: [src:192.168.61.0/25] Ext-0: mtu:9000 next:1
ffffffffffffe4434be52a100806
192.168.61.51/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:16 buckets:1 uRPF:18 to:[0:0]]
[0] [@2]: dpo-receive: 192.168.61.51 on Ext-0
192.168.61.127/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:15 buckets:1 uRPF:16 to:[0:0]]
[0] [@0]: dpo-drop ip4
224.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:11 buckets:1 uRPF:10 to:[0:0]]
[0] [@0]: dpo-drop ip4
240.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:9 to:[0:0]]
[0] [@0]: dpo-drop ip4
255.255.255.255/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:12 buckets:1 uRPF:11 to:[0:0]]
[0] [@0]: dpo-drop ip4
If I read the trace correctly, ip4-mfib-forward-lookup goes into index 11
instead of index 13 and therefore is not reaching local, vrrp4-input.
(Again, if I configure SUT using FIB 0, I can see that ip4-mfib-forward-lookup
is pointing to the FIB entry 192.168.61.0/25 and then reaches local,
vrrp4-input)
If I start VR at this point in time (vrrp proto start Ext-0 vr_id 61), VRRP is
changing to BACKUP, then immediately to MASTER while the peer stays on MASTER.
So, we have MASTER/MASTER. And 'show errors' still doesn't list vrrp4-input.
Is this a bug in VPP or is VRRP not supported with FIBs other than 0? Do you
have a suggestion how to fix/solve this?
Thank you and best regards,
Mechthild Buescher
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19629): https://lists.fd.io/g/vpp-dev/message/19629
Mute This Topic: https://lists.fd.io/mt/83756732/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
