Hi David, please find the ip link show output(for ifindex) and ping and its corresponding perf fib events output. OIF seems MGMT(ifindex 5) always, not enslaved interfaces ?
ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master MGMT state UP mode DEFAULT group default qlen 1000 link/ether 08:00:27:ee:c2:f8 brd ff:ff:ff:ff:ff:ff 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master MGMT state UP mode DEFAULT group default qlen 1000 link/ether 08:00:27:0e:75:05 brd ff:ff:ff:ff:ff:ff 4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 08:00:27:08:6c:37 brd ff:ff:ff:ff:ff:ff 5: MGMT: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether c2:08:e9:2b:8a:a4 brd ff:ff:ff:ff:ff:ff 6: LOMGMT: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master MGMT state UNKNOWN mode DEFAULT group default qlen 1000 link/ether c2:04:90:b4:2d:9e brd ff:ff:ff:ff:ff:ff ping 1.1.1.100 -I enp0s3 -c 1 sudo perf report Samples: 27 of event 'fib:fib_table_lookup', Event count (approx.): 27 Overhead Trace output 11.11% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 ◆ 7.41% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 192.168.1.11/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 ▒ 7.41% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 192.168.1.11/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 4 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 1 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 5 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 1 oif 5 iif 1 proto 17 0.0.0.0/36297 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 1 oif 5 iif 1 proto 17 2.2.2.100/36297 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 ▒ 3.70% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0 ▒ 3.70% table 254 oif 0 iif 1 proto 0 192.168.1.2/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 ▒ 3.70% table 254 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0 ▒ 3.70% table 254 oif 0 iif 4 proto 0 192.168.1.11/0 -> 192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 ▒ 3.70% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 0 iif 1 proto 0 192.168.1.2/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.11/0 -> 192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.12/0 -> 192.168.1.2/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 ▒ 3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 5 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 5 iif 1 proto 17 0.0.0.0/36297 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ 3.70% table 255 oif 5 iif 1 proto 17 2.2.2.100/36297 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 ▒ ping 1.1.1.100 -I enp0s8 -c 1 sudo perf report Samples: 27 of event 'fib:fib_table_lookup', Event count (approx.): 27 Overhead Trace output 11.11% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 4 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 5 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 5 iif 1 proto 17 0.0.0.0/51188 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 1 oif 5 iif 1 proto 17 2.2.2.100/51188 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0 3.70% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0 3.70% table 254 oif 0 iif 1 proto 0 192.168.1.2/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 3.70% table 254 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0 3.70% table 254 oif 0 iif 4 proto 0 192.168.1.1/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 3.70% table 254 oif 0 iif 4 proto 0 192.168.1.11/0 -> 192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 3.70% table 254 oif 0 iif 4 proto 0 192.168.1.8/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 3.70% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 0 iif 1 proto 0 192.168.1.2/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.1/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.11/0 -> 192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.12/0 -> 192.168.1.2/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0 3.70% table 255 oif 0 iif 4 proto 0 192.168.1.8/0 -> 192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0 scope 0 flags 5 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 5 iif 1 proto 17 0.0.0.0/51188 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 3.70% table 255 oif 5 iif 1 proto 17 2.2.2.100/51188 -> 1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11 Regards, Bala. On Fri, Apr 16, 2021 at 1:11 AM David Ahern <dsah...@gmail.com> wrote: > > On 4/15/21 12:15 AM, Bala Sajja wrote: > > When interfaces are not part of VRF and below ip address config is > > done on these interfaces, ping with -I (interface) option, we see > > packets transmitting out of the right interfaces. > > > > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s3 > > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s8 > > > > ping 1.1.1.100 -I enp0s3 , packet always goes out of enp0s3 > > ping 1.1.1.100 -I enp0s8, packet always goes out of enp0s8 > > > > When interfaces are enslaved to VRF as below and ip are configured > > on these interfaces, packets go out of one interface only. > > > > ip link add MGMT type vrf table 1 > > ip link set dev MGMT up > > ip link set dev enp0s3 up > > ip link set dev enp0s3 master MGMT > > ip link set dev enp0s8 up > > ip link set dev enp0s8 master MGMT > > ip link set dev enp0s9 up > > > > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s3 > > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s8 > > > > ping 1.1.1.100 -I enp0s3 , packet always goes out of enp0s3 > > ping 1.1.1.100 -I enp0s8, packet always goes out of enp0s3 > > > > > > I believe this use case will not work since the FIB lookup loses the > original device binding (the -I argument). take a look at the FIB lookup > argument and result: > > perf record -e fib:* > perf script