Hi Ales, Thanks for the reply. Please, see my comment inline.
Ales Musil wrote: > On Sat, Apr 11, 2026 at 12:01 PM Odintsov Vladislav <[email protected]> > wrote: >> Hi all, >> >> I'm currently trying to adopt OVN native BGP/EVPN support to our cloud' >> usecases and faced one problem: >> >> Imagine topology of one "external" VTEP node and 2 OVN-managed hosts: >> one of them is a gateway chassis, and another is just a normal workload >> chassis (without EVPN connectivity). >> >> XXXXXXXXXXXXXXX >> X X >> XXXXXXXXXXXXXXXXX IP fabric XXXXXXXXXXXXXXX >> X X X X >> X XXXXXXXXXXXXXXX X >> X X X >> X X X >> XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX >> X X X X X X >> X Ext VTEP X X OVN GW X X OVN w/l X >> X X X X X X >> XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX >> 10.128.0.5 10.128.0.14 10.128.0.6 >> >> > > Hi Vladislav, > > the graph is a bit distorted but I think I have an idea what might > be wrong, see down below. > >> >> >> VTEP node has configured VRF linked to VNI 300 and announces Type-5 >> route with 172.31.0.0/24<http://172.31.0.0/24> IP prefix. >> >> [root@vtep01 ~]# ip a s br-300 >> 19: br-300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue >> master ovnvrf300 state UP group default qlen 1000 >> link/ether 00:04:00:00:00:01 brd ff:ff:ff:ff:ff:ff >> inet 172.31.0.1/24<http://172.31.0.1/24> scope global br-300 >> valid_lft forever preferred_lft forever >> [root@vtep01 ~]# ip a s ovnvrf300 >> 18: ovnvrf300: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state >> UP group default qlen 1000 >> link/ether c2:9d:fb:b7:74:6f brd ff:ff:ff:ff:ff:ff >> [root@vtep01 ~]# ip a s vxlan300 >> 23: vxlan300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue >> master br-300 state UNKNOWN group default qlen 1000 >> link/ether 2a:39:b8:a7:b6:80 brd ff:ff:ff:ff:ff:ff >> inet6 fe80::2839:b8ff:fea7:b680/64 scope link >> valid_lft forever preferred_lft forever >> >> OVN has EVPN-configured edge LR and localnet-enabled LS with static >> route "10.1.0.0/24" with IPv6 LLA nexthop and output LRP which leads to LR1. >> >> LR1 has two LRPs: one to edge LR, another is a link to "workloads". >> >> "workload" LS, which has 2 LSPs: one on gateway node, another on the >> "workload" node. >> >> OVN topology is as next: >> >> switch b534bd4c-02bb-4428-a834-fbb21b59d79f (ls1) >> port ls1-lr1 >> type: router >> router-port: lr1-downlink >> port lsp2 >> addresses: ["00:00:00:00:00:02 10.1.0.102"] >> port lsp1 >> addresses: ["00:00:00:00:00:01 10.1.0.101"] >> switch f03fa0e5-46a9-49b8-91b7-2e506977170c (test-ovn1-uplink) >> port ln-node2 >> type: router >> router-port: edge1-uplink2 >> port test-ovn1-uplink-lsp >> type: localnet >> addresses: ["unknown"] >> port test-ovn1-edge1 >> type: router >> router-port: edge1-uplink >> router d322ecc0-c225-49f0-aa37-7c3003097714 (edge1) >> port edge1-lr1 >> mac: "00:02:00:00:00:01" >> ipv6-lla: "fe80::202:ff:fe00:1" >> port edge1-uplink >> mac: "0a:00:aa:21:e5:e0" > > I think the problem here is actually the MAC address, it should > be same as the br-300. Think of the LRP as br-300 representation > in the OVN topology. > br-300 interface configuration on the ovn side and corresponding ovn edge LR are next: [root@test-ovn1 ~]# ip li show br-300 43: br-300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovnvrf300 state UP mode DEFAULT group default qlen 1000 link/ether 0a:00:aa:21:e5:e0 brd ff:ff:ff:ff:ff:ff [root@test-ovn1 ~]# ovn-nbctl show edge1 router d322ecc0-c225-49f0-aa37-7c3003097714 (edge1) port edge1-lr1 mac: "00:02:00:00:00:01" ipv6-lla: "fe80::202:ff:fe00:1" port edge1-uplink mac: "0a:00:aa:21:e5:e0" ipv6-lla: "fe80::800:aaff:fe21:e5e0" networks: ["10.128.0.101/20"] gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9] So, their mac addresses are the same. Moreover, IIUC, the problem is MAC/IP resolving for the remote side ovn the ovn chassis, which has no EVPN connectivity. Isn't it? I've checked multinode testsuite and couldn't find any test, which correlates to partially-connected (only GW nodes) scenarios. Only 'ovn multinode bgp L3 EVPN'. If you point me to the test - I'd be very happy and appreciate it. >> ipv6-lla: "fe80::800:aaff:fe21:e5e0" >> networks: ["10.128.0.101/20<http://10.128.0.101/20>"] <--- I'm >> not sure it is correct >> gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9] >> router fab999fe-6ef7-478e-9b42-494cafe8f4b1 (lr1) >> port lr1-downlink >> mac: "00:03:00:00:00:01" >> ipv6-lla: "fe80::203:ff:fe00:1" >> networks: ["10.1.0.1/24<http://10.1.0.1/24>"] >> port lr1-edge1 >> mac: "00:02:00:00:00:02" >> ipv6-lla: "fe80::202:ff:fe00:2" >> >> # ovn-nbctl lr-route-list edge1 >> IPv4 Routes >> Route Table <main>: >> 10.1.0.0/16<http://10.1.0.0/16> fe80::202:ff:fe00:2 >> dst-ip edge1-lr1 >> # ovn-nbctl lr-route-list lr1 >> IPv4 Routes >> Route Table <main>: >> 0.0.0.0/0<http://0.0.0.0/0> fe80::202:ff:fe00:1 >> dst-ip lr1-edge1 >> >> # ovn-nbctl get logical-switch test-ovn1-uplink other_config >> {dynamic-routing-advertise-ifname=lo-300, >> dynamic-routing-bridge-ifname=br-300, >> dynamic-routing-redistribute="fdb,ip", dynamic-routing-vni="300", >> dynamic-routing-vxlan-ifname=vxlan300} >> >> # ovn-nbctl get logical-router edge1 options >> {dynamic-routing="true", dynamic-routing-redistribute=static, >> dynamic-routing-vrf-id="300"} >> >> # ovn-nbctl get logical-router-port edge1-uplink options >> {dynamic-routing-maintain-vrf="true", >> dynamic-routing-redistribute=connected} >> >> So, if I start pinging from VTEP's VRF (from 172.31.0.0/24) to LSP1 >> (10.1.0.101), which is on the gateway node, it works fine. >> >> On the OVN GW node the evpn commands have next output: >> >> # ovn-appctl evpn/remote-vtep-list >> IP: 10.128.0.5, port: 4789, vni: 300 >> >> # ovn-appctl evpn/vtep-arp-list >> UUID: 7e3f9194-0699-4ed2-83c3-73b474c1b2b1, VNI: 300, MAC: >> 00:04:00:00:00:01, IP: 10.128.0.5, dp_key: 5 >> >> # ovn-appctl evpn/vtep-binding-list >> UUID: 826cd9a5-d582-4a3a-85d9-2f6471c0afa6, Remote IP: 10.128.0.5, vni: >> 300, binding_key: 0x80000015, tunnel_ofport: 10, dp_key: 5 >> >> # ovn-appctl evpn/vtep-fdb-list >> UUID: c44b7556-e8f9-4bce-9d8b-9c916b79d105, MAC: 00:04:00:00:00:01, vni: >> 300, binding_key: 0x80000015, dp_key: 5 >> >> # ovn-appctl evpn/vtep-multicast-group-list >> UUID: 81f25768-6a40-497e-b48f-e17faa91e0e7, Remote IPs: 10.128.0.5, vni: 300 >> >> >> PROBLEM: >> >> BUT, if I start pinging from VTEP node to LSP2 (10.1.0.102), it fails. >> ICMP request goes from VTEP node to OVN GW node via VXLAN VNI300 (fine), >> then OVN GW sends this traffic inside GENEVE to OVN w/l node and ICMP >> request is seen on the VIF (lsp2). >> >> ICMP response is sent from lsp2, but I see ARP request in the GENEVE - >> OVN w/l node doesn't know br-300 MAC address on the VTEP node (which is >> the dst MAC for ICMP response). >> >> On then one hand that is expected - EVPN mac addresses are stored in the >> ovn-controller runtime and not committed to OVN SB, so OVN w/l node >> doesn't know which mac to use. >> >> On the other hand, maybe I've lost some configuration hints and such >> topology can work as well? > > Using the br-300 MAC in the LRP should actually fix this. > >> >> >> P.S. in ipv6 scenario, if I don't set 'networks' on the gateway LRP, >> ovn-northd generated incorrect logical flow (example): > > Yeah, this is a known bug we aim to fix. > >> >> # ovn-sbctl dump-flows edge1 | grep 'null' >> table=16(lr_in_ip_routing ), priority=514 , match=(reg7 == 0 && >> ip6.dst == 2002::/64), action=(ip.ttl--; reg8[0..15] = 0; reg0 = >> 10.128.0.5; reg5 = (null); eth.src = 0a:00:aa:21:e5:e0; outport = >> "edge1-uplink"; flags.loopback = 1; reg9[9] = 1; next;) >> >> So, I'm very concerned about which IP addresses should be used in GW LRP >> 'networks'? Why do we need them at all? >> >> >> -- >> regards, >> Vladislav Odintsov >> >> _______________________________________________ >> dev mailing list >> [email protected]<mailto:[email protected]> >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> > > For more details check the multinode.at<http://multinode.at> tests where > we have test with similar configuration. > > Regards, > Ales > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
