Hi Vladislav, Ales,
On 4/15/26 6:11 PM, Odintsov Vladislav wrote:
> Hi Ales,
>
> Thanks for the reply. Please, see my comment inline.
>
> Ales Musil wrote:
>> On Sat, Apr 11, 2026 at 12:01 PM Odintsov Vladislav <[email protected]>
>> wrote:
>>> Hi all,
>>>
>>> I'm currently trying to adopt OVN native BGP/EVPN support to our cloud'
>>> usecases and faced one problem:
>>>
>>> Imagine topology of one "external" VTEP node and 2 OVN-managed hosts:
>>> one of them is a gateway chassis, and another is just a normal workload
>>> chassis (without EVPN connectivity).
>>>
>>> XXXXXXXXXXXXXXX
>>> X X
>>> XXXXXXXXXXXXXXXXX IP fabric XXXXXXXXXXXXXXX
>>> X X X X
>>> X XXXXXXXXXXXXXXX X
>>> X X X
>>> X X X
>>> XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX
>>> X X X X X X
>>> X Ext VTEP X X OVN GW X X OVN w/l X
>>> X X X X X X
>>> XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX
>>> 10.128.0.5 10.128.0.14 10.128.0.6
>>>
>>>
>>
>> Hi Vladislav,
>>
>> the graph is a bit distorted but I think I have an idea what might
>> be wrong, see down below.
>>
>>>
>>>
>>> VTEP node has configured VRF linked to VNI 300 and announces Type-5
>>> route with 172.31.0.0/24<http://172.31.0.0/24> IP prefix.
>>>
>>> [root@vtep01 ~]# ip a s br-300
>>> 19: br-300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>> master ovnvrf300 state UP group default qlen 1000
>>> link/ether 00:04:00:00:00:01 brd ff:ff:ff:ff:ff:ff
>>> inet 172.31.0.1/24<http://172.31.0.1/24> scope global br-300
>>> valid_lft forever preferred_lft forever
>>> [root@vtep01 ~]# ip a s ovnvrf300
>>> 18: ovnvrf300: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state
>>> UP group default qlen 1000
>>> link/ether c2:9d:fb:b7:74:6f brd ff:ff:ff:ff:ff:ff
>>> [root@vtep01 ~]# ip a s vxlan300
>>> 23: vxlan300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>> master br-300 state UNKNOWN group default qlen 1000
>>> link/ether 2a:39:b8:a7:b6:80 brd ff:ff:ff:ff:ff:ff
>>> inet6 fe80::2839:b8ff:fea7:b680/64 scope link
>>> valid_lft forever preferred_lft forever
>>>
>>> OVN has EVPN-configured edge LR and localnet-enabled LS with static
>>> route "10.1.0.0/24" with IPv6 LLA nexthop and output LRP which leads to LR1.
>>>
>>> LR1 has two LRPs: one to edge LR, another is a link to "workloads".
>>>
>>> "workload" LS, which has 2 LSPs: one on gateway node, another on the
>>> "workload" node.
>>>
>>> OVN topology is as next:
>>>
>>> switch b534bd4c-02bb-4428-a834-fbb21b59d79f (ls1)
>>> port ls1-lr1
>>> type: router
>>> router-port: lr1-downlink
>>> port lsp2
>>> addresses: ["00:00:00:00:00:02 10.1.0.102"]
>>> port lsp1
>>> addresses: ["00:00:00:00:00:01 10.1.0.101"]
>>> switch f03fa0e5-46a9-49b8-91b7-2e506977170c (test-ovn1-uplink)
>>> port ln-node2
>>> type: router
>>> router-port: edge1-uplink2
>>> port test-ovn1-uplink-lsp
>>> type: localnet
>>> addresses: ["unknown"]
>>> port test-ovn1-edge1
>>> type: router
>>> router-port: edge1-uplink
>>> router d322ecc0-c225-49f0-aa37-7c3003097714 (edge1)
>>> port edge1-lr1
>>> mac: "00:02:00:00:00:01"
>>> ipv6-lla: "fe80::202:ff:fe00:1"
>>> port edge1-uplink
>>> mac: "0a:00:aa:21:e5:e0"
>>
>> I think the problem here is actually the MAC address, it should
>> be same as the br-300. Think of the LRP as br-300 representation
>> in the OVN topology.
>>
>
> br-300 interface configuration on the ovn side and corresponding ovn
> edge LR are next:
>
> [root@test-ovn1 ~]# ip li show br-300
> 43: br-300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> master ovnvrf300 state UP mode DEFAULT group default qlen 1000
> link/ether 0a:00:aa:21:e5:e0 brd ff:ff:ff:ff:ff:ff
> [root@test-ovn1 ~]# ovn-nbctl show edge1
> router d322ecc0-c225-49f0-aa37-7c3003097714 (edge1)
> port edge1-lr1
> mac: "00:02:00:00:00:01"
> ipv6-lla: "fe80::202:ff:fe00:1"
> port edge1-uplink
> mac: "0a:00:aa:21:e5:e0"
> ipv6-lla: "fe80::800:aaff:fe21:e5e0"
> networks: ["10.128.0.101/20"]
> gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9]
>
> So, their mac addresses are the same. Moreover, IIUC, the problem is
> MAC/IP resolving for the remote side ovn the ovn chassis, which has no
> EVPN connectivity. Isn't it?
> I've checked multinode testsuite and couldn't find any test, which
> correlates to partially-connected (only GW nodes) scenarios. Only 'ovn
> multinode bgp L3 EVPN'. If you point me to the test - I'd be very happy
> and appreciate it.
>
>>> ipv6-lla: "fe80::800:aaff:fe21:e5e0"
>>> networks: ["10.128.0.101/20<http://10.128.0.101/20>"] <--- I'm
>>> not sure it is correct
>>> gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9]
>>> router fab999fe-6ef7-478e-9b42-494cafe8f4b1 (lr1)
>>> port lr1-downlink
>>> mac: "00:03:00:00:00:01"
>>> ipv6-lla: "fe80::203:ff:fe00:1"
>>> networks: ["10.1.0.1/24<http://10.1.0.1/24>"]
>>> port lr1-edge1
>>> mac: "00:02:00:00:00:02"
>>> ipv6-lla: "fe80::202:ff:fe00:2"
>>>
>>> # ovn-nbctl lr-route-list edge1
>>> IPv4 Routes
>>> Route Table <main>:
>>> 10.1.0.0/16<http://10.1.0.0/16> fe80::202:ff:fe00:2
>>> dst-ip edge1-lr1
>>> # ovn-nbctl lr-route-list lr1
>>> IPv4 Routes
>>> Route Table <main>:
>>> 0.0.0.0/0<http://0.0.0.0/0> fe80::202:ff:fe00:1
>>> dst-ip lr1-edge1
>>>
>>> # ovn-nbctl get logical-switch test-ovn1-uplink other_config
>>> {dynamic-routing-advertise-ifname=lo-300,
>>> dynamic-routing-bridge-ifname=br-300,
>>> dynamic-routing-redistribute="fdb,ip", dynamic-routing-vni="300",
>>> dynamic-routing-vxlan-ifname=vxlan300}
>>>
>>> # ovn-nbctl get logical-router edge1 options
>>> {dynamic-routing="true", dynamic-routing-redistribute=static,
>>> dynamic-routing-vrf-id="300"}
>>>
>>> # ovn-nbctl get logical-router-port edge1-uplink options
>>> {dynamic-routing-maintain-vrf="true",
>>> dynamic-routing-redistribute=connected}
>>>
>>> So, if I start pinging from VTEP's VRF (from 172.31.0.0/24) to LSP1
>>> (10.1.0.101), which is on the gateway node, it works fine.
>>>
>>> On the OVN GW node the evpn commands have next output:
>>>
>>> # ovn-appctl evpn/remote-vtep-list
>>> IP: 10.128.0.5, port: 4789, vni: 300
>>>
>>> # ovn-appctl evpn/vtep-arp-list
>>> UUID: 7e3f9194-0699-4ed2-83c3-73b474c1b2b1, VNI: 300, MAC:
>>> 00:04:00:00:00:01, IP: 10.128.0.5, dp_key: 5
>>>
>>> # ovn-appctl evpn/vtep-binding-list
>>> UUID: 826cd9a5-d582-4a3a-85d9-2f6471c0afa6, Remote IP: 10.128.0.5, vni:
>>> 300, binding_key: 0x80000015, tunnel_ofport: 10, dp_key: 5
>>>
>>> # ovn-appctl evpn/vtep-fdb-list
>>> UUID: c44b7556-e8f9-4bce-9d8b-9c916b79d105, MAC: 00:04:00:00:00:01, vni:
>>> 300, binding_key: 0x80000015, dp_key: 5
>>>
>>> # ovn-appctl evpn/vtep-multicast-group-list
>>> UUID: 81f25768-6a40-497e-b48f-e17faa91e0e7, Remote IPs: 10.128.0.5, vni: 300
>>>
>>>
>>> PROBLEM:
>>>
>>> BUT, if I start pinging from VTEP node to LSP2 (10.1.0.102), it fails.
>>> ICMP request goes from VTEP node to OVN GW node via VXLAN VNI300 (fine),
>>> then OVN GW sends this traffic inside GENEVE to OVN w/l node and ICMP
>>> request is seen on the VIF (lsp2).
>>>
>>> ICMP response is sent from lsp2, but I see ARP request in the GENEVE -
>>> OVN w/l node doesn't know br-300 MAC address on the VTEP node (which is
>>> the dst MAC for ICMP response).
I think what's going on here is that, because "edge1" is a distributed
router with a DGP ("edge1-uplink") when the ICMP reply from lsp2 hits
the "edge1" ingress pipeline that happens on the ovn-workload chassis
(so not the one where the DGP is bound, also not the one where OVN's
EVPN is running).
That packet will hit the learned route (that's synced through the
Southbound) with a next hop IP reachable via EVPN.
>>>
>>> On then one hand that is expected - EVPN mac addresses are stored in the
>>> ovn-controller runtime and not committed to OVN SB, so OVN w/l node
>>> doesn't know which mac to use.
>>>
As you said above, the nexthop MAC address is only stored on the OVN GW
node, in ovn-controller memory (and in OpenFlow rules to implement the
ARP proxy behavior).
So, still in the ingress pipeline of "edge1", on ovn-workload, we'll try
to resolve the next hop IP and fail, because as you said there's no SB
mac binding for it and no local OpenFlow rules for the ARP proxy behavior.
The IP packet goes then to ovn-controller _still on ovn-workload_, which
buffers it, and crafts an ARP request for the EVPN next hop IP and
injects it into the router pipeline. This packet is sent through GENEVE
and that's what you're seeing.
Claude summarized this a bit better than I did :)
```
For the ICMP reply from lsp2 (10.1.0.102) → VTEP (172.31.0.1):
1. Workload chassis processes: ls1 → lr1 → edge1 (logical ingress
pipeline runs on the originating chassis)
2. edge1's routing determines: outport = edge1-uplink, nexthop =
10.128.0.5 (VTEP IP)
3. In lr_in_arp_resolve, the workload chassis executes
lookup_arp(edge1-uplink, 10.128.0.5) against OFTABLE_MAC_BINDING
4. This lookup fails because the EVPN-learned MAC (00:04:00:00:00:01)
for IP 10.128.0.5 only exists as an OpenFlow flow on the gateway chassis
5. The workload chassis falls back to sending an ARP request — which
is exactly what the user observes in the GENEVE tunnel
```
>>> On the other hand, maybe I've lost some configuration hints and such
>>> topology can work as well?
I'm not sure it's possible or that easy to sync the learned evpn/arps in
the SB database. We should look into it though.
Nevertheless, what we'd need to change in the config today to try to
make it work for your use case is to ensure that the ingress pipeline of
"edge1" is ran on the OVN GW chassis.
I think we have two options:
A. Turn "edge1" into a "gateway router" (options:chassis="<OVN-GW>") ->
you lose the HA you might have today for your edge1-uplink router port.
NOTE: there's no real HA configured now in your setup for that port,
it's only "gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9]" but
that could change I guess.
B. Force "edge1"'s downlink router port to be bound to the same chassis
as the uplink port, i.e., configure "edge1-lr1" with the same "gateway
chassis" as "edge1-uplink" (or both with the same HA_Chassis_Group).
I think I'd prefer B if it works and if this were my deployment.
Vladislav, would you be able to test it out by any chance?
If these two config changes work, it would actually be nice if we could
make sure this is validated with multinode tests too - so we could add
two new scenarios using the configs you suggested + A or B above.
Regards,
Dumitru
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
