On 6/3/26 5:23 PM, Mark Michelson wrote: > Hi Dumitru, > > This seems much better than v1 to me, but I have some notes. >
Hi Mark, Thanks for the review! > On Mon, Jun 1, 2026 at 9:50 AM Dumitru Ceara <[email protected]> wrote: >> >> Add a configuration and usage guide for OVN's dynamic routing >> feature covering IP route exchange with external routing >> daemons. >> >> The guide is incomplete on purpose, all of these configuration >> options are already described in the main ovn-nb man page and >> in the architecture document. The added value is some more >> details and a few worked examples. >> >> The guide includes: >> - Two worked examples: gateway router with BGP and >> distributed router with gateway ports. >> - Verification/troubleshooting commands and best practices. >> >> Also update the architecture document with information that >> was missing: >> - Add hub-spoke to the redistribution options list. >> - Fix VRF table ID description to list the exact excluded >> values (0, 252, 253, 254, 255, 4294967295). >> - Add Routing Protocol Redirect section. >> - Add Port-to-Interface Mapping section. >> - Improve connected-as-host + local-only grouping. >> >> Also expand the dynamic-routing-v4-prefix-nexthop description >> in ovn-nb.xml to explain the RFC 5549 use case. >> >> Reported-at: https://redhat.atlassian.net/browse/FDP-3119 >> Assisted-by: Claude Opus 4.6, Claude Code >> Signed-off-by: Dumitru Ceara <[email protected]> >> --- >> v2: >> - Addressed Mark's comments: >> - Renamed configuration.rst to configuration-examples.rst >> and stripped the configuration reference sections that >> duplicated the architecture document. The new document >> focuses on worked examples with explanatory prose, >> verification commands, and best practices. >> - Added hub-spoke to the redistribution options list in >> architecture.rst. >> - Fixed VRF table ID accuracy in architecture.rst to list >> all excluded values (0, 252-255, 4294967295) matching >> the TABLE_ID_VALID macro. >> - Improved connected-as-host + local-only grouping in >> architecture.rst. >> - Added Routing Protocol Redirect section to >> architecture.rst. >> - Added Port-to-Interface Mapping section to >> architecture.rst. >> - Expanded dynamic-routing-v4-prefix-nexthop description >> in ovn-nb.xml with RFC 5549 explanation. >> --- >> Documentation/automake.mk | 1 + >> .../topics/dynamic-routing/architecture.rst | 63 ++- >> .../configuration-examples.rst | 366 ++++++++++++++++++ >> .../topics/dynamic-routing/index.rst | 1 + >> ovn-nb.xml | 8 +- >> 5 files changed, 434 insertions(+), 5 deletions(-) >> create mode 100644 >> Documentation/topics/dynamic-routing/configuration-examples.rst >> >> diff --git a/Documentation/automake.mk b/Documentation/automake.mk >> index aa286ebd9c..91582163e1 100644 >> --- a/Documentation/automake.mk >> +++ b/Documentation/automake.mk >> @@ -27,6 +27,7 @@ DOC_SOURCE = \ >> Documentation/topics/test-development.rst \ >> Documentation/topics/high-availability.rst \ >> Documentation/topics/dynamic-routing/architecture.rst \ >> + Documentation/topics/dynamic-routing/configuration-examples.rst \ >> Documentation/topics/dynamic-routing/index.rst \ >> Documentation/topics/incremental-processing/datapath-sync-graph.png \ >> Documentation/topics/incremental-processing/evpn-arp-graph.png \ >> diff --git a/Documentation/topics/dynamic-routing/architecture.rst >> b/Documentation/topics/dynamic-routing/architecture.rst >> index e7c968330f..b436d4aa99 100644 >> --- a/Documentation/topics/dynamic-routing/architecture.rst >> +++ b/Documentation/topics/dynamic-routing/architecture.rst >> @@ -288,6 +288,9 @@ accepts a comma-separated list of the following values: >> - ``lb`` --- The VIP address of each load balancer associated with this >> router and neighboring routers. >> >> +- ``hub-spoke`` --- Routes learned through OVN Interconnection (OVN-IC) >> + from other routers, enabling hub-and-spoke propagation. >> + >> These options can also be set per logical router port, overriding the >> router-level setting for routes associated with that specific port. >> >> @@ -331,6 +334,11 @@ this behavior: when set to ``true``, ``ovn-controller`` >> only installs >> routes on the chassis where the ``tracked_port`` is locally bound, >> preventing other chassis from advertising the route at all. >> >> +This is particularly useful with ``connected-as-host`` redistribution >> +to ensure host routes are only announced from the chassis that owns the >> +workload, providing optimal traffic forwarding and avoiding unnecessary >> +traffic tromboning. >> + > > I think I wasn't clear with my intent with regards to this change. In > this version, you've added the note saying that "This is particularly > useful with connected-as-host..." after the paragraph about > "dynamic-routing-redistribute-local-only" option. This is good! > However, this same information is still in the document earlier in the > paragraph explaining the "tracked_port" field. The result is that the > documentation seems weirdly repetitive now. I think the sentence "This > mechanism is particularly useful for host routes generated by the > ``connected-as-host`` redistribution mode." should be removed from the > paragraph discussing the "tracked_port" field. > Ah I missed that part, fixed up in v3. >> IP Route Learning >> ----------------- >> >> @@ -391,8 +399,9 @@ The VRF routing table ID is determined by one of the >> following, in order >> of precedence: >> >> 1. The ``dynamic-routing-vrf-id`` option on the logical router, if set to >> - a valid integer (1-4294967295, excluding reserved table IDs such as >> - ``RT_TABLE_MAIN`` and ``RT_TABLE_LOCAL``). >> + a valid integer in the range (1-4294967294), excluding the reserved >> + table IDs 252 (``RT_TABLE_COMPAT``), 253 (``RT_TABLE_DEFAULT``), >> + 254 (``RT_TABLE_MAIN``), and 255 (``RT_TABLE_LOCAL``). >> >> 2. The tunnel key of the logical router datapath, used as a fallback >> when ``dynamic-routing-vrf-id`` is not configured. >> @@ -423,6 +432,56 @@ If ``dynamic-routing-maintain-vrf`` is ``false`` (the >> default), the VRF >> is expected to already exist on the chassis, managed by external tooling >> or configuration management. >> >> +Routing Protocol Redirect >> +------------------------- >> + >> +OVN can optionally redirect routing protocol control plane traffic from >> +a logical router port to a logical switch port where an external routing >> +daemon is listening. This allows the routing daemon to peer using the >> +router port's IP addresses through an OVN-managed logical switch port. >> + >> +Routing protocol redirect is entirely optional. The routing daemon can >> +establish its BGP or BFD sessions completely outside of OVN --- for >> +example, on a separate physical interface, a loopback, or any other >> +interface that is not managed by OVN. Use this feature only when you >> +want the routing daemon to peer using the logical router port's IP >> +addresses through an OVN-managed logical switch port. >> + >> +The ``routing-protocols`` option on a ``Logical_Router_Port`` specifies >> +which control plane protocols to redirect: >> + >> +- ``BGP`` --- Redirects TCP port 179 (BGP session establishment and >> + keepalives). >> + >> +- ``BFD`` --- Redirects UDP port 3784 (Bidirectional Forwarding >> + Detection). >> + >> +The ``routing-protocol-redirect`` option on the same port identifies the >> +logical switch port where the routing daemon is bound. OVN installs >> +logical flows to forward matching control plane traffic between the >> +router port and the specified switch port. >> + >> +For BGP unnumbered deployments (RFC 5549), periodic IPv6 Router >> +Advertisements can be enabled on the logical router port via the >> +``ipv6_ra_configs`` column. This provides automatic link-local address >> +discovery, which is required for establishing BGP sessions over >> +IPv6-only peering links that carry both IPv4 and IPv6 routes. >> + >> +Port-to-Interface Mapping >> +------------------------- >> + >> +When a chassis has multiple links toward the fabric, each running BGP >> +independently, the ``dynamic-routing-port-name`` option on a logical >> +router port restricts route learning to a specific Linux interface. >> +``ovn-controller`` resolves the port name to a Linux interface. If the >> +referenced port is bound locally, the interface name is discovered >> +automatically. >> + >> +When automatic discovery is not available (for example, the port is not >> +bound locally), the ``dynamic-routing-port-mapping`` option on the >> +``Open_vSwitch`` table provides an explicit mapping from logical port >> +names to Linux interface names. >> + >> EVPN (Ethernet VPN) Integration >> ------------------------------- >> >> diff --git a/Documentation/topics/dynamic-routing/configuration-examples.rst >> b/Documentation/topics/dynamic-routing/configuration-examples.rst >> new file mode 100644 >> index 0000000000..4fc9be4718 >> --- /dev/null >> +++ b/Documentation/topics/dynamic-routing/configuration-examples.rst >> @@ -0,0 +1,366 @@ >> +.. >> + Licensed under the Apache License, Version 2.0 (the "License"); you >> may >> + not use this file except in compliance with the License. You may >> obtain >> + a copy of the License at >> + >> + http://www.apache.org/licenses/LICENSE-2.0 >> + >> + Unless required by applicable law or agreed to in writing, software >> + distributed under the License is distributed on an "AS IS" BASIS, >> WITHOUT >> + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See >> the >> + License for the specific language governing permissions and >> limitations >> + under the License. >> + >> + Convention for heading levels in OVN documentation: >> + >> + ======= Heading 0 (reserved for the title in a document) >> + ------- Heading 1 >> + ~~~~~~~ Heading 2 >> + +++++++ Heading 3 >> + ''''''' Heading 4 >> + >> + Avoid deeper levels because they do not render well. >> + >> +====================================== >> +Dynamic Routing Configuration Examples >> +====================================== >> + >> +Introduction >> +------------ >> + >> +This document provides worked configuration examples for OVN's dynamic > > This may be because my brain is American-English-pilled, but "worked > configuration examples" sounds weird to me. In my head "working" would > get the point across that these examples function properly. However, I > think removing the word altogether and just saying "This document > provides configuration examples" would be appropriate. > Yeah, that has nothing to do with American-English or any kind of English actually. It was my AI helper making things up and me missing it. I'll remove "worked", it makes no sense. >> +routing feature. For the underlying architecture, data flow, and full >> +description of each option, see >> +:doc:`/topics/dynamic-routing/architecture` and ``ovn-nb``\(5). >> + >> +Configuration Examples >> +---------------------- >> + >> +The following examples demonstrate complete configurations for common >> +deployment scenarios. >> + >> +Example: Gateway Router with BGP >> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> + >> +A centralized gateway router pinned to a specific chassis, peering >> +with an external BGP speaker to advertise connected and static >> +routes. >> + >> +:: >> + >> + External BGP Peer (AS 65001) >> + | >> + | eBGP peering >> + | >> + +---------+---------+ >> + | Gateway Router | (lr-gw, pinned to chassis-1) >> + | 10.0.0.1/24 | dynamic-routing = true >> + +---------+---------+ >> + | >> + +---------+---------+ >> + | Logical Switch | (ls-internal) >> + | 10.0.0.0/24 | >> + +---------+---------+ >> + | >> + VM1 VM2 VM3 > > Maybe I'm being overly nitpicky, but it seems odd that ls-fabric is > not in the above diagram, especially since ls-fabric is included in > later diagrams in the document. > You're right, it's better to have similar diagrams. >> + >> +**Create the logical topology.** >> +:: >> + >> + $ ovn-nbctl lr-add lr-gw >> + $ ovn-nbctl ls-add ls-internal >> + $ ovn-nbctl ls-add ls-fabric >> + >> + # Router port toward the internal network. >> + $ ovn-nbctl lrp-add lr-gw lrp-internal \ >> + 00:00:00:00:00:01 10.0.0.1/24 >> + $ ovn-nbctl lsp-add-router-port ls-internal \ >> + lsp-internal-to-gw lrp-internal >> + >> + # Router port toward the fabric. >> + $ ovn-nbctl lrp-add lr-gw lrp-fabric \ >> + 00:00:00:00:00:02 192.168.1.1/24 >> + $ ovn-nbctl lsp-add-router-port ls-fabric \ >> + lsp-fabric-to-gw lrp-fabric >> + >> + # Localnet port for physical connectivity. >> + $ ovn-nbctl lsp-add-localnet-port ls-fabric \ >> + lsp-fabric-ln physnet-fabric >> + >> +**Pin the router to a chassis and enable dynamic routing.** >> + >> +Enable IP route exchange on this logical router with >> +``dynamic-routing=true``. The ``dynamic-routing-redistribute`` option >> +controls which route types are advertised to external peers --- >> +``connected`` advertises the subnet prefixes of the router's ports >> +and ``static`` advertises all configured static routes. Set an >> +explicit ``dynamic-routing-vrf-id`` so the VRF table ID is >> +predictable in FRR configuration. >> + >> +:: >> + >> + $ ovn-nbctl set Logical_Router lr-gw \ >> + options:chassis=chassis-1 \ >> + options:dynamic-routing=true \ >> + options:dynamic-routing-redistribute=connected,static \ >> + options:dynamic-routing-vrf-id=100 >> + >> +**Configure VRF management and routing protocol redirect.** >> + >> +Setting ``dynamic-routing-maintain-vrf=true`` on the fabric-facing >> +port lets ``ovn-controller`` create and delete the VRF automatically >> +when the port is bound or unbound. >> + >> +A dedicated logical switch port (``lsp-bgp``) is added for the BGP >> +daemon. The ``routing-protocols`` and ``routing-protocol-redirect`` >> +options on the router port tell OVN to forward BGP (TCP 179) and BFD >> +(UDP 3784) control plane traffic to that switch port, so FRR can peer >> +using the router port's IP addresses. >> + >> +Periodic IPv6 Router Advertisements are enabled to support BGP >> +unnumbered (RFC 5549), where the link-local nexthop is discovered >> +automatically. >> + >> +:: >> + >> + # Let ovn-controller create/delete the VRF automatically. >> + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ >> + options:dynamic-routing-maintain-vrf=true >> + >> + # Add a logical switch port for the BGP daemon. >> + $ ovn-nbctl lsp-add ls-fabric lsp-bgp >> + $ ovn-nbctl lsp-set-addresses lsp-bgp unknown >> + >> + # Redirect BGP and BFD control plane traffic to lsp-bgp. >> + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ >> + options:routing-protocols=BGP,BFD \ >> + options:routing-protocol-redirect=lsp-bgp >> + >> + # Enable periodic RAs for BGP unnumbered peer discovery. >> + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ >> + ipv6_ra_configs:send_periodic=true \ >> + ipv6_ra_configs:address_mode=slaac \ >> + ipv6_ra_configs:max_interval=10 \ >> + ipv6_ra_configs:min_interval=5 >> + >> +**Bind the BGP interface on the chassis.** >> +:: >> + >> + # Create an OVS internal port bound to the BGP LSP. >> + $ ovs-vsctl add-port br-int ext0-bgp -- \ >> + set Interface ext0-bgp type=internal \ >> + external-ids:iface-id=lsp-bgp >> + >> + # Place the interface into the VRF. >> + $ ip link set dev ext0-bgp master ovnvrf100 >> + $ ip link set dev ext0-bgp up >> + >> +**Configure FRR on the chassis.** >> +:: >> + >> + configure terminal >> + >> + vrf ovnvrf100 >> + exit-vrf >> + >> + router bgp 65000 vrf ovnvrf100 >> + bgp router-id 192.168.1.1 >> + neighbor ext0-bgp interface remote-as external >> + address-family ipv4 unicast >> + redistribute kernel >> + exit-address-family >> + address-family ipv6 unicast >> + redistribute kernel >> + neighbor ext0-bgp activate >> + exit-address-family >> + >> +Example: Distributed Router with Gateway Ports >> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> + >> +A distributed logical router with a distributed gateway port, >> +advertising connected, NAT, and load balancer routes. The >> +``local-only`` option ensures host routes are announced only from the >> +chassis hosting the workload. >> + >> +:: >> + >> + External BGP Peer >> + | >> + | eBGP peering >> + | >> + +---------+---------+ >> + | LS-Fabric | >> + | 172.16.0.0/24 | >> + +---------+---------+ >> + | >> + +-------------+-------------------+ >> + | Distributed LR | >> + | (lr-dist) | >> + | dynamic-routing = true | >> + | NAT: 172.16.0.10 -> 10.0.1.10 | >> + | LB VIP: 172.16.0.100 | >> + +--------+----------+-------------+ >> + | | >> + +-----+------+ +-+---------+ >> + | LS-A | | LS-B | >> + | 10.0.1/24 | | 10.0.2/24 | >> + +-----+------+ +-----+-----+ >> + | | >> + VM-A1 VM-B1 >> + (chassis-1) (chassis-2) >> + >> +**Create the distributed router with a gateway port.** >> +:: >> + >> + $ ovn-nbctl lr-add lr-dist >> + $ ovn-nbctl ls-add ls-a >> + $ ovn-nbctl ls-add ls-b >> + $ ovn-nbctl ls-add ls-fabric >> + >> + # Internal ports. >> + $ ovn-nbctl lrp-add lr-dist lrp-a \ >> + 00:00:00:00:01:01 10.0.1.1/24 >> + $ ovn-nbctl lsp-add-router-port ls-a lsp-a-to-lr lrp-a >> + >> + $ ovn-nbctl lrp-add lr-dist lrp-b \ >> + 00:00:00:00:01:02 10.0.2.1/24 >> + $ ovn-nbctl lsp-add-router-port ls-b lsp-b-to-lr lrp-b >> + >> + # Distributed gateway port. >> + $ ovn-nbctl lrp-add lr-dist lrp-gw \ >> + 00:00:00:00:02:01 172.16.0.1/24 >> + $ ovn-nbctl lsp-add-router-port ls-fabric \ >> + lsp-fabric-to-lr lrp-gw >> + >> + # Configure HA chassis group for the gateway port. >> + $ ovn-nbctl ha-chassis-group-add ha-gw >> + $ ovn-nbctl ha-chassis-group-add-chassis ha-gw \ >> + chassis-1 10 >> + $ ovn-nbctl ha-chassis-group-add-chassis ha-gw \ >> + chassis-2 5 >> + $ GRP=$(ovn-nbctl --bare --columns=_uuid \ >> + find HA_Chassis_Group name=ha-gw) >> + $ ovn-nbctl set Logical_Router_Port lrp-gw \ >> + ha_chassis_group=$GRP >> + >> +**Add NAT and load balancer.** >> +:: >> + >> + $ ovn-nbctl lr-nat-add lr-dist dnat_and_snat \ >> + 172.16.0.10 10.0.1.10 >> + $ ovn-nbctl lb-add lb-web 172.16.0.100:80 \ >> + 10.0.1.10:8080,10.0.2.10:8080 >> + $ ovn-nbctl lr-lb-add lr-dist lb-web >> + >> +**Enable dynamic routing with NAT and LB redistribution.** >> + >> +Here ``dynamic-routing-redistribute`` includes ``nat`` (NAT external >> +IPs) and ``lb`` (load balancer VIPs) in addition to ``connected`` >> +subnets. Setting ``dynamic-routing-redistribute-local-only=true`` on >> +the gateway port ensures these host routes are only advertised from >> +the chassis where the tracked workload port is locally bound, >> +avoiding unnecessary traffic tromboning. >> + >> +:: >> + >> + $ ovn-nbctl set Logical_Router lr-dist \ >> + options:dynamic-routing=true \ >> + options:dynamic-routing-redistribute=connected,nat,lb \ >> + options:dynamic-routing-vrf-id=200 >> + >> + $ ovn-nbctl set Logical_Router_Port lrp-gw \ >> + options:dynamic-routing-maintain-vrf=true \ >> + options:dynamic-routing-redistribute-local-only=true >> + >> +With ``local-only`` enabled, NAT and LB host routes are only >> +advertised on the chassis where their traffic is processed, >> +ensuring optimal traffic forwarding. >> + >> +**Set up routing protocol redirect and FRR** (same pattern as the >> +gateway router example above). >> + >> +Verification and Troubleshooting >> +-------------------------------- > > IMO, this section does a good job with verification, but not > troubleshooting. Troubleshooting implies that if something is wrong, > we will provide guidance about how to fix what is wrong. If, for > instance, we don't see the expected rows in the Advertised_Route > table, nothing in the guide says what to do to fix this issue. > > The easiest fix would be to remove the word "Troubleshooting" from the > section title. IMO, trying to guess how to troubleshoot such a complex > setup would be tremendously difficult. > I'll do that. I'll see if I can think of troubleshooting guidelines for the future. Thanks, all, for the reviews! I posted v3 here: https://patchwork.ozlabs.org/project/ovn/patch/[email protected]/ Regards, Dumitru _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
