Hi Dumitru, This seems much better than v1 to me, but I have some notes.
On Mon, Jun 1, 2026 at 9:50 AM Dumitru Ceara <[email protected]> wrote: > > Add a configuration and usage guide for OVN's dynamic routing > feature covering IP route exchange with external routing > daemons. > > The guide is incomplete on purpose, all of these configuration > options are already described in the main ovn-nb man page and > in the architecture document. The added value is some more > details and a few worked examples. > > The guide includes: > - Two worked examples: gateway router with BGP and > distributed router with gateway ports. > - Verification/troubleshooting commands and best practices. > > Also update the architecture document with information that > was missing: > - Add hub-spoke to the redistribution options list. > - Fix VRF table ID description to list the exact excluded > values (0, 252, 253, 254, 255, 4294967295). > - Add Routing Protocol Redirect section. > - Add Port-to-Interface Mapping section. > - Improve connected-as-host + local-only grouping. > > Also expand the dynamic-routing-v4-prefix-nexthop description > in ovn-nb.xml to explain the RFC 5549 use case. > > Reported-at: https://redhat.atlassian.net/browse/FDP-3119 > Assisted-by: Claude Opus 4.6, Claude Code > Signed-off-by: Dumitru Ceara <[email protected]> > --- > v2: > - Addressed Mark's comments: > - Renamed configuration.rst to configuration-examples.rst > and stripped the configuration reference sections that > duplicated the architecture document. The new document > focuses on worked examples with explanatory prose, > verification commands, and best practices. > - Added hub-spoke to the redistribution options list in > architecture.rst. > - Fixed VRF table ID accuracy in architecture.rst to list > all excluded values (0, 252-255, 4294967295) matching > the TABLE_ID_VALID macro. > - Improved connected-as-host + local-only grouping in > architecture.rst. > - Added Routing Protocol Redirect section to > architecture.rst. > - Added Port-to-Interface Mapping section to > architecture.rst. > - Expanded dynamic-routing-v4-prefix-nexthop description > in ovn-nb.xml with RFC 5549 explanation. > --- > Documentation/automake.mk | 1 + > .../topics/dynamic-routing/architecture.rst | 63 ++- > .../configuration-examples.rst | 366 ++++++++++++++++++ > .../topics/dynamic-routing/index.rst | 1 + > ovn-nb.xml | 8 +- > 5 files changed, 434 insertions(+), 5 deletions(-) > create mode 100644 > Documentation/topics/dynamic-routing/configuration-examples.rst > > diff --git a/Documentation/automake.mk b/Documentation/automake.mk > index aa286ebd9c..91582163e1 100644 > --- a/Documentation/automake.mk > +++ b/Documentation/automake.mk > @@ -27,6 +27,7 @@ DOC_SOURCE = \ > Documentation/topics/test-development.rst \ > Documentation/topics/high-availability.rst \ > Documentation/topics/dynamic-routing/architecture.rst \ > + Documentation/topics/dynamic-routing/configuration-examples.rst \ > Documentation/topics/dynamic-routing/index.rst \ > Documentation/topics/incremental-processing/datapath-sync-graph.png \ > Documentation/topics/incremental-processing/evpn-arp-graph.png \ > diff --git a/Documentation/topics/dynamic-routing/architecture.rst > b/Documentation/topics/dynamic-routing/architecture.rst > index e7c968330f..b436d4aa99 100644 > --- a/Documentation/topics/dynamic-routing/architecture.rst > +++ b/Documentation/topics/dynamic-routing/architecture.rst > @@ -288,6 +288,9 @@ accepts a comma-separated list of the following values: > - ``lb`` --- The VIP address of each load balancer associated with this > router and neighboring routers. > > +- ``hub-spoke`` --- Routes learned through OVN Interconnection (OVN-IC) > + from other routers, enabling hub-and-spoke propagation. > + > These options can also be set per logical router port, overriding the > router-level setting for routes associated with that specific port. > > @@ -331,6 +334,11 @@ this behavior: when set to ``true``, ``ovn-controller`` > only installs > routes on the chassis where the ``tracked_port`` is locally bound, > preventing other chassis from advertising the route at all. > > +This is particularly useful with ``connected-as-host`` redistribution > +to ensure host routes are only announced from the chassis that owns the > +workload, providing optimal traffic forwarding and avoiding unnecessary > +traffic tromboning. > + I think I wasn't clear with my intent with regards to this change. In this version, you've added the note saying that "This is particularly useful with connected-as-host..." after the paragraph about "dynamic-routing-redistribute-local-only" option. This is good! However, this same information is still in the document earlier in the paragraph explaining the "tracked_port" field. The result is that the documentation seems weirdly repetitive now. I think the sentence "This mechanism is particularly useful for host routes generated by the ``connected-as-host`` redistribution mode." should be removed from the paragraph discussing the "tracked_port" field. > IP Route Learning > ----------------- > > @@ -391,8 +399,9 @@ The VRF routing table ID is determined by one of the > following, in order > of precedence: > > 1. The ``dynamic-routing-vrf-id`` option on the logical router, if set to > - a valid integer (1-4294967295, excluding reserved table IDs such as > - ``RT_TABLE_MAIN`` and ``RT_TABLE_LOCAL``). > + a valid integer in the range (1-4294967294), excluding the reserved > + table IDs 252 (``RT_TABLE_COMPAT``), 253 (``RT_TABLE_DEFAULT``), > + 254 (``RT_TABLE_MAIN``), and 255 (``RT_TABLE_LOCAL``). > > 2. The tunnel key of the logical router datapath, used as a fallback > when ``dynamic-routing-vrf-id`` is not configured. > @@ -423,6 +432,56 @@ If ``dynamic-routing-maintain-vrf`` is ``false`` (the > default), the VRF > is expected to already exist on the chassis, managed by external tooling > or configuration management. > > +Routing Protocol Redirect > +------------------------- > + > +OVN can optionally redirect routing protocol control plane traffic from > +a logical router port to a logical switch port where an external routing > +daemon is listening. This allows the routing daemon to peer using the > +router port's IP addresses through an OVN-managed logical switch port. > + > +Routing protocol redirect is entirely optional. The routing daemon can > +establish its BGP or BFD sessions completely outside of OVN --- for > +example, on a separate physical interface, a loopback, or any other > +interface that is not managed by OVN. Use this feature only when you > +want the routing daemon to peer using the logical router port's IP > +addresses through an OVN-managed logical switch port. > + > +The ``routing-protocols`` option on a ``Logical_Router_Port`` specifies > +which control plane protocols to redirect: > + > +- ``BGP`` --- Redirects TCP port 179 (BGP session establishment and > + keepalives). > + > +- ``BFD`` --- Redirects UDP port 3784 (Bidirectional Forwarding > + Detection). > + > +The ``routing-protocol-redirect`` option on the same port identifies the > +logical switch port where the routing daemon is bound. OVN installs > +logical flows to forward matching control plane traffic between the > +router port and the specified switch port. > + > +For BGP unnumbered deployments (RFC 5549), periodic IPv6 Router > +Advertisements can be enabled on the logical router port via the > +``ipv6_ra_configs`` column. This provides automatic link-local address > +discovery, which is required for establishing BGP sessions over > +IPv6-only peering links that carry both IPv4 and IPv6 routes. > + > +Port-to-Interface Mapping > +------------------------- > + > +When a chassis has multiple links toward the fabric, each running BGP > +independently, the ``dynamic-routing-port-name`` option on a logical > +router port restricts route learning to a specific Linux interface. > +``ovn-controller`` resolves the port name to a Linux interface. If the > +referenced port is bound locally, the interface name is discovered > +automatically. > + > +When automatic discovery is not available (for example, the port is not > +bound locally), the ``dynamic-routing-port-mapping`` option on the > +``Open_vSwitch`` table provides an explicit mapping from logical port > +names to Linux interface names. > + > EVPN (Ethernet VPN) Integration > ------------------------------- > > diff --git a/Documentation/topics/dynamic-routing/configuration-examples.rst > b/Documentation/topics/dynamic-routing/configuration-examples.rst > new file mode 100644 > index 0000000000..4fc9be4718 > --- /dev/null > +++ b/Documentation/topics/dynamic-routing/configuration-examples.rst > @@ -0,0 +1,366 @@ > +.. > + Licensed under the Apache License, Version 2.0 (the "License"); you may > + not use this file except in compliance with the License. You may obtain > + a copy of the License at > + > + http://www.apache.org/licenses/LICENSE-2.0 > + > + Unless required by applicable law or agreed to in writing, software > + distributed under the License is distributed on an "AS IS" BASIS, > WITHOUT > + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See > the > + License for the specific language governing permissions and limitations > + under the License. > + > + Convention for heading levels in OVN documentation: > + > + ======= Heading 0 (reserved for the title in a document) > + ------- Heading 1 > + ~~~~~~~ Heading 2 > + +++++++ Heading 3 > + ''''''' Heading 4 > + > + Avoid deeper levels because they do not render well. > + > +====================================== > +Dynamic Routing Configuration Examples > +====================================== > + > +Introduction > +------------ > + > +This document provides worked configuration examples for OVN's dynamic This may be because my brain is American-English-pilled, but "worked configuration examples" sounds weird to me. In my head "working" would get the point across that these examples function properly. However, I think removing the word altogether and just saying "This document provides configuration examples" would be appropriate. > +routing feature. For the underlying architecture, data flow, and full > +description of each option, see > +:doc:`/topics/dynamic-routing/architecture` and ``ovn-nb``\(5). > + > +Configuration Examples > +---------------------- > + > +The following examples demonstrate complete configurations for common > +deployment scenarios. > + > +Example: Gateway Router with BGP > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +A centralized gateway router pinned to a specific chassis, peering > +with an external BGP speaker to advertise connected and static > +routes. > + > +:: > + > + External BGP Peer (AS 65001) > + | > + | eBGP peering > + | > + +---------+---------+ > + | Gateway Router | (lr-gw, pinned to chassis-1) > + | 10.0.0.1/24 | dynamic-routing = true > + +---------+---------+ > + | > + +---------+---------+ > + | Logical Switch | (ls-internal) > + | 10.0.0.0/24 | > + +---------+---------+ > + | > + VM1 VM2 VM3 Maybe I'm being overly nitpicky, but it seems odd that ls-fabric is not in the above diagram, especially since ls-fabric is included in later diagrams in the document. > + > +**Create the logical topology.** > +:: > + > + $ ovn-nbctl lr-add lr-gw > + $ ovn-nbctl ls-add ls-internal > + $ ovn-nbctl ls-add ls-fabric > + > + # Router port toward the internal network. > + $ ovn-nbctl lrp-add lr-gw lrp-internal \ > + 00:00:00:00:00:01 10.0.0.1/24 > + $ ovn-nbctl lsp-add-router-port ls-internal \ > + lsp-internal-to-gw lrp-internal > + > + # Router port toward the fabric. > + $ ovn-nbctl lrp-add lr-gw lrp-fabric \ > + 00:00:00:00:00:02 192.168.1.1/24 > + $ ovn-nbctl lsp-add-router-port ls-fabric \ > + lsp-fabric-to-gw lrp-fabric > + > + # Localnet port for physical connectivity. > + $ ovn-nbctl lsp-add-localnet-port ls-fabric \ > + lsp-fabric-ln physnet-fabric > + > +**Pin the router to a chassis and enable dynamic routing.** > + > +Enable IP route exchange on this logical router with > +``dynamic-routing=true``. The ``dynamic-routing-redistribute`` option > +controls which route types are advertised to external peers --- > +``connected`` advertises the subnet prefixes of the router's ports > +and ``static`` advertises all configured static routes. Set an > +explicit ``dynamic-routing-vrf-id`` so the VRF table ID is > +predictable in FRR configuration. > + > +:: > + > + $ ovn-nbctl set Logical_Router lr-gw \ > + options:chassis=chassis-1 \ > + options:dynamic-routing=true \ > + options:dynamic-routing-redistribute=connected,static \ > + options:dynamic-routing-vrf-id=100 > + > +**Configure VRF management and routing protocol redirect.** > + > +Setting ``dynamic-routing-maintain-vrf=true`` on the fabric-facing > +port lets ``ovn-controller`` create and delete the VRF automatically > +when the port is bound or unbound. > + > +A dedicated logical switch port (``lsp-bgp``) is added for the BGP > +daemon. The ``routing-protocols`` and ``routing-protocol-redirect`` > +options on the router port tell OVN to forward BGP (TCP 179) and BFD > +(UDP 3784) control plane traffic to that switch port, so FRR can peer > +using the router port's IP addresses. > + > +Periodic IPv6 Router Advertisements are enabled to support BGP > +unnumbered (RFC 5549), where the link-local nexthop is discovered > +automatically. > + > +:: > + > + # Let ovn-controller create/delete the VRF automatically. > + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ > + options:dynamic-routing-maintain-vrf=true > + > + # Add a logical switch port for the BGP daemon. > + $ ovn-nbctl lsp-add ls-fabric lsp-bgp > + $ ovn-nbctl lsp-set-addresses lsp-bgp unknown > + > + # Redirect BGP and BFD control plane traffic to lsp-bgp. > + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ > + options:routing-protocols=BGP,BFD \ > + options:routing-protocol-redirect=lsp-bgp > + > + # Enable periodic RAs for BGP unnumbered peer discovery. > + $ ovn-nbctl set Logical_Router_Port lrp-fabric \ > + ipv6_ra_configs:send_periodic=true \ > + ipv6_ra_configs:address_mode=slaac \ > + ipv6_ra_configs:max_interval=10 \ > + ipv6_ra_configs:min_interval=5 > + > +**Bind the BGP interface on the chassis.** > +:: > + > + # Create an OVS internal port bound to the BGP LSP. > + $ ovs-vsctl add-port br-int ext0-bgp -- \ > + set Interface ext0-bgp type=internal \ > + external-ids:iface-id=lsp-bgp > + > + # Place the interface into the VRF. > + $ ip link set dev ext0-bgp master ovnvrf100 > + $ ip link set dev ext0-bgp up > + > +**Configure FRR on the chassis.** > +:: > + > + configure terminal > + > + vrf ovnvrf100 > + exit-vrf > + > + router bgp 65000 vrf ovnvrf100 > + bgp router-id 192.168.1.1 > + neighbor ext0-bgp interface remote-as external > + address-family ipv4 unicast > + redistribute kernel > + exit-address-family > + address-family ipv6 unicast > + redistribute kernel > + neighbor ext0-bgp activate > + exit-address-family > + > +Example: Distributed Router with Gateway Ports > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +A distributed logical router with a distributed gateway port, > +advertising connected, NAT, and load balancer routes. The > +``local-only`` option ensures host routes are announced only from the > +chassis hosting the workload. > + > +:: > + > + External BGP Peer > + | > + | eBGP peering > + | > + +---------+---------+ > + | LS-Fabric | > + | 172.16.0.0/24 | > + +---------+---------+ > + | > + +-------------+-------------------+ > + | Distributed LR | > + | (lr-dist) | > + | dynamic-routing = true | > + | NAT: 172.16.0.10 -> 10.0.1.10 | > + | LB VIP: 172.16.0.100 | > + +--------+----------+-------------+ > + | | > + +-----+------+ +-+---------+ > + | LS-A | | LS-B | > + | 10.0.1/24 | | 10.0.2/24 | > + +-----+------+ +-----+-----+ > + | | > + VM-A1 VM-B1 > + (chassis-1) (chassis-2) > + > +**Create the distributed router with a gateway port.** > +:: > + > + $ ovn-nbctl lr-add lr-dist > + $ ovn-nbctl ls-add ls-a > + $ ovn-nbctl ls-add ls-b > + $ ovn-nbctl ls-add ls-fabric > + > + # Internal ports. > + $ ovn-nbctl lrp-add lr-dist lrp-a \ > + 00:00:00:00:01:01 10.0.1.1/24 > + $ ovn-nbctl lsp-add-router-port ls-a lsp-a-to-lr lrp-a > + > + $ ovn-nbctl lrp-add lr-dist lrp-b \ > + 00:00:00:00:01:02 10.0.2.1/24 > + $ ovn-nbctl lsp-add-router-port ls-b lsp-b-to-lr lrp-b > + > + # Distributed gateway port. > + $ ovn-nbctl lrp-add lr-dist lrp-gw \ > + 00:00:00:00:02:01 172.16.0.1/24 > + $ ovn-nbctl lsp-add-router-port ls-fabric \ > + lsp-fabric-to-lr lrp-gw > + > + # Configure HA chassis group for the gateway port. > + $ ovn-nbctl ha-chassis-group-add ha-gw > + $ ovn-nbctl ha-chassis-group-add-chassis ha-gw \ > + chassis-1 10 > + $ ovn-nbctl ha-chassis-group-add-chassis ha-gw \ > + chassis-2 5 > + $ GRP=$(ovn-nbctl --bare --columns=_uuid \ > + find HA_Chassis_Group name=ha-gw) > + $ ovn-nbctl set Logical_Router_Port lrp-gw \ > + ha_chassis_group=$GRP > + > +**Add NAT and load balancer.** > +:: > + > + $ ovn-nbctl lr-nat-add lr-dist dnat_and_snat \ > + 172.16.0.10 10.0.1.10 > + $ ovn-nbctl lb-add lb-web 172.16.0.100:80 \ > + 10.0.1.10:8080,10.0.2.10:8080 > + $ ovn-nbctl lr-lb-add lr-dist lb-web > + > +**Enable dynamic routing with NAT and LB redistribution.** > + > +Here ``dynamic-routing-redistribute`` includes ``nat`` (NAT external > +IPs) and ``lb`` (load balancer VIPs) in addition to ``connected`` > +subnets. Setting ``dynamic-routing-redistribute-local-only=true`` on > +the gateway port ensures these host routes are only advertised from > +the chassis where the tracked workload port is locally bound, > +avoiding unnecessary traffic tromboning. > + > +:: > + > + $ ovn-nbctl set Logical_Router lr-dist \ > + options:dynamic-routing=true \ > + options:dynamic-routing-redistribute=connected,nat,lb \ > + options:dynamic-routing-vrf-id=200 > + > + $ ovn-nbctl set Logical_Router_Port lrp-gw \ > + options:dynamic-routing-maintain-vrf=true \ > + options:dynamic-routing-redistribute-local-only=true > + > +With ``local-only`` enabled, NAT and LB host routes are only > +advertised on the chassis where their traffic is processed, > +ensuring optimal traffic forwarding. > + > +**Set up routing protocol redirect and FRR** (same pattern as the > +gateway router example above). > + > +Verification and Troubleshooting > +-------------------------------- IMO, this section does a good job with verification, but not troubleshooting. Troubleshooting implies that if something is wrong, we will provide guidance about how to fix what is wrong. If, for instance, we don't see the expected rows in the Advertised_Route table, nothing in the guide says what to do to fix this issue. The easiest fix would be to remove the word "Troubleshooting" from the section title. IMO, trying to guess how to troubleshoot such a complex setup would be tremendously difficult. > + > +Checking OVN State > +~~~~~~~~~~~~~~~~~~ > + > +Verify that ``ovn-northd`` has populated the ``Advertised_Route`` > +table:: > + > + $ ovn-sbctl list Advertised_Route > + > +Check for learned routes from external peers:: > + > + $ ovn-sbctl list Learned_Route > + > +Checking the VRF and Kernel Routes > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Verify the VRF exists and is up:: > + > + $ ip link show type vrf > + > +List routes in the VRF table. Routes marked ``proto ovn`` were > +installed by ``ovn-controller``; routes with other protocol values > +(e.g., ``proto bgp``) were learned from the routing daemon:: > + > + $ ip route show table 100 > + > +Checking FRR State > +~~~~~~~~~~~~~~~~~~ > + > +Verify BGP peering is established:: > + > + $ vtysh -c "show bgp summary" > + > +Check received and advertised routes:: > + > + $ vtysh -c "show bgp ipv4 unicast" > + $ vtysh -c "show bgp ipv6 unicast" > + > +Best Practices > +-------------- > + > +**Use explicit VRF IDs.** > +Set ``dynamic-routing-vrf-id`` rather than relying on the datapath > +tunnel key. Explicit IDs make the VRF configuration predictable and > +easier to reference in FRR configurations and monitoring. > + > +**Use local-only for host routes.** > +When using ``connected-as-host`` redistribution, combine it with > +``dynamic-routing-redistribute-local-only=true`` to ensure host > +routes are only announced from the chassis that owns the workload. > +This provides optimal traffic forwarding and avoids unnecessary > +traffic tromboning. > + > +**Let ovn-controller manage VRFs when possible.** > +Set ``dynamic-routing-maintain-vrf=true`` on the logical router port > +to let ``ovn-controller`` handle VRF creation and deletion. This > +simplifies chassis provisioning and ensures VRF lifecycle matches > +port binding. > + > +**Configure BFD for fast failure detection.** > +Include ``BFD`` in the ``routing-protocols`` option and enable BFD > +in FRR for sub-second failure detection. This significantly improves > +convergence time compared to relying on BGP hold timers alone. > + > +**Use per-port redistribution for multi-homed setups.** > +When a chassis has multiple fabric links, use per-port > +``dynamic-routing-redistribute`` and ``dynamic-routing-port-name`` to > +control exactly which routes are advertised and learned on each link. > + > +See Also > +-------- > + > +- :doc:`/topics/dynamic-routing/architecture` --- Architecture and > + internal design of the dynamic routing feature. > + > +- ``ovn-nb``\(5) --- Full reference for all ``Logical_Router`` > + and ``Logical_Router_Port`` dynamic routing options. > + > +- ``ovn-sb``\(5) --- Documentation of the ``Advertised_Route`` > + and ``Learned_Route`` tables. > + > +- ``ovn-controller``\(8) --- Controller-side configuration options > + including ``dynamic-routing-port-mapping``. > diff --git a/Documentation/topics/dynamic-routing/index.rst > b/Documentation/topics/dynamic-routing/index.rst > index eb61afb4a3..d5ef53d8ab 100644 > --- a/Documentation/topics/dynamic-routing/index.rst > +++ b/Documentation/topics/dynamic-routing/index.rst > @@ -29,3 +29,4 @@ Dynamic Routing > :maxdepth: 2 > > architecture > + configuration-examples > diff --git a/ovn-nb.xml b/ovn-nb.xml > index 38c6a84a29..22a53a15ad 100644 > --- a/ovn-nb.xml > +++ b/ovn-nb.xml > @@ -3655,9 +3655,11 @@ or > </p> > > <p> > - This defines nexthop used by IPv4 advertised routes instead of the > - routes being advertised as blackhole. This can be either valid IPv4 > - or IPv6 address. > + This defines the nexthop used by IPv4 advertised routes instead of > + the routes being advertised as blackhole. This can be either a > + valid IPv4 or IPv6 address. When an IPv6 address is used, it > + enables RFC 5549-style routing where IPv4 traffic is forwarded > + over an IPv6-only peering link. > </p> > </column> > > -- > 2.54.0 > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
