I'll just provide my overall opinion here. I'll use Cisco, Juniper, Linux
routing as examples. My sequential order of thought:
1.
OVN allows you to configure source based routes in the routing table. This is
not possible in any the other 3 routing implementations.
2.
I consider routing based on source IP as PBR.
3.
PBR pipelining: Cisco forces PBR evaluation before RIB lookup. Juniper allows
PBR evaluation before RIB lookup, as well as after. But evaluation after,
results in sending to another RIB table. Linux PBR is evaluated before routing
lookup as IP rules.
4.
OVN offers PBR expressed as routes in the route table, as well as PBR done by
logical route policies after route lookup.
So if we examine these implementations. All of the traditional options offer a
way to do PBR before RIB lookup. In my opinion, this is the most valuable use
case of PBR. By preferring dest over src, it is no longer possible to do the
same in OVN.
I agree that the current way of comparing longest prefix match is not ideal.
Personally, I think the ideal solution would be to offer PBR stage before RIB
lookup in OVN, and deprecate being able to program source routes in the RIB.
Practically I think it makes the most sense to leave the default behavior the
way it was, then offer options to set source or dest as the higher priority for
now.
For OVNK, we leveraged the longest prefix quirky behavior to do some silly
things that always bothered me. Routes like this:
[root@ovn-worker ~]# ovn-nbctl lr-route-list ovn_cluster_router
IPv4 Routes
Route Table <main>:
100.64.0.2 100.88.0.2 dst-ip
100.64.0.3 100.88.0.3 dst-ip
100.64.0.4 100.64.0.4 dst-ip <---- LOL
10.244.0.0/24 100.88.0.2 dst-ip
10.244.2.0/24 100.88.0.3 dst-ip
10.244.1.0/24 100.64.0.4 src-ip
10.244.0.0/16 100.64.0.4 src-ip
Where we use a /32 route to itself to override the src IP route later in the
list. This is an example of perferring "dest over source" routes. With IC we
don't need to use these source routes anymore, so plan on removing them.
We will still need source routes for MEG, so that a pod can be ecmp'ed to
different external gateways based on its source IP. In that case we will need
to use "source over dest" in OVNK.
Thanks,
-Tim
________________________________
From: Ilya Maximets <[email protected]>
Sent: Friday, September 5, 2025 9:01 AM
To: Dumitru Ceara <[email protected]>; Surya Seetharaman <[email protected]>;
Ales Musil <[email protected]>
Cc: [email protected] <[email protected]>; ovs dev <[email protected]>;
Jaime Caamaño Ruiz <[email protected]>; Girish Moodalbail
<[email protected]>; Nadia Pinaeva <[email protected]>; Tim Rozet
<[email protected]>
Subject: Re: [ovs-dev] OVN src route priority issue regarding 25.09 release
[Some people who received this message don't often get email from
[email protected]. Learn why this is important at
https://aka.ms/LearnAboutSenderIdentification ]
External email: Use caution opening links or attachments
On 9/5/25 2:44 PM, Dumitru Ceara wrote:
> On 9/5/25 2:39 PM, Dumitru Ceara wrote:
>> On 9/5/25 2:33 PM, Ilya Maximets wrote:
>>> On 9/5/25 2:13 PM, Dumitru Ceara wrote:
>>>> Hi Ilya,
>>>>
>>>> On 9/5/25 1:41 PM, Ilya Maximets wrote:
>>>>> On 9/5/25 9:16 AM, Surya Seetharaman via dev wrote:
>>>>>>
>>>>>> Also a side question for anyone who can answer (no need to derail the
>>>>>> original intent
>>>>>> of the thread), in real world routers how do things work? Is there a
>>>>>> reliable link someone
>>>>>> can provide pointing to a correct source page around this that I can read
>>>>>> just for my own understanding?
>>>>>
>>>>> I'm not an expert on how router appliances work, but I know a bit how
>>>>> routing
>>>>> works in linux and so in linux-based routers. From my understanding, all
>>>>> routing in linux is pretty much policy-based routing. There is a set of
>>>>> policy rules (ip rule) that performs packet matching using different
>>>>> criteria
>>>>> including the input port or source address. And this is pretty much the
>>>>> only place where you can do source based decision. Rules have priorities
>>>>> and the actions. Priorities make the logic transparent, i.e. it's obvious
>>>>> which rule will be evaluated first. Typical action is 'lookup ID' that
>>>>> performs a route lookup in the corresponding routing table. If there is
>>>>> no
>>>>> route, then the next priority policy rule is evaluated. Inside the
>>>>> routing
>>>>> table, the match is dst-only, so there is no problem there.
>>>>>
>>>>
>>>> What happens if there's a route match in the corresponding routing
>>>> table? I'm assuming the packet is just forwarded according to the next
>>>> hop of the route.
>>>
>>> Yes, if there is a match it just gets routed according to the match.
>>>
>>>>
>>>> In OVN (leaving aside source ip based routing) one can configure:
>>>> 1. a static route, e.g., dst=42.42.42.0/24 via 1.1.1.1
>>>> 2. a router policy (evaluated in the next stage, after static routes):
>>>> if dst == 42.42.42.42 reroute 1.1.1.2
>>>
>>> This policy just overrides the routing decision entirely. Why it needs
>>> to be a policy and not just another route? It has a larger prefix, so
>>> it will have higher priority. Or you can also keep it as a policy rule
>>> that matches on 42.42.42.42 destination and performs a lookup in a
>>> table with the only route being 1.1.1.2 default route.
>>>
>>
>> Sorry, I over-simplified. Let me try again with a different type of config:
>>
>> 1. a static route, e.g., dst=42.42.42.0/24 via 1.1.1.1
>> 2. a router policy:
>> if tcp.dst == 80 && dst == 42.0.0.0/8 reroute 1.1.1.2
>>
>
> rule: prio 1000, if tcp.dst == 80 then lookup 1000
> rule: prio 500, lookup 500
>
> table 1000: dst=42.0.0.0/8 via 1.1.1.2
> table 500: dst=42.42.42.0/24 via 1.1.1.1
>
> So we're probably OK. :)
Yep, should work.
Note: kernel priorities are inverted, i.e. 0 is the highest. And the
kernel also doesn't allow matching on L4 in the rules. The workaround
is to use netfilter to set a flow mark and then match on it in the rule,
but OVN can support any kind of matches in the policy rules, so it will
be more flexible than a kernel in that regard.
>
>> Note the larger subnet in the policy.
>>
>> I'm not sure ovn-k8s has something like this configured but I'm pretty
>> sure OVN would support it just fine today.
>>
>>>>
>>>> Essentially changing the routing decision (1) for a subset of the
>>>> traffic (dst 42.42.42.42).
>>>>
>>>> I'm trying to figure out how that would be configured in linux but i'm
>>>> having a hard time figuring it out.
>>>>
>>>> I'm asking because this is something commonly used in ovn-kubernetes today.
>>>>
>>>>> And there are sane defaults for the policy rules:
>>>>>
>>>>> $ ip rule
>>>>> 0: from all lookup local
>>>>> 32766: from all lookup main
>>>>> 32767: from all lookup default
>>>>>
>>>>> AFAIU, every router appliance manufacturer has their own logic on how
>>>>> policy
>>>>> based routing interacts with routing tables and in which order the source
>>>>> and destination matches are evaluated, so there is no standard there.
>>>>> But mixing the source and the destination during the prefix-based lookup
>>>>> seems unique to OVN and kind of strange indeed.
>>>>>
>>>>> For me it seems like what the linux kernel does is fairly flexible and
>>>>> less
>>>>> ambiguous. There is single entry point (policy rules), clear priorities
>>>>> and no mixing of src and dst during prefix lookups, as there is just no
>>>>> prefix lookup for src. So, maybe that's the architecture OVN should
>>>>> consider
>>>>> moving towards.
>>>>>
>>>>
>>>> It might be but, I guess, we need to make sure we're also flexible
>>>> enough to easily support all the use cases CMS currently have.
>>>>
>>>>> Best regards, Ilya Maximets.
>>>>>
>>>>
>>>> Regards,
>>>> Dumitru
>>>>
>>>
>
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
