On Fri, Sep 18, 2020 at 12:58 AM Alexander Constantinescu <
acons...@redhat.com> wrote:
> Hi Han
>
> Sorry for the late reply.
>
> Is this the current situation?
>>
>
> Yes, it is.
>
> When you say there are too many default routes, what do you mean in the
>> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>>
>
> Each corresponds to a node in our cluster, like this:
>
> ip4.src == && ip4.dst ==
> , allow
> ip6.src == && ip6.dst ==
> , allow
> ip4.src == && ip4.dst ==
> , allow
> ...
> ip6.src == && ip6.dst ==
> , allow
>
> so on large clusters (say 1000 nodes) with IPv6 and IPv4 enabled we can
> reach ~2000 logical router policies. By having the SOUTH_TO_NORTH_IP we can
> completely remove all of them and have the "default route" logical router
> policy specify:
>
> default route (lowest priority): ip4.src ==
> ip4.dst == SOUTH_TO_NORTH_IP, nexthop =
>
> In addition, if SOUTH_TO_NORTH_IP is a user defined IP,
>>
>
> I didn't think it should be user defined, more so "system defined", like
> 0.0.0.0/0
>
> I am not sure how would it work, because ip4.dst is the dst IP from
>> packet header
>>
>
> I didn't intend for such an IP to be used solely as a destination IP, but
> as source too, if the user requires it.
>
> Comparing it with SOUTH_TO_NORTH_IP would just result in mismatch, unless
>> all south-to-north traffic really has this IP as destination (I guess
>> that's not the case).
>>
>
> Sure, I just wanted to assess the feasibility of such an IP from OVN's
> point of view. Obviously the real destination IP would be different, but I
> (without knowing the underlying works of OVN) thought there might be a
> programmable way of saying: "this IP is unknown to my network topology, so
> I could use identifier/alias grouping all such IPs under an umbrella
> identifier such as X.X.X.X/X"
>
>
I see. You don't really want the packet's dst IP to be replaced, so what
you need is some value in metadata and then you could define a policy to
match that metadata and specify the nexthop. This is supported by the
policy route table.
However, what really matters here is how to define "unknown IPs". We can't
say an IP is unknown just because there is no port with the IP assigned.
There can still be east-west traffic for such IP, e.g. for nested workload
in a VM, etc. From OVN's point of view this is completely user defined, and
the best way for the user to tell OVN this information is through the route
policies (which means you don't really need that extra metadata to achieve
this). I think it is the way how k8s-ovn is using OVN that causes this many
routes required. If the IP allocation can be better managed so that a big
range of IP is for east-west and otherwise north-south, then you probably
end up with a much smaller number of policies.
>
> On Wed, Sep 16, 2020 at 11:09 PM Han Zhou wrote:
>
>>
>>
>> On Wed, Sep 16, 2020 at 10:07 AM Alexander Constantinescu <
>> acons...@redhat.com> wrote:
>>
>>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>>>
>>> Yes, I was just using it as an example (though, granted, noop example)
>>>
>>> Some background to help steer the discussion:
>>>
>>> Essentially the functionality here is to have south -> north traffic
>>> from certain logical switch ports exit the cluster through a dedicated node
>>> (an egress node if you will). To do this we currently have a set of default
>>> logical router policies, intended to leave east <-> west traffic untouched,
>>> and then logical router policies with a lower priority, which specify
>>> reroute actions for this functionality to happen. However, on large
>>> clusters, there's this concern that the default logical router policies
>>> will become too many. Hence why the idea here would be to drop them
>>> completely and have this "special IP" that we can use to filter on the
>>> destination, south -> north, traffic .
>>>
>>> If you have a default route, anything "unknown" would just hit the
default route, right? Why would you need another IP for this purpose?
>>>
>>> As to remove the default logical router policies, which can become a
>>> lot, on big clusters - as described above. With only reroute policies of
>>> type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things
>>> would become lighter.
>>>
>>
>> Thanks for the background. So you have:
>>
>>
>> ...
>> default route (lowest priority): ip4.src == ,
>> nexthop =
>> default route (lowest priority): ip4.src == ,
>> nexthop =
>>
>> Is this the current situation?
>> When you say there are too many default routes, what do you mean in the
>> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>>
>> In addition, if SOUTH_TO_NORTH_IP is a user defined IP, I am not sure how
>> would it work, because ip4.dst is the dst IP from packet header. Comparing
>> it with SOUTH_TO_NORTH_IP would just result in mismatch, unless all
>> south-to-north traffic really has this IP as destination (I guess that's
