Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack

[Vladislav Odintsov]

Hi Han,

thanks for your comment.
You’re right. disabling ct.inv didn’t help in my case.
I’ve mixed allow-stateful with allow-related rules in port_group ACLs. 
Currently testing...

At first glance it works as expected.

Regards,
Vladislav Odintsov

> On 25 Aug 2021, at 01:18, Han Zhou  wrote:
> 
> 
> 
> On Tue, Aug 24, 2021 at 3:02 PM Vladislav Odintsov  > wrote:
> 
> 
> Regards,
> Vladislav Odintsov
> 
>> On 25 Aug 2021, at 00:47, Numan Siddique > > wrote:
>> 
>> On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov > > wrote:
>>> 
>>> Hi,
>>> 
>>> we’ve faced an issue where asymmetric-routed traffic is used. Please help 
>>> understand what options do we have to allow such traffic.
>>> 
>>> Topology is next:
>>> 
>>> client lsp (10.0.0.1/24 )
>>> |
>>>ls-external
>>>/ \
>>> lsp router vm1 eth0: 10.0.0.2/24  lsp router vm2 
>>> eth0: 10.0.0.3/24 
>>> lsp router vm1 eth1: 192.168.0.1/24   lsp router vm2 
>>> eth1: 192.168.0.2/24 
>>>\ /
>>>ls-internal
>>> |
>>>server lsp (192.168.0.10/24 )
>>> 
>>> 
>>> All LSPs have port_security configured with " 0.0.0.0/0 
>>>  ::/0" and belong to port group pg1.
>>> 
>>> There are two ACLs within this PG:
>>> from-lport 0.0.0.0/0  allow-related
>>> to-lport 0.0.0.0/0  allow-related
>>> 
>>> The problem is when traffic from client to server goes through router vm1 
>>> and returns through router vm2, there is no connectivity. I see reply 
>>> traffic on the server interface, which is going to router vm2 mac address, 
>>> but I don't see it on the router vm2 interface.
>>> I guess the reason for this is that conntrack first time sees packet for 
>>> the connection and ACK+SYN flags are set and treats this packet as invalid, 
>>> right?
>> 
>> I think so.
>> 
>>> 
>>> If yes, is there any option how to use asymmetric-routed topologies inside 
>>> OVN with stateful ACLs?
>>> I found there is an ability to replace ct.inv field check: 
>>> https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4
>>>  
>>> 
>>> Is it good idea to use this option to solve the issue or this is intended 
>>> specifically to use with smart NICs without invalid state support and can 
>>> be removed in future?
>>> 
> 
> 
> I am afraid disabling ct.inv will not help here. In your use case the 
> connections won't become established in conntrack, which means stateful ACL 
> wouldn't work. This will be the case even with physical stateful FWs. I'd 
> suggest either disable ACLs or use stateless ACLs (i.e. allow/allow-stateless 
> instead of allow-related).
>> 
>> I do not understand your use case completely.  I'm not quite clear
>> from the diagram which all resources are external
>> and which all are part of OVN.  Have you tried using the ECMP routes feature 
>> ?
>> 
> Logical Routers are not used in this topology. Only two logical switches - 
> ls-external and ls-internal.
> 
> In OVN we have:
> 1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24 
> ), "lsp-router-vm2-eth0" (10.0.0.3/24 
> ) and "lsp-client" (10.0.0.1/24 )
> 2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24 
> ), "lsp-router-vm2-eth1" (192.168.0.2/24 
> ) and "lsp-server" (192.168.0.10/24 
> )
> 3. Port group pg1 with mentioned above LSPs
> 4. Ingress and egress ACLs 0.0.0.0/0  with allow-related 
> action.
> 
> External:
> Linux client VM (10.0.0.1), which has static ECMP route:
> 192.168.0.0/24  via nexthop 10.0.0.2 via nexthop 
> 10.0.0.3
> 
> sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server).
> Traffic to server can go either through router VM1 (10.0.0.2) or through 
> router VM2 (10.0.0.3).
> 
> Server also has static ECMP route to reverse direction:
> 10.0.0.0/24  via nexthop 192.168.0.1 via nexthop 
> 192.168.0.2
> 
> So, when traffic in both directions goes through same router VM, it passes 
> normally.
> When traffic goes from client to server through one router VM and returns 
> back through another - SYN-ACK is blocked somewhere in OVS/conntrack. On 
> router VM’s interface we don’t see SYN+ACK.
> 
> So, OVN-based ECMP feature is not relevant for this case since it doesn’t 
> invoke logical routers.
> 
>> Regarding the ct.inv flag, does it work when you disable the usage of ct.inv 
>> ?
>> 
> Not yet. Firs

Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack

[Han Zhou]

On Tue, Aug 24, 2021 at 3:02 PM Vladislav Odintsov 
wrote:

>
>
> Regards,
> Vladislav Odintsov
>
> On 25 Aug 2021, at 00:47, Numan Siddique  wrote:
>
> On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov 
> wrote:
>
>
> Hi,
>
> we’ve faced an issue where asymmetric-routed traffic is used. Please help
> understand what options do we have to allow such traffic.
>
> Topology is next:
>
> client lsp (10.0.0.1/24)
> |
>ls-external
>/ \
> lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24
> lsp router vm1 eth1: 192.168.0.1/24  lsp router vm2 eth1: 192.168.0.2/24
>\ /
>ls-internal
> |
>server lsp (192.168.0.10/24)
>
>
> All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and
> belong to port group pg1.
>
> There are two ACLs within this PG:
> from-lport 0.0.0.0/0 allow-related
> to-lport 0.0.0.0/0 allow-related
>
> The problem is when traffic from client to server goes through router vm1
> and returns through router vm2, there is no connectivity. I see reply
> traffic on the server interface, which is going to router vm2 mac address,
> but I don't see it on the router vm2 interface.
> I guess the reason for this is that conntrack first time sees packet for
> the connection and ACK+SYN flags are set and treats this packet as invalid,
> right?
>
>
> I think so.
>
>
> If yes, is there any option how to use asymmetric-routed topologies inside
> OVN with stateful ACLs?
> I found there is an ability to replace ct.inv field check:
> https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4
> Is it good idea to use this option to solve the issue or this is intended
> specifically to use with smart NICs without invalid state support and can
> be removed in future?
>
>
I am afraid disabling ct.inv will not help here. In your use case the
connections won't become established in conntrack, which means stateful ACL
wouldn't work. This will be the case even with physical stateful FWs. I'd
suggest either disable ACLs or use stateless ACLs (i.e.
allow/allow-stateless instead of allow-related).

>
> I do not understand your use case completely.  I'm not quite clear
> from the diagram which all resources are external
> and which all are part of OVN.  Have you tried using the ECMP routes
> feature ?
>
> Logical Routers are not used in this topology. Only two logical switches -
> ls-external and ls-internal.
>
> In OVN we have:
> 1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24),
> "lsp-router-vm2-eth0" (10.0.0.3/24) and "lsp-client" (10.0.0.1/24)
> 2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24),
> "lsp-router-vm2-eth1" (192.168.0.2/24) and "lsp-server" (192.168.0.10/24)
> 3. Port group pg1 with mentioned above LSPs
> 4. Ingress and egress ACLs 0.0.0.0/0 with allow-related action.
>
> External:
> Linux client VM (10.0.0.1), which has static ECMP route:
> 192.168.0.0/24 via nexthop 10.0.0.2 via nexthop 10.0.0.3
>
> sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server).
> Traffic to server can go either through router VM1 (10.0.0.2) or through
> router VM2 (10.0.0.3).
>
> Server also has static ECMP route to reverse direction:
> 10.0.0.0/24 via nexthop 192.168.0.1 via nexthop 192.168.0.2
>
> So, when traffic in both directions goes through same router VM, it passes
> normally.
> When traffic goes from client to server through one router VM and returns
> back through another - SYN-ACK is blocked somewhere in OVS/conntrack. On
> router VM’s interface we don’t see SYN+ACK.
>
> So, OVN-based ECMP feature is not relevant for this case since it doesn’t
> invoke logical routers.
>
> Regarding the ct.inv flag, does it work when you disable the usage of
> ct.inv ?
>
> Not yet. First wanted to understand options.
>
> Thanks
> Numan
>
>
> If these routes are configured in the logical router, then
>
> Thanks.
>
> Regards,
> Vladislav Odintsov
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack

[Vladislav Odintsov]

Regards,
Vladislav Odintsov

> On 25 Aug 2021, at 00:47, Numan Siddique  wrote:
> 
> On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov  wrote:
>> 
>> Hi,
>> 
>> we’ve faced an issue where asymmetric-routed traffic is used. Please help 
>> understand what options do we have to allow such traffic.
>> 
>> Topology is next:
>> 
>> client lsp (10.0.0.1/24)
>> |
>>ls-external
>>/ \
>> lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24
>> lsp router vm1 eth1: 192.168.0.1/24  lsp router vm2 eth1: 192.168.0.2/24
>>\ /
>>ls-internal
>> |
>>server lsp (192.168.0.10/24)
>> 
>> 
>> All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and 
>> belong to port group pg1.
>> 
>> There are two ACLs within this PG:
>> from-lport 0.0.0.0/0 allow-related
>> to-lport 0.0.0.0/0 allow-related
>> 
>> The problem is when traffic from client to server goes through router vm1 
>> and returns through router vm2, there is no connectivity. I see reply 
>> traffic on the server interface, which is going to router vm2 mac address, 
>> but I don't see it on the router vm2 interface.
>> I guess the reason for this is that conntrack first time sees packet for the 
>> connection and ACK+SYN flags are set and treats this packet as invalid, 
>> right?
> 
> I think so.
> 
>> 
>> If yes, is there any option how to use asymmetric-routed topologies inside 
>> OVN with stateful ACLs?
>> I found there is an ability to replace ct.inv field check: 
>> https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4
>> Is it good idea to use this option to solve the issue or this is intended 
>> specifically to use with smart NICs without invalid state support and can be 
>> removed in future?
>> 
> 
> I do not understand your use case completely.  I'm not quite clear
> from the diagram which all resources are external
> and which all are part of OVN.  Have you tried using the ECMP routes feature ?
> 
Logical Routers are not used in this topology. Only two logical switches - 
ls-external and ls-internal.

In OVN we have:
1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24), 
"lsp-router-vm2-eth0" (10.0.0.3/24) and "lsp-client" (10.0.0.1/24)
2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24), 
"lsp-router-vm2-eth1" (192.168.0.2/24) and "lsp-server" (192.168.0.10/24)
3. Port group pg1 with mentioned above LSPs
4. Ingress and egress ACLs 0.0.0.0/0 with allow-related action.

External:
Linux client VM (10.0.0.1), which has static ECMP route:
192.168.0.0/24 via nexthop 10.0.0.2 via nexthop 10.0.0.3

sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server).
Traffic to server can go either through router VM1 (10.0.0.2) or through router 
VM2 (10.0.0.3).

Server also has static ECMP route to reverse direction:
10.0.0.0/24 via nexthop 192.168.0.1 via nexthop 192.168.0.2

So, when traffic in both directions goes through same router VM, it passes 
normally.
When traffic goes from client to server through one router VM and returns back 
through another - SYN-ACK is blocked somewhere in OVS/conntrack. On router VM’s 
interface we don’t see SYN+ACK.

So, OVN-based ECMP feature is not relevant for this case since it doesn’t 
invoke logical routers.

> Regarding the ct.inv flag, does it work when you disable the usage of ct.inv ?
> 
Not yet. First wanted to understand options.

> Thanks
> Numan
> 
> 
> If these routes are configured in the logical router, then
>> Thanks.
>> 
>> Regards,
>> Vladislav Odintsov
>> 
>> ___
>> discuss mailing list
>> disc...@openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] [ovn] OVN asymmetric routing with conntrack

[Numan Siddique]

On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov  wrote:
>
> Hi,
>
> we’ve faced an issue where asymmetric-routed traffic is used. Please help 
> understand what options do we have to allow such traffic.
>
> Topology is next:
>
>  client lsp (10.0.0.1/24)
>  |
> ls-external
> / \
> lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24
> lsp router vm1 eth1: 192.168.0.1/24  lsp router vm2 eth1: 192.168.0.2/24
> \ /
> ls-internal
>  |
> server lsp (192.168.0.10/24)
>
>
> All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and belong 
> to port group pg1.
>
> There are two ACLs within this PG:
> from-lport 0.0.0.0/0 allow-related
> to-lport 0.0.0.0/0 allow-related
>
> The problem is when traffic from client to server goes through router vm1 and 
> returns through router vm2, there is no connectivity. I see reply traffic on 
> the server interface, which is going to router vm2 mac address, but I don't 
> see it on the router vm2 interface.
> I guess the reason for this is that conntrack first time sees packet for the 
> connection and ACK+SYN flags are set and treats this packet as invalid, right?

I think so.

>
> If yes, is there any option how to use asymmetric-routed topologies inside 
> OVN with stateful ACLs?
> I found there is an ability to replace ct.inv field check: 
> https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4
> Is it good idea to use this option to solve the issue or this is intended 
> specifically to use with smart NICs without invalid state support and can be 
> removed in future?
>

I do not understand your use case completely.  I'm not quite clear
from the diagram which all resources are external
and which all are part of OVN.  Have you tried using the ECMP routes feature ?

Regarding the ct.inv flag, does it work when you disable the usage of ct.inv ?

Thanks
Numan


If these routes are configured in the logical router, then
> Thanks.
>
> Regards,
> Vladislav Odintsov
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

[ovs-discuss] [ovn] OVN asymmetric routing with conntrack

[Vladislav Odintsov]

Hi,

we’ve faced an issue where asymmetric-routed traffic is used. Please help 
understand what options do we have to allow such traffic.

Topology is next:

 client lsp (10.0.0.1/24)
 |
ls-external
/ \
lsp router vm1 eth0: 10.0.0.2/24 lsp router vm2 eth0: 10.0.0.3/24
lsp router vm1 eth1: 192.168.0.1/24  lsp router vm2 eth1: 192.168.0.2/24
\ /
ls-internal
 |
server lsp (192.168.0.10/24)


All LSPs have port_security configured with " 0.0.0.0/0 ::/0" and belong 
to port group pg1.

There are two ACLs within this PG:
from-lport 0.0.0.0/0 allow-related
to-lport 0.0.0.0/0 allow-related

The problem is when traffic from client to server goes through router vm1 and 
returns through router vm2, there is no connectivity. I see reply traffic on 
the server interface, which is going to router vm2 mac address, but I don't see 
it on the router vm2 interface.
I guess the reason for this is that conntrack first time sees packet for the 
connection and ACK+SYN flags are set and treats this packet as invalid, right?

If yes, is there any option how to use asymmetric-routed topologies inside OVN 
with stateful ACLs?
I found there is an ability to replace ct.inv field check: 
https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4
Is it good idea to use this option to solve the issue or this is intended 
specifically to use with smart NICs without invalid state support and can be 
removed in future?

Thanks.

Regards,
Vladislav Odintsov

___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss