On Thu, Apr 21, 2022 at 5:02 AM Frode Nordahl <frode.nord...@canonical.com> wrote: > > On Wed, Apr 20, 2022 at 10:10 PM Numan Siddique <num...@ovn.org> wrote: > > > > On Tue, Apr 12, 2022 at 11:01 AM Numan Siddique <num...@ovn.org> wrote: > > > > > > On Mon, Apr 11, 2022 at 12:37 PM Frode Nordahl > > > <frode.nord...@canonical.com> wrote: > > > > > > > > tor. 7. apr. 2022, 19:40 skrev Frode Nordahl > > > > <frode.nord...@canonical.com>: > > > > > > > > > On Thu, Apr 7, 2022 at 6:51 PM Numan Siddique <num...@ovn.org> wrote: > > > > > > > > > > > > On Wed, Apr 6, 2022 at 12:11 PM Numan Siddique <num...@ovn.org> > > > > > > wrote: > > > > > > > > > > > > > > On Wed, Apr 6, 2022 at 4:58 AM Frode Nordahl > > > > > > > <frode.nord...@canonical.com> wrote: > > > > > > > > > > > > > > > > On Wed, Mar 30, 2022 at 4:54 PM Numan Siddique <num...@ovn.org> > > > > > wrote: > > > > > > > > > > > > > > > > > > On Wed, Mar 30, 2022 at 6:21 AM Frode Nordahl > > > > > > > > > <frode.nord...@canonical.com> wrote: > > > > > > > > > > > > > > > > > > > > On Wed, Mar 30, 2022 at 11:45 AM Frode Nordahl > > > > > > > > > > <frode.nord...@canonical.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hello Numan, > > > > > > > > > > > > > > > > > > > > > > This patch does unfortunately break gateway routers in > > > > > > > > > > > some > > > > > > > > > > > circumstances, (but not all!), at least for the way > > > > > > > > > > > OpenStack > > > > > consumes > > > > > > > > > > > them. > > > > > > > > > > > > > > > > > > > > > > Will dig more and try to figure out what is going on, but > > > > > sending this > > > > > > > > > > > e-mail proactively in case you have any ideas from the > > > > > > > > > > > top of > > > > > your > > > > > > > > > > > head or if anyone else has run into the same issue. > > > > > > > > > > > > > > > > > > > > > > The gateway appears to handle conntrack as it should: > > > > > > > > > > > $ sudo conntrack -E --dst 194.169.254.178 > > > > > > > > > > > [NEW] tcp 6 120 SYN_SENT src=10.11.2.11 > > > > > dst=194.169.254.178 > > > > > > > > > > > sport=60234 dport=22 [UNREPLIED] src=10.42.3.34 > > > > > > > > > > > dst=10.11.2.11 > > > > > > > > > > > sport=22 dport=60234 zone=52 > > > > > > > > > > > [UPDATE] tcp 6 60 SYN_RECV src=10.11.2.11 > > > > > dst=194.169.254.178 > > > > > > > > > > > sport=60234 dport=22 src=10.42.3.34 dst=10.11.2.11 > > > > > > > > > > > sport=22 > > > > > > > > > > > dport=60234 zone=52 > > > > > > > > > > > [UPDATE] tcp 6 432000 ESTABLISHED src=10.11.2.11 > > > > > > > > > > > dst=194.169.254.178 sport=60234 dport=22 src=10.42.3.34 > > > > > dst=10.11.2.11 > > > > > > > > > > > sport=22 dport=60234 [ASSURED] zone=52 > > > > > > > > > > > > > > > > > > > > > > However as you can see traffic going to the instance > > > > > > > > > > > suddenly > > > > > gets its > > > > > > > > > > > source address mangled/set to 0.0.0.0, and the connection > > > > > > > > > > > can > > > > > never > > > > > > > > > > > establish. > > > > > > > > > > > $ sudo tcpdump -nevvi tape5c1862d-b4 > > > > > > > > > > > tcpdump: listening on tape5c1862d-b4, link-type EN10MB > > > > > (Ethernet), > > > > > > > > > > > capture size 262144 bytes > > > > > > > > > > > 09:40:34.920187 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 74: (tos 0x0, ttl 62, id 24824, offset 0, > > > > > flags [DF], > > > > > > > > > > > proto TCP (6), length 60) > > > > > > > > > > > 10.11.2.11.60820 > 10.42.3.34.22: Flags [S], cksum > > > > > > > > > > > 0x9f1b > > > > > > > > > > > (correct), seq 2926328730, win 64240, options [mss > > > > > 1460,sackOK,TS val > > > > > > > > > > > 870680827 ecr 0,nop,wscale 7], length 0 > > > > > > > > > > > 09:40:34.920537 fa:16:3e:fc:82:be > fa:16:3e:c8:19:af, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, > > > > > > > > > > > flags > > > > > [DF], > > > > > > > > > > > proto TCP (6), length 60) > > > > > > > > > > > 10.42.3.34.22 > 10.11.2.11.60820: Flags [S.], cksum > > > > > > > > > > > 0x1990 > > > > > > > > > > > (incorrect -> 0x284c), seq 1596962125, ack 2926328731, win > > > > > 62230, > > > > > > > > > > > options [mss 8902,sackOK,TS val 3490675961 ecr > > > > > 870680827,nop,wscale > > > > > > > > > > > 7], length 0 > > > > > > > > > > > 09:40:34.968033 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 107: (tos 0x0, ttl 62, id 24826, offset > > > > > > > > > > > 0, > > > > > flags > > > > > > > > > > > [DF], proto TCP (6), length 93) > > > > > > > > > > > 10.11.2.11.60820 > 10.42.3.34.22: Flags [P.], cksum > > > > > > > > > > > 0x4293 > > > > > > > > > > > (correct), seq 1:42, ack 1, win 502, options [nop,nop,TS > > > > > > > > > > > val > > > > > 870680922 > > > > > > > > > > > ecr 3490675961], length 41 > > > > > > > > > > > 09:40:34.968243 fa:16:3e:fc:82:be > fa:16:3e:c8:19:af, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 66: (tos 0x0, ttl 64, id 30244, offset 0, > > > > > flags [DF], > > > > > > > > > > > proto TCP (6), length 52) > > > > > > > > > > > 10.42.3.34.22 > 10.11.2.11.60820: Flags [.], cksum > > > > > > > > > > > 0x1988 > > > > > > > > > > > (incorrect -> 0x64a4), seq 1, ack 42, win 486, options > > > > > [nop,nop,TS val > > > > > > > > > > > 3490676008 ecr 870680922], length 0 > > > > > > > > > > > 09:40:34.968857 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 66: (tos 0x0, ttl 62, id 24825, offset 0, > > > > > flags [DF], > > > > > > > > > > > proto TCP (6), length 52) > > > > > > > > > > > 10.11.2.11.60820 > 10.42.3.34.22: Flags [.], cksum > > > > > > > > > > > 0x64ec > > > > > > > > > > > (correct), seq 1, ack 1, win 502, options [nop,nop,TS val > > > > > 870680922 > > > > > > > > > > > ecr 3490675961], length 0 > > > > > > > > > > > 09:40:34.968932 fa:16:3e:fc:82:be > fa:16:3e:c8:19:af, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 66: (tos 0x0, ttl 64, id 30245, offset 0, > > > > > flags [DF], > > > > > > > > > > > proto TCP (6), length 52) > > > > > > > > > > > 10.42.3.34.22 > 10.11.2.11.60820: Flags [.], cksum > > > > > > > > > > > 0x1988 > > > > > > > > > > > (incorrect -> 0x64a3), seq 1, ack 42, win 486, options > > > > > [nop,nop,TS val > > > > > > > > > > > 3490676009 ecr 870680922], length 0 > > > > > > > > > > > 09:40:34.977991 fa:16:3e:fc:82:be > fa:16:3e:c8:19:af, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 107: (tos 0x0, ttl 64, id 30246, offset > > > > > > > > > > > 0, > > > > > flags > > > > > > > > > > > [DF], proto TCP (6), length 93) > > > > > > > > > > > 10.42.3.34.22 > 10.11.2.11.60820: Flags [P.], cksum > > > > > > > > > > > 0x19b1 > > > > > > > > > > > (incorrect -> 0x4241), seq 1:42, ack 42, win 486, options > > > > > [nop,nop,TS > > > > > > > > > > > val 3490676018 ecr 870680922], length 41 > > > > > > > > > > > 09:40:34.978323 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 66: (tos 0x0, ttl 62, id 24827, offset 0, > > > > > flags [DF], > > > > > > > > > > > proto TCP (6), length 52) > > > > > > > > > > > 0.0.0.0.60820 > 10.42.3.34.22: Flags [.], cksum 0x706d > > > > > (correct), > > > > > > > > > > > seq 2926328772, ack 1596962167, win 502, options > > > > > > > > > > > [nop,nop,TS > > > > > val > > > > > > > > > > > 870680932 ecr 3490676018], length 0 > > > > > > > > > > > 09:40:34.979089 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 1578: (tos 0x0, ttl 62, id 24828, offset > > > > > > > > > > > 0, > > > > > flags > > > > > > > > > > > [DF], proto TCP (6), length 1564) > > > > > > > > > > > 0.0.0.0.60820 > 10.42.3.34.22: Flags [P.], cksum > > > > > > > > > > > 0x135a > > > > > (incorrect > > > > > > > > > > > -> 0xd379), seq 0:1512, ack 1, win 502, options > > > > > > > > > > > [nop,nop,TS val > > > > > > > > > > > 870680933 ecr 3490676018], length 1512 > > > > > > > > > > > 09:40:35.157196 fa:16:3e:c8:19:af > fa:16:3e:fc:82:be, > > > > > ethertype IPv4 > > > > > > > > > > > (0x0800), length 130: (tos 0x0, ttl 62, id 24830, offset > > > > > > > > > > > 0, > > > > > flags > > > > > > > > > > > [DF], proto TCP (6), length 116) > > > > > > > > > > > 0.0.0.0.60820 > 10.42.3.34.22: Flags [P.], cksum > > > > > > > > > > > 0x5f70 > > > > > (correct), > > > > > > > > > > > seq 1448:1512, ack 1, win 502, options [nop,nop,TS val > > > > > 870681111 ecr > > > > > > > > > > > 3490676018], length 64 > > > > > > > > > > > > > > > > > > > > Turning hardware offload all the way off appears to resolve > > > > > > > > > > the > > > > > issue, > > > > > > > > > > so this might as well be driver/firmware related. May I ask > > > > > > > > > > what > > > > > > > > > > card/driver/firmware you've been using this with? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Frode, > > > > > > > > > > > > > > > > > > This patch attempted to resolve this BZ - > > > > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1984953. > > > > > > > > > This BZ is related to HWOL not working. > > > > > > > > > > > > > > > > > > I didn't test myself but I think other colleagues of mine > > > > > > > > > tested > > > > > with > > > > > > > > > NVidia/Mellanox CX5. > > > > > > > > > I don't think we saw the source address getting mangled in our > > > > > testing. > > > > > > > > > > > > > > > > Thank you for sharing those details. We see the source mangling > > > > > > > > with > > > > > > > > CX6 here but only with distributed FIPs off, we'll park this > > > > > > > > for now > > > > > > > > until we receive information from Nvidia/Mellanox as to what's > > > > > > > > going > > > > > > > > on from a driver/firmware perspective. > > > > > > > > > > > > > > > > > Please let me know if you have any more questions. > > > > > > > > > > > > > > > > We did unfortunately run into another issue which appears to > > > > > > > > have its > > > > > > > > root in this patch. The symptomps are described in > > > > > > > > https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/1967856 > > > > > > > > > > > > > > > > This rudimentary patch does appear to fix it, but it may require > > > > > > > > acquiring and passing in the `distributed` variable from the > > > > > > > > `lrouter_check_nat_entry` function two new places, so I wanted > > > > > > > > to air > > > > > > > > > > > > > > Thanks for finding this issue. I think we should use separate > > > > > > > zones > > > > > > > for hairpin traffic. > > > > > > > And I thought I had addressed it. To handle this hairpin issue, > > > > > > > the > > > > > > > patch sets the register bit REGBIT_DST_NAT_IP_LOCAL > > > > > > > and if this bit is set, then ct_snat() action is used instead of > > > > > > > ct_snat_in_czone(). > > > > > > > Let me test it out and update here. > > > > > > > > > > > > I tested it locally and it seems to work for me. > > > > > > > > > > > > Can you please share your OVN dbs if possible ? I've left a comment > > > > > > on the launchpad bug. You can attach it there too if it's possible. > > > > > > > > > > Sure thing, I attached them to the LP bug [0]. In this DB the active > > > > > gateway chassis is `deep-ferret.maas` and the instance on > > > > > `comic-perch.maas` is unable to have two ping sessions to itself using > > > > > non-distributed FIP 10.78.95.196. > > > > > > > > > > I'll keep looking as well, thank you for your help so far! > > > > > > > > > > 0: https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/1967856 > > > > > > > > > > Updated OVN to main and it unfortunately made no difference. > > > > > > > > The combination of stateless on the NAT rule and the allow-related ACLs > > > > does indeed look strange, but this is how OpenStack sets it up. Have not > > > > looked into whether that makes sense or not yet. > > > > > > > > To ensure we're looking at the same thing I made this modification to > > > > the > > > > `DNAT LR hairpin IPv4` system test [2] > > > > > > > > And executed it like this: > > > > > > > > sudo make check-kernel TESTSUITEFLAGS="337" > > > > > > > > It fails consistently here. If I either revert [1] or remove the check > > > > for > > > > the second ping from the test it succeeds. > > > > > > > > 2: > > > > https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/1967856/+attachment/5579267/+files/test-synthesis.patch > > > > > > Thanks for this reproducer. The test case is failing for me too. > > > I'll debug further and get back to you. > > > > > > Numan > > > > > > I debugged a bit. I can reproduce the issue 100% of the time. The > > fact that the first ping works and the subsequent one fails indicates > > that it is not an OVN issue. > > From what I could see so far, it looks like conntrack issue to me. I > > need to debug this further to root cause it. > > Thank you for the update. If we're sure the change of how OVN programs > the flows is correct and should work that sounds reasonable to me. > > Looking at conntrack events or dumping the conntrack table on the > first and second attempt (the synthetic test currently does this) does > indeed indicate that CT responds differently to the two sessions, but > it was unclear to me whether the root of that is the change in OVN or > not. > > So essentially our current line of thought should be that the change > in OVN has uncovered a CT-related bug either in OVS, the OVS kernel > datapath or kernel CT in general?
Yes. That's my understanding too. This needs to be debugged further. We probably didn't see this issue earlier, since the packet was going into 2 zones. Thanks Numan > > -- > Frode Nordahl > > > Numan > > > > > > > > > > > > > > > > > -- > > > > > > > > Frode Nordahl > > > > > > > > > > Numan > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > Numan > > > > > > > > > > > > > > > > > > > > > > this with you before proceeding with the fix: > > > > > > > > diff --git a/northd/northd.c b/northd/northd.c > > > > > > > > index 2fb0a93c2..5fae010c0 100644 > > > > > > > > --- a/northd/northd.c > > > > > > > > +++ b/northd/northd.c > > > > > > > > @@ -9891,6 +9891,7 @@ build_lrouter_nat_flows_for_lb(struct > > > > > ovn_lb_vip *lb_vip, > > > > > > > > undnat_match_p, > > > > > > > > est_actions, > > > > > > > > &lb->nlb->header_); > > > > > > > > } else { > > > > > > > > + /* XXX do we need to check for distributed here? */ > > > > > > > > ovn_lflow_add_with_hint( > > > > > > > > lflows, od, S_ROUTER_OUT_UNDNAT, 120, > > > > > undnat_match_p, > > > > > > > > od->is_gw_router ? "ct_dnat;" : > > > > > > > > "ct_dnat_in_czone;", > > > > > > > > @@ -12851,7 +12852,9 @@ build_lrouter_out_undnat_flow(struct > > > > > > > > hmap > > > > > > > > *lflows, struct ovn_datapath *od, > > > > > > > > is_v6 ? "6" : "4", nat->external_ip); > > > > > > > > } else { > > > > > > > > ds_put_format(actions, > > > > > > > > - od->is_gw_router ? "ct_dnat;" : > > > > > "ct_dnat_in_czone;"); > > > > > > > > + (od->is_gw_router || > > > > > > > > + (!distributed && od->n_l3dgw_ports)) ? > > > > > > > > + "ct_dnat;" : "ct_dnat_in_czone;"); > > > > > > > > } > > > > > > > > > > > > > > > > ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, > > > > > > > > 100, > > > > > > > > @@ -13250,7 +13253,10 @@ build_lrouter_nat_defrag_and_lb(struct > > > > > > > > ovn_datapath *od, struct hmap *lflows, > > > > > > > > * not committed, it would produce ongoing datapath flows > > > > > > > > with > > > > > the ct.new > > > > > > > > * flag set. Some NICs are unable to offload these flows. > > > > > > > > */ > > > > > > > > - if (od->is_gw_router && (od->nbr->n_nat || > > > > > > > > od->has_lb_vip)) { > > > > > > > > + /* XXX we probably need to get the distributed variable > > > > > > > > passed > > > > > in here to > > > > > > > > + * XXX retain the proposed optimization */ > > > > > > > > + if ((od->is_gw_router || od->n_l3dgw_ports) && > > > > > > > > + (od->nbr->n_nat || od->has_lb_vip)) { > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, > > > > > > > > "ip", "flags.loopback = 1; ct_dnat;"); > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, > > > > > > > > > > > > > > > > The environment in question is using distributed gateway ports > > > > > > > > by > > > > > > > > associating multiple Gateway_Chassis with a Logical_Router_Port > > > > > > > > and > > > > > at > > > > > > > > the same time does not use distributed FIPs. > > > > > > > > > > > > > > > > -- > > > > > > > > Frode Nordahl > > > > > > > > > > > > > > > > > Numan > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Frode Nordahl > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Frode Nordahl > > > > > > > > > > > > > > > > > > > > > > On Sat, Nov 20, 2021 at 12:48 AM Numan Siddique < > > > > > num...@ovn.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Nov 19, 2021 at 1:11 PM Mark Michelson < > > > > > mmich...@redhat.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the update, Numan. > > > > > > > > > > > > > > > > > > > > > > > > > > Acked-by: Mark Michelson <mmich...@redhat.com> > > > > > > > > > > > > > > > > > > > > > > > > Thanks. I applied both the patches to the main branch. > > > > > > > > > > > > > > > > > > > > > > > > Numan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/19/21 11:32, num...@ovn.org wrote: > > > > > > > > > > > > > > From: Numan Siddique <num...@ovn.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > > Make of use of these new actions for the distributed > > > > > routers > > > > > > > > > > > > > > for NAT. These new actions ensure that both sNAT > > > > > > > > > > > > > > and > > > > > dNAT > > > > > > > > > > > > > > happens in the same zone. This approach solves a > > > > > > > > > > > > > > couple > > > > > of > > > > > > > > > > > > > > problems: > > > > > > > > > > > > > > > > > > > > > > > > > > > > - The datapath flows generated for external > > > > > > > > > > > > > > traffic > > > > > which requires > > > > > > > > > > > > > > dNAT (N -> S) or sNAT (S -> N) are completely > > > > > HWOL'able. > > > > > > > > > > > > > > > > > > > > > > > > > > > > - Since there is only one zone, it would avoid > > > > > multiple recirculations > > > > > > > > > > > > > > (improving the performance). > > > > > > > > > > > > > > > > > > > > > > > > > > > > If the packet needs to be both sNATted and dNATted > > > > > > > > > > > > > > (for > > > > > hairpin traffic > > > > > > > > > > > > > > with source and destination on the same chassis), > > > > > > > > > > > > > > then > > > > > sNAT is done > > > > > > > > > > > > > > in a separate zone. To detect this scenario, this > > > > > > > > > > > > > > patch > > > > > adds a few > > > > > > > > > > > > > > extra logical flows. For each dnat_and_snat entry > > > > > > > > > > > > > > prior > > > > > to this patch > > > > > > > > > > > > > > ovn-northd was generating 9 logical flows and with > > > > > > > > > > > > > > this > > > > > patch it now > > > > > > > > > > > > > > generates 12 logical flows. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Similar approach can be taken for gateway routers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Reported-at: > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1984953 > > > > > > > > > > > > > > Signed-off-by: Numan Siddique <num...@ovn.org> > > > > > > > > > > > > > > --- > > > > > > > > > > > > > > > > > > > > > > > > > > > > v2 -> v3 > > > > > > > > > > > > > > ------ > > > > > > > > > > > > > > * Addressed Mark's comments and updated the > > > > > documentation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > v1 -> v2 > > > > > > > > > > > > > > ------ > > > > > > > > > > > > > > * Rebased and resolved conflicts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > include/ovn/logical-fields.h | 1 + > > > > > > > > > > > > > > lib/logical-fields.c | 4 + > > > > > > > > > > > > > > northd/northd.c | 147 +++++++-- > > > > > > > > > > > > > > northd/ovn-northd.8.xml | 205 ++++++++++--- > > > > > > > > > > > > > > tests/ovn-northd.at | 575 > > > > > +++++++++++++++++++---------------- > > > > > > > > > > > > > > tests/ovn.at | 2 +- > > > > > > > > > > > > > > tests/system-ovn.at | 64 ++-- > > > > > > > > > > > > > > 7 files changed, 610 insertions(+), 388 > > > > > > > > > > > > > > deletions(-) > > > > > > > > > > > > > > > > > > > > > > > > > > > > diff --git a/include/ovn/logical-fields.h > > > > > b/include/ovn/logical-fields.h > > > > > > > > > > > > > > index c9675f81c..2118f7933 100644 > > > > > > > > > > > > > > --- a/include/ovn/logical-fields.h > > > > > > > > > > > > > > +++ b/include/ovn/logical-fields.h > > > > > > > > > > > > > > @@ -70,6 +70,7 @@ enum mff_log_flags_bits { > > > > > > > > > > > > > > MLF_LOOKUP_FDB_BIT = 8, > > > > > > > > > > > > > > MLF_SKIP_SNAT_FOR_LB_BIT = 9, > > > > > > > > > > > > > > MLF_LOCALPORT_BIT = 10, > > > > > > > > > > > > > > + MLF_USE_SNAT_ZONE = 11, > > > > > > > > > > > > > > }; > > > > > > > > > > > > > > > > > > > > > > > > > > > > /* MFF_LOG_FLAGS_REG flag assignments */ > > > > > > > > > > > > > > diff --git a/lib/logical-fields.c > > > > > > > > > > > > > > b/lib/logical-fields.c > > > > > > > > > > > > > > index 7b3d431e0..352a48c89 100644 > > > > > > > > > > > > > > --- a/lib/logical-fields.c > > > > > > > > > > > > > > +++ b/lib/logical-fields.c > > > > > > > > > > > > > > @@ -125,6 +125,10 @@ ovn_init_symtab(struct shash > > > > > *symtab) > > > > > > > > > > > > > > MLF_SKIP_SNAT_FOR_LB_BIT); > > > > > > > > > > > > > > expr_symtab_add_subfield(symtab, > > > > > "flags.skip_snat_for_lb", NULL, > > > > > > > > > > > > > > flags_str); > > > > > > > > > > > > > > + snprintf(flags_str, sizeof flags_str, > > > > > > > > > > > > > > "flags[%d]", > > > > > > > > > > > > > > + MLF_USE_SNAT_ZONE); > > > > > > > > > > > > > > + expr_symtab_add_subfield(symtab, > > > > > "flags.use_snat_zone", NULL, > > > > > > > > > > > > > > + flags_str); > > > > > > > > > > > > > > > > > > > > > > > > > > > > /* Connection tracking state. */ > > > > > > > > > > > > > > expr_symtab_add_field_scoped(symtab, > > > > > > > > > > > > > > "ct_mark", > > > > > MFF_CT_MARK, NULL, false, > > > > > > > > > > > > > > diff --git a/northd/northd.c b/northd/northd.c > > > > > > > > > > > > > > index 0ff61deec..e4d051a94 100644 > > > > > > > > > > > > > > --- a/northd/northd.c > > > > > > > > > > > > > > +++ b/northd/northd.c > > > > > > > > > > > > > > @@ -159,11 +159,14 @@ enum ovn_stage { > > > > > > > > > > > > > > PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, > > > > > > > > > > > > > > 18, > > > > > "lr_in_arp_request") \ > > > > > > > > > > > > > > > > > > > \ > > > > > > > > > > > > > > /* Logical router egress stages. */ > > > > > \ > > > > > > > > > > > > > > - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, > > > > > "lr_out_undnat") \ > > > > > > > > > > > > > > - PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 1, > > > > > "lr_out_post_undnat") \ > > > > > > > > > > > > > > - PIPELINE_STAGE(ROUTER, OUT, SNAT, 2, > > > > > "lr_out_snat") \ > > > > > > > > > > > > > > - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 3, > > > > > "lr_out_egr_loop") \ > > > > > > > > > > > > > > - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 4, > > > > > "lr_out_delivery") > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, > > > > > > > > > > > > > > 0, > > > > > \ > > > > > > > > > > > > > > + "lr_out_chk_dnat_local") > > > > > \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, > > > > > > > > > > > > > > 1, > > > > > "lr_out_undnat") \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, > > > > > > > > > > > > > > 2, > > > > > "lr_out_post_undnat") \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, SNAT, > > > > > > > > > > > > > > 3, > > > > > "lr_out_snat") \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, > > > > > > > > > > > > > > 4, > > > > > "lr_out_post_snat") \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, > > > > > > > > > > > > > > 5, > > > > > "lr_out_egr_loop") \ > > > > > > > > > > > > > > + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, > > > > > > > > > > > > > > 6, > > > > > "lr_out_delivery") > > > > > > > > > > > > > > > > > > > > > > > > > > > > #define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, > > > > > TABLE, NAME) \ > > > > > > > > > > > > > > S_##DP_TYPE##_##PIPELINE##_##STAGE > > > > > \ > > > > > > > > > > > > > > @@ -210,6 +213,7 @@ enum ovn_stage { > > > > > > > > > > > > > > #define REGBIT_PKT_LARGER "reg9[1]" > > > > > > > > > > > > > > #define REGBIT_LOOKUP_NEIGHBOR_RESULT "reg9[2]" > > > > > > > > > > > > > > #define REGBIT_LOOKUP_NEIGHBOR_IP_RESULT "reg9[3]" > > > > > > > > > > > > > > +#define REGBIT_DST_NAT_IP_LOCAL "reg9[4]" > > > > > > > > > > > > > > > > > > > > > > > > > > > > /* Register to store the eth address associated > > > > > > > > > > > > > > to a > > > > > router port for packets > > > > > > > > > > > > > > * received in S_ROUTER_IN_ADMISSION. > > > > > > > > > > > > > > @@ -9568,9 +9572,10 @@ > > > > > build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, > > > > > > > > > > > > > > > > > > > > > > > > > > > > undnat_match_p, > > > > > est_actions, > > > > > > > > > > > > > > > > > > > > > > > > > > > > &lb->nlb->header_); > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > - ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_OUT_UNDNAT, 120, > > > > > > > > > > > > > > - undnat_match_p, > > > > > "ct_dnat;", > > > > > > > > > > > > > > - > > > > > > > > > > > > > > &lb->nlb->header_); > > > > > > > > > > > > > > + ovn_lflow_add_with_hint( > > > > > > > > > > > > > > + lflows, od, S_ROUTER_OUT_UNDNAT, > > > > > > > > > > > > > > 120, > > > > > undnat_match_p, > > > > > > > > > > > > > > + od->is_gw_router ? "ct_dnat;" : > > > > > "ct_dnat_in_czone;", > > > > > > > > > > > > > > + &lb->nlb->header_); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > free(undnat_match_p); > > > > > > > > > > > > > > next: > > > > > > > > > > > > > > @@ -9865,7 +9870,7 @@ > > > > > lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, > > > > > > > > > > > > > > uint16_t priority; > > > > > > > > > > > > > > > > > > > > > > > > > > > > /* Priority of logical flows > > > > > > > > > > > > > > corresponding to > > > > > exempted_ext_ips is > > > > > > > > > > > > > > - * +1 of the corresponding regulr NAT rule. > > > > > > > > > > > > > > + * +2 of the corresponding regular NAT > > > > > > > > > > > > > > rule. > > > > > > > > > > > > > > * For example, if we have following NAT > > > > > > > > > > > > > > rule > > > > > and we associate > > > > > > > > > > > > > > * exempted external ips to it: > > > > > > > > > > > > > > * "ovn-nbctl lr-nat-add router > > > > > > > > > > > > > > dnat_and_snat > > > > > 10.15.24.139 50.0.0.11" > > > > > > > > > > > > > > @@ -9873,17 +9878,17 @@ > > > > > lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, > > > > > > > > > > > > > > * And now we associate exempted external > > > > > > > > > > > > > > ip > > > > > address set to it. > > > > > > > > > > > > > > * Now corresponding to above rule we will > > > > > have following logical > > > > > > > > > > > > > > * flows: > > > > > > > > > > > > > > - * lr_out_snat...priority=162, > > > > > > > > > > > > > > match=(..ip4.dst > > > > > == $exempt_range), > > > > > > > > > > > > > > + * lr_out_snat...priority=163, > > > > > > > > > > > > > > match=(..ip4.dst > > > > > == $exempt_range), > > > > > > > > > > > > > > * > > > > > > > > > > > > > > action=(next;) > > > > > > > > > > > > > > * lr_out_snat...priority=161, match=(..), > > > > > action=(ct_snat(....);) > > > > > > > > > > > > > > * > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > if (is_src) { > > > > > > > > > > > > > > /* S_ROUTER_IN_DNAT uses priority 100 > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > - priority = 100 + 1; > > > > > > > > > > > > > > + priority = 100 + 2; > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > /* S_ROUTER_OUT_SNAT uses priority > > > > > > > > > > > > > > (mask + > > > > > 1 + 128 + 1) */ > > > > > > > > > > > > > > - priority = count_1bits(ntohl(mask)) + > > > > > > > > > > > > > > 2; > > > > > > > > > > > > > > + priority = count_1bits(ntohl(mask)) + > > > > > > > > > > > > > > 3; > > > > > > > > > > > > > > > > > > > > > > > > > > > > if (!od->is_gw_router) { > > > > > > > > > > > > > > priority += 128; > > > > > > > > > > > > > > @@ -12268,9 +12273,9 @@ > > > > > build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > /* Traffic received on l3dgw_port is > > > > > > > > > > > > > > subject > > > > > to NAT. */ > > > > > > > > > > > > > > ds_clear(match); > > > > > > > > > > > > > > ds_clear(actions); > > > > > > > > > > > > > > - ds_put_format(match, "ip && ip%s.dst == %s > > > > > > > > > > > > > > && > > > > > inport == %s", > > > > > > > > > > > > > > - is_v6 ? "6" : "4", > > > > > nat->external_ip, > > > > > > > > > > > > > > - > > > > > > > > > > > > > > od->l3dgw_ports[0]->json_key); > > > > > > > > > > > > > > + ds_put_format(match, "ip && ip%s.dst == %s > > > > > > > > > > > > > > && > > > > > inport == %s && " > > > > > > > > > > > > > > + "flags.loopback == 0", is_v6 > > > > > > > > > > > > > > ? > > > > > "6" : "4", > > > > > > > > > > > > > > + nat->external_ip, > > > > > od->l3dgw_ports[0]->json_key); > > > > > > > > > > > > > > if (!distributed && od->n_l3dgw_ports) { > > > > > > > > > > > > > > /* Flows for NAT rules that are > > > > > centralized are only > > > > > > > > > > > > > > * programmed on the gateway chassis. > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > @@ -12282,12 +12287,31 @@ > > > > > build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > ds_put_format(actions, "ip%s.dst=%s; > > > > > next;", > > > > > > > > > > > > > > is_v6 ? "6" : "4", > > > > > nat->logical_ip); > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > - ds_put_cstr(actions, "ct_snat;"); > > > > > > > > > > > > > > + ds_put_cstr(actions, > > > > > > > > > > > > > > "ct_snat_in_czone;"); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_IN_UNSNAT, > > > > > > > > > > > > > > 100, > > > > > > > > > > > > > > ds_cstr(match), > > > > > ds_cstr(actions), > > > > > > > > > > > > > > &nat->header_); > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + if (!stateless) { > > > > > > > > > > > > > > + ds_clear(match); > > > > > > > > > > > > > > + ds_clear(actions); > > > > > > > > > > > > > > + ds_put_format(match, "ip && ip%s.dst > > > > > > > > > > > > > > == %s > > > > > && inport == %s && " > > > > > > > > > > > > > > + "flags.loopback == 1 && > > > > > flags.use_snat_zone == 1", > > > > > > > > > > > > > > + is_v6 ? "6" : "4", > > > > > nat->external_ip, > > > > > > > > > > > > > > + > > > > > > > > > > > > > > od->l3dgw_ports[0]->json_key); > > > > > > > > > > > > > > + if (!distributed && od->n_l3dgw_ports) > > > > > > > > > > > > > > { > > > > > > > > > > > > > > + /* Flows for NAT rules that are > > > > > centralized are only > > > > > > > > > > > > > > + * programmed on the gateway > > > > > > > > > > > > > > chassis. */ > > > > > > > > > > > > > > + ds_put_format(match, " && > > > > > is_chassis_resident(%s)", > > > > > > > > > > > > > > + > > > > > od->l3dgw_ports[0]->cr_port->json_key); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > + ds_put_cstr(actions, "ct_snat;"); > > > > > > > > > > > > > > + ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_IN_UNSNAT, > > > > > > > > > > > > > > + 100, > > > > > ds_cstr(match), ds_cstr(actions), > > > > > > > > > > > > > > + &nat->header_); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -12364,7 +12388,7 @@ > > > > > build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > ds_put_format(actions, > > > > > > > > > > > > > > "ip%s.dst=%s; > > > > > next;", > > > > > > > > > > > > > > is_v6 ? "6" : "4", > > > > > nat->logical_ip); > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > - ds_put_format(actions, > > > > > > > > > > > > > > "ct_dnat(%s", > > > > > nat->logical_ip); > > > > > > > > > > > > > > + ds_put_format(actions, > > > > > "ct_dnat_in_czone(%s", nat->logical_ip); > > > > > > > > > > > > > > if (nat->external_port_range[0]) { > > > > > > > > > > > > > > ds_put_format(actions, ",%s", > > > > > nat->external_port_range); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > @@ -12417,7 +12441,8 @@ > > > > > build_lrouter_out_undnat_flow(struct hmap *lflows, struct > > > > > ovn_datapath *od, > > > > > > > > > > > > > > ds_put_format(actions, "ip%s.src=%s; > > > > > > > > > > > > > > next;", > > > > > > > > > > > > > > is_v6 ? "6" : "4", > > > > > nat->external_ip); > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > - ds_put_format(actions, "ct_dnat;"); > > > > > > > > > > > > > > + ds_put_format(actions, > > > > > > > > > > > > > > + od->is_gw_router ? > > > > > > > > > > > > > > "ct_dnat;" : > > > > > "ct_dnat_in_czone;"); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_OUT_UNDNAT, 100, > > > > > > > > > > > > > > @@ -12425,6 +12450,36 @@ > > > > > build_lrouter_out_undnat_flow(struct hmap *lflows, struct > > > > > ovn_datapath *od, > > > > > > > > > > > > > > &nat->header_); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > +static void > > > > > > > > > > > > > > +build_lrouter_out_is_dnat_local(struct hmap > > > > > > > > > > > > > > *lflows, > > > > > struct ovn_datapath *od, > > > > > > > > > > > > > > + const struct > > > > > > > > > > > > > > nbrec_nat > > > > > *nat, struct ds *match, > > > > > > > > > > > > > > + struct ds *actions, > > > > > bool distributed, > > > > > > > > > > > > > > + bool is_v6) > > > > > > > > > > > > > > +{ > > > > > > > > > > > > > > + /* Note that this only applies for NAT on a > > > > > distributed router. > > > > > > > > > > > > > > + */ > > > > > > > > > > > > > > + if (!od->n_l3dgw_ports) { > > > > > > > > > > > > > > + return; > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + ds_clear(match); > > > > > > > > > > > > > > + ds_put_format(match, "ip && ip%s.dst == %s && > > > > > > > > > > > > > > ", > > > > > > > > > > > > > > + is_v6 ? "6" : "4", > > > > > > > > > > > > > > nat->external_ip); > > > > > > > > > > > > > > + if (distributed) { > > > > > > > > > > > > > > + ds_put_format(match, > > > > > "is_chassis_resident(\"%s\")", nat->logical_port); > > > > > > > > > > > > > > + } else { > > > > > > > > > > > > > > + ds_put_format(match, > > > > > > > > > > > > > > "is_chassis_resident(%s)", > > > > > > > > > > > > > > + > > > > > od->l3dgw_ports[0]->cr_port->json_key); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + ds_clear(actions); > > > > > > > > > > > > > > + ds_put_cstr(actions, REGBIT_DST_NAT_IP_LOCAL" > > > > > > > > > > > > > > = 1; > > > > > next;"); > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_OUT_CHECK_DNAT_LOCAL, > > > > > > > > > > > > > > + 50, ds_cstr(match), > > > > > ds_cstr(actions), > > > > > > > > > > > > > > + &nat->header_); > > > > > > > > > > > > > > +} > > > > > > > > > > > > > > + > > > > > > > > > > > > > > static void > > > > > > > > > > > > > > build_lrouter_out_snat_flow(struct hmap *lflows, > > > > > struct ovn_datapath *od, > > > > > > > > > > > > > > const struct nbrec_nat > > > > > *nat, struct ds *match, > > > > > > > > > > > > > > @@ -12478,16 +12533,19 @@ > > > > > build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > ds_put_format(match, "ip && ip%s.src == > > > > > > > > > > > > > > %s && > > > > > outport == %s", > > > > > > > > > > > > > > is_v6 ? "6" : "4", > > > > > nat->logical_ip, > > > > > > > > > > > > > > > > > > > > > > > > > > > > od->l3dgw_ports[0]->json_key); > > > > > > > > > > > > > > - if (!distributed && od->n_l3dgw_ports) { > > > > > > > > > > > > > > - /* Flows for NAT rules that are > > > > > > > > > > > > > > centralized > > > > > are only > > > > > > > > > > > > > > - * programmed on the gateway chassis. */ > > > > > > > > > > > > > > - priority += 128; > > > > > > > > > > > > > > - ds_put_format(match, " && > > > > > is_chassis_resident(%s)", > > > > > > > > > > > > > > - > > > > > od->l3dgw_ports[0]->cr_port->json_key); > > > > > > > > > > > > > > - } else if (distributed) { > > > > > > > > > > > > > > - priority += 128; > > > > > > > > > > > > > > - ds_put_format(match, " && > > > > > is_chassis_resident(\"%s\")", > > > > > > > > > > > > > > - nat->logical_port); > > > > > > > > > > > > > > + if (od->n_l3dgw_ports) { > > > > > > > > > > > > > > + if (distributed) { > > > > > > > > > > > > > > + ovs_assert(nat->logical_port); > > > > > > > > > > > > > > + priority += 128; > > > > > > > > > > > > > > + ds_put_format(match, " && > > > > > is_chassis_resident(\"%s\")", > > > > > > > > > > > > > > + nat->logical_port); > > > > > > > > > > > > > > + } else { > > > > > > > > > > > > > > + /* Flows for NAT rules that are > > > > > centralized are only > > > > > > > > > > > > > > + * programmed on the gateway > > > > > > > > > > > > > > chassis. */ > > > > > > > > > > > > > > + priority += 128; > > > > > > > > > > > > > > + ds_put_format(match, " && > > > > > is_chassis_resident(%s)", > > > > > > > > > > > > > > + > > > > > od->l3dgw_ports[0]->cr_port->json_key); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > ds_clear(actions); > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -12505,7 +12563,7 @@ > > > > > build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > ds_put_format(actions, "ip%s.src=%s; > > > > > next;", > > > > > > > > > > > > > > is_v6 ? "6" : "4", > > > > > nat->external_ip); > > > > > > > > > > > > > > } else { > > > > > > > > > > > > > > - ds_put_format(actions, "ct_snat(%s", > > > > > > > > > > > > > > + ds_put_format(actions, > > > > > "ct_snat_in_czone(%s", > > > > > > > > > > > > > > nat->external_ip); > > > > > > > > > > > > > > if (nat->external_port_range[0]) { > > > > > > > > > > > > > > ds_put_format(actions, ",%s", > > > > > nat->external_port_range); > > > > > > > > > > > > > > @@ -12519,6 +12577,24 @@ > > > > > build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath > > > > > *od, > > > > > > > > > > > > > > ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_OUT_SNAT, > > > > > > > > > > > > > > priority, > > > > > ds_cstr(match), > > > > > > > > > > > > > > ds_cstr(actions), > > > > > &nat->header_); > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + if (!stateless) { > > > > > > > > > > > > > > + ds_put_cstr(match, " && > > > > > "REGBIT_DST_NAT_IP_LOCAL" == 1"); > > > > > > > > > > > > > > + ds_clear(actions); > > > > > > > > > > > > > > + if (distributed) { > > > > > > > > > > > > > > + ds_put_format(actions, "eth.src = > > > > > "ETH_ADDR_FMT"; ", > > > > > > > > > > > > > > + ETH_ADDR_ARGS(mac)); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > + ds_put_format(actions, > > > > > REGBIT_DST_NAT_IP_LOCAL" = 0; ct_snat(%s", > > > > > > > > > > > > > > + nat->external_ip); > > > > > > > > > > > > > > + if (nat->external_port_range[0]) { > > > > > > > > > > > > > > + ds_put_format(actions, ",%s", > > > > > nat->external_port_range); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > + ds_put_format(actions, ");"); > > > > > > > > > > > > > > + ovn_lflow_add_with_hint(lflows, od, > > > > > S_ROUTER_OUT_SNAT, > > > > > > > > > > > > > > + priority + 1, > > > > > ds_cstr(match), > > > > > > > > > > > > > > + > > > > > > > > > > > > > > ds_cstr(actions), > > > > > &nat->header_); > > > > > > > > > > > > > > + } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -12749,10 +12825,13 @@ > > > > > build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap > > > > > *lflows, > > > > > > > > > > > > > > /* Packets are allowed by default. */ > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, > > > > > > > > > > > > > > 0, > > > > > "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, > > > > > > > > > > > > > > 0, > > > > > "1", "next;"); > > > > > > > > > > > > > > + ovn_lflow_add(lflows, od, > > > > > S_ROUTER_OUT_CHECK_DNAT_LOCAL, 0, "1", > > > > > > > > > > > > > > + REGBIT_DST_NAT_IP_LOCAL" = 0; > > > > > > > > > > > > > > next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, > > > > > > > > > > > > > > 0, > > > > > "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, > > > > > "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > > > > > > > > > > S_ROUTER_OUT_UNDNAT, 0, > > > > > "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); > > > > > > > > > > > > > > + ovn_lflow_add(lflows, od, > > > > > > > > > > > > > > S_ROUTER_OUT_POST_SNAT, > > > > > 0, "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > > > > > > > > > > S_ROUTER_OUT_EGR_LOOP, > > > > > 0, "1", "next;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -12765,8 +12844,7 @@ > > > > > build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap > > > > > *lflows, > > > > > > > > > > > > > > * not committed, it would produce ongoing > > > > > datapath flows with the ct.new > > > > > > > > > > > > > > * flag set. Some NICs are unable to offload > > > > > > > > > > > > > > these > > > > > flows. > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > - if ((od->is_gw_router || od->n_l3dgw_ports) && > > > > > > > > > > > > > > - (od->nbr->n_nat || od->has_lb_vip)) { > > > > > > > > > > > > > > + if (od->is_gw_router && (od->nbr->n_nat || > > > > > od->has_lb_vip)) { > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > > > > > > > > > > S_ROUTER_OUT_UNDNAT, > > > > > 50, > > > > > > > > > > > > > > "ip", "flags.loopback = 1; > > > > > ct_dnat;"); > > > > > > > > > > > > > > ovn_lflow_add(lflows, od, > > > > > S_ROUTER_OUT_POST_UNDNAT, 50, > > > > > > > > > > > > > > @@ -12839,6 +12917,10 @@ > > > > > build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap > > > > > *lflows, > > > > > > > > > > > > > > } > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > + /* S_ROUTER_OUT_DNAT_LOCAL */ > > > > > > > > > > > > > > + build_lrouter_out_is_dnat_local(lflows, od, > > > > > nat, match, actions, > > > > > > > > > > > > > > + > > > > > > > > > > > > > > distributed, > > > > > is_v6); > > > > > > > > > > > > > > + > > > > > > > > > > > > > > /* S_ROUTER_OUT_UNDNAT */ > > > > > > > > > > > > > > build_lrouter_out_undnat_flow(lflows, od, > > > > > > > > > > > > > > nat, > > > > > match, actions, distributed, > > > > > > > > > > > > > > mac, is_v6); > > > > > > > > > > > > > > @@ -12912,7 +12994,8 @@ > > > > > build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap > > > > > *lflows, > > > > > > > > > > > > > > "clone { ct_clear; " > > > > > > > > > > > > > > "inport = outport; > > > > > > > > > > > > > > outport = > > > > > \"\"; " > > > > > > > > > > > > > > "eth.dst <-> eth.src; " > > > > > > > > > > > > > > - "flags = 0; > > > > > > > > > > > > > > flags.loopback = > > > > > 1; "); > > > > > > > > > > > > > > + "flags = 0; > > > > > > > > > > > > > > flags.loopback = > > > > > 1; " > > > > > > > > > > > > > > + "flags.use_snat_zone = > > > > > "REGBIT_DST_NAT_IP_LOCAL"; "); > > > > > > > > > > > > > > for (int j = 0; j < MFF_N_LOG_REGS; > > > > > > > > > > > > > > j++) { > > > > > > > > > > > > > > ds_put_format(actions, "reg%d = > > > > > > > > > > > > > > 0; ", > > > > > j); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > diff --git a/northd/ovn-northd.8.xml > > > > > b/northd/ovn-northd.8.xml > > > > > > > > > > > > > > index 21d83718c..e39e6e805 100644 > > > > > > > > > > > > > > --- a/northd/ovn-northd.8.xml > > > > > > > > > > > > > > +++ b/northd/ovn-northd.8.xml > > > > > > > > > > > > > > @@ -2879,23 +2879,65 @@ icmp6 { > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > For each configuration in the OVN > > > > > > > > > > > > > > Northbound > > > > > database, that asks > > > > > > > > > > > > > > to change the source IP address of a > > > > > > > > > > > > > > packet > > > > > from <var>A</var> to > > > > > > > > > > > > > > - <var>B</var>, a priority-100 flow matches > > > > > <code>ip && > > > > > > > > > > > > > > - ip4.dst == <var>B</var> && > > > > > > > > > > > > > > inport == > > > > > <var>GW</var></code> or > > > > > > > > > > > > > > - <code>ip && > > > > > > > > > > > > > > - ip6.dst == <var>B</var> && > > > > > > > > > > > > > > inport == > > > > > <var>GW</var></code> > > > > > > > > > > > > > > - where <var>GW</var> is the logical router > > > > > gateway port, with an > > > > > > > > > > > > > > - action <code>ct_snat;</code>. If the NAT > > > > > > > > > > > > > > rule > > > > > is of type > > > > > > > > > > > > > > - dnat_and_snat and has > > > > > <code>stateless=true</code> in the > > > > > > > > > > > > > > - options, then the action would be > > > > > <code>ip4/6.dst= > > > > > > > > > > > > > > - (<var>B</var>)</code>. > > > > > > > > > > > > > > + <var>B</var>, two priority-100 flows are > > > > > added. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > If the NAT rule cannot be handled in a > > > > > distributed manner, then > > > > > > > > > > > > > > - the priority-100 flow above is only > > > > > programmed on the > > > > > > > > > > > > > > + the below priority-100 flows are only > > > > > programmed on the > > > > > > > > > > > > > > gateway chassis. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > + <ul> > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + The first flow matches <code>ip > > > > > > > > > > > > > > && > > > > > > > > > > > > > > + ip4.dst == <var>B</var> && > > > > > > > > > > > > > > inport > > > > > == <var>GW</var> > > > > > > > > > > > > > > + && flags.loopback == > > > > > > > > > > > > > > 0</code> or > > > > > > > > > > > > > > + <code>ip && > > > > > > > > > > > > > > + ip6.dst == <var>B</var> && > > > > > > > > > > > > > > inport > > > > > == <var>GW</var> > > > > > > > > > > > > > > + && flags.loopback == 0</code> > > > > > > > > > > > > > > + where <var>GW</var> is the logical > > > > > > > > > > > > > > router > > > > > gateway port, with an > > > > > > > > > > > > > > + action > > > > > > > > > > > > > > <code>ct_snat_in_czone;</code> to > > > > > unSNAT in the common > > > > > > > > > > > > > > + zone. If the NAT rule is of type > > > > > dnat_and_snat and has > > > > > > > > > > > > > > + <code>stateless=true</code> in the > > > > > options, then the action > > > > > > > > > > > > > > + would be > > > > > <code>ip4/6.dst=(<var>B</var>)</code>. > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + If the NAT entry is of type > > > > > <code>snat</code>, then there is an > > > > > > > > > > > > > > + additional match > > > > > <code>is_chassis_resident(<var>cr-GW</var>) > > > > > > > > > > > > > > + </code> where <var>cr-GW</var> is the > > > > > chassis resident port of > > > > > > > > > > > > > > + <var>GW</var>. > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + The second flow matches <code>ip > > > > > && > > > > > > > > > > > > > > + ip4.dst == <var>B</var> && > > > > > > > > > > > > > > inport > > > > > == <var>GW</var> > > > > > > > > > > > > > > + && flags.loopback == 1 > > > > > > > > > > > > > > && > > > > > > > > > > > > > > + flags.use_snat_zone == 1</code> or > > > > > > > > > > > > > > + <code>ip && > > > > > > > > > > > > > > + ip6.dst == <var>B</var> && > > > > > > > > > > > > > > inport > > > > > == <var>GW</var> > > > > > > > > > > > > > > + && flags.loopback == 0 > > > > > > > > > > > > > > && > > > > > > > > > > > > > > + flags.use_snat_zone == 1</code> > > > > > > > > > > > > > > + where <var>GW</var> is the logical > > > > > > > > > > > > > > router > > > > > gateway port, with an > > > > > > > > > > > > > > + action <code>ct_snat;</code> to > > > > > > > > > > > > > > unSNAT in > > > > > the snat zone. If the > > > > > > > > > > > > > > + NAT rule is of type dnat_and_snat > > > > > > > > > > > > > > and has > > > > > > > > > > > > > > + <code>stateless=true</code> in the > > > > > options, then the action > > > > > > > > > > > > > > + would be > > > > > <code>ip4/6.dst=(<var>B</var>)</code>. > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + If the NAT entry is of type > > > > > <code>snat</code>, then there is an > > > > > > > > > > > > > > + additional match > > > > > <code>is_chassis_resident(<var>cr-GW</var>) > > > > > > > > > > > > > > + </code> where <var>cr-GW</var> is the > > > > > chassis resident port of > > > > > > > > > > > > > > + <var>GW</var>. > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + </ul> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > A priority-0 logical flow with match > > > > > <code>1</code> has actions > > > > > > > > > > > > > > <code>next;</code>. > > > > > > > > > > > > > > @@ -4031,7 +4073,43 @@ nd_ns { > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <h3>Egress Table 0: UNDNAT</h3> > > > > > > > > > > > > > > + <h3>Egress Table 0: Check DNAT local </h3> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + This table checks if the packet needs to be > > > > > DNATed in the router ingress > > > > > > > > > > > > > > + table <code>lr_in_dnat</code> after it is > > > > > > > > > > > > > > SNATed > > > > > and looped back > > > > > > > > > > > > > > + to the ingress pipeline. This check is done > > > > > > > > > > > > > > only > > > > > for routers configured > > > > > > > > > > > > > > + with distributed gateway ports and NAT > > > > > > > > > > > > > > entries. > > > > > This check is done > > > > > > > > > > > > > > + so that SNAT and DNAT is done in different > > > > > > > > > > > > > > zones > > > > > instead of a common > > > > > > > > > > > > > > + zone. > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <ul> > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + <p> > > > > > > > > > > > > > > + For each NAT rule in the OVN Northbound > > > > > database on a > > > > > > > > > > > > > > + distributed router, a priority-50 logical > > > > > flow with match > > > > > > > > > > > > > > + <code>ip4.dst == <var>E</var> && > > > > > > > > > > > > > > + is_chassis_resident(<var>P</var>)</code>, > > > > > where <var>E</var> is the > > > > > > > > > > > > > > + external IP address specified in the NAT > > > > > rule, <var>GW</var> > > > > > > > > > > > > > > + is the logical router distributed gateway > > > > > port. For dnat_and_snat > > > > > > > > > > > > > > + NAT rule, <var>P</var> is the logical > > > > > > > > > > > > > > port > > > > > specified in the NAT rule. > > > > > > > > > > > > > > + If <ref column="logical_port" > > > > > > > > > > > > > > + table="NAT" db="OVN_Northbound"/> column > > > > > > > > > > > > > > of > > > > > > > > > > > > > > + <ref table="NAT" db="OVN_Northbound"/> > > > > > > > > > > > > > > table > > > > > is NOT set, then > > > > > > > > > > > > > > + <var>P</var> is the <code>chassisredirect > > > > > port</code> of > > > > > > > > > > > > > > + <var>GW</var> with the actions: > > > > > > > > > > > > > > + <code>REGBIT_DST_NAT_IP_LOCAL = 1; next; > > > > > </code> > > > > > > > > > > > > > > + </p> > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + A priority-0 logical flow with match > > > > > <code>1</code> has actions > > > > > > > > > > > > > > + <code>REGBIT_DST_NAT_IP_LOCAL = 0; > > > > > > > > > > > > > > next;</code>. > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + </ul> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <h3>Egress Table 1: UNDNAT</h3> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > This is for already established connections' > > > > > reverse traffic. > > > > > > > > > > > > > > @@ -4040,6 +4118,23 @@ nd_ns { > > > > > > > > > > > > > > is unDNATed here. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > + <ul> > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + A priority-0 logical flow with match > > > > > <code>1</code> has actions > > > > > > > > > > > > > > + <code>next;</code>. > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + </ul> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <h3>Egress Table 1: UNDNAT on Gateway > > > > > > > > > > > > > > Routers</h3> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <ul> > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + For all IP packets, a priority-50 flow > > > > > > > > > > > > > > with an > > > > > action > > > > > > > > > > > > > > + <code>flags.loopback = 1; ct_dnat;</code>. > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + </ul> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <h3>Egress Table 1: UNDNAT on Distributed > > > > > Routers</h3> > > > > > > > > > > > > > > <ul> > > > > > > > > > > > > > > <li> > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > @@ -4050,9 +4145,9 @@ nd_ns { > > > > > > > > > > > > > > gateway chassis that matches > > > > > > > > > > > > > > <code>ip && ip4.src == > > > > > > > > > > > > > > <var>B</var> > > > > > && > > > > > > > > > > > > > > outport == <var>GW</var></code>, where > > > > > <var>GW</var> is the logical > > > > > > > > > > > > > > - router gateway port with an action > > > > > <code>ct_dnat;</code>. If the > > > > > > > > > > > > > > - backend IPv4 address <var>B</var> is also > > > > > configured with L4 port > > > > > > > > > > > > > > - <var>PORT</var> of protocol <var>P</var>, > > > > > then the > > > > > > > > > > > > > > + router gateway port with an action > > > > > <code>ct_dnat_in_czone;</code>. > > > > > > > > > > > > > > + If the backend IPv4 address <var>B</var> > > > > > > > > > > > > > > is > > > > > also configured with > > > > > > > > > > > > > > + L4 port <var>PORT</var> of protocol > > > > > <var>P</var>, then the > > > > > > > > > > > > > > match also includes <code>P.src</code> > > > > > > > > > > > > > > == > > > > > <var>PORT</var>. These > > > > > > > > > > > > > > flows are not added for load balancers > > > > > > > > > > > > > > with > > > > > IPv6 <var>VIPs</var>. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > @@ -4072,7 +4167,7 @@ nd_ns { > > > > > > > > > > > > > > matches <code>ip && ip4.src == > > > > > <var>B</var> > > > > > > > > > > > > > > && outport == > > > > > > > > > > > > > > <var>GW</var></code>, > > > > > where <var>GW</var> > > > > > > > > > > > > > > is the logical router gateway port, > > > > > > > > > > > > > > with an > > > > > action > > > > > > > > > > > > > > - <code>ct_dnat;</code>. If the NAT rule > > > > > > > > > > > > > > is of > > > > > type > > > > > > > > > > > > > > + <code>ct_dnat_in_czone;</code>. If the > > > > > > > > > > > > > > NAT > > > > > rule is of type > > > > > > > > > > > > > > dnat_and_snat and has > > > > > <code>stateless=true</code> in the > > > > > > > > > > > > > > options, then the action would be > > > > > <code>ip4/6.src= > > > > > > > > > > > > > > (<var>B</var>)</code>. > > > > > > > > > > > > > > @@ -4081,7 +4176,7 @@ nd_ns { > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > If the NAT rule cannot be handled in a > > > > > distributed manner, then > > > > > > > > > > > > > > the priority-100 flow above is only > > > > > programmed on the > > > > > > > > > > > > > > - gateway chassis. > > > > > > > > > > > > > > + gateway chassis with the action > > > > > <code>ct_dnat_in_czone</code>. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > @@ -4094,26 +4189,17 @@ nd_ns { > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <li> > > > > > > > > > > > > > > - For all IP packets, a priority-50 flow > > > > > > > > > > > > > > with an > > > > > action > > > > > > > > > > > > > > - <code>flags.loopback = 1; ct_dnat;</code>. > > > > > > > > > > > > > > - </li> > > > > > > > > > > > > > > - > > > > > > > > > > > > > > - <li> > > > > > > > > > > > > > > - A priority-0 logical flow with match > > > > > <code>1</code> has actions > > > > > > > > > > > > > > - <code>next;</code>. > > > > > > > > > > > > > > - </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <h3>Egress Table 1: Post UNDNAT</h3> > > > > > > > > > > > > > > + <h3>Egress Table 2: Post UNDNAT</h3> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > <ul> > > > > > > > > > > > > > > <li> > > > > > > > > > > > > > > A priority-50 logical flow is added that > > > > > commits any untracked flows > > > > > > > > > > > > > > - from the previous table > > > > > <code>lr_out_undnat</code>. This flow > > > > > > > > > > > > > > - matches on <code>ct.new && > > > > > > > > > > > > > > ip</code> > > > > > with action > > > > > > > > > > > > > > - <code>ct_commit { } ; next; </code>. > > > > > > > > > > > > > > + from the previous table > > > > > <code>lr_out_undnat</code> for Gateway > > > > > > > > > > > > > > + routers. This flow matches on > > > > > > > > > > > > > > <code>ct.new > > > > > && ip</code> > > > > > > > > > > > > > > + with action <code>ct_commit { } ; next; > > > > > </code>. > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <li> > > > > > > > > > > > > > > @@ -4124,7 +4210,7 @@ nd_ns { > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <h3>Egress Table 2: SNAT</h3> > > > > > > > > > > > > > > + <h3>Egress Table 3: SNAT</h3> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > Packets that are configured to be SNATed get > > > > > their source IP address > > > > > > > > > > > > > > @@ -4140,7 +4226,7 @@ nd_ns { > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <p>Egress Table 2: SNAT on Gateway Routers</p> > > > > > > > > > > > > > > + <p>Egress Table 3: SNAT on Gateway Routers</p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <ul> > > > > > > > > > > > > > > <li> > > > > > > > > > > > > > > @@ -4239,7 +4325,7 @@ nd_ns { > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <p>Egress Table 2: SNAT on Distributed > > > > > > > > > > > > > > Routers</p> > > > > > > > > > > > > > > + <p>Egress Table 3: SNAT on Distributed > > > > > > > > > > > > > > Routers</p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <ul> > > > > > > > > > > > > > > <li> > > > > > > > > > > > > > > @@ -4247,28 +4333,47 @@ nd_ns { > > > > > > > > > > > > > > For each configuration in the OVN > > > > > > > > > > > > > > Northbound > > > > > database, that asks > > > > > > > > > > > > > > to change the source IP address of a > > > > > > > > > > > > > > packet > > > > > from an IP address of > > > > > > > > > > > > > > <var>A</var> or to change the source IP > > > > > address of a packet that > > > > > > > > > > > > > > - belongs to network <var>A</var> to > > > > > <var>B</var>, a flow matches > > > > > > > > > > > > > > - <code>ip && ip4.src == > > > > > > > > > > > > > > <var>A</var> > > > > > && > > > > > > > > > > > > > > - outport == <var>GW</var></code>, where > > > > > <var>GW</var> is the > > > > > > > > > > > > > > - logical router gateway port, with an > > > > > > > > > > > > > > action > > > > > > > > > > > > > > - <code>ct_snat(<var>B</var>);</code>. The > > > > > priority of the flow > > > > > > > > > > > > > > - is calculated based on the mask of > > > > > <var>A</var>, with matches > > > > > > > > > > > > > > - having larger masks getting higher > > > > > priorities. If the NAT rule > > > > > > > > > > > > > > - is of type dnat_and_snat and has > > > > > <code>stateless=true</code> > > > > > > > > > > > > > > - in the options, then the action would be > > > > > <code>ip4/6.src= > > > > > > > > > > > > > > - (<var>B</var>)</code>. > > > > > > > > > > > > > > + belongs to network <var>A</var> to > > > > > <var>B</var>, two flows are > > > > > > > > > > > > > > + added. The priority <var>P</var> of > > > > > > > > > > > > > > these > > > > > flows are calculated > > > > > > > > > > > > > > + based on the mask of <var>A</var>, with > > > > > matches having larger > > > > > > > > > > > > > > + masks getting higher priorities. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > If the NAT rule cannot be handled in a > > > > > distributed manner, then > > > > > > > > > > > > > > - the flow above is only programmed on the > > > > > > > > > > > > > > - gateway chassis increasing flow priority > > > > > > > > > > > > > > by > > > > > 128 in > > > > > > > > > > > > > > - order to be run first > > > > > > > > > > > > > > + the below flows are only programmed on > > > > > > > > > > > > > > the > > > > > gateway chassis increasing > > > > > > > > > > > > > > + flow priority by 128 in order to be run > > > > > > > > > > > > > > first. > > > > > > > > > > > > > > </p> > > > > > > > > > > > > > > > > > > > > > > > > > > > > + <ul> > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + The first flow is added with the > > > > > > > > > > > > > > calculated > > > > > priority <var>P</var> > > > > > > > > > > > > > > + and match <code>ip && ip4.src > > > > > > > > > > > > > > == > > > > > <var>A</var> && > > > > > > > > > > > > > > + outport == <var>GW</var></code>, where > > > > > <var>GW</var> is the > > > > > > > > > > > > > > + logical router gateway port, with an > > > > > > > > > > > > > > action > > > > > > > > > > > > > > + > > > > > <code>ct_snat_in_czone(<var>B</var>);</code> to SNATed in the > > > > > > > > > > > > > > + common zone. If the NAT rule is of > > > > > > > > > > > > > > type > > > > > dnat_and_snat and has > > > > > > > > > > > > > > + <code>stateless=true</code> in the > > > > > > > > > > > > > > options, > > > > > then the action > > > > > > > > > > > > > > + would be > > > > > <code>ip4/6.src=(<var>B</var>)</code>. > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > + <li> > > > > > > > > > > > > > > + The second flow is added with the > > > > > calculated priority > > > > > > > > > > > > > > + <code><var>P</var> + 1 </code> and > > > > > > > > > > > > > > match > > > > > > > > > > > > > > + <code>ip && ip4.src == > > > > > > > > > > > > > > <var>A</var> > > > > > && > > > > > > > > > > > > > > + outport == <var>GW</var> && > > > > > > > > > > > > > > + REGBIT_DST_NAT_IP_LOCAL == 0</code>, > > > > > > > > > > > > > > where > > > > > <var>GW</var> is the > > > > > > > > > > > > > > + logical router gateway port, with an > > > > > > > > > > > > > > action > > > > > > > > > > > > > > + <code>ct_snat(<var>B</var>);</code> to > > > > > > > > > > > > > > SNAT > > > > > in the snat zone. > > > > > > > > > > > > > > + If the NAT rule is of type > > > > > > > > > > > > > > dnat_and_snat > > > > > and has > > > > > > > > > > > > > > + <code>stateless=true</code> in the > > > > > > > > > > > > > > options, > > > > > then the action would > > > > > > > > > > > > > > + be > > > > > > > > > > > > > > <code>ip4/6.src=(<var>B</var>)</code>. > > > > > > > > > > > > > > + </li> > > > > > > > > > > > > > > + </ul> > > > > > > > > > > > > > > + > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > If the NAT rule can be handled in a > > > > > distributed manner, then > > > > > > > > > > > > > > - there is an additional action > > > > > > > > > > > > > > + there is an additional action (for both > > > > > > > > > > > > > > the > > > > > flows) > > > > > > > > > > > > > > <code>eth.src = <var>EA</var>;</code>, > > > > > > > > > > > > > > where > > > > > <var>EA</var> > > > > > > > > > > > > > > is the ethernet address associated with > > > > > > > > > > > > > > the > > > > > IP address > > > > > > > > > > > > > > <var>A</var> in the NAT rule. This > > > > > > > > > > > > > > allows > > > > > upstream MAC > > > > > > > > > > > > > > @@ -4284,7 +4389,8 @@ nd_ns { > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > If the NAT rule has > > > > > <code>exempted_ext_ips</code> set, then > > > > > > > > > > > > > > - there is an additional flow configured > > > > > > > > > > > > > > at the > > > > > priority + 1 of > > > > > > > > > > > > > > + there is an additional flow configured > > > > > > > > > > > > > > at the > > > > > priority > > > > > > > > > > > > > > + <code><var>P</var> + 2 </code> of > > > > > > > > > > > > > > corresponding NAT rule. The flow > > > > > > > > > > > > > > matches if > > > > > destination ip > > > > > > > > > > > > > > is an <code>exempted_ext_ip</code> and > > > > > > > > > > > > > > the > > > > > action is <code>next; > > > > > > > > > > > > > > </code>. This flow is used to bypass the > > > > > ct_snat action for a flow > > > > > > > > > > > > > > @@ -4299,7 +4405,7 @@ nd_ns { > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <h3>Egress Table 3: Egress Loopback</h3> > > > > > > > > > > > > > > + <h3>Egress Table 4: Egress Loopback</h3> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > For distributed logical routers where one > > > > > > > > > > > > > > of the > > > > > logical router > > > > > > > > > > > > > > @@ -4344,6 +4450,7 @@ clone { > > > > > > > > > > > > > > outport = ""; > > > > > > > > > > > > > > flags = 0; > > > > > > > > > > > > > > flags.loopback = 1; > > > > > > > > > > > > > > + flags.use_snat_zone = REGBIT_DST_NAT_IP_LOCAL; > > > > > > > > > > > > > > reg0 = 0; > > > > > > > > > > > > > > reg1 = 0; > > > > > > > > > > > > > > ... > > > > > > > > > > > > > > @@ -4368,7 +4475,7 @@ clone { > > > > > > > > > > > > > > </li> > > > > > > > > > > > > > > </ul> > > > > > > > > > > > > > > > > > > > > > > > > > > > > - <h3>Egress Table 4: Delivery</h3> > > > > > > > > > > > > > > + <h3>Egress Table 5: Delivery</h3> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <p> > > > > > > > > > > > > > > Packets that reach this table are ready for > > > > > delivery. It contains: > > > > > > > > > > > > > > diff --git a/tests/ovn-northd.at > > > > > > > > > > > > > > b/tests/ovn-northd.at > > > > > > > > > > > > > > index 85b47a18f..70ec5e2e3 100644 > > > > > > > > > > > > > > --- a/tests/ovn-northd.at > > > > > > > > > > > > > > +++ b/tests/ovn-northd.at > > > > > > > > > > > > > > @@ -877,25 +877,25 @@ check_flow_match_sets() { > > > > > > > > > > > > > > echo > > > > > > > > > > > > > > echo "IPv4: stateful" > > > > > > > > > > > > > > ovn-nbctl --wait=sb lr-nat-add R1 dnat_and_snat > > > > > 172.16.1.1 50.0.0.11 > > > > > > > > > > > > > > -check_flow_match_sets 2 2 3 0 0 0 0 > > > > > > > > > > > > > > +check_flow_match_sets 3 4 2 0 0 0 0 > > > > > > > > > > > > > > ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > echo > > > > > > > > > > > > > > echo "IPv4: stateless" > > > > > > > > > > > > > > ovn-nbctl --wait=sb --stateless lr-nat-add R1 > > > > > dnat_and_snat 172.16.1.1 50.0.0.11 > > > > > > > > > > > > > > -check_flow_match_sets 2 0 1 2 2 0 0 > > > > > > > > > > > > > > +check_flow_match_sets 2 0 0 2 2 0 0 > > > > > > > > > > > > > > ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > echo > > > > > > > > > > > > > > echo "IPv6: stateful" > > > > > > > > > > > > > > ovn-nbctl --wait=sb lr-nat-add R1 dnat_and_snat > > > > > fd01::1 fd11::2 > > > > > > > > > > > > > > -check_flow_match_sets 2 2 3 0 0 0 0 > > > > > > > > > > > > > > +check_flow_match_sets 3 4 2 0 0 0 0 > > > > > > > > > > > > > > ovn-nbctl lr-nat-del R1 dnat_and_snat fd01::1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > echo > > > > > > > > > > > > > > echo "IPv6: stateless" > > > > > > > > > > > > > > ovn-nbctl --wait=sb --stateless lr-nat-add R1 > > > > > dnat_and_snat fd01::1 fd11::2 > > > > > > > > > > > > > > -check_flow_match_sets 2 0 1 0 0 2 2 > > > > > > > > > > > > > > +check_flow_match_sets 2 0 0 0 0 2 2 > > > > > > > > > > > > > > > > > > > > > > > > > > > > AT_CLEANUP > > > > > > > > > > > > > > ]) > > > > > > > > > > > > > > @@ -924,9 +924,9 @@ echo "CR-LRP UUID is: " $uuid > > > > > > > > > > > > > > ovn-nbctl --portrange lr-nat-add R1 dnat_and_snat > > > > > 172.16.1.1 50.0.0.11 1-3000 > > > > > > > > > > > > > > > > > > > > > > > > > > > > AT_CAPTURE_FILE([sbflows]) > > > > > > > > > > > > > > -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows > > > > > > > > > > > > > > && > > > > > test 2 = `grep -c lr_in_unsnat sbflows`]) > > > > > > > > > > > > > > +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows > > > > > > > > > > > > > > && > > > > > test 3 = `grep -c lr_in_unsnat sbflows`]) > > > > > > > > > > > > > > AT_CHECK([grep -c 'ct_snat.*3000' sbflows && grep > > > > > > > > > > > > > > -c > > > > > 'ct_dnat.*3000' sbflows], > > > > > > > > > > > > > > - [0], [1 > > > > > > > > > > > > > > + [0], [2 > > > > > > > > > > > > > > 1 > > > > > > > > > > > > > > ]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -934,9 +934,9 @@ ovn-nbctl lr-nat-del R1 > > > > > dnat_and_snat 172.16.1.1 > > > > > > > > > > > > > > ovn-nbctl --wait=sb --portrange lr-nat-add R1 snat > > > > > 172.16.1.1 50.0.0.11 1-3000 > > > > > > > > > > > > > > > > > > > > > > > > > > > > AT_CAPTURE_FILE([sbflows2]) > > > > > > > > > > > > > > -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows2 > > > > > > > > > > > > > > && > > > > > test 2 = `grep -c lr_in_unsnat sbflows`]) > > > > > > > > > > > > > > +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows2 > > > > > > > > > > > > > > && > > > > > test 3 = `grep -c lr_in_unsnat sbflows`]) > > > > > > > > > > > > > > AT_CHECK([grep -c 'ct_snat.*3000' sbflows2 && > > > > > > > > > > > > > > grep -c > > > > > 'ct_dnat.*3000' sbflows2], > > > > > > > > > > > > > > - [1], [1 > > > > > > > > > > > > > > + [1], [2 > > > > > > > > > > > > > > 0 > > > > > > > > > > > > > > ]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -944,7 +944,7 @@ ovn-nbctl lr-nat-del R1 snat > > > > > 172.16.1.1 > > > > > > > > > > > > > > ovn-nbctl --wait=sb --portrange --stateless > > > > > > > > > > > > > > lr-nat-add > > > > > R1 dnat_and_snat 172.16.1.2 50.0.0.12 1-3000 > > > > > > > > > > > > > > > > > > > > > > > > > > > > AT_CAPTURE_FILE([sbflows3]) > > > > > > > > > > > > > > -OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows3 > > > > > > > > > > > > > > && > > > > > test 3 = `grep -c lr_in_unsnat sbflows3`]) > > > > > > > > > > > > > > +OVS_WAIT_UNTIL([ovn-sbctl dump-flows R1 > sbflows3 > > > > > > > > > > > > > > && > > > > > test 4 = `grep -c lr_in_unsnat sbflows3`]) > > > > > > > > > > > > > > AT_CHECK([grep 'ct_[s]dnat.*172\.16\.1\.2.*3000' > > > > > sbflows3], [1]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > ovn-nbctl lr-nat-del R1 dnat_and_snat 172.16.1.1 > > > > > > > > > > > > > > @@ -1008,17 +1008,20 @@ AT_CAPTURE_FILE([drflows]) > > > > > > > > > > > > > > ovn-sbctl dump-flows CR > crflows > > > > > > > > > > > > > > AT_CAPTURE_FILE([crflows]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > -AT_CHECK([ > > > > > > > > > > > > > > - grep -c lr_out_snat drflows > > > > > > > > > > > > > > - grep -c lr_out_snat crflows > > > > > > > > > > > > > > - grep lr_out_snat drflows | grep "ip4.src == > > > > > 50.0.0.11" | grep -c "ip4.dst == $allowed_range" > > > > > > > > > > > > > > - grep lr_out_snat crflows | grep "ip4.src == > > > > > 50.0.0.11" | grep -c "ip4.dst == $allowed_range"], [0], [dnl > > > > > > > > > > > > > > -3 > > > > > > > > > > > > > > -3 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > +AT_CHECK([grep -e "lr_out_snat" drflows | sed > > > > > 's/table=../table=??/' | sort], [0], [dnl > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=0 , > > > > > match=(1), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=120 , > > > > > match=(nd_ns), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=161 , > > > > > match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && > > > > > is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range), > > > > > action=(ct_snat_in_czone(172.16.1.1);) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=162 , > > > > > match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && > > > > > is_chassis_resident("cr-DR-S1") && ip4.dst == $allowed_range && > > > > > reg9[[4]] > > > > > == 1), action=(reg9[[4]] = 0; ct_snat(172.16.1.1);) > > > > > > > > > > > > > > +]) > > > > > > > > > > > > > > + > > > > > > > > > > > > > > +AT_CHECK([grep -e "lr_out_snat" crflows | sed > > > > > 's/table=../table=??/' | sort], [0], [dnl > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=0 , > > > > > match=(1), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=120 , > > > > > match=(nd_ns), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=33 , > > > > > match=(ip && ip4.src == 50.0.0.11 && ip4.dst == $allowed_range), > > > > > action=(ct_snat(172.16.1.1);) > > > > > > > > > > > > > > ]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > + > > > > > > > > > > > > > > # SNAT with DISALLOWED_IPs > > > > > > > > > > > > > > check ovn-nbctl lr-nat-del DR snat 50.0.0.11 > > > > > > > > > > > > > > check ovn-nbctl lr-nat-del CR snat 50.0.0.11 > > > > > > > > > > > > > > @@ -1036,19 +1039,19 @@ AT_CAPTURE_FILE([drflows2]) > > > > > > > > > > > > > > ovn-sbctl dump-flows CR > crflows2 > > > > > > > > > > > > > > AT_CAPTURE_FILE([crflows2]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > -AT_CHECK([ > > > > > > > > > > > > > > - grep -c lr_out_snat drflows2 > > > > > > > > > > > > > > - grep -c lr_out_snat crflows2 > > > > > > > > > > > > > > - grep lr_out_snat drflows2 | grep "ip4.src == > > > > > 50.0.0.11" | grep "ip4.dst == $disallowed_range" | grep -c > > > > > "priority=162" > > > > > > > > > > > > > > - grep lr_out_snat drflows2 | grep "ip4.src == > > > > > 50.0.0.11" | grep -c "priority=161" > > > > > > > > > > > > > > - grep lr_out_snat crflows2 | grep "ip4.src == > > > > > 50.0.0.11" | grep "ip4.dst == $disallowed_range" | grep -c > > > > > "priority=34" > > > > > > > > > > > > > > - grep lr_out_snat crflows2 | grep "ip4.src == > > > > > 50.0.0.11" | grep -c "priority=33"], [0], [dnl > > > > > > > > > > > > > > -4 > > > > > > > > > > > > > > -4 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > -1 > > > > > > > > > > > > > > +AT_CHECK([grep -e "lr_out_snat" drflows2 | sed > > > > > 's/table=../table=??/' | sort], [0], [dnl > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=0 , > > > > > match=(1), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=120 , > > > > > match=(nd_ns), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=161 , > > > > > match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && > > > > > is_chassis_resident("cr-DR-S1")), > > > > > action=(ct_snat_in_czone(172.16.1.1);) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=162 , > > > > > match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && > > > > > is_chassis_resident("cr-DR-S1") && reg9[[4]] == 1), action=(reg9[[4]] > > > > > = 0; > > > > > ct_snat(172.16.1.1);) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=163 , > > > > > match=(ip && ip4.src == 50.0.0.11 && outport == "DR-S1" && > > > > > is_chassis_resident("cr-DR-S1") && ip4.dst == $disallowed_range), > > > > > action=(next;) > > > > > > > > > > > > > > +]) > > > > > > > > > > > > > > + > > > > > > > > > > > > > > +AT_CHECK([grep -e "lr_out_snat" crflows2 | sed > > > > > 's/table=../table=??/' | sort], [0], [dnl > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=0 , > > > > > match=(1), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=120 , > > > > > match=(nd_ns), action=(next;) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=33 , > > > > > match=(ip && ip4.src == 50.0.0.11), action=(ct_snat(172.16.1.1);) > > > > > > > > > > > > > > + table=??(lr_out_snat ), priority=35 , > > > > > match=(ip && ip4.src == 50.0.0.11 && ip4.dst == $disallowed_range), > > > > > action=(next;) > > > > > > > > > > > > > > ]) > > > > > > > > > > > > > > > > > > > > > > > > > > > > # Stateful FIP with ALLOWED_IPs > > > > > > > > > > > > > > @@ -1059,2 > > > > _______________________________________________ > > > > dev mailing list > > > > d...@openvswitch.org > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > > > > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev