From: Numan Siddique <num...@ovn.org> When a Gateway router is configured with a load balancer and it is also configured with options:lb_force_snat_ip=<IP>, OVN after load balancing the destination IP to one of the backend also does a NAT on the source ip with the lb_force_snat_ip if the packet is destined to a load balancer VIP.
There is a problem with the snat of source ip to 'lb_force_snat_ip' in one particular usecase. When the packet enters the Gateway router from a provider logical switch destined to the load balancer VIP, then it is first load balanced to one of the backend and then the source ip is snatted to 'lb_force_snat_ip'. If the chosen backend is reachable via the provider logical switch, then the packet is hairpinned back and it may hit the wire with the source ip 'lb_force_snat_ip'. If 'lb_force_snat_ip' happens to be an OVN internal IP then the packet may be dropped. This patch addresses this issue by providing the option to set the option - 'lb_force_snat_ip=router_ip'. If 'router_ip' is set, then OVN will snat the load balanced packet to the router ip of the logical router port which chosen as 'outport' in lr_in_ip_routing stage. Example. If the gateway router is ovn-nbctl show lr0 router 68f20092-5563-44b8-9ccb-b11de3e3a66c (lr0) port lr0-sw0 mac: "00:00:00:00:ff:01" networks: ["10.0.0.1/24"] port lr0-public mac: "00:00:20:20:12:13" networks: ["172.168.0.100/24"] Then the below logical flows are added if 'lb_force_snat_ip' is configured to 'router_ip'. table=1 (lr_out_snat), priority=110 match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) table=1 (lr_out_snat), priority=110 match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0") action=(ct_snat(10.0.0.1);) For the above described scenario, the packet will have source ip as 172.168.0.100 which belongs to the provider logical switch CIDR. Reported-by: Tim Rozet <tro...@redhat.com> Signed-off-by: Numan Siddique <num...@ovn.org> --- northd/ovn-northd.8.xml | 35 ++++++++++++++++++ northd/ovn-northd.c | 66 ++++++++++++++++++++++++++++++++-- tests/ovn-northd.at | 79 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 3 deletions(-) diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml index 70065a36d9..27b28aff93 100644 --- a/northd/ovn-northd.8.xml +++ b/northd/ovn-northd.8.xml @@ -3653,6 +3653,32 @@ nd_ns { <code>flags.force_snat_for_dnat == 1 && ip</code> with an action <code>ct_snat(<var>B</var>);</code>. </p> + </li> + + <li> + <p> + If the Gateway router in the OVN Northbound database has been + configured to force SNAT a packet (that has been previously + load-balanced) using router IP (i.e <ref column="options" + table="Logical_Router"/>:lb_force_snat_ip=router_ip), then for + each logical router port <var>P</var> attached to the Gateway + router, a priority-110 flow matches + <code>flags.force_snat_for_lb == 1 && outport == <var>P</var> + </code> with an action <code>ct_snat(<var>R</var>);</code> + where <var>R</var> is the router port IP configured. + If <code>R</code> is an IPv4 address then the match will also + include <code>ip4</code> and if it is an IPv6 address, then the + match will also include <code>ip6</code>. + </p> + + <p> + If the logical router port <var>P</var> is configured with multiple + IPv4 and multiple IPv6 addresses, only the first IPv4 and first IPv6 + address is considered. + </p> + </li> + + <li> <p> If the Gateway router in the OVN Northbound database has been configured to force SNAT a packet (that has been previously @@ -3660,6 +3686,9 @@ nd_ns { <code>flags.force_snat_for_lb == 1 && ip</code> with an action <code>ct_snat(<var>B</var>);</code>. </p> + </li> + + <li> <p> For each configuration in the OVN Northbound database, that asks to change the source IP address of a packet from an IP address of @@ -3673,14 +3702,18 @@ nd_ns { options, then the action would be <code>ip4/6.src= (<var>B</var>)</code>. </p> + </li> + <li> <p> If the NAT rule has <code>allowed_ext_ips</code> configured, then there is an additional match <code>ip4.dst == <var>allowed_ext_ips </var></code>. Similarly, for IPV6, match would be <code>ip6.dst == <var>allowed_ext_ips</var></code>. </p> + </li> + <li> <p> If the NAT rule has <code>exempted_ext_ips</code> set, then there is an additional flow configured at the priority + 1 of @@ -3689,7 +3722,9 @@ nd_ns { </code>. This flow is used to bypass the ct_snat action for a packet which is destinted to <code>exempted_ext_ips</code>. </p> + </li> + <li> <p> A priority-0 logical flow with match <code>1</code> has actions <code>next;</code>. diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index db6572a62b..ece158b71e 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -622,6 +622,7 @@ struct ovn_datapath { struct lport_addresses dnat_force_snat_addrs; struct lport_addresses lb_force_snat_addrs; + bool lb_force_snat_router_ip; struct ovn_port **localnet_ports; size_t n_localnet_ports; @@ -721,6 +722,17 @@ init_nat_entries(struct ovn_datapath *od) snat_ip_add(od, od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, NULL); } + } else { + const char *lb_force_snat = + smap_get(&od->nbr->options, "lb_force_snat_ip"); + if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") + && smap_get(&od->nbr->options, "chassis")) { + /* Set it to true only if its gateway router and + * options:lb_force_snat_ip=router_ip. */ + od->lb_force_snat_router_ip = true; + } else { + od->lb_force_snat_router_ip = false; + } } if (!od->nbr->n_nat) { @@ -8365,9 +8377,12 @@ get_force_snat_ip(struct ovn_datapath *od, const char *key_type, } if (!extract_ip_address(addresses, laddrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", - addresses, UUID_ARGS(&od->key)); + if (strcmp(addresses, "router_ip") || strcmp(key_type, "lb")) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", + addresses, UUID_ARGS(&od->key)); + } + return false; } @@ -8943,6 +8958,48 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, ds_destroy(&actions); } +static void +build_lrouter_force_snat_flows_op(struct ovn_port *op, + struct hmap *lflows, + struct ds *match, struct ds *actions) +{ + if (!op->nbrp || !op->peer || !op->od->lb_force_snat_router_ip) { + return; + } + + if (op->lrp_networks.n_ipv4_addrs) { + ds_clear(match); + ds_clear(actions); + + /* Higher priority rules to force SNAT with the router port ip. + * This only takes effect when the packet has already been + * load balanced once. */ + ds_put_format(match, "flags.force_snat_for_lb == 1 && ip4 && " + "outport == %s", op->json_key); + ds_put_format(actions, "ct_snat(%s);", + op->lrp_networks.ipv4_addrs[0].addr_s); + ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, + ds_cstr(match), ds_cstr(actions)); + } + + /* op->lrp_networks.ipv6_addrs will always have LLA and that will be + * last in the list. So add the flows only if n_ipv6_addrs > 1. */ + if (op->lrp_networks.n_ipv6_addrs > 1) { + ds_clear(match); + ds_clear(actions); + + /* Higher priority rules to force SNAT with the router port ip. + * This only takes effect when the packet has already been + * load balanced once. */ + ds_put_format(match, "flags.force_snat_for_lb == 1 && ip6 && " + "outport == %s", op->json_key); + ds_put_format(actions, "ct_snat(%s);", + op->lrp_networks.ipv6_addrs[0].addr_s); + ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, + ds_cstr(match), ds_cstr(actions)); + } +} + static void build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op) { @@ -11278,6 +11335,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, "dnat"); } } + if (lb_force_snat_ip) { if (od->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", @@ -11490,6 +11548,8 @@ build_lswitch_and_lrouter_iterate_by_op(struct ovn_port *op, &lsi->match, &lsi->actions); build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions); + build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, + &lsi->actions); } static void diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 7240e22baf..fd03b1fb66 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2443,3 +2443,82 @@ check ovn-sbctl set chassis hv1 other_config:port-up-notif=true wait_row_count nb:Logical_Switch_Port 1 up=false name=lsp1 AT_CLEANUP + +AT_SETUP([ovn -- lb_force_snat_ip for Gateway Routers]) +ovn_start + +check ovn-nbctl ls-add sw0 +check ovn-nbctl ls-add sw1 + +# Create a logical router and attach both logical switches +check ovn-nbctl lr-add lr0 +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +check ovn-nbctl lsp-add sw0 sw0-lr0 +check ovn-nbctl lsp-set-type sw0-lr0 router +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 + +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 +check ovn-nbctl lsp-add sw1 sw1-lr0 +check ovn-nbctl lsp-set-type sw1-lr0 router +check ovn-nbctl lsp-set-addresses sw1-lr0 00:00:00:00:ff:02 +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1 + +check ovn-nbctl ls-add public +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +check ovn-nbctl lsp-add public public-lr0 +check ovn-nbctl lsp-set-type public-lr0 router +check ovn-nbctl lsp-set-addresses public-lr0 router +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public + +check ovn-nbctl set logical_router lr0 options:chassis=ch1 + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_out_snat" lr0flows | grep force_snat_for_lb | sort], [0], [dnl +]) + +check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="20.0.0.4 aef0::4" + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_out_snat" lr0flows | grep force_snat_for_lb | sort], [0], [dnl + table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip4), action=(ct_snat(20.0.0.4);) + table=1 (lr_out_snat ), priority=100 , match=(flags.force_snat_for_lb == 1 && ip6), action=(ct_snat(aef0::4);) +]) + +check ovn-nbctl --wait=sb set logical_router lr0 options:lb_force_snat_ip="router_ip" + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_out_snat" lr0flows | grep force_snat_for_lb | sort], [0], [dnl + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) +]) + +check ovn-nbctl --wait=sb remove logical_router lr0 options chassis + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_out_snat" lr0flows | grep force_snat_for_lb | sort], [0], [dnl +]) + +check ovn-nbctl set logical_router lr0 options:chassis=ch1 +check ovn-nbctl --wait=sb add logical_router_port lr0-sw1 networks "bef0\:\:1/64" + +ovn-sbctl dump-flows lr0 > lr0flows +AT_CAPTURE_FILE([lr0flows]) + +AT_CHECK([grep "lr_out_snat" lr0flows | grep force_snat_for_lb | sort], [0], [dnl + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-public"), action=(ct_snat(172.168.0.100);) + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw0"), action=(ct_snat(10.0.0.1);) + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip4 && outport == "lr0-sw1"), action=(ct_snat(20.0.0.1);) + table=1 (lr_out_snat ), priority=110 , match=(flags.force_snat_for_lb == 1 && ip6 && outport == "lr0-sw1"), action=(ct_snat(bef0::1);) +]) + +AT_CLEANUP -- 2.29.2 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev