On 8/26/25 9:43 PM, Lucas Vargas Dias via dev wrote:
> FIN or FIN-ACK packets from client was dropping because the client
> side was using conntrack. Connection is in SYN_SENT state because the
> response from backends bypass the conntrack, when client sends a FIN
> or FIN-ACK, the conntrack is invalid and packet is dropped.
> To fix it, remove the client side from conntrack, calculating the hash
> from packet to choice the backend.
> REG_CT_TP_DST is reused to store the index from backend
> because some flows the depends from conntracks use this register.
>
> Fixes: 264c8310c399 ("northd: Fix logical router load-balancer nat rules when
> using DGP.")
> Signed-off-by: Lucas Vargas Dias <[email protected]>
> ---
Hi Lucas,
Thanks for the patch and sorry for the delay in reviewing it.
> northd/northd.c | 131 +++++++++++++++++++-------
> tests/multinode.at | 99 ++++++++++++--------
> tests/ovn-northd.at | 218 +++++++++++++++++++++++++++++++++++++++++---
> 3 files changed, 367 insertions(+), 81 deletions(-)
>
> diff --git a/northd/northd.c b/northd/northd.c
> index 2cb69f9aa..f7c8053dd 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -11955,37 +11955,13 @@ build_distr_lrouter_nat_flows_for_lb(struct
> lrouter_nat_lb_flows_ctx *ctx,
> bool stateless_nat)
> {
> struct ds dnat_action = DS_EMPTY_INITIALIZER;
> + struct ds new_action_stateless_nat = DS_EMPTY_INITIALIZER;
> + struct ds new_match_stateless_nat = DS_EMPTY_INITIALIZER;
>
> /* Store the match lengths, so we can reuse the ds buffer. */
> size_t new_match_len = ctx->new_match->length;
> size_t undnat_match_len = ctx->undnat_match->length;
>
> - /* (NOTE) dnat_action: Add the first LB backend IP as a destination
> - * action of the lr_in_dnat NAT rule. Including the backend IP is useful
> - * for accepting packets coming from a chassis that does not have
> - * previously established conntrack entries. This means that the actions
> - * (ip4.dst + ct_lb_mark) are executed in addition and ip4.dst is not
> - * useful when traffic passes through the same chassis for ingress/egress
> - * packets. However, the actions are complementary in cases where traffic
> - * enters from one chassis, the ack response comes from another chassis,
> - * and the final ack step of the TCP handshake comes from the first
> - * chassis used. Without using stateless NAT, the connection will not be
> - * established because the return packet followed a path through another
> - * chassis and only ct_lb_mark will not be able to receive the ack and
> - * forward it to the right backend. With using stateless NAT, the packet
> - * will be accepted and forwarded to the same backend that corresponds to
> - * the previous conntrack entry that is in the SYN_SENT state
> - * (created by ct_lb_mark for the first rcv packet in this flow).
> - */
> - if (stateless_nat) {
> - if (!vector_is_empty(&ctx->lb_vip->backends)) {
> - const struct ovn_lb_backend *backend =
> - vector_get_ptr(&ctx->lb_vip->backends, 0);
> - bool ipv6 = !IN6_IS_ADDR_V4MAPPED(&backend->ip);
> - ds_put_format(&dnat_action, "%s.dst = %s; ", ipv6 ? "ip6" :
> "ip4",
> - backend->ip_str);
> - }
> - }
> ds_put_format(&dnat_action, "%s", ctx->new_action[type]);
>
> const char *meter = NULL;
> @@ -11996,18 +11972,77 @@ build_distr_lrouter_nat_flows_for_lb(struct
> lrouter_nat_lb_flows_ctx *ctx,
>
> if (!vector_is_empty(&ctx->lb_vip->backends) ||
> !ctx->lb_vip->empty_backend_rej) {
> + ds_put_format(&new_match_stateless_nat, "is_chassis_resident(%s)",
> + dgp->cr_port->json_key);
> ds_put_format(ctx->new_match, " && is_chassis_resident(%s)",
> dgp->cr_port->json_key);
> }
>
> - ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio,
> - ds_cstr(ctx->new_match), ds_cstr(&dnat_action),
> - NULL, meter, &ctx->lb->nlb->header_,
> - lflow_ref);
> + if (stateless_nat) {
This is becoming quite hard to follow. We need at least a comment
explaining the differences between the "stateless" case and the regular
case. I think we should also create separate functions for the two
types of LBs.
> + bool ipv4 = ctx->lb_vip->address_family == AF_INET;
> + const char *ip_match = ipv4 ? "ip4" : "ip6";
> + ds_put_format(&new_match_stateless_nat, " && %s && %s.dst == %s",
> + ip_match, ip_match, ctx->lb_vip->vip_str);
> + if (ctx->lb_vip->port_str) {
> + ds_put_format(&new_match_stateless_nat,
> + " && %s && %s.dst == %s",
> + ctx->lb->proto, ctx->lb->proto,
> + ctx->lb_vip->port_str);
> + }
>
> + const struct ovn_lb_backend *backend;
> + if (vector_len(&ctx->lb_vip->backends) == 1) {
> + backend = vector_get_ptr(&ctx->lb_vip->backends, 0);
> + ds_put_format(&new_action_stateless_nat, "%s.dst = %s; ",
> + ip_match, backend->ip_str);
Nit: one space too many.
> + if (ctx->lb_vip->port_str) {
> + ds_put_format(&new_action_stateless_nat, "%s.dst = %s; ",
> + ctx->lb->proto, backend->port_str);
Nit: one space too many.
> + }
> + ds_put_format(&new_action_stateless_nat, "next;");
> + ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT,
> + ctx->prio,
> + ds_cstr(&new_match_stateless_nat),
> + ds_cstr(&new_action_stateless_nat),
> + NULL, meter, &ctx->lb->nlb->header_,
> + lflow_ref);
> + }
> + size_t match_len = new_match_stateless_nat.length;
> + size_t i = 0;
> + VECTOR_FOR_EACH_PTR (&ctx->lb_vip->backends, backend) {
> + if (vector_len(&ctx->lb_vip->backends) <= 1) {
> + break;
> + }
> + ds_put_format(&new_match_stateless_nat, " && "REG_CT_TP_DST" == "
> + "%"PRIuSIZE, i++);
This is becoming a bit confusing. REG_CT_TP_DST stands for "register
for conntrack transport protocol destination". But now it's used to
store an index. Can we add a new definition for this case? That is,
another alias for "reg1[0..15]"?
Also nit: one space too many.
> + ds_put_format(&new_action_stateless_nat, "%s.dst = %s; ",
> + ip_match, backend->ip_str);
Nit: one space too many.
> + if (ctx->lb_vip->port_str) {
> + ds_put_format(&new_action_stateless_nat, "%s.dst = %s; ",
> + ctx->lb->proto, backend->port_str);
Nit: one space too many.
> + }
> + ds_put_format(&new_action_stateless_nat, "next;");
> + ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT,
> + ctx->prio,
> + ds_cstr(&new_match_stateless_nat),
> + ds_cstr(&new_action_stateless_nat),
> + NULL, meter, &ctx->lb->nlb->header_,
> + lflow_ref);
> + ds_clear(&new_action_stateless_nat);
> + ds_truncate(&new_match_stateless_nat, match_len);
> + }
> + } else {
> + ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT,
> + ctx->prio, ds_cstr(ctx->new_match),
> + ds_cstr(&dnat_action), NULL, meter,
> + &ctx->lb->nlb->header_, lflow_ref);
> + }
> ds_truncate(ctx->new_match, new_match_len);
>
> ds_destroy(&dnat_action);
> + ds_destroy(&new_match_stateless_nat);
> + ds_destroy(&new_action_stateless_nat);
> +
> if (vector_is_empty(&ctx->lb_vip->backends)) {
> return;
> }
> @@ -12401,21 +12436,51 @@ build_lrouter_defrag_flows_for_lb(struct
> ovn_lb_datapaths *lb_dps,
> if (!lb_dps->n_nb_lr) {
> return;
> }
> + struct ds action = DS_EMPTY_INITIALIZER;
Instead of adding a local 'action' variable you could just pass the
existing one from the caller of build_lrouter_defrag_flows_for_lb() like
we do for 'match'.
For some context: that's a (micro) optimization that happened at some
point in northd to avoid reallocating strings when possible.
> + struct ds values = DS_EMPTY_INITIALIZER;
>
We don't really need this temporary string, do we? Can't we just
populate 'action' directly?
> + bool use_stateless_nat = smap_get_bool(&lb_dps->lb->nlb->options,
> + "use_stateless_nat", false);
This should actually happen in ovn_northd_lb_init() and should be stored
as a field in 'struct ovn_northd_lb'. There's another instance of this
in build_lrouter_nat_flows_for_lb(). That should be fixed up too.
> for (size_t i = 0; i < lb_dps->lb->n_vips; i++) {
> struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i];
> bool ipv6 = lb_vip->address_family == AF_INET6;
> int prio = 100;
> -
> + enum ovn_stage stage = S_ROUTER_IN_DEFRAG;
> ds_clear(match);
> ds_put_format(match, "ip && ip%c.dst == %s", ipv6 ? '6' : '4',
> lb_vip->vip_str);
> -
> + if (use_stateless_nat) {
We need at least a comment here explaining the difference between the
stateless nat and "stateful" nat case. Maybe even separate functions to
populate the action string?
> + stage = S_ROUTER_IN_CT_EXTRACT;
> + prio = 120;
> + if (vector_len(&lb_vip->backends) > 1) {
> + ds_put_format(&action, REG_CT_TP_DST" = select(");
Don't we need to set REG_CT_PROTO too?
> + for (size_t idx_backend = 0; idx_backend <
> + vector_len(&lb_vip->backends); idx_backend++){
The correct way to break the line here is (IMO):
for (size_t idx_backend = 0;
idx_backend < vector_len(&lb_vip->backends);
idx_backend++) {
ds_put_format(&values, "%"PRIuSIZE",", idx_backend);
}
> + ds_put_format(&values, "%"PRIuSIZE",", idx_backend);
> + }
> + ds_truncate(&values, values.length - 1);
Nit: across the code base in most similar cases we'd do something like:
ds_chomp(&values, ',');
> + if (lb_dps->lb->selection_fields) {
> + ds_put_format(&action, "values=(%s); hash_fields=\"%s\"",
> + ds_cstr(&values),
> + lb_dps->lb->selection_fields);
> + } else {
> + ds_put_format(&action, "%s", ds_cstr(&values));
Just a note: if selection_fields is not set, the select() might return a
random backend if !ovs_feature_is_supported(OVS_DP_HASH_L4_SYM_SUPPORT)
(e.g., when TCP retransmits happen).
> + }
> + ds_put_format(&action,"); ");
> + }
> + ds_put_format(&action, "next;");
> + } else {
> + ds_put_format(&action, "ct_dnat;");
> + }
> ovn_lflow_add_with_dp_group(
> lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths),
> - S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;",
> + stage, prio, ds_cstr(match), ds_cstr(&action),
> &lb_dps->lb->nlb->header_, lb_dps->lflow_ref);
> + ds_clear(&action);
> + ds_clear(&values);
> }
> + ds_destroy(&action);
> + ds_destroy(&values);
> }
>
> static void
> diff --git a/tests/multinode.at b/tests/multinode.at
Thanks for adding and updating the multinode e2e tests for the feature!
> index cf748af49..25e3193f7 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1643,6 +1643,7 @@ check multinode_nbctl ls-lb-add sw0 lb0
>
> # Set use_stateless_nat to true
> check multinode_nbctl set load_balancer lb0 options:use_stateless_nat=true
> +check multinode_nbctl set load_balancer lb0
> selection_fields="ip_src,tp_src,ip_dst,tp_dst"
>
> # Start backend http services
> M_NS_DAEMONIZE([ovn-chassis-1], [sw0p1], [python3 -m http.server --bind
> 10.0.1.3 80 >/dev/null 2>&1], [http1.pid])
> @@ -1674,14 +1675,12 @@ m_as ovn-gw-2 ovs-appctl dpctl/flush-conntrack
> M_NS_EXEC([ovn-chassis-3], [publicp1], [sh -c 'curl -v 172.16.0.100:80
> --retry 0 --connect-timeout 1 --max-time 1 --local-port 59001'])
> OVS_WAIT_FOR_OUTPUT([m_as ovn-gw-1 ovs-appctl dpctl/dump-conntrack |
> M_FORMAT_CT(20.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59001,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59001),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59001,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59001),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> M_NS_EXEC([ovn-chassis-4], [publicp2], [sh -c 'curl -v 172.16.0.100:80
> --retry 0 --connect-timeout 1 --max-time 1 --local-port 59000'])
> OVS_WAIT_FOR_OUTPUT([m_as ovn-gw-2 ovs-appctl dpctl/dump-conntrack |
> M_FORMAT_CT(30.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59000,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59000),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59000,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59000),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> @@ -1712,7 +1711,6 @@ Connected to 172.16.0.100 (172.16.0.100) port 80
> # Check if we have only one backend for the same connection - orig + dest
> ports
> OVS_WAIT_FOR_OUTPUT([echo -e $gw1_ct | M_FORMAT_CT(20.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59004,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59004),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59004,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59004),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> @@ -1778,7 +1776,6 @@ Connected to 172.16.0.100 (172.16.0.100) port 80
> # Check if we have only one backend for the same connection - orig + dest
> ports
> OVS_WAIT_FOR_OUTPUT([echo -e $gw1_ct | M_FORMAT_CT(20.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59005,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59005),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59005,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59005),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> @@ -1834,7 +1831,6 @@ Connected to 172.16.0.100 (172.16.0.100) port 80
> # Check if we have only one backend for the same connection - orig + dest
> ports
> OVS_WAIT_FOR_OUTPUT([echo -e $gw2_ct | M_FORMAT_CT(30.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59006,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59006),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59006,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59006),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> @@ -1900,7 +1896,6 @@ Connected to 172.16.0.100 (172.16.0.100) port 80
> # Check if we have only one backend for the same connection - orig + dest
> ports
> OVS_WAIT_FOR_OUTPUT([echo -e $gw2_ct | M_FORMAT_CT(30.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
> -tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59007,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59007),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
>
> tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59007,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59007),zone=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> @@ -1987,7 +1982,6 @@ M_NS_EXEC([ovn-chassis-3], [publicp1], [sh -c 'curl -v
> -O 172.16.0.100:9000/down
> OVS_WAIT_FOR_OUTPUT([m_as ovn-gw-1 ovs-appctl dpctl/dump-conntrack |
> M_FORMAT_CT(20.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
>
> tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59008,dport=80),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59008),zone=<cleared>,protoinfo=(state=<cleared>)
> -tcp,orig=(src=20.0.1.3,dst=<cleared>,sport=59008,dport=9000),reply=(src=<cleared>,dst=20.0.1.3,sport=80,dport=59008),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [9000])], [0], [dnl
> @@ -2003,7 +1997,6 @@ M_NS_EXEC([ovn-chassis-4], [publicp2], [sh -c 'curl -v
> -O 172.16.0.100:9000/down
> OVS_WAIT_FOR_OUTPUT([m_as ovn-gw-2 ovs-appctl dpctl/dump-conntrack |
> M_FORMAT_CT(30.0.1.3) | \
> grep tcp | sed -E -e 's/10.0.1.3|10.0.1.4/<cleared>/g' | sort], [0], [dnl
>
> tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59008,dport=80),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59008),zone=<cleared>,protoinfo=(state=<cleared>)
> -tcp,orig=(src=30.0.1.3,dst=<cleared>,sport=59008,dport=9000),reply=(src=<cleared>,dst=30.0.1.3,sport=80,dport=59008),zone=<cleared>,mark=<cleared>,protoinfo=(state=<cleared>)
> ])
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [9000])], [0], [dnl
> @@ -2036,14 +2029,14 @@ done
> # |
> #
> +.............................|.............................+
> # |
> |
> -# DGP publicp3 (ovn-gw-3) (20.0.2.3/24) DGP
> publicp4 (ovn-gw-4) (20.0.2.4/24)
> +# DGP publicp3 (ovn-gw-3) (20.0.1.2/24) DGP
> publicp4 (ovn-gw-4) (20.0.2.4/24)
> # |
> |
> #
> +.............................+.............................+
> -# |
> -# | (overlay)
> +# |
> |
> +# | (public_right)
> (public_left)|
> #
> +.............................+.............................+
> # |
> |
> -# DGP public1 (ovn-gw-1) (20.0.2.1/24) DGP public2
> (ovn-gw-2) (20.0.2.2/24)
> +# DGP public1 (ovn-gw-1) (20.0.1.1/24) DGP public2
> (ovn-gw-2) (20.0.2.2/24)
> # |
> |
> #
> +.............................+.............................+
> # |
> @@ -2100,17 +2093,18 @@ check multinode_nbctl lsp-set-addresses sw0-lr0 router
> check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
>
> # create external connection for N/S traffic using multiple DGPs
> -check multinode_nbctl ls-add public
> +check multinode_nbctl ls-add public_right
> +check multinode_nbctl ls-add public_left
>
> # create external connection for N/S traffic
> # DGP public1
> -check multinode_nbctl lsp-add public ln-public-1
> +check multinode_nbctl lsp-add public_right ln-public-1
> check multinode_nbctl lsp-set-type ln-public-1 localnet
> check multinode_nbctl lsp-set-addresses ln-public-1 unknown
> check multinode_nbctl lsp-set-options ln-public-1 network_name=public1
>
> # DGP public2
> -check multinode_nbctl lsp-add public ln-public-2
> +check multinode_nbctl lsp-add public_left ln-public-2
> check multinode_nbctl lsp-set-type ln-public-2 localnet
> check multinode_nbctl lsp-set-addresses ln-public-2 unknown
> check multinode_nbctl lsp-set-options ln-public-2 network_name=public2
> @@ -2121,8 +2115,8 @@ m_as ovn-gw-1 ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=public1:br-e
> # Attach DGP public2 to GW-2 public2 (overlay connectivity)
> m_as ovn-gw-2 ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=public2:br-ex
>
> -check multinode_nbctl lrp-add lr0 lr0-public-p1 40:54:00:00:00:01
> 20.0.2.1/24 2000::1/64
> -check multinode_nbctl lsp-add public public-lr0-p1
> +check multinode_nbctl lrp-add lr0 lr0-public-p1 40:54:00:00:00:01
> 20.0.1.1/24 2000::1/64
> +check multinode_nbctl lsp-add public_right public-lr0-p1
> check multinode_nbctl lsp-set-type public-lr0-p1 router
> check multinode_nbctl lsp-set-addresses public-lr0-p1 router
> check multinode_nbctl lsp-set-options public-lr0-p1 router-port=lr0-public-p1
> @@ -2130,13 +2124,13 @@ check multinode_nbctl lrp-set-gateway-chassis
> lr0-public-p1 ovn-gw-1 10
>
> m_wait_for_ports_up
>
> -M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 20.0.2.1
> | FORMAT_PING], \
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 20.0.1.1
> | FORMAT_PING], \
> [0], [dnl
> 3 packets transmitted, 3 received, 0% packet loss, time 0ms
> ])
>
> check multinode_nbctl lrp-add lr0 lr0-public-p2 40:54:00:00:00:02
> 20.0.2.2/24 2000::2/64
> -check multinode_nbctl lsp-add public public-lr0-p2
> +check multinode_nbctl lsp-add public_left public-lr0-p2
> check multinode_nbctl lsp-set-type public-lr0-p2 router
> check multinode_nbctl lsp-set-addresses public-lr0-p2 router
> check multinode_nbctl lsp-set-options public-lr0-p2 router-port=lr0-public-p2
> @@ -2157,7 +2151,7 @@ check multinode_nbctl lsp-set-options sw1-lr1
> router-port=lr1-sw1
>
> # create external connection for N/S traffic
> # DGP public3
> -check multinode_nbctl lsp-add public ln-public-3
> +check multinode_nbctl lsp-add public_right ln-public-3
> check multinode_nbctl lsp-set-type ln-public-3 localnet
> check multinode_nbctl lsp-set-addresses ln-public-3 unknown
> check multinode_nbctl lsp-set-options ln-public-3 network_name=public3
> @@ -2165,8 +2159,8 @@ check multinode_nbctl lsp-set-options ln-public-3
> network_name=public3
> # Attach DGP public3 to GW-3 public3 (overlay connectivity)
> m_as ovn-gw-3 ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=public3:br-ex
>
> -check multinode_nbctl lrp-add lr1 lr1-public-p3 40:54:00:00:00:03
> 20.0.2.3/24 2000::3/64
> -check multinode_nbctl lsp-add public public-lr1-p3
> +check multinode_nbctl lrp-add lr1 lr1-public-p3 40:54:00:00:00:03
> 20.0.1.2/24 2000::3/64
> +check multinode_nbctl lsp-add public_right public-lr1-p3
> check multinode_nbctl lsp-set-type public-lr1-p3 router
> check multinode_nbctl lsp-set-addresses public-lr1-p3 router
> check multinode_nbctl lsp-set-options public-lr1-p3 router-port=lr1-public-p3
> @@ -2177,21 +2171,16 @@ M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c
> 3 -i 0.3 -w 2 40.0.2.1 | F
> 3 packets transmitted, 3 received, 0% packet loss, time 0ms
> ])
>
> -M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 20.0.2.3
> | FORMAT_PING], \
> +M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 20.0.1.2
> | FORMAT_PING], \
> [0], [dnl
> 3 packets transmitted, 3 received, 0% packet loss, time 0ms
> ])
>
> # Add a default route for multiple DGPs using ECMP - first step
> -check multinode_nbctl --ecmp lr-route-add lr0 0.0.0.0/0 20.0.2.3
> -check multinode_nbctl --ecmp lr-route-add lr1 0.0.0.0/0 20.0.2.1
> -
> -# Add SNAT rules using gateway-port
> -check multinode_nbctl --gateway-port lr0-public-p1 lr-nat-add lr0 snat
> 20.0.2.1 10.0.2.0/24
> -check multinode_nbctl --gateway-port lr0-public-p2 lr-nat-add lr0 snat
> 20.0.2.2 10.0.2.0/24
> -check multinode_nbctl --gateway-port lr1-public-p3 lr-nat-add lr1 snat
> 20.0.2.3 40.0.2.0/24
> +check multinode_nbctl --ecmp lr-route-add lr0 0.0.0.0/0 20.0.1.2
> +check multinode_nbctl --ecmp lr-route-add lr1 0.0.0.0/0 20.0.1.1
>
> -M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 20.0.2.1
> | FORMAT_PING], \
> +M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 20.0.1.1
> | FORMAT_PING], \
> [0], [dnl
> 3 packets transmitted, 3 received, 0% packet loss, time 0ms
> ])
> @@ -2203,7 +2192,7 @@ M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3
> -i 0.3 -w 2 20.0.2.2 | F
>
> # Configure the second DGP for the lr1
> # DGP public4
> -check multinode_nbctl lsp-add public ln-public-4
> +check multinode_nbctl lsp-add public_left ln-public-4
> check multinode_nbctl lsp-set-type ln-public-4 localnet
> check multinode_nbctl lsp-set-addresses ln-public-4 unknown
> check multinode_nbctl lsp-set-options ln-public-4 network_name=public4
> @@ -2212,7 +2201,7 @@ check multinode_nbctl lsp-set-options ln-public-4
> network_name=public4
> m_as ovn-gw-4 ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=public4:br-ex
>
> check multinode_nbctl lrp-add lr1 lr1-public-p4 40:54:00:00:00:04
> 20.0.2.4/24 2000::4/64
> -check multinode_nbctl lsp-add public public-lr1-p4
> +check multinode_nbctl lsp-add public_left public-lr1-p4
> check multinode_nbctl lsp-set-type public-lr1-p4 router
> check multinode_nbctl lsp-set-addresses public-lr1-p4 router
> check multinode_nbctl lsp-set-options public-lr1-p4 router-port=lr1-public-p4
> @@ -2223,9 +2212,6 @@ M_NS_CHECK_EXEC([ovn-chassis-3], [sw1p1], [ping -q -c 3
> -i 0.3 -w 2 20.0.2.4 | F
> 3 packets transmitted, 3 received, 0% packet loss, time 0ms
> ])
>
> -# Add SNAT rules using gateway-port
> -check multinode_nbctl --gateway-port lr1-public-p4 lr-nat-add lr1 snat
> 20.0.2.4 40.0.2.0/24
> -
> # Add a default route for multiple DGPs using ECMP - second step (multipath)
> check multinode_nbctl --ecmp lr-route-add lr0 0.0.0.0/0 20.0.2.4
> check multinode_nbctl --ecmp lr-route-add lr1 0.0.0.0/0 20.0.2.2
> @@ -2359,36 +2345,77 @@ fi
> # Set use_stateless_nat to true
> # Now, if the traffic passes through both gateways (GW-1 and GW-2) it will
> be forwarded successfully
> check multinode_nbctl set load_balancer lb0 options:use_stateless_nat=true
> +check multinode_nbctl --wait=sb set load_balancer lb0
> selection_fields="ip_src,tp_src,ip_dst,tp_dst"
>
> # Check the flows again for the LB VIP - always needs to be successful
> regardless of the datapath (one or two gw chassis)
> M_NS_EXEC([ovn-chassis-3], [sw1p1], [sh -c 'curl -v -O
> 172.16.0.100:80/download_file --retry 0 --connect-timeout 1 --max-time 1
> 2>curl.out'])
> +M_NS_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'ss -nn >connections.out'])
> +M_NS_EXEC([ovn-chassis-2], [sw0p2], [sh -c 'ss -nn >connections.out'])
This overrides the connections.out file from the previous command. Did
you mean to use >> instead of > ?
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [80])], [0], [dnl
> Connected to 172.16.0.100 (172.16.0.100) port 80
> 200 OK
> ])
>
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-1 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-2 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> M_NS_EXEC([ovn-chassis-3], [sw1p1], [sh -c 'curl -v -O
> 172.16.0.100:80/download_file --retry 0 --connect-timeout 1 --max-time 1
> 2>curl.out'])
> +M_NS_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'ss -nn >connections.out'])
> +M_NS_EXEC([ovn-chassis-2], [sw0p2], [sh -c 'ss -nn >connections.out'])
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [80])], [0], [dnl
> Connected to 172.16.0.100 (172.16.0.100) port 80
> 200 OK
> ])
>
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-1 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-2 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> M_NS_EXEC([ovn-chassis-3], [sw1p1], [sh -c 'curl -v -O
> 172.16.0.100:80/download_file --retry 0 --connect-timeout 1 --max-time 1
> 2>curl.out'])
This now failed in CI in my fork:
https://github.com/dceara/ovn/actions/runs/18527590496/job/52802379445#step:19:5473
multinode.at:2355: waiting for output from m_as ovn-chassis-3 cat -v
curl.out | sed 's/\(.*\)Connected to 172.16.0.100 (172.16.0.100) port
80/Connected to 172.16.0.100 (172.16.0.100) port 80\n/' | sed
's/\(.*\)200 OK/200 OK\n/' | grep -i -e connected -e 200...
multinode.at:2355: wait failed after 30 seconds
--- wait-expected-stdout 2025-10-15 12:03:20.195100146 +0000
+++ wait-stdout 2025-10-15 12:03:53.652853417 +0000
@@ -1,2 +1 @@
Connected to 172.16.0.100 (172.16.0.100) port 80
-200 OK
> +M_NS_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'ss -nn >connections.out'])
> +M_NS_EXEC([ovn-chassis-2], [sw0p2], [sh -c 'ss -nn >connections.out'])
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [80])], [0], [dnl
> Connected to 172.16.0.100 (172.16.0.100) port 80
> 200 OK
> ])
>
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-1 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-2 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> M_NS_EXEC([ovn-chassis-3], [sw1p1], [sh -c 'curl -v -O
> 172.16.0.100:80/download_file --retry 0 --connect-timeout 1 --max-time 1
> 2>curl.out'])
> +M_NS_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'ss -nn >connections.out'])
> +M_NS_EXEC([ovn-chassis-2], [sw0p2], [sh -c 'ss -nn >connections.out'])
>
> OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-3 cat -v curl.out |
> M_FORMAT_CURL([172.16.0.100], [80])], [0], [dnl
> Connected to 172.16.0.100 (172.16.0.100) port 80
> 200 OK
> ])
>
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-1 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> +OVS_WAIT_FOR_OUTPUT([m_as ovn-chassis-2 cat connections.out | grep
> "FIN-WAIT-2" | wc -l], [0], [dnl
> +0
> +])
> +
> # Direct backend traffic using the same LB ports needs to be dropped
> M_NS_EXEC([ovn-chassis-3], [sw1p1], [sh -c 'curl -v -O
> 10.0.2.3:80/download_file --retry 0 --connect-timeout 1 --max-time 1
> 2>curl.out'])
>
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 05acd6f4d..cebf9d6ba 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -14331,14 +14331,66 @@ ovn-sbctl dump-flows lr1 > lr1flows
> AT_CAPTURE_FILE([lr1flows])
>
> # Check stateless NAT rules for load balancer with multiple DGP
> -# 1. Check if the backend IPs are in the ipX.dst action
> +# 1. Check if the reg1[0..15] will select one of backends
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "30.0.0.1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip4.dst ==
> 30.0.0.1), action=(reg1[[0..15]] = select(0,1,2); next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep "30.0.0.1"],
> [0], [dnl
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1-ts1")),
> action=(ip4.dst = 172.16.0.103;
> ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);)
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1-ts2")),
> action=(ip4.dst = 172.16.0.103;
> ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);)
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip4 && ip4.dst == 30.0.0.1 && is_chassis_resident("cr-lr1_public")),
> action=(ip4.dst = 172.16.0.103;
> ct_lb_mark(backends=172.16.0.103,172.16.0.102,172.16.0.101);)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> +AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> ])
>
> -# 2. Check if the DGP ports are in the match with action next
> +# 4. Check if the VIP IP is in the ipX.src action
> +AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(ip4.src = 30.0.0.1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(ip4.src = 30.0.0.1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(ip4.src = 30.0.0.1;
> next;)
> +])
> +
> +# Set selection fields
> +check ovn-nbctl --wait=sb set load_balancer lb1
> selection_fields="ip_src,tp_src,ip_dst,tp_dst"
> +
> +ovn-sbctl dump-flows lr1 > lr1flows
> +AT_CAPTURE_FILE([lr1flows])
> +
> +# 1. Check if the reg1[0..15] will select one of backends using hash_fields
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "30.0.0.1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip4.dst ==
> 30.0.0.1), action=(reg1[[0..15]] = select(values=(0,1,2);
> hash_fields="ip_dst,ip_src,tcp_dst,tcp_src"); next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> +AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep "30.0.0.1"],
> [0], [dnl
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 0), action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 1), action=(ip4.dst = 172.16.0.102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1 &&
> reg1[[0..15]] == 2), action=(ip4.dst = 172.16.0.101; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> @@ -14346,7 +14398,7 @@ AT_CHECK([grep "lr_out_undnat" lr1flows |
> ovn_strip_lflows], [0], [dnl
> table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> ])
>
> -# 3. Check if the VIP IP is in the ipX.src action
> +# 4. Check if the VIP IP is in the ipX.src action
> AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> @@ -14355,6 +14407,51 @@ AT_CHECK([grep "lr_out_snat" lr1flows |
> ovn_strip_lflows], [0], [dnl
> table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103) || (ip4.src == 172.16.0.102) || (ip4.src == 172.16.0.101)) &&
> (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(ip4.src = 30.0.0.1;
> next;)
> ])
>
> +# Delete LB and create with one backend
> +check ovn-nbctl --wait=sb lb-del lb1
> +check ovn-nbctl --wait=sb lb-add lb1 "30.0.0.1" "172.16.0.103"
> +
> +# Set use_stateless_nat to true
> +check ovn-nbctl --wait=sb set load_balancer lb1
> options:use_stateless_nat=true
> +
> +# Associate load balancer to s1
> +check ovn-nbctl ls-lb-add s1 lb1
> +check ovn-nbctl lr-lb-add lr1 lb1
> +check ovn-nbctl --wait=sb sync
> +
> +ovn-sbctl dump-flows lr1 > lr1flows
> +AT_CAPTURE_FILE([lr1flows])
> +
> +# 1. Check if the reg1[0..15] will select one of backends using hash_fields
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "30.0.0.1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip4.dst ==
> 30.0.0.1), action=(next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> +AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep "30.0.0.1"],
> [0], [dnl
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip4 && ip4.dst == 30.0.0.1),
> action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip4 && ip4.dst == 30.0.0.1),
> action=(ip4.dst = 172.16.0.103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip4 && ip4.dst == 30.0.0.1),
> action=(ip4.dst = 172.16.0.103; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> +AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> +])
> +
> +# 4. Check if the VIP IP is in the ipX.src action
> +AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(ip4.src = 30.0.0.1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(ip4.src = 30.0.0.1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip4 && ((ip4.src ==
> 172.16.0.103)) && (inport == "lr1_public" || outport == "lr1_public") &&
> is_chassis_resident("cr-lr1_public") && tcp), action=(ip4.src = 30.0.0.1;
> next;)
> +])
> +
> +
> AT_CLEANUP
> ])
>
> @@ -14445,14 +14542,67 @@ ovn-sbctl dump-flows lr1 > lr1flows
> AT_CAPTURE_FILE([lr1flows])
>
> # Check stateless NAT rules for load balancer with multiple DGP
> -# 1. Check if the backend IPs are in the ipX.dst action
> +# 1. Check if the reg1[0..15] will select one of backends
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip6.dst ==
> 2001:db8:cccc::1), action=(reg1[[0..15]] = select(0,1,2); next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip6 && ip6.dst == 2001:db8:cccc::1 && is_chassis_resident("cr-lr1-ts1")),
> action=(ip6.dst = 2001:db8:aaaa:3::103;
> ct_lb_mark(backends=2001:db8:aaaa:3::103,2001:db8:aaaa:3::102,2001:db8:aaaa:3::101);)
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip6 && ip6.dst == 2001:db8:cccc::1 && is_chassis_resident("cr-lr1-ts2")),
> action=(ip6.dst = 2001:db8:aaaa:3::103;
> ct_lb_mark(backends=2001:db8:aaaa:3::103,2001:db8:aaaa:3::102,2001:db8:aaaa:3::101);)
> - table=??(lr_in_dnat ), priority=110 , match=(ct.new && !ct.rel &&
> ip6 && ip6.dst == 2001:db8:cccc::1 && is_chassis_resident("cr-lr1_public")),
> action=(ip6.dst = 2001:db8:aaaa:3::103;
> ct_lb_mark(backends=2001:db8:aaaa:3::103,2001:db8:aaaa:3::102,2001:db8:aaaa:3::101);)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> +AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> ])
>
> -# 2. Check if the DGP ports are in the match with action next
> +# 4. Check if the VIP IP is in the ipX.src action
> +AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> +])
> +
> +# Set selection fields
> +check ovn-nbctl --wait=sb set load_balancer lb1
> selection_fields="ip_src,tp_src,ip_dst,tp_dst"
> +
> +ovn-sbctl dump-flows lr1 > lr1flows
> +AT_CAPTURE_FILE([lr1flows])
> +
> +# Check stateless NAT rules for load balancer with multiple DGP
> +# 1. Check if the reg1[0..15] will select one of backends
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip6.dst ==
> 2001:db8:cccc::1), action=(reg1[[0..15]] = select(values=(0,1,2);
> hash_fields="ip_dst,ip_src,tcp_dst,tcp_src"); next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> +AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 0), action=(ip6.dst =
> 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 1), action=(ip6.dst =
> 2001:db8:aaaa:3::102; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1 && reg1[[0..15]] == 2), action=(ip6.dst =
> 2001:db8:aaaa:3::101; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> @@ -14460,7 +14610,7 @@ AT_CHECK([grep "lr_out_undnat" lr1flows |
> ovn_strip_lflows], [0], [dnl
> table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> ])
>
> -# 3. Check if the VIP IP is in the ipX.src action
> +# 4. Check if the VIP IP is in the ipX.src action
> AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> @@ -14469,6 +14619,50 @@ AT_CHECK([grep "lr_out_snat" lr1flows |
> ovn_strip_lflows], [0], [dnl
> table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103) || (ip6.src == 2001:db8:aaaa:3::102) || (ip6.src ==
> 2001:db8:aaaa:3::101)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> ])
>
> +# Delete LB and create with one backend
> +check ovn-nbctl --wait=sb lb-del lb1
> +check ovn-nbctl --wait=sb lb-add lb1 "2001:db8:cccc::1"
> "2001:db8:aaaa:3::103"
> +
> +# Set use_stateless_nat to true
> +check ovn-nbctl --wait=sb set load_balancer lb1
> options:use_stateless_nat=true
> +
> +# Associate load balancer to s1
> +check ovn-nbctl ls-lb-add s1 lb1
> +check ovn-nbctl lr-lb-add lr1 lb1
> +check ovn-nbctl --wait=sb sync
> +
> +ovn-sbctl dump-flows lr1 > lr1flows
> +AT_CAPTURE_FILE([lr1flows])
> +# Check stateless NAT rules for load balancer with multiple DGP
> +# 1. Check if the reg1[0..15] will select one of backends
> +AT_CHECK([grep "lr_in_ct_extract" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> + table=??(lr_in_ct_extract ), priority=120 , match=(ip && ip6.dst ==
> 2001:db8:cccc::1), action=(next;)
> +])
> +
> +# 2. Check if the backend IPs are in the ipX.dst action
> +AT_CHECK([grep "lr_in_dnat" lr1flows | ovn_strip_lflows | grep
> "2001:db8:cccc::1"], [0], [dnl
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts1") && ip6 && ip6.dst ==
> 2001:db8:cccc::1), action=(ip6.dst = 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1-ts2") && ip6 && ip6.dst ==
> 2001:db8:cccc::1), action=(ip6.dst = 2001:db8:aaaa:3::103; next;)
> + table=??(lr_in_dnat ), priority=110 ,
> match=(is_chassis_resident("cr-lr1_public") && ip6 && ip6.dst ==
> 2001:db8:cccc::1), action=(ip6.dst = 2001:db8:aaaa:3::103; next;)
> +])
> +
> +# 3. Check if the DGP ports are in the match with action next
> +AT_CHECK([grep "lr_out_undnat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_undnat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(next;)
> + table=??(lr_out_undnat ), priority=120 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(next;)
> +])
> +
> +# 4. Check if the VIP IP is in the ipX.src action
> +AT_CHECK([grep "lr_out_snat" lr1flows | ovn_strip_lflows], [0], [dnl
> + table=??(lr_out_snat ), priority=0 , match=(1), action=(next;)
> + table=??(lr_out_snat ), priority=120 , match=(nd_ns),
> action=(next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1-ts1" || outport == "lr1-ts1") &&
> is_chassis_resident("cr-lr1-ts1") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1-ts2" || outport == "lr1-ts2") &&
> is_chassis_resident("cr-lr1-ts2") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> + table=??(lr_out_snat ), priority=160 , match=(ip6 && ((ip6.src ==
> 2001:db8:aaaa:3::103)) && (inport == "lr1_public" || outport == "lr1_public")
> && is_chassis_resident("cr-lr1_public") && tcp), action=(ip6.src =
> 2001:db8:cccc::1; next;)
> +])
> +
> AT_CLEANUP
> ])
>
Regards,
Dumitru
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
