On Tue, Feb 11, 2025 at 01:21:17PM -0500, Numan Siddique wrote: > On Tue, Feb 11, 2025 at 5:15 AM Felix Huettner > <[email protected]> wrote: > > > > On Mon, Feb 10, 2025 at 10:37:25AM -0500, [email protected] wrote: > > > From: Numan Siddique <[email protected]> > > > > > > Consider the below logical topology > > > > > > sw0-p1 - > > > | > > > sw0-p2 - -> sw0 -> lr0 ---- > > > ... | | > > > sw0-pn - | > > > | > > > sw1-p1 - | > > > | | > > > sw1p-2 - -> sw1 -> lr1 ---- --- public (provider switch) > > > ... | | > > > sw1-pn- | > > > | > > > swn-p1 - | > > > | | > > > swn-p2- -> swn -> lrn ---- > > > ... | > > > swn-pn - > > > > > > All the routers are connected to the provider switch via > > > a ditributed gateway port. > > > > > > If sw0-p1 is resident on the chassis C1, then since there is a path > > > to all the switches and the routers, ovn-controller will add all > > > these datapaths to its 'local_datapaths' map. This in turn results > > > in processing all the logical flows and installing all the openflows > > > and in turn wasting the CPU time. This can be very costly in > > > a highly scaled deployment. > > > > > > Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath > > > binding for a provider switch (with only dgp peer ports). > > > > > > In this commit, ovn-controller makes use of this flag and stops > > > adding other datapaths connected to the public provider switch > > > to the 'local_datapaths'. > > > > > > For example, when it claims sw0-p1, it adds sw0, lr0 and public > > > to the local_datapaths and stops there. If it later claims > > > sw1-p1, it will add sw1 and lr1. > > > > > > This reduces the recompute time and the number of openflow rules > > > added to ovs-vswitchd significantly. > > > > > > I tested this patch with a deployment of below logical resources: > > > > > > No of logical switches - 778 > > > No of logical routers - 871 > > > No of logical flows - 85626 > > > No of 'ovn-sbctl dump-flows' - 208631 > > > > > > Without this patch, afte claiming sw0-p1, ovn-controller adds > > > 269098 openflow rules and it takes approx 2500 milli seconds > > > for a recompute. > > > > > > With this patch, after claiming sw0-p1, ovn-controller adds > > > 21350 openflow rules and it takes approx 280 milli seconds > > > for a recompute. > > > > > > There is approx 90% reduction in the openflow rules and > > > 88% reduction in recompute time when a comoute node has > > > VIFs from one logical switch. > > > > Hi Numan, > > > > that sounds really great. We have a similar setup that would heavily > > benefit from this. > > > > However when reading through the patch i have one concern about the > > correctness of not considering these ports, but that might just come > > from misunderstanding something. > > > > I assumed that we have one chassis c1 that has claimed sw0-p1. It should > > then know sw0, lr0 and public. > > I also assumed that there is chassis c2 claiming sw1-p1. That would then > > know sw1, lr1 and public. > > Then i assumed there is a chassis g1 that has claimed the lr0 CR port. > > Also chassis g2 has claimed the lr1 CR port. > > > > Now we add a NAT rule to lr0. It is configured with dnat_and_snat and > > the settings necessary to process it in a distributed way (i think by > > setting logical_port and stateless, but i may be wrong here). > > It's actually logical_port and external_mac > > It maps > > the sw0-p1 IP to some IP on public. > > > > On lr1 we have a similar setup of NAT entries for the sw1-p1 IP. > > > > If i understand these NAT settins correctly this should now lead to traffic > > from sw0-p1 to sw1-p1 to flow directly (when using the external ip of the > > sw1-p1 NAT rule). So the traffic would flow from chassic c1 to c2 > > without any involvement of g1 and g2. > > > That's correct. > > > In this case wouldn't chassis c1 need to know about the lr1 and sw1 > > datapath? > > Not really. Packet from sw0-p1 on C1 will enter lr0 pipeline first. > If there is a mac_binding entry > for the public IP of sw1-p1 in the lr0 datapath, it will set the > eth.dst to this mac and sends > the packet to public switch datapath and then to the wire via the localnet > port. > The fabric will make sure that the packet reaches C2. > If there is no mac_binding entry, ovn-controller will generate an ARP > request for the sw1-p1 public IP. > And then eventually reinjects the packet after it learns the mac.
Hi Numan, thanks a lot for the explanation. I always had the misunderstanding that such traffic would go via the overlay instead of via the localnet port. With the localnet port that makes a lot more sense. > > The same happens on the C2 for the reply path. > > Even without this patch, I think this is the behavior. > > There is a multinode system test to cover this scenario. > > Hope this addresses your concern. Let me know if you have further questions. Yes this was really helpful. Thanks a lot, Felix > > Numan > > > > > I guess i missed something somewhere, but since i am not sure i wanted > > to share this concern. > > > > Thanks a lot, > > Felix > > > > > > > > Signed-off-by: Numan Siddique <[email protected]> > > > --- > > > controller/binding.c | 249 +++++++++-- > > > controller/binding.h | 2 + > > > controller/local_data.c | 84 +++- > > > controller/local_data.h | 6 + > > > controller/lport.c | 12 + > > > controller/lport.h | 4 + > > > controller/ovn-controller.c | 38 ++ > > > tests/multinode.at | 178 ++++++++ > > > tests/ovn-performance.at | 6 +- > > > tests/ovn.at | 853 ++++++++++++++++++++++++++++++++++++ > > > 10 files changed, 1394 insertions(+), 38 deletions(-) > > > > > > diff --git a/controller/binding.c b/controller/binding.c > > > index ea5bf5a9fd..88c9640c3a 100644 > > > --- a/controller/binding.c > > > +++ b/controller/binding.c > > > @@ -825,6 +825,16 @@ static bool binding_lport_update_port_sec( > > > static bool ovs_iface_matches_lport_iface_id_ver( > > > const struct ovsrec_interface *, > > > const struct sbrec_port_binding *); > > > +static bool cleanup_patch_port_local_dps( > > > + const struct sbrec_port_binding *, const struct sbrec_port_binding > > > *cr_pb, > > > + const struct sbrec_port_binding *peer, struct local_datapath *ld, > > > + struct binding_ctx_in *b_ctx_in, > > > + struct binding_ctx_out *b_ctx_out, > > > + bool *cleanup); > > > +static bool local_datapath_is_relevant( > > > + struct local_datapath *, struct local_datapath *ignore_peer_ld, > > > + struct hmap *local_datapaths, int *depth, const struct sbrec_chassis > > > *, > > > + struct ovsdb_idl_index *); > > > > > > void > > > related_lports_init(struct related_lports *rp) > > > @@ -1062,6 +1072,13 @@ binding_dump_related_lports(struct related_lports > > > *related_lports, > > > } > > > } > > > > > > +struct dp_binding { > > > + struct hmap_node key_node; > > > + > > > + uint32_t dp_key; > > > + struct hmapx binding_lports; > > > +}; > > > + > > > void > > > binding_dump_local_bindings(struct local_binding_data *lbinding_data, > > > struct ds *out_data) > > > @@ -1130,6 +1147,19 @@ binding_dump_local_bindings(struct > > > local_binding_data *lbinding_data, > > > free(nodes); > > > } > > > > > > +void > > > +binding_dump_local_datapaths(struct hmap *local_datapaths, > > > + struct ds *out_data) > > > +{ > > > + ds_put_cstr(out_data, "Local datapaths:\n"); > > > + struct local_datapath *ld; > > > + HMAP_FOR_EACH (ld, hmap_node, local_datapaths) { > > > + ds_put_format(out_data, "Datapath: %s, type: %s\n", > > > + smap_get(&ld->datapath->external_ids, "name"), > > > + ld->is_switch ? "switch" : "router"); > > > + } > > > +} > > > + > > > void > > > set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb, > > > const struct sbrec_chassis *chassis_rec, > > > @@ -2137,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in > > > *b_ctx_in, > > > > > > static bool consider_patch_port_for_local_datapaths( > > > const struct sbrec_port_binding *, > > > - struct binding_ctx_in *, struct binding_ctx_out *); > > > + const struct sbrec_port_binding *cr_pb, > > > + struct binding_ctx_in *, struct binding_ctx_out *, > > > + bool check_and_remove_localdps); > > > > > > void > > > binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out > > > *b_ctx_out) > > > @@ -2182,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct > > > binding_ctx_out *b_ctx_out) > > > switch (lport_type) { > > > case LP_PATCH: > > > update_related_lport(pb, b_ctx_out); > > > - consider_patch_port_for_local_datapaths(pb, b_ctx_in, > > > b_ctx_out); > > > + consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in, > > > + b_ctx_out, false); > > > break; > > > > > > case LP_VTEP: > > > @@ -2238,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct > > > binding_ctx_out *b_ctx_out) > > > struct lport *lnet_lport = xmalloc(sizeof *lnet_lport); > > > lnet_lport->pb = pb; > > > ovs_list_push_back(&localnet_lports, &lnet_lport->list_node); > > > + if (pb->chassis == b_ctx_in->chassis_rec) { > > > + sbrec_port_binding_set_chassis(pb, NULL); > > > + } > > > break; > > > } > > > > > > @@ -2566,7 +2602,6 @@ consider_iface_release(const struct > > > ovsrec_interface *iface_rec, > > > lbinding->iface->name, > > > > > > &lbinding->iface->header_.uuid); > > > } > > > - > > > } else if (b_lport && b_lport->type == LP_LOCALPORT) { > > > /* lbinding is associated with a localport. Remove it from > > > the > > > * related lports. */ > > > @@ -2956,12 +2991,27 @@ handle_updated_vif_lport(const struct > > > sbrec_port_binding *pb, > > > > > > static bool > > > consider_patch_port_for_local_datapaths(const struct sbrec_port_binding > > > *pb, > > > + const struct sbrec_port_binding > > > *cr_pb, > > > struct binding_ctx_in *b_ctx_in, > > > - struct binding_ctx_out > > > *b_ctx_out) > > > + struct binding_ctx_out > > > *b_ctx_out, > > > + bool check_and_remove_localdps) > > > { > > > - struct local_datapath *ld = > > > - get_local_datapath(b_ctx_out->local_datapaths, > > > - pb->datapath->tunnel_key); > > > + const struct sbrec_port_binding *peer; > > > + struct local_datapath *peer_ld = NULL; > > > + struct local_datapath *ld = NULL; > > > + > > > + ld = get_local_datapath(b_ctx_out->local_datapaths, > > > + pb->datapath->tunnel_key); > > > + if (ld && ld->has_only_dgp_peer_ports) { > > > + /* Nothing much to do. */ > > > + return true; > > > + } > > > + > > > + peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name); > > > + if (peer) { > > > + peer_ld = get_local_datapath(b_ctx_out->local_datapaths, > > > + peer->datapath->tunnel_key); > > > + } > > > > > > if (!ld) { > > > /* If 'ld' for this lport is not present, then check if > > > @@ -2969,17 +3019,9 @@ consider_patch_port_for_local_datapaths(const > > > struct sbrec_port_binding *pb, > > > * and peer's datapath is already in the local datapaths, > > > * then add this lport's datapath to the local_datapaths. > > > * */ > > > - const struct sbrec_port_binding *peer; > > > - struct local_datapath *peer_ld = NULL; > > > - peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name); > > > - if (peer) { > > > - peer_ld = > > > - get_local_datapath(b_ctx_out->local_datapaths, > > > - peer->datapath->tunnel_key); > > > - } > > > - if (peer_ld && need_add_peer_to_local( > > > - b_ctx_in->sbrec_port_binding_by_name, peer, > > > - b_ctx_in->chassis_rec)) { > > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports && > > > + need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, > > > peer, > > > + b_ctx_in->chassis_rec)) { > > > ld = add_local_datapath( > > > b_ctx_in->sbrec_datapath_binding_by_key, > > > b_ctx_in->sbrec_port_binding_by_datapath, > > > @@ -2992,7 +3034,7 @@ consider_patch_port_for_local_datapaths(const > > > struct sbrec_port_binding *pb, > > > /* Add the peer datapath to the local datapaths if it's > > > * not present yet. > > > */ > > > - if (need_add_peer_to_local( > > > + if (peer && need_add_peer_to_local( > > > b_ctx_in->sbrec_port_binding_by_name, pb, > > > b_ctx_in->chassis_rec)) { > > > add_local_datapath_peer_port( > > > @@ -3003,6 +3045,18 @@ consider_patch_port_for_local_datapaths(const > > > struct sbrec_port_binding *pb, > > > ld, b_ctx_out->local_datapaths, > > > b_ctx_out->tracked_dp_bindings); > > > } > > > + > > > + if (check_and_remove_localdps) { > > > + bool cleanedup = false; > > > + if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, > > > b_ctx_in, > > > + b_ctx_out, &cleanedup)) { > > > + return false; > > > + } > > > + > > > + if (cleanedup) { > > > + ld = NULL; > > > + } > > > + } > > > } > > > > > > /* If this chassis is requested - try to claim. */ > > > @@ -3021,12 +3075,10 @@ consider_patch_port_for_local_datapaths(const > > > struct sbrec_port_binding *pb, > > > || if_status_is_port_claimed(b_ctx_out->if_mgr, > > > pb->logical_port)) { > > > > > > remove_local_lports(pb->logical_port, b_ctx_out); > > > - if (!release_lport(pb, ld, b_ctx_in->chassis_rec, > > > - !b_ctx_in->ovnsb_idl_txn, > > > - b_ctx_out->tracked_dp_bindings, > > > - b_ctx_out->if_mgr)) { > > > - return false; > > > - } > > > + return release_lport(pb, ld, b_ctx_in->chassis_rec, > > > + !b_ctx_in->ovnsb_idl_txn, > > > + b_ctx_out->tracked_dp_bindings, > > > + b_ctx_out->if_mgr); > > > } > > > return true; > > > } > > > @@ -3086,8 +3138,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in, > > > > > > case LP_PATCH: > > > update_related_lport(pb, b_ctx_out); > > > - handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in, > > > - b_ctx_out); > > > + handled = consider_patch_port_for_local_datapaths(pb, NULL, > > > b_ctx_in, > > > + b_ctx_out, > > > true); > > > break; > > > > > > case LP_VTEP: > > > @@ -3130,8 +3182,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in, > > > break; > > > } > > > handled = consider_patch_port_for_local_datapaths(distributed_pb, > > > - b_ctx_in, > > > - b_ctx_out); > > > + pb, b_ctx_in, > > > + b_ctx_out, > > > true); > > > break; > > > > > > case LP_EXTERNAL: > > > @@ -3901,3 +3953,142 @@ binding_destroy(void) > > > shash_destroy_free_data(&_qos_ports); > > > sset_clear(&_postponed_ports); > > > } > > > + > > > +static bool > > > +is_patch_pb_chassis_relevant( > > > + const struct sbrec_port_binding *pb, > > > + const struct sbrec_chassis *chassis, > > > + struct ovsdb_idl_index *sbrec_port_binding_by_name) > > > +{ > > > + if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) { > > > + return true; > > > + } > > > + > > > + const struct sbrec_port_binding *pb_crp = > > > + lport_get_cr_port(sbrec_port_binding_by_name, pb); > > > + if (pb_crp) { > > > + return ha_chassis_group_contains(pb_crp->ha_chassis_group, > > > chassis); > > > + } > > > + > > > + return false; > > > +} > > > + > > > +static bool > > > +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb, > > > + const struct sbrec_port_binding *cr_pb, > > > + const struct sbrec_port_binding *peer, > > > + struct local_datapath *ld, > > > + struct binding_ctx_in *b_ctx_in, > > > + struct binding_ctx_out *b_ctx_out, > > > + bool *cleanedup) > > > +{ > > > + *cleanedup = false; > > > + if (!peer) { > > > + /* Remove 'pb' from the ld's peer ports as it has no peer. */ > > > + remove_local_datapath_peer_port(pb, ld, > > > + b_ctx_out->local_datapaths); > > > + } > > > + > > > + /* We can consider removing the 'ld' of the patch port 'pb' from the > > > + * local datapaths, if all the below conditions are met > > > + * - 'pb' doesn't have a peer or ld' is a router datapath > > > + * - if 'pb' is a distributed gateway port (dgp), then > > > + * its chassisredirect port's ha chassis group doesn't > > > + * contain our 'chassis rec' > > > + * - and finally 'ld' is not relevant any more. See > > > + * local_datapath_is_relevant() for more details. > > > + * > > > + * Note: If 'ld' can be removed, then all its connected local > > > datapaths > > > + * can also be removed. > > > + * > > > + * For example, if we had sw1-port1 -> sw1 -> lr1 -> sw2 and if > > > + * sw1-port1 resides on this chassis, and if the link between sw1 and > > > + * lr1 is broken, then we can remove lr1 and sw2 from the > > > + * local_datapaths. > > > + * */ > > > + > > > + bool consider_ld_for_removal = !peer || !ld->is_switch; > > > + if (consider_ld_for_removal && cr_pb) { > > > + consider_ld_for_removal = !ha_chassis_group_contains( > > > + cr_pb->ha_chassis_group, b_ctx_in->chassis_rec); > > > + } > > > + > > > + if (!consider_ld_for_removal) { > > > + return true; > > > + } > > > + > > > + int depth = 0; > > > + > > > + bool is_relevant = local_datapath_is_relevant( > > > + ld, NULL, b_ctx_out->local_datapaths, > > > + &depth, b_ctx_in->chassis_rec, > > > + b_ctx_in->sbrec_port_binding_by_name); > > > + > > > + if (depth >= 100) { > > > + /* datapaths are too deeply nested. Fall back to recompute. */ > > > + return false; > > > + } > > > + > > > + if (!is_relevant) { > > > + /* This 'ld' can be removed from the local datapaths as > > > + * - its a router datapath and > > > + * - it has no peers locally. */ > > > + local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths, > > > + > > > b_ctx_out->tracked_dp_bindings); > > > + *cleanedup = true; > > > + } > > > + > > > + return true; > > > +} > > > + > > > +static bool > > > +local_datapath_is_relevant(struct local_datapath *ld, > > > + struct local_datapath *ignore_peer_ld, > > > + struct hmap *local_datapaths, int *depth, > > > + const struct sbrec_chassis *chassis, > > > + struct ovsdb_idl_index *sbrec_pb_by_name) > > > +{ > > > + if (!sset_is_empty(&ld->claimed_lports) || > > > + !shash_is_empty(&ld->external_ports) || > > > + !shash_is_empty(&ld->multichassis_ports) || > > > + ld->vtep_port) { > > > + return true; > > > + } > > > + > > > + bool relevant = false; > > > + > > > + if (*depth >= 100) { > > > + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); > > > + VLOG_WARN_RL(&rl, "datapaths nested too deep"); > > > + return true; > > > + } > > > + > > > + for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) { > > > + const struct sbrec_port_binding *remote = > > > ld->peer_ports[i].remote; > > > + const struct sbrec_port_binding *local = ld->peer_ports[i].local; > > > + > > > + if (is_patch_pb_chassis_relevant(local, chassis, > > > + sbrec_pb_by_name)) { > > > + return true; > > > + } > > > + > > > + if (is_patch_pb_chassis_relevant(remote, chassis, > > > + sbrec_pb_by_name)) { > > > + return true; > > > + } > > > + > > > + struct local_datapath *peer_ld; > > > + uint32_t remote_peer_ld_key; > > > + remote_peer_ld_key = > > > ld->peer_ports[i].remote->datapath->tunnel_key; > > > + peer_ld = get_local_datapath(local_datapaths, > > > remote_peer_ld_key); > > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports && > > > + peer_ld != ignore_peer_ld) { > > > + *depth = *depth + 1; > > > + relevant = local_datapath_is_relevant(peer_ld, ld, > > > + local_datapaths, depth, > > > + chassis, > > > sbrec_pb_by_name); > > > + } > > > + } > > > + > > > + return relevant; > > > +} > > > diff --git a/controller/binding.h b/controller/binding.h > > > index d13ae36c79..a4346c3e10 100644 > > > --- a/controller/binding.h > > > +++ b/controller/binding.h > > > @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct > > > binding_ctx_in *, > > > void binding_tracked_dp_destroy(struct hmap *tracked_datapaths); > > > > > > void binding_dump_local_bindings(struct local_binding_data *, struct ds > > > *); > > > +void binding_dump_local_datapaths(struct hmap *local_datapaths, > > > + struct ds *out_data); > > > > > > void binding_dump_related_lports(struct related_lports *related_lports, > > > struct ds *); > > > diff --git a/controller/local_data.c b/controller/local_data.c > > > index e19b2bf865..bb5740f1d6 100644 > > > --- a/controller/local_data.c > > > +++ b/controller/local_data.c > > > @@ -53,6 +53,13 @@ static struct tracked_datapath > > > *tracked_datapath_create( > > > > > > static bool datapath_is_switch(const struct sbrec_datapath_binding *); > > > static bool datapath_is_transit_switch(const struct > > > sbrec_datapath_binding *); > > > +static bool datapath_has_only_dgp_peer_ports( > > > + const struct sbrec_datapath_binding *); > > > +static void local_datapath_remove_and_destroy__( > > > + struct local_datapath *ld, > > > + const struct sbrec_port_binding *ignore_peer_port, > > > + struct hmap *local_datapaths, > > > + struct hmap *tracked_datapaths); > > > > > > static uint64_t local_datapath_usage; > > > > > > @@ -86,6 +93,7 @@ local_datapath_alloc(const struct > > > sbrec_datapath_binding *dp) > > > ld->datapath = dp; > > > ld->is_switch = datapath_is_switch(dp); > > > ld->is_transit_switch = datapath_is_transit_switch(dp); > > > + ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp); > > > shash_init(&ld->external_ports); > > > shash_init(&ld->multichassis_ports); > > > sset_init(&ld->claimed_lports); > > > @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld) > > > free(ld); > > > } > > > > > > +void local_datapath_remove_and_destroy(struct local_datapath *ld, > > > + struct hmap *local_datapaths, > > > + struct hmap *tracked_datapaths) > > > +{ > > > + local_datapath_remove_and_destroy__(ld, NULL, local_datapaths, > > > + tracked_datapaths); > > > +} > > > + > > > /* Checks if pb is running on local gw router or pb is a patch port > > > * and the peer datapath should be added to local datapaths. */ > > > bool > > > @@ -226,12 +242,12 @@ add_local_datapath_peer_port( > > > get_local_datapath(local_datapaths, > > > peer->datapath->tunnel_key); > > > if (!peer_ld) { > > > - add_local_datapath__(sbrec_datapath_binding_by_key, > > > - sbrec_port_binding_by_datapath, > > > - sbrec_port_binding_by_name, 1, > > > - peer->datapath, chassis, local_datapaths, > > > - tracked_datapaths); > > > - return; > > > + peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key, > > > + sbrec_port_binding_by_datapath, > > > + sbrec_port_binding_by_name, 1, > > > + peer->datapath, chassis, > > > + local_datapaths, > > > + tracked_datapaths); > > > } > > > > > > local_datapath_peer_port_add(peer_ld, peer, pb); > > > @@ -613,6 +629,17 @@ add_local_datapath__(struct ovsdb_idl_index > > > *sbrec_datapath_binding_by_key, > > > tracked_datapaths); > > > } > > > > > > + if (ld->has_only_dgp_peer_ports) { > > > + /* If this flag is set, it means this 'switch' datapath has > > > + * - one ore many localnet ports. > > > + * - all the router ports it is connected to are > > > + * distributed gateway ports (DGPs). > > > + * There is no need to add the routers of the dgps to > > > + * the local datapaths. > > > + * */ > > > + return ld; > > > + } > > > + > > > if (depth >= 100) { > > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); > > > VLOG_WARN_RL(&rl, "datapaths nested too deep"); > > > @@ -705,6 +732,13 @@ datapath_is_transit_switch(const struct > > > sbrec_datapath_binding *ldp) > > > return smap_get(&ldp->external_ids, "interconn-ts") != NULL; > > > } > > > > > > +static bool > > > +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding > > > *ldp) > > > +{ > > > + return datapath_is_switch(ldp) && > > > + smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", > > > false); > > > +} > > > + > > > bool > > > lb_is_local(const struct sbrec_load_balancer *sbrec_lb, > > > const struct hmap *local_datapaths) > > > @@ -745,3 +779,41 @@ lb_is_local(const struct sbrec_load_balancer > > > *sbrec_lb, > > > > > > return false; > > > } > > > + > > > +static void > > > +local_datapath_remove_and_destroy__(struct local_datapath *ld, > > > + const struct sbrec_port_binding > > > *ignore_pb, > > > + struct hmap *local_datapaths, > > > + struct hmap *tracked_datapaths) > > > +{ > > > + for (size_t i = 0; i < ld->n_peer_ports; i++) { > > > + const struct sbrec_port_binding *remote = > > > ld->peer_ports[i].remote; > > > + const struct sbrec_port_binding *local = ld->peer_ports[i].local; > > > + > > > + if (local == ignore_pb) { > > > + continue; > > > + } > > > + > > > + struct local_datapath *peer_ld; > > > + uint32_t remote_peer_ld_key; > > > + > > > + remote_peer_ld_key = > > > ld->peer_ports[i].remote->datapath->tunnel_key; > > > + peer_ld = get_local_datapath(local_datapaths, > > > remote_peer_ld_key); > > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports) { > > > + local_datapath_remove_and_destroy__(peer_ld, remote, > > > + local_datapaths, > > > + tracked_datapaths); > > > + } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) { > > > + remove_local_datapath_peer_port(ld->peer_ports[i].remote, > > > + peer_ld, local_datapaths); > > > + } > > > + } > > > + > > > + hmap_remove(local_datapaths, &ld->hmap_node); > > > + if (tracked_datapaths) { > > > + tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED, > > > + tracked_datapaths); > > > + } > > > + > > > + local_datapath_destroy(ld); > > > +} > > > diff --git a/controller/local_data.h b/controller/local_data.h > > > index 857d63a51d..ef01f4259e 100644 > > > --- a/controller/local_data.h > > > +++ b/controller/local_data.h > > > @@ -46,6 +46,8 @@ struct local_datapath { > > > const struct sbrec_datapath_binding *datapath; > > > bool is_switch; > > > bool is_transit_switch; > > > + /* Valid only for 'is_switch' local datapath. */ > > > + bool has_only_dgp_peer_ports; > > > > > > /* The localnet port in this datapath, if any (at most one is > > > allowed). */ > > > const struct sbrec_port_binding *localnet_port; > > > @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath( > > > > > > void local_datapaths_destroy(struct hmap *local_datapaths); > > > void local_datapath_destroy(struct local_datapath *ld); > > > +void local_datapath_remove_and_destroy(struct local_datapath *, > > > + struct hmap *local_datapaths, > > > + struct hmap *tracked_datapaths); > > > + > > > void add_local_datapath_peer_port( > > > const struct sbrec_port_binding *, > > > const struct sbrec_chassis *, > > > diff --git a/controller/lport.c b/controller/lport.c > > > index 8bc230e896..f164803528 100644 > > > --- a/controller/lport.c > > > +++ b/controller/lport.c > > > @@ -102,6 +102,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding > > > *pb, > > > return get_peer_lport(pb, sbrec_port_binding_by_name); > > > } > > > > > > +const struct sbrec_port_binding * > > > +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name, > > > + const struct sbrec_port_binding *pb) > > > +{ > > > + const char *crp = smap_get(&pb->options, "chassis-redirect-port"); > > > + if (crp) { > > > + return lport_lookup_by_name(sbrec_port_binding_by_name, crp); > > > + } > > > + > > > + return NULL; > > > +} > > > + > > > enum can_bind > > > lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec, > > > const struct sbrec_port_binding *pb) > > > diff --git a/controller/lport.h b/controller/lport.h > > > index 3a58ebe39e..069bd59096 100644 > > > --- a/controller/lport.h > > > +++ b/controller/lport.h > > > @@ -73,4 +73,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer( > > > bool > > > lport_is_activated_by_activation_strategy(const struct > > > sbrec_port_binding *pb, > > > const struct sbrec_chassis > > > *chassis); > > > +const struct sbrec_port_binding *lport_get_cr_port( > > > + struct ovsdb_idl_index *sbrec_port_binding_by_name, > > > + const struct sbrec_port_binding *); > > > + > > > #endif /* controller/lport.h */ > > > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c > > > index da942abaa9..dc8ceec076 100644 > > > --- a/controller/ovn-controller.c > > > +++ b/controller/ovn-controller.c > > > @@ -101,6 +101,7 @@ static unixctl_cb_func debug_pause_execution; > > > static unixctl_cb_func debug_resume_execution; > > > static unixctl_cb_func debug_status_execution; > > > static unixctl_cb_func debug_dump_local_bindings; > > > +static unixctl_cb_func debug_dump_local_datapaths; > > > static unixctl_cb_func debug_dump_related_lports; > > > static unixctl_cb_func debug_dump_local_template_vars; > > > static unixctl_cb_func debug_dump_local_mac_bindings; > > > @@ -1688,6 +1689,22 @@ runtime_data_sb_datapath_binding_handler(struct > > > engine_node *node OVS_UNUSED, > > > return false; > > > } > > > } > > > + > > > + if (sbrec_datapath_binding_is_updated( > > > + dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) && > > > + !sbrec_datapath_binding_is_new(dp)) { > > > + struct local_datapath *ld = > > > + get_local_datapath(&rt_data->local_datapaths, > > > + dp->tunnel_key); > > > + if (ld && ld->is_switch) { > > > + bool only_dgp_peer_ports = > > > + smap_get_bool(&dp->external_ids, > > > "only_dgp_peer_ports", > > > + false); > > > + if (ld->has_only_dgp_peer_ports != > > > only_dgp_peer_ports) { > > > + return false; > > > + } > > > + } > > > + } > > > } > > > > > > return true; > > > @@ -4357,6 +4374,12 @@ lflow_output_runtime_data_handler(struct > > > engine_node *node, > > > init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out); > > > > > > struct tracked_datapath *tdp; > > > + HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) { > > > + if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) { > > > + return false; > > > + } > > > + } > > > + > > > HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) { > > > if (tdp->tracked_type == TRACKED_RESOURCE_NEW) { > > > if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in, > > > @@ -5552,6 +5575,10 @@ main(int argc, char *argv[]) > > > debug_dump_local_bindings, > > > &runtime_data->lbinding_data); > > > > > > + unixctl_command_register("debug/dump-local-datapaths", "", 0, 0, > > > + debug_dump_local_datapaths, > > > + &runtime_data->local_datapaths); > > > + > > > unixctl_command_register("debug/dump-related-ports", "", 0, 0, > > > debug_dump_related_lports, > > > &runtime_data->related_lports); > > > @@ -6517,6 +6544,17 @@ debug_dump_local_bindings(struct unixctl_conn > > > *conn, int argc OVS_UNUSED, > > > ds_destroy(&binding_data); > > > } > > > > > > +static void > > > +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc > > > OVS_UNUSED, > > > + const char *argv[] OVS_UNUSED, > > > + void *local_datapaths) > > > +{ > > > + struct ds local_dps_data = DS_EMPTY_INITIALIZER; > > > + binding_dump_local_datapaths(local_datapaths, &local_dps_data); > > > + unixctl_command_reply(conn, ds_cstr(&local_dps_data)); > > > + ds_destroy(&local_dps_data); > > > +} > > > + > > > static void > > > debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED, > > > const char *argv[] OVS_UNUSED, void > > > *related_lports) > > > diff --git a/tests/multinode.at b/tests/multinode.at > > > index c1bd3123ac..b828f16135 100644 > > > --- a/tests/multinode.at > > > +++ b/tests/multinode.at > > > @@ -2776,3 +2776,181 @@ for i in 1 2; do > > > done > > > > > > AT_CLEANUP > > > + > > > +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch > > > functionality]) > > > + > > > +# Check that ovn-fake-multinode setup is up and running > > > +check_fake_multinode_setup > > > + > > > +# Delete the multinode NB and OVS resources before starting the test. > > > +cleanup_multinode_resources > > > + > > > +check multinode_nbctl ls-add sw0 > > > +check multinode_nbctl lsp-add sw0 sw0-port1 > > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 > > > 10.0.0.3 1000::3" > > > +check multinode_nbctl lsp-add sw0 sw0-port2 > > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 > > > 10.0.0.4 1000::4" > > > + > > > +check multinode_nbctl lr-add lr0 > > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > > 1000::1/64 > > > +check multinode_nbctl lsp-add sw0 sw0-lr0 > > > +check multinode_nbctl lsp-set-type sw0-lr0 router > > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router > > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 > > > + > > > +check multinode_nbctl ls-add public > > > +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 > > > 172.16.1.100/24 2000::1/64 > > > +check multinode_nbctl lsp-add public public-lr0 > > > +check multinode_nbctl lsp-set-type public-lr0 router > > > +check multinode_nbctl lsp-set-addresses public-lr0 router > > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public > > > + > > > +# localnet port > > > +check multinode_nbctl lsp-add public ln-public > > > +check multinode_nbctl lsp-set-type ln-public localnet > > > +check multinode_nbctl lsp-set-addresses ln-public unknown > > > +check multinode_nbctl lsp-set-options ln-public network_name=public > > > + > > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20 > > > +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24 > > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 > > > sw0-port1 50:54:00:00:00:03 > > > +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64 > > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 > > > sw0-port1 50:54:00:00:00:03 > > > + > > > +check multinode_nbctl --wait=hv sync > > > + > > > +check multinode_nbctl ls-add sw1 > > > +check multinode_nbctl lsp-add sw1 sw1-port1 > > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 > > > 20.0.0.3 2000::3" > > > + > > > +check multinode_nbctl lr-add lr1 > > > +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 > > > 2000::a/64 > > > +check multinode_nbctl lsp-add sw1 sw1-lr1 > > > +check multinode_nbctl lsp-set-type sw1-lr1 router > > > +check multinode_nbctl lsp-set-addresses sw1-lr1 router > > > +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1 > > > + > > > +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 > > > 172.16.1.101/24 > > > +check multinode_nbctl lsp-add public public-lr1 > > > +check multinode_nbctl lsp-set-type public-lr1 router > > > +check multinode_nbctl lsp-set-addresses public-lr1 router > > > +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public > > > + > > > +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24 > > > +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3 > > > +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20 > > > + > > > +check multinode_nbctl --wait=hv sync > > > + > > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 > > > 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a > > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 > > > 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a > > > + > > > +m_wait_for_ports_up sw0-port1 > > > +m_wait_for_ports_up sw1-port1 > > > + > > > +m_as ovn-central-az1-1 ovn-sbctl show > > > + > > > +m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | sort > > > +m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | sort > > > + > > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | > > > sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > > sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([m_as ovn-gw-1 ovn-appctl debug/dump-local-datapaths | sort], > > > [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# ping lr0-public IP - 172.168.0.100 > > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.100 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# ping lr1-public IP - 172.168.0.101 from sw0p1 > > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.101 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1 > > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.120 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1 > > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.110 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# Bind sw0-port2 on chassis-2 > > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 > > > 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a > > > +m_wait_for_ports_up sw0-port2 > > > + > > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > > sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2 > > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.110 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2 > > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 > > > 172.16.1.120 | FORMAT_PING], \ > > > +[0], [dnl > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > > +]) > > > + > > > +# Create a normal router port in public with its peer as a normal > > > distributed router port. > > > +check multinode_nbctl lsp-add public public-lr2 > > > +check multinode_nbctl lsp-set-type public-lr2 router > > > +check multinode_nbctl lsp-set-addresses public-lr2 router > > > +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public > > > +check multinode_nbctl lr-add lr2 > > > +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 > > > 172.16.1.102/24 3000::a/64 > > > + > > > +check multinode_nbctl --wait=hv sync > > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | > > > sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > > sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CLEANUP > > > diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at > > > index 7d480c20c8..a003fc36bc 100644 > > > --- a/tests/ovn-performance.at > > > +++ b/tests/ovn-performance.at > > > @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT( > > > ) > > > > > > OVN_CONTROLLER_EXPECT_HIT_COND( > > > - [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0], > > > + [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0], > > > [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && > > > ovn-nbctl --wait=hv sync] > > > ) > > > > > > @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list > > > chassis hv5) > > > OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public > > > chassis=$hv5_ch]) > > > check ovn-nbctl --wait=hv sync > > > # Delete hv5 from gateway chassis. There should be no lflow_run. > > > -OVN_CONTROLLER_EXPECT_NO_HIT( > > > - [hv1 hv2 hv3 hv4 hv5], [lflow_run], > > > +OVN_CONTROLLER_EXPECT_HIT_COND( > > > + [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0] > > > [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5] > > > ) > > > > > > diff --git a/tests/ovn.at b/tests/ovn.at > > > index d105ed2535..d4e87212a5 100644 > > > --- a/tests/ovn.at > > > +++ b/tests/ovn.at > > > @@ -41481,6 +41481,859 @@ OVN_CLEANUP([hv1]) > > > AT_CLEANUP > > > ]) > > > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB > > > datapath_binding]) > > > +AT_KEYWORDS([multiple-l3dgw-ports]) > > > +ovn_start > > > +net_add n1 > > > +sim_add hv1 > > > +as hv1 > > > +check ovs-vsctl add-br br-phys > > > +ovn_attach n1 br-phys 192.168.0.1 > > > + > > > +sim_add hv2 > > > +as hv2 > > > +check ovs-vsctl add-br br-phys > > > +ovn_attach n1 br-phys 192.168.0.2 > > > + > > > +sim_add gw1 > > > +as gw1 > > > +check ovs-vsctl add-br br-phys > > > +ovn_attach n1 br-phys 192.168.0.3 > > > + > > > +sim_add gw2 > > > +as gw2 > > > +check ovs-vsctl add-br br-phys > > > +ovn_attach n1 br-phys 192.168.0.4 > > > + > > > +check ovn-nbctl ls-add sw0 > > > +check ovn-nbctl lsp-add sw0 sw0-port1 > > > +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 > > > 1000::3" > > > +check ovn-nbctl lsp-add sw0 sw0-port2 > > > +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 > > > 1000::4" > > > + > > > +check ovn-nbctl lr-add lr0 > > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > > 1000::1/64 > > > +check ovn-nbctl lsp-add sw0 sw0-lr0 > > > +check ovn-nbctl lsp-set-type sw0-lr0 router > > > +check ovn-nbctl lsp-set-addresses sw0-lr0 router > > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 > > > + > > > +check ovn-nbctl ls-add public > > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 > > > 172.168.0.100/24 2000::1/64 > > > +check ovn-nbctl lsp-add public public-lr0 > > > +check ovn-nbctl lsp-set-type public-lr0 router > > > +check ovn-nbctl lsp-set-addresses public-lr0 router > > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public > > > + > > > +# localnet port > > > +check ovn-nbctl lsp-add public ln-public > > > +check ovn-nbctl lsp-set-type ln-public localnet > > > +check ovn-nbctl lsp-set-addresses ln-public unknown > > > +check ovn-nbctl lsp-set-options ln-public network_name=phys > > > + > > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20 > > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24 > > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 > > > sw0-port2 f0:00:00:01:02:04 > > > +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64 > > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 > > > f0:00:00:01:02:04 > > > + > > > +check ovn-nbctl --wait=hv sync > > > + > > > +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=sw0)) > > > +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=lr0)) > > > +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=public)) > > > + > > > +check_offlows_for_datapath() { > > > + hv=$1 > > > + dp_key=$2 > > > + should_be_present=$3 > > > + > > > + if [[ "$should_be_present" == "yes" ]]; then > > > + echo "Flows should be present for hv - $hv : datapath - $dp_key" > > > + OVS_WAIT_UNTIL( > > > + [test $(as $hv ovs-ofctl dump-flows br-int | grep -c > > > metadata=0x$dp_key) -gt 0] > > > + ) > > > + else > > > + echo "Flows should NOT be present for hv - $hv : datapath - > > > $dp_key" > > > + OVS_WAIT_UNTIL( > > > + [test $(as $hv ovs-ofctl dump-flows br-int | grep -c > > > metadata=0x$dp_key) -eq 0] > > > + ) > > > + fi > > > +} > > > + > > > +AT_CHECK([ovn-sbctl get datapath_binding public > > > external_ids:only_dgp_peer_ports], [0], [dnl > > > +"true" > > > +]) > > > + > > > +check_offlows_for_datapath hv1 $sw0_dp_key no > > > +check_offlows_for_datapath hv1 $lr0_dp_key no > > > +check_offlows_for_datapath hv1 $public_dp_key no > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key no > > > + > > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > > +check_offlows_for_datapath gw1 $public_dp_key yes > > > + > > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > > +check_offlows_for_datapath gw2 $public_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +# Create a VIF on hv1 for sw0-port1 > > > +AS_BOX([create a VIF on hv1 for sw0-port1]) > > > + > > > +as hv1 > > > +ovs-vsctl -- add-port br-int hv1-vif1 -- \ > > > + set interface hv1-vif1 external-ids:iface-id=sw0-port1 \ > > > + options:tx_pcap=hv1/vif1-tx.pcap \ > > > + options:rxq_pcap=hv1/vif1-rx.pcap \ > > > + ofport-request=1 > > > + > > > +wait_for_ports_up sw0-port1 > > > + > > > +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for > > > sw0, lr0 and public]) > > > + > > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > + > > > +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public]) > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 > > > to public]) > > > + > > > +check ovn-nbctl ls-add sw1 > > > +check ovn-nbctl lsp-add sw1 sw1-port1 > > > +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3" > > > + > > > +check ovn-nbctl lr-add lr1 > > > +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 > > > +check ovn-nbctl lsp-add sw1 sw1-lr1 > > > +check ovn-nbctl lsp-set-type sw1-lr1 router > > > +check ovn-nbctl lsp-set-addresses sw1-lr1 router > > > +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1 > > > + > > > +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24 > > > +check ovn-nbctl lsp-add public public-lr1 > > > +check ovn-nbctl lsp-set-type public-lr1 router > > > +check ovn-nbctl lsp-set-addresses public-lr1 router > > > +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public > > > + > > > +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=sw1)) > > > +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=lr1)) > > > + > > > +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24 > > > +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 > > > + > > > +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 > > > to public]) > > > + > > > +check ovn-nbctl ls-add sw2 > > > +check ovn-nbctl lsp-add sw2 sw2-port1 > > > +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3" > > > + > > > +check ovn-nbctl lr-add lr2 > > > +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24 > > > +check ovn-nbctl lsp-add sw2 sw2-lr2 > > > +check ovn-nbctl lsp-set-type sw2-lr2 router > > > +check ovn-nbctl lsp-set-addresses sw2-lr2 router > > > +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2 > > > + > > > +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24 > > > +check ovn-nbctl lsp-add public public-lr2 > > > +check ovn-nbctl lsp-set-type public-lr2 router > > > +check ovn-nbctl lsp-set-addresses public-lr2 router > > > +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public > > > + > > > +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=sw2)) > > > +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > > external_ids:name=lr2)) > > > + > > > +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24 > > > +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3 > > > + > > > +check ovn-nbctl --wait=hv sync > > > + > > > +# Since lr1-public is not a DGP, public is not a "only_dgp_peer_ports". > > > +AT_CHECK([ovn-sbctl get datapath_binding public > > > external_ids:only_dgp_peer_ports], [1], [ignore], [ignore]) > > > + > > > +check_offlows_for_datapath hv1 $sw1_dp_key yes > > > +check_offlows_for_datapath hv1 $lr1_dp_key yes > > > +check_offlows_for_datapath hv1 $sw2_dp_key yes > > > +check_offlows_for_datapath hv1 $lr2_dp_key yes > > > + > > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath gw1 $sw1_dp_key yes > > > +check_offlows_for_datapath gw1 $lr1_dp_key yes > > > +check_offlows_for_datapath gw1 $sw2_dp_key yes > > > +check_offlows_for_datapath gw1 $lr2_dp_key yes > > > + > > > +check_offlows_for_datapath gw2 $sw1_dp_key no > > > +check_offlows_for_datapath gw2 $lr1_dp_key no > > > +check_offlows_for_datapath gw2 $sw2_dp_key no > > > +check_offlows_for_datapath gw2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public]) > > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20 > > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30 > > > +wait_row_count Port_Binding 1 logical_port=cr-lr1-public > > > +wait_row_count Port_Binding 1 logical_port=cr-lr2-public > > > + > > > +AT_CHECK([ovn-sbctl get datapath_binding public > > > external_ids:only_dgp_peer_ports], [0], [dnl > > > +"true" > > > +]) > > > + > > > +check ovn-nbctl --wait=hv sync > > > + > > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key no > > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > > +check_offlows_for_datapath gw1 $public_dp_key yes > > > +check_offlows_for_datapath gw1 $sw1_dp_key no > > > +check_offlows_for_datapath gw1 $lr1_dp_key no > > > +check_offlows_for_datapath gw1 $sw2_dp_key no > > > +check_offlows_for_datapath gw1 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller > > > debug/dump-local-datapaths], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Create a VIF on hv2 for sw1-port1]) > > > + > > > +as hv2 > > > +ovs-vsctl -- add-port br-int hv2-vif1 -- \ > > > + set interface hv2-vif1 external-ids:iface-id=sw1-port1 \ > > > + options:tx_pcap=hv2/vif1-tx.pcap \ > > > + options:rxq_pcap=hv2/vif1-rx.pcap \ > > > + ofport-request=1 > > > + > > > +wait_for_ports_up sw1-port1 > > > + > > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > + > > > +# Since there are no distributed dnat_and_snat entries > > > +# in lr1, hv2 will not have "public" in its > > > +# local datapaths. > > > +check_offlows_for_datapath hv2 $public_dp_key no > > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > > +check_offlows_for_datapath gw1 $public_dp_key yes > > > +check_offlows_for_datapath gw1 $sw1_dp_key no > > > +check_offlows_for_datapath gw1 $lr1_dp_key no > > > +check_offlows_for_datapath gw1 $sw2_dp_key no > > > +check_offlows_for_datapath gw1 $lr2_dp_key no > > > + > > > +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local > > > datapaths. > > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > > +check_offlows_for_datapath gw2 $public_dp_key yes > > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# Add distributed dnat_and_snat in lr1. hv2 should have > > > +# public in its local datapaths. > > > +AS_BOX([ Add distributed dnat_and_snat in lr1]) > > > + > > > +check ovn-nbctl lr-nat-del lr1 dnat_and_snat > > > +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 > > > 20.0.0.3 sw1-port1 10:00:00:01:02:14 > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key yes > > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Create a VIF on hv2 for sw0-port2]) > > > + > > > +as hv2 > > > +ovs-vsctl -- add-port br-int hv2-vif2 -- \ > > > + set interface hv2-vif2 external-ids:iface-id=sw0-port2 \ > > > + options:tx_pcap=hv2/vif2-tx.pcap \ > > > + options:rxq_pcap=hv2/vif2-rx.pcap \ > > > + ofport-request=2 > > > + > > > +wait_for_ports_up sw0-port2 > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key yes > > > +check_offlows_for_datapath hv2 $lr0_dp_key yes > > > +check_offlows_for_datapath hv2 $public_dp_key yes > > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: lr1, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Delete the VIF for sw1-port1 in hv2]) > > > + > > > +as hv2 ovs-vsctl del-port hv2-vif1 > > > +check ovn-nbctl --wait=hv sync > > > +check_column "false" Port_Binding up logical_port=sw1-port1 > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key yes > > > +check_offlows_for_datapath hv2 $lr0_dp_key yes > > > +check_offlows_for_datapath hv2 $public_dp_key yes > > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Delete the VIF for sw0-port2 in hv2]) > > > + > > > +# Presently when a port binding is released we are not > > > +# deleting its datapath from the local_datapaths if it > > > +# is not relevant anymore. > > > + > > > +as hv2 ovs-vsctl del-port hv2-vif2 > > > +check ovn-nbctl --wait=hv sync > > > +check_column "false" Port_Binding up logical_port=sw0-port2 > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# hv2 would still have public, sw0 and lr0 in its local datapaths. > > > +# Next recompute should delete these datapaths. > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# Trigger a recompute > > > +AS_BOX([Trigger a recompute in hv2]) > > > +check as hv2 ovn-appctl inc-engine/recompute > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key no > > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Disconnect sw2 from lr2]) > > > + > > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx > > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Reconnect sw2 to lr2 again]) > > > + > > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2 > > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > > +check_offlows_for_datapath hv1 $public_dp_key yes > > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Create a VIF on gw2 for sw1-port1]) > > > + > > > +as gw2 > > > +ovs-vsctl -- add-port br-int gw2-vif2 -- \ > > > + set interface gw2-vif2 external-ids:iface-id=sw1-port1 \ > > > + options:tx_pcap=gw2/vif2-tx.pcap \ > > > + options:rxq_pcap=gw2/vif2-rx.pcap \ > > > + ofport-request=2 > > > + > > > +wait_for_ports_up sw1-port1 > > > + > > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > > +check_offlows_for_datapath gw2 $public_dp_key yes > > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Delete the VIF for sw1-port1 in gw2]) > > > + > > > +as gw2 ovs-vsctl del-port gw2-vif2 > > > +check ovn-nbctl --wait=hv sync > > > +check_column "false" Port_Binding up logical_port=sw1-port1 > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +# hv2 would still have public in its local datapaths. Next recompute > > > should > > > +# delete this datapath from the local datapaths. > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AS_BOX([Create a logical port for public and bind it on hv2]) > > > +# hv2 will only have public in its local datapaths. > > > +check ovn-nbctl lsp-add public public-p1 > > > + > > > +as hv2 > > > +ovs-vsctl -- add-port br-int hv2-vif3 -- \ > > > + set interface hv2-vif3 external-ids:iface-id=public-p1 \ > > > + options:tx_pcap=hv2/vif3-tx.pcap \ > > > + options:rxq_pcap=hv2/vif3-rx.pcap \ > > > + ofport-request=2 > > > + > > > +wait_for_ports_up public-p1 > > > + > > > +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute > > > +# check ovn-nbctl --wait=hv sync > > > + > > > +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort > > > + > > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > > +check_offlows_for_datapath hv2 $public_dp_key yes > > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > > + > > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: public, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr0, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw0, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths > > > | sort], [0], [dnl > > > +Datapath: lr1, type: router > > > +Datapath: lr2, type: router > > > +Datapath: public, type: switch > > > +Datapath: sw1, type: switch > > > +Datapath: sw2, type: switch > > > +Local datapaths: > > > +]) > > > + > > > +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2]) > > > +AT_CLEANUP > > > +]) > > > + > > > OVN_FOR_EACH_NORTHD([ > > > AT_SETUP([requested-tnl-key-recompute]) > > > AT_KEYWORDS([requested-tnl-key-recompute]) > > > -- > > > 2.48.1 > > > > > > _______________________________________________ > > > dev mailing list > > > [email protected] > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
