Move the load-balancer health-check sync out of ovn-ic.c into the en_service_monitor node and give the node a change handler for its busiest input.
The sync reads an SB Port_Binding only through create_synced_svcs_mon(), which - for each IC-SB Service_Monitor targeting this AZ - looks up the backing port by 'logical_port' and reads its 'up' and 'chassis'. SB Port_Binding is the busiest table in an availability zone, so answering every change on it with a full recompute charges the whole health-check sync to churn that cannot affect it. Add a handler that falls back only when the change can matter: the row is new or deleted, or its 'up'/'chassis' column changed, and the port backs a Service_Monitor targeting this AZ. The second test resolves through the existing by-target-AZ/logical-port index, so it costs one lookup per changed row. Every other port-binding change is answered as handled and unchanged. Drop the en_icsb_availability_zone input from en_service_monitor and en_address_set, and stop modelling the Availability_Zone table as an engine input altogether: these were its last two consumers. Both nodes need only the AZ identity that en_az reports and neither reads the table, so the input only exposed them to the nb_ic_cfg sequence number bumped on its rows. en_gateway, en_port_binding and en_route already avoid it for the same reason; this completes the pattern. Add tests/ovn-ic.at tests for the port-binding handler scoping and for the sequence-number churn no longer recomputing the node. Assisted-by: Claude Opus 4.8, Claude Code Signed-off-by: Paulo Guilherme Silva <[email protected]> --- NEWS | 7 + ic/en-service-monitor.c | 500 +++++++++++++++++++++++++++++++++++++++- ic/en-service-monitor.h | 4 + ic/inc-proc-ic.c | 34 ++- ic/ovn-ic.c | 441 ----------------------------------- ic/ovn-ic.h | 2 - tests/ovn-ic.at | 76 ++++++ 7 files changed, 610 insertions(+), 454 deletions(-) diff --git a/NEWS b/NEWS index 860bd8191..49be2df5c 100644 --- a/NEWS +++ b/NEWS @@ -126,6 +126,13 @@ OVN v26.09.0 - xxx xx xxxx coalescing bursts of change-driven runs into fewer, larger runs. It defaults to 0, which keeps running as soon as there is work to do. A forced full recompute is never delayed. + - ovn-ic now processes database changes incrementally. Its processing + engine is split into one node per subsystem - availability zone, + gateways, transit switches and routers, datapath tunnel keys, port + bindings, routes, service monitors and address sets - and each node has + change handlers, so a change is reconciled in place instead of re-running + the whole ovn-ic computation. The "inc-engine/show-stats" command + reports the per-node recompute counters. OVN v26.03.0 - xxx xx xxxx -------------------------- diff --git a/ic/en-service-monitor.c b/ic/en-service-monitor.c index d0aab60bb..bd903e709 100644 --- a/ic/en-service-monitor.c +++ b/ic/en-service-monitor.c @@ -14,14 +14,452 @@ #include <config.h> -#include "en-service-monitor.h" #include "en-az.h" +#include "en-service-monitor.h" #include "lib/inc-proc-eng.h" +#include "lib/ovn-ic-sb-idl.h" +#include "lib/ovn-sb-idl.h" #include "openvswitch/vlog.h" #include "ovn-ic.h" +#include "ovsdb-idl.h" VLOG_DEFINE_THIS_MODULE(en_ic_service_monitor); +struct service_monitor_info { + struct hmap_node hmap_node; + union { + const struct sbrec_service_monitor *sb_rec; + const struct icsbrec_service_monitor *ic_rec; + } db_rec; + /* Destination availability zone name. */ + char *dst_az_name; + /* Source availability zone name. */ + char *src_az_name; + /* Chassis name associated with monitor logical port. */ + char *chassis_name; +}; + +struct sync_service_monitor_data { + /* Map of service monitors to be pushed to other AZs. */ + struct hmap pushed_svcs_map; + /* Map of service monitors synced from other AZs to our. */ + struct hmap synced_svcs_map; + /* Map of local service monitors in the ICSBDB. */ + struct hmap local_ic_svcs_map; + /* Map of local service monitors in SBDB. */ + struct hmap local_sb_svcs_map; + /* MAC address used for service monitor. */ + char *prpg_svc_monitor_mac; +}; + +static void +create_service_monitor_info(struct hmap *svc_map, + const void *db_rec, + const struct uuid *uuid, + const char *src_az_name, + const char *target_az_name, + const char *chassis_name, + bool ic_rec) +{ + struct service_monitor_info *svc_mon = xzalloc(sizeof(*svc_mon)); + size_t hash = uuid_hash(uuid); + + if (ic_rec) { + svc_mon->db_rec.ic_rec = + (const struct icsbrec_service_monitor *) db_rec; + } else { + svc_mon->db_rec.sb_rec = + (const struct sbrec_service_monitor *) db_rec; + } + + svc_mon->dst_az_name = target_az_name ? xstrdup(target_az_name) : NULL; + svc_mon->chassis_name = chassis_name ? xstrdup(chassis_name) : NULL; + svc_mon->src_az_name = xstrdup(src_az_name); + + hmap_insert(svc_map, &svc_mon->hmap_node, hash); +} + +static void +destroy_service_monitor_info(struct service_monitor_info *svc_mon) +{ + free(svc_mon->src_az_name); + free(svc_mon->dst_az_name); + free(svc_mon->chassis_name); + free(svc_mon); +} + +static void +refresh_sb_record_cache(struct hmap *svc_mon_map, + const struct sbrec_service_monitor *lookup_rec) +{ + size_t hash = uuid_hash(&lookup_rec->header_.uuid); + struct service_monitor_info *svc_mon; + + HMAP_FOR_EACH_WITH_HASH (svc_mon, hmap_node, hash, svc_mon_map) { + ovs_assert(svc_mon->db_rec.sb_rec); + if (svc_mon->db_rec.sb_rec == lookup_rec) { + hmap_remove(svc_mon_map, &svc_mon->hmap_node); + destroy_service_monitor_info(svc_mon); + return; + } + } +} + +static void +refresh_ic_record_cache(struct hmap *svc_mon_map, + const struct icsbrec_service_monitor *lookup_rec) +{ + size_t hash = uuid_hash(&lookup_rec->header_.uuid); + struct service_monitor_info *svc_mon; + + HMAP_FOR_EACH_WITH_HASH (svc_mon, hmap_node, hash, svc_mon_map) { + ovs_assert(svc_mon->db_rec.ic_rec); + if (svc_mon->db_rec.ic_rec == lookup_rec) { + hmap_remove(svc_mon_map, &svc_mon->hmap_node); + destroy_service_monitor_info(svc_mon); + return; + } + } +} + +static void +remove_unused_ic_records(struct hmap *local_ic_svcs_map) +{ + struct service_monitor_info *svc_mon; + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, local_ic_svcs_map) { + icsbrec_service_monitor_delete(svc_mon->db_rec.ic_rec); + destroy_service_monitor_info(svc_mon); + } + + hmap_destroy(local_ic_svcs_map); +} + +static void +remove_unused_sb_records(struct hmap *local_sb_svcs_map) +{ + struct service_monitor_info *svc_mon; + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, local_sb_svcs_map) { + sbrec_service_monitor_delete(svc_mon->db_rec.sb_rec); + destroy_service_monitor_info(svc_mon); + } + + hmap_destroy(local_sb_svcs_map); +} + +static void +create_pushed_svcs_mon(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + struct hmap *pushed_svcs_map) +{ + struct sbrec_service_monitor *key = + sbrec_service_monitor_index_init_row( + ctx->sbrec_service_monitor_by_remote_type); + + sbrec_service_monitor_index_set_remote(key, true); + + const struct sbrec_service_monitor *sb_rec; + SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, + ctx->sbrec_service_monitor_by_remote_type) { + const char *target_az_name = smap_get(&sb_rec->options, + "az-name"); + if (!target_az_name) { + continue; + } + create_service_monitor_info(pushed_svcs_map, sb_rec, + &sb_rec->header_.uuid, + runned_az->name, target_az_name, + NULL, false); + } + + sbrec_service_monitor_index_destroy_row(key); +} + +static void +create_synced_svcs_mon(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + struct hmap *synced_svcs_map) +{ + struct icsbrec_service_monitor *key = + icsbrec_service_monitor_index_init_row( + ctx->icsbrec_service_monitor_by_target_az); + + icsbrec_service_monitor_index_set_target_availability_zone( + key, runned_az->name); + + const struct icsbrec_service_monitor *ic_rec; + ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, + ctx->icsbrec_service_monitor_by_target_az) { + + const struct sbrec_port_binding *pb = + find_sb_pb_by_name(ctx->sbrec_port_binding_by_name, + ic_rec->logical_port); + + if (!pb || !pb->up) { + continue; + } + + const char *chassis_name = pb->chassis ? pb->chassis->name : NULL; + create_service_monitor_info(synced_svcs_map, ic_rec, + &ic_rec->header_.uuid, + runned_az->name, + NULL, chassis_name, true); + } + + icsbrec_service_monitor_index_destroy_row(key); +} + +static void +create_local_ic_svcs_map(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + struct hmap *owned_svc_map) +{ + struct icsbrec_service_monitor *key = + icsbrec_service_monitor_index_init_row( + ctx->icsbrec_service_monitor_by_source_az); + + icsbrec_service_monitor_index_set_source_availability_zone( + key, runned_az->name); + + const struct icsbrec_service_monitor *ic_rec; + ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, + ctx->icsbrec_service_monitor_by_source_az) { + create_service_monitor_info(owned_svc_map, ic_rec, + &ic_rec->header_.uuid, + runned_az->name, NULL, + NULL, true); + } + + icsbrec_service_monitor_index_destroy_row(key); +} + +static void +create_local_sb_svcs_map(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + struct hmap *owned_svc_map) +{ + struct sbrec_service_monitor *key = + sbrec_service_monitor_index_init_row( + ctx->sbrec_service_monitor_by_ic_learned); + + sbrec_service_monitor_index_set_ic_learned( + key, true); + + const struct sbrec_service_monitor *sb_rec; + SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, + ctx->sbrec_service_monitor_by_ic_learned) { + create_service_monitor_info(owned_svc_map, sb_rec, + &sb_rec->header_.uuid, + runned_az->name, NULL, + NULL, false); + } + + sbrec_service_monitor_index_destroy_row(key); +} + +static const struct sbrec_service_monitor * +lookup_sb_svc_rec(struct ic_context *ctx, + const struct service_monitor_info *svc_mon) +{ + const struct icsbrec_service_monitor *db_rec = + svc_mon->db_rec.ic_rec; + struct sbrec_service_monitor *key = + sbrec_service_monitor_index_init_row( + ctx->sbrec_service_monitor_by_remote_type_logical_port); + + sbrec_service_monitor_index_set_remote(key, false); + sbrec_service_monitor_index_set_logical_port(key, db_rec->logical_port); + + const struct sbrec_service_monitor *sb_rec; + SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, + ctx->sbrec_service_monitor_by_remote_type_logical_port) { + if (db_rec->port == sb_rec->port && + ((db_rec->type && sb_rec->type && + !strcmp(db_rec->type, sb_rec->type)) || + (!db_rec->type && !sb_rec->type)) && + !strcmp(db_rec->ip, sb_rec->ip) && + !strcmp(db_rec->src_ip, sb_rec->src_ip) && + !strcmp(db_rec->protocol, sb_rec->protocol)) { + sbrec_service_monitor_index_destroy_row(key); + return sb_rec; + } + } + + sbrec_service_monitor_index_destroy_row(key); + + return NULL; +} + +static const struct icsbrec_service_monitor * +lookup_icsb_svc_rec(struct ic_context *ctx, + const struct service_monitor_info *svc_mon) +{ + const struct sbrec_service_monitor *db_rec = + svc_mon->db_rec.sb_rec; + struct icsbrec_service_monitor *key = + icsbrec_service_monitor_index_init_row( + ctx->icsbrec_service_monitor_by_target_az_logical_port); + + ovs_assert(svc_mon->dst_az_name); + icsbrec_service_monitor_index_set_target_availability_zone( + key, svc_mon->dst_az_name); + + icsbrec_service_monitor_index_set_logical_port( + key, db_rec->logical_port); + + const struct icsbrec_service_monitor *ic_rec; + ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, + ctx->icsbrec_service_monitor_by_target_az_logical_port) { + if (db_rec->port == ic_rec->port && + ((db_rec->type && ic_rec->type && + !strcmp(db_rec->type, ic_rec->type)) || + (!db_rec->type && !ic_rec->type)) && + !strcmp(db_rec->ip, ic_rec->ip) && + !strcmp(db_rec->src_ip, ic_rec->src_ip) && + !strcmp(db_rec->protocol, ic_rec->protocol) && + !strcmp(db_rec->logical_port, ic_rec->logical_port)) { + icsbrec_service_monitor_index_destroy_row(key); + return ic_rec; + } + } + + icsbrec_service_monitor_index_destroy_row(key); + + return NULL; +} + +static void +create_service_monitor_data(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + struct sync_service_monitor_data *sync_data) +{ + const struct sbrec_sb_global *ic_sb = sbrec_sb_global_first( + ctx->ovnsb_idl); + const char *svc_monitor_mac = smap_get(&ic_sb->options, + "svc_monitor_mac"); + + if (!svc_monitor_mac) { + return; + } + + sync_data->prpg_svc_monitor_mac = xstrdup(svc_monitor_mac); + create_pushed_svcs_mon(ctx, runned_az, &sync_data->pushed_svcs_map); + create_synced_svcs_mon(ctx, runned_az, &sync_data->synced_svcs_map); + create_local_ic_svcs_map(ctx, runned_az, &sync_data->local_ic_svcs_map); + create_local_sb_svcs_map(ctx, runned_az, &sync_data->local_sb_svcs_map); +} + +static void +destroy_service_monitor_data(struct sync_service_monitor_data *sync_data) +{ + struct service_monitor_info *svc_mon; + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data->pushed_svcs_map) { + destroy_service_monitor_info(svc_mon); + } + + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data->synced_svcs_map) { + destroy_service_monitor_info(svc_mon); + } + + hmap_destroy(&sync_data->pushed_svcs_map); + hmap_destroy(&sync_data->synced_svcs_map); + free(sync_data->prpg_svc_monitor_mac); +} + +static void +sync_service_monitor(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az) +{ + if (!ctx->ovnisb_unlocked_txn || !ctx->ovnsb_txn) { + return; + } + + struct sync_service_monitor_data sync_data; + memset(&sync_data, 0, sizeof(sync_data)); + hmap_init(&sync_data.pushed_svcs_map); + hmap_init(&sync_data.synced_svcs_map); + hmap_init(&sync_data.local_ic_svcs_map); + hmap_init(&sync_data.local_sb_svcs_map); + + create_service_monitor_data(ctx, runned_az, &sync_data); + + struct service_monitor_info *svc_mon; + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data.pushed_svcs_map) { + const struct sbrec_service_monitor *db_rec = svc_mon->db_rec.sb_rec; + const struct icsbrec_service_monitor *ic_rec = + lookup_icsb_svc_rec(ctx, svc_mon); + + if (ic_rec) { + sbrec_service_monitor_set_status(db_rec, ic_rec->status); + } else { + ic_rec = icsbrec_service_monitor_insert(ctx->ovnisb_unlocked_txn); + icsbrec_service_monitor_set_type(ic_rec, db_rec->type); + icsbrec_service_monitor_set_ip(ic_rec, db_rec->ip); + icsbrec_service_monitor_set_port(ic_rec, db_rec->port); + icsbrec_service_monitor_set_src_ip(ic_rec, db_rec->src_ip); + icsbrec_service_monitor_set_src_mac(ic_rec, + sync_data.prpg_svc_monitor_mac); + icsbrec_service_monitor_set_protocol(ic_rec, db_rec->protocol); + icsbrec_service_monitor_set_logical_port(ic_rec, + db_rec->logical_port); + icsbrec_service_monitor_set_target_availability_zone(ic_rec, + svc_mon->dst_az_name); + icsbrec_service_monitor_set_source_availability_zone(ic_rec, + svc_mon->src_az_name); + } + + /* Always update options because they change from NB. */ + icsbrec_service_monitor_set_options(ic_rec, &db_rec->options); + refresh_ic_record_cache(&sync_data.local_ic_svcs_map, ic_rec); + } + + HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data.synced_svcs_map) { + const struct icsbrec_service_monitor *db_rec = + svc_mon->db_rec.ic_rec; + const struct sbrec_service_monitor *sb_rec = + lookup_sb_svc_rec(ctx, svc_mon); + + if (sb_rec) { + icsbrec_service_monitor_set_status(svc_mon->db_rec.ic_rec, + sb_rec->status); + } else { + sb_rec = sbrec_service_monitor_insert(ctx->ovnsb_txn); + sbrec_service_monitor_set_type(sb_rec, db_rec->type); + sbrec_service_monitor_set_ip(sb_rec, db_rec->ip); + sbrec_service_monitor_set_port(sb_rec, db_rec->port); + sbrec_service_monitor_set_src_ip(sb_rec, db_rec->src_ip); + /* Set svc_monitor_mac from local SBDB. */ + sbrec_service_monitor_set_src_mac(sb_rec, + sync_data.prpg_svc_monitor_mac); + sbrec_service_monitor_set_protocol(sb_rec, + db_rec->protocol); + sbrec_service_monitor_set_logical_port(sb_rec, + db_rec->logical_port); + sbrec_service_monitor_set_remote(sb_rec, false); + sbrec_service_monitor_set_ic_learned(sb_rec, true); + } + + /* Only update if ic owns it */ + if (sb_rec->ic_learned) { + /* Always update options since they may change via + * NB configuration. Also update chassis_name if + * the port has been reassigned to a different chassis. + */ + if (svc_mon->chassis_name) { + sbrec_service_monitor_set_chassis_name(sb_rec, + svc_mon->chassis_name); + } + sbrec_service_monitor_set_options(sb_rec, &db_rec->options); + refresh_sb_record_cache(&sync_data.local_sb_svcs_map, sb_rec); + } + } + + /* Delete local created records that are no longer used. */ + remove_unused_ic_records(&sync_data.local_ic_svcs_map); + remove_unused_sb_records(&sync_data.local_sb_svcs_map); + + destroy_service_monitor_data(&sync_data); +} + enum engine_node_state en_service_monitor_run(struct engine_node *node, void *data OVS_UNUSED) @@ -41,6 +479,66 @@ en_service_monitor_run(struct engine_node *node, return EN_UPDATED; } +/* True if 'logical_port' backs a service monitor targeting this AZ - i.e. a + * port whose 'up'/'chassis' state create_synced_svcs_mon() reads. */ +static bool +svc_monitor_target_lport(struct ic_context *ctx, + const struct icsbrec_availability_zone *runned_az, + const char *logical_port) +{ + struct icsbrec_service_monitor *key = + icsbrec_service_monitor_index_init_row( + ctx->icsbrec_service_monitor_by_target_az_logical_port); + icsbrec_service_monitor_index_set_target_availability_zone( + key, runned_az->name); + icsbrec_service_monitor_index_set_logical_port(key, logical_port); + const struct icsbrec_service_monitor *ic_rec = + icsbrec_service_monitor_index_find( + ctx->icsbrec_service_monitor_by_target_az_logical_port, key); + icsbrec_service_monitor_index_destroy_row(key); + return ic_rec != NULL; +} + +/* SB Port_Binding: sync_service_monitor() reads a port binding only through + * create_synced_svcs_mon(), which - for each IC-SB service monitor targeting + * this AZ - looks up the backing SB port binding by logical_port and reads its + * 'up' and 'chassis'. So a port-binding change matters only when it is + * new/deleted or its 'up'/'chassis' changed AND the port backs such a service + * monitor. Every other SB port-binding change (other columns, or ports that + * are not service-monitor backends - the vast majority of SB churn) is a + * no-op, instead of forcing a full recompute of this node. */ +enum engine_input_handler_result +en_service_monitor_sb_port_binding_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct ic_context *ctx = eng_ctx->client_ctx; + const struct ed_type_az *az = engine_get_input_data("az", node); + + if (!az->runned_az) { + return EN_HANDLED_UNCHANGED; + } + + const struct sbrec_port_binding_table *tbl = + EN_OVSDB_GET(engine_get_input("SB_port_binding", node)); + const struct sbrec_port_binding *pb; + SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (pb, tbl) { + if (!sbrec_port_binding_is_new(pb) && + !sbrec_port_binding_is_deleted(pb) && + !ovsdb_idl_track_is_updated(&pb->header_, + &sbrec_port_binding_col_up) && + !ovsdb_idl_track_is_updated(&pb->header_, + &sbrec_port_binding_col_chassis)) { + continue; + } + if (svc_monitor_target_lport(ctx, az->runned_az, pb->logical_port)) { + return EN_UNHANDLED; + } + } + + return EN_HANDLED_UNCHANGED; +} + void * en_service_monitor_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/ic/en-service-monitor.h b/ic/en-service-monitor.h index 55103635e..ff6cc9c6f 100644 --- a/ic/en-service-monitor.h +++ b/ic/en-service-monitor.h @@ -9,4 +9,8 @@ void *en_service_monitor_init(struct engine_node *node, struct engine_arg *arg); void en_service_monitor_cleanup(void *data); +enum engine_input_handler_result +en_service_monitor_sb_port_binding_handler(struct engine_node *node, + void *data); + #endif /* EN_IC_SERVICE_MONITOR_H */ diff --git a/ic/inc-proc-ic.c b/ic/inc-proc-ic.c index 92b368153..af0060ae5 100644 --- a/ic/inc-proc-ic.c +++ b/ic/inc-proc-ic.c @@ -118,12 +118,14 @@ VLOG_DEFINE_THIS_MODULE(inc_proc_ic); ICNB_NODES #undef ICNB_NODE -/* Note: the ic_sb_global table is intentionally not modeled as an engine input - * node. It only carries IC-SB sequence numbers, which are written by - * update_sequence_numbers() in the main loop (outside the engine) and are not - * read by any subsystem node. */ +/* Note: the ic_sb_global and availability_zone tables are intentionally not + * modeled as engine input nodes. ic_sb_global only carries IC-SB sequence + * numbers, written by update_sequence_numbers() in the main loop (outside the + * engine). availability_zone is consumed by the en_az node, which reads it + * directly every iteration; the subsystem nodes depend on en_az for the AZ + * identity rather than on the (sequence-number-bumped) Availability_Zone + * table. */ #define ICSB_NODES \ - ICSB_NODE(availability_zone, "availability_zone") \ ICSB_NODE(service_monitor, "service_monitor") \ ICSB_NODE(route, "route") \ ICSB_NODE(datapath_binding, "datapath_binding") \ @@ -351,13 +353,23 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_route, &en_sb_learned_route, route_sb_learned_route_handler); - /* en_service_monitor: sync load-balancer health checks across AZs. */ + /* en_service_monitor: sync load-balancer health checks across AZs. + * + * Like the other AZ-scoped nodes it uses only the AZ identity + * (en_az's resolved AZ name and the by-source/target-AZ indexes) + * and does not read the Availability_Zone table, so it does not depend on + * en_icsb_availability_zone and is not churned by its nb_ic_cfg sequence + * number. */ engine_add_input(&en_service_monitor, &en_az, NULL); - engine_add_input(&en_service_monitor, &en_icsb_availability_zone, NULL); engine_add_input(&en_service_monitor, &en_icsb_service_monitor, NULL); engine_add_input(&en_service_monitor, &en_sb_sb_global, NULL); engine_add_input(&en_service_monitor, &en_sb_service_monitor, NULL); - engine_add_input(&en_service_monitor, &en_sb_port_binding, NULL); + /* SB port bindings are the busiest table in the AZ, but the sync reads + * only the 'up'/'chassis' of ports backing a service monitor targeting + * this AZ, so a dedicated handler scopes that churn out instead of forcing + * a full recompute on every port-binding change. */ + engine_add_input(&en_service_monitor, &en_sb_port_binding, + en_service_monitor_sb_port_binding_handler); /* en_address_set: advertise/learn address sets across AZs. * @@ -365,9 +377,11 @@ void inc_proc_ic_init(struct ovsdb_idl_loop *nb, * address sets into local/remote by comparing their availability_zone * against this instance's AZ, so it depends on en_az (which reports * EN_UPDATED only when the AZ identity changes) to be re-run when the - * resolved AZ changes. */ + * resolved AZ changes. It uses only that AZ identity and does not read + * the Availability_Zone table, so it does not depend on + * en_icsb_availability_zone and is not churned by its nb_ic_cfg sequence + * number. */ engine_add_input(&en_address_set, &en_az, NULL); - engine_add_input(&en_address_set, &en_icsb_availability_zone, NULL); engine_add_input(&en_address_set, &en_nb_nb_global, NULL); engine_add_input(&en_address_set, &en_nb_address_set, NULL); engine_add_input(&en_address_set, &en_sb_address_set, NULL); diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index a73a30723..6730c4c79 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -595,447 +595,6 @@ port_binding_collect_lsp_ts(const struct shash *lsp_ts_map, } } -/* - * Data structures and functions related to - * synchronize health checks for load balancers - * between availability zones. - */ -struct sync_service_monitor_data { - /* Map of service monitors to be pushed to other AZs. */ - struct hmap pushed_svcs_map; - /* Map of service monitors synced from other AZs to our. */ - struct hmap synced_svcs_map; - /* Map of local service monitors in the ICSBDB. */ - struct hmap local_ic_svcs_map; - /* Map of local service monitors in SBDB. */ - struct hmap local_sb_svcs_map; - /* MAC address used for service monitor. */ - char *prpg_svc_monitor_mac; -}; - -struct service_monitor_info { - struct hmap_node hmap_node; - union { - const struct sbrec_service_monitor *sb_rec; - const struct icsbrec_service_monitor *ic_rec; - } db_rec; - /* Destination availability zone name. */ - char *dst_az_name; - /* Source availability zone name. */ - char *src_az_name; - /* Chassis name associated with monitor logical port. */ - char *chassis_name; -}; - -static void -create_service_monitor_info(struct hmap *svc_map, - const void *db_rec, - const struct uuid *uuid, - const char *src_az_name, - const char *target_az_name, - const char *chassis_name, - bool ic_rec) -{ - struct service_monitor_info *svc_mon = xzalloc(sizeof(*svc_mon)); - size_t hash = uuid_hash(uuid); - - if (ic_rec) { - svc_mon->db_rec.ic_rec = - (const struct icsbrec_service_monitor *) db_rec; - } else { - svc_mon->db_rec.sb_rec = - (const struct sbrec_service_monitor *) db_rec; - } - - svc_mon->dst_az_name = target_az_name ? xstrdup(target_az_name) : NULL; - svc_mon->chassis_name = chassis_name ? xstrdup(chassis_name) : NULL; - svc_mon->src_az_name = xstrdup(src_az_name); - - hmap_insert(svc_map, &svc_mon->hmap_node, hash); -} - -static void -destroy_service_monitor_info(struct service_monitor_info *svc_mon) -{ - free(svc_mon->src_az_name); - free(svc_mon->dst_az_name); - free(svc_mon->chassis_name); - free(svc_mon); -} - -static void -refresh_sb_record_cache(struct hmap *svc_mon_map, - const struct sbrec_service_monitor *lookup_rec) -{ - size_t hash = uuid_hash(&lookup_rec->header_.uuid); - struct service_monitor_info *svc_mon; - - HMAP_FOR_EACH_WITH_HASH (svc_mon, hmap_node, hash, svc_mon_map) { - ovs_assert(svc_mon->db_rec.sb_rec); - if (svc_mon->db_rec.sb_rec == lookup_rec) { - hmap_remove(svc_mon_map, &svc_mon->hmap_node); - destroy_service_monitor_info(svc_mon); - return; - } - } -} - -static void -refresh_ic_record_cache(struct hmap *svc_mon_map, - const struct icsbrec_service_monitor *lookup_rec) -{ - size_t hash = uuid_hash(&lookup_rec->header_.uuid); - struct service_monitor_info *svc_mon; - - HMAP_FOR_EACH_WITH_HASH (svc_mon, hmap_node, hash, svc_mon_map) { - ovs_assert(svc_mon->db_rec.ic_rec); - if (svc_mon->db_rec.ic_rec == lookup_rec) { - hmap_remove(svc_mon_map, &svc_mon->hmap_node); - destroy_service_monitor_info(svc_mon); - return; - } - } -} - -static void -remove_unused_ic_records(struct hmap *local_ic_svcs_map) -{ - struct service_monitor_info *svc_mon; - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, local_ic_svcs_map) { - icsbrec_service_monitor_delete(svc_mon->db_rec.ic_rec); - destroy_service_monitor_info(svc_mon); - } - - hmap_destroy(local_ic_svcs_map); -} - -static void -remove_unused_sb_records(struct hmap *local_sb_svcs_map) -{ - struct service_monitor_info *svc_mon; - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, local_sb_svcs_map) { - sbrec_service_monitor_delete(svc_mon->db_rec.sb_rec); - destroy_service_monitor_info(svc_mon); - } - - hmap_destroy(local_sb_svcs_map); -} - -static void -create_pushed_svcs_mon(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az, - struct hmap *pushed_svcs_map) -{ - struct sbrec_service_monitor *key = - sbrec_service_monitor_index_init_row( - ctx->sbrec_service_monitor_by_remote_type); - - sbrec_service_monitor_index_set_remote(key, true); - - const struct sbrec_service_monitor *sb_rec; - SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, - ctx->sbrec_service_monitor_by_remote_type) { - const char *target_az_name = smap_get(&sb_rec->options, - "az-name"); - if (!target_az_name) { - continue; - } - create_service_monitor_info(pushed_svcs_map, sb_rec, - &sb_rec->header_.uuid, - runned_az->name, - target_az_name, NULL, false); - } - - sbrec_service_monitor_index_destroy_row(key); -} - -static void -create_synced_svcs_mon(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az, - struct hmap *synced_svcs_map) -{ - struct icsbrec_service_monitor *key = - icsbrec_service_monitor_index_init_row( - ctx->icsbrec_service_monitor_by_target_az); - - icsbrec_service_monitor_index_set_target_availability_zone( - key, runned_az->name); - - const struct icsbrec_service_monitor *ic_rec; - ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, - ctx->icsbrec_service_monitor_by_target_az) { - - const struct sbrec_port_binding *pb = - find_sb_pb_by_name(ctx->sbrec_port_binding_by_name, - ic_rec->logical_port); - - if (!pb || !pb->up) { - continue; - } - - const char *chassis_name = pb->chassis ? pb->chassis->name : NULL; - create_service_monitor_info(synced_svcs_map, ic_rec, - &ic_rec->header_.uuid, - runned_az->name, - NULL, chassis_name, true); - } - - icsbrec_service_monitor_index_destroy_row(key); -} - -static void -create_local_ic_svcs_map(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az, - struct hmap *owned_svc_map) -{ - struct icsbrec_service_monitor *key = - icsbrec_service_monitor_index_init_row( - ctx->icsbrec_service_monitor_by_source_az); - - icsbrec_service_monitor_index_set_source_availability_zone( - key, runned_az->name); - - const struct icsbrec_service_monitor *ic_rec; - ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, - ctx->icsbrec_service_monitor_by_source_az) { - create_service_monitor_info(owned_svc_map, ic_rec, - &ic_rec->header_.uuid, - runned_az->name, NULL, - NULL, true); - } - - icsbrec_service_monitor_index_destroy_row(key); -} - -static void -create_local_sb_svcs_map(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az, - struct hmap *owned_svc_map) -{ - struct sbrec_service_monitor *key = - sbrec_service_monitor_index_init_row( - ctx->sbrec_service_monitor_by_ic_learned); - - sbrec_service_monitor_index_set_ic_learned( - key, true); - - const struct sbrec_service_monitor *sb_rec; - SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, - ctx->sbrec_service_monitor_by_ic_learned) { - create_service_monitor_info(owned_svc_map, sb_rec, - &sb_rec->header_.uuid, - runned_az->name, NULL, - NULL, false); - } - - sbrec_service_monitor_index_destroy_row(key); -} - -static const struct sbrec_service_monitor * -lookup_sb_svc_rec(struct ic_context *ctx, - const struct service_monitor_info *svc_mon) -{ - const struct icsbrec_service_monitor *db_rec = - svc_mon->db_rec.ic_rec; - struct sbrec_service_monitor *key = - sbrec_service_monitor_index_init_row( - ctx->sbrec_service_monitor_by_remote_type_logical_port); - - sbrec_service_monitor_index_set_remote(key, false); - sbrec_service_monitor_index_set_logical_port(key, db_rec->logical_port); - - const struct sbrec_service_monitor *sb_rec; - SBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (sb_rec, key, - ctx->sbrec_service_monitor_by_remote_type_logical_port) { - if (db_rec->port == sb_rec->port && - ((db_rec->type && sb_rec->type && - !strcmp(db_rec->type, sb_rec->type)) || - (!db_rec->type && !sb_rec->type)) && - !strcmp(db_rec->ip, sb_rec->ip) && - !strcmp(db_rec->src_ip, sb_rec->src_ip) && - !strcmp(db_rec->protocol, sb_rec->protocol)) { - sbrec_service_monitor_index_destroy_row(key); - return sb_rec; - } - } - - sbrec_service_monitor_index_destroy_row(key); - - return NULL; -} - -static const struct icsbrec_service_monitor * -lookup_icsb_svc_rec(struct ic_context *ctx, - const struct service_monitor_info *svc_mon) -{ - const struct sbrec_service_monitor *db_rec = - svc_mon->db_rec.sb_rec; - struct icsbrec_service_monitor *key = - icsbrec_service_monitor_index_init_row( - ctx->icsbrec_service_monitor_by_target_az_logical_port); - - ovs_assert(svc_mon->dst_az_name); - icsbrec_service_monitor_index_set_target_availability_zone( - key, svc_mon->dst_az_name); - - icsbrec_service_monitor_index_set_logical_port( - key, db_rec->logical_port); - - const struct icsbrec_service_monitor *ic_rec; - ICSBREC_SERVICE_MONITOR_FOR_EACH_EQUAL (ic_rec, key, - ctx->icsbrec_service_monitor_by_target_az_logical_port) { - if (db_rec->port == ic_rec->port && - ((db_rec->type && ic_rec->type && - !strcmp(db_rec->type, ic_rec->type)) || - (!db_rec->type && !ic_rec->type)) && - !strcmp(db_rec->ip, ic_rec->ip) && - !strcmp(db_rec->src_ip, ic_rec->src_ip) && - !strcmp(db_rec->protocol, ic_rec->protocol) && - !strcmp(db_rec->logical_port, ic_rec->logical_port)) { - icsbrec_service_monitor_index_destroy_row(key); - return ic_rec; - } - } - - icsbrec_service_monitor_index_destroy_row(key); - - return NULL; -} - -static void -create_service_monitor_data(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az, - struct sync_service_monitor_data *sync_data) -{ - const struct sbrec_sb_global *ic_sb = sbrec_sb_global_first( - ctx->ovnsb_idl); - const char *svc_monitor_mac = smap_get(&ic_sb->options, - "svc_monitor_mac"); - - if (!svc_monitor_mac) { - return; - } - - sync_data->prpg_svc_monitor_mac = xstrdup(svc_monitor_mac); - create_pushed_svcs_mon(ctx, runned_az, &sync_data->pushed_svcs_map); - create_synced_svcs_mon(ctx, runned_az, &sync_data->synced_svcs_map); - create_local_ic_svcs_map(ctx, runned_az, &sync_data->local_ic_svcs_map); - create_local_sb_svcs_map(ctx, runned_az, - &sync_data->local_sb_svcs_map); -} - -static void -destroy_service_monitor_data(struct sync_service_monitor_data *sync_data) -{ - struct service_monitor_info *svc_mon; - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data->pushed_svcs_map) { - destroy_service_monitor_info(svc_mon); - } - - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data->synced_svcs_map) { - destroy_service_monitor_info(svc_mon); - } - - hmap_destroy(&sync_data->pushed_svcs_map); - hmap_destroy(&sync_data->synced_svcs_map); - free(sync_data->prpg_svc_monitor_mac); -} - -void -sync_service_monitor(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az) -{ - if (!ctx->ovnisb_unlocked_txn || !ctx->ovnsb_txn) { - return; - } - - struct sync_service_monitor_data sync_data; - memset(&sync_data, 0, sizeof(sync_data)); - hmap_init(&sync_data.pushed_svcs_map); - hmap_init(&sync_data.synced_svcs_map); - hmap_init(&sync_data.local_ic_svcs_map); - hmap_init(&sync_data.local_sb_svcs_map); - - create_service_monitor_data(ctx, runned_az, &sync_data); - - struct service_monitor_info *svc_mon; - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data.pushed_svcs_map) { - const struct sbrec_service_monitor *db_rec = svc_mon->db_rec.sb_rec; - const struct icsbrec_service_monitor *ic_rec = - lookup_icsb_svc_rec(ctx, svc_mon); - - if (ic_rec) { - sbrec_service_monitor_set_status(db_rec, ic_rec->status); - } else { - ic_rec = icsbrec_service_monitor_insert(ctx->ovnisb_unlocked_txn); - icsbrec_service_monitor_set_type(ic_rec, db_rec->type); - icsbrec_service_monitor_set_ip(ic_rec, db_rec->ip); - icsbrec_service_monitor_set_port(ic_rec, db_rec->port); - icsbrec_service_monitor_set_src_ip(ic_rec, db_rec->src_ip); - icsbrec_service_monitor_set_src_mac(ic_rec, - sync_data.prpg_svc_monitor_mac); - icsbrec_service_monitor_set_protocol(ic_rec, db_rec->protocol); - icsbrec_service_monitor_set_logical_port(ic_rec, - db_rec->logical_port); - icsbrec_service_monitor_set_target_availability_zone(ic_rec, - svc_mon->dst_az_name); - icsbrec_service_monitor_set_source_availability_zone(ic_rec, - svc_mon->src_az_name); - } - - /* Always update options because they change from NB. */ - icsbrec_service_monitor_set_options(ic_rec, &db_rec->options); - refresh_ic_record_cache(&sync_data.local_ic_svcs_map, ic_rec); - } - - HMAP_FOR_EACH_SAFE (svc_mon, hmap_node, &sync_data.synced_svcs_map) { - const struct icsbrec_service_monitor *db_rec = - svc_mon->db_rec.ic_rec; - const struct sbrec_service_monitor *sb_rec = - lookup_sb_svc_rec(ctx, svc_mon); - - if (sb_rec) { - icsbrec_service_monitor_set_status(svc_mon->db_rec.ic_rec, - sb_rec->status); - } else { - sb_rec = sbrec_service_monitor_insert(ctx->ovnsb_txn); - sbrec_service_monitor_set_type(sb_rec, db_rec->type); - sbrec_service_monitor_set_ip(sb_rec, db_rec->ip); - sbrec_service_monitor_set_port(sb_rec, db_rec->port); - sbrec_service_monitor_set_src_ip(sb_rec, db_rec->src_ip); - /* Set svc_monitor_mac from local SBDB. */ - sbrec_service_monitor_set_src_mac(sb_rec, - sync_data.prpg_svc_monitor_mac); - sbrec_service_monitor_set_protocol(sb_rec, - db_rec->protocol); - sbrec_service_monitor_set_logical_port(sb_rec, - db_rec->logical_port); - sbrec_service_monitor_set_remote(sb_rec, false); - sbrec_service_monitor_set_ic_learned(sb_rec, true); - } - - /* Only update if ic owns it */ - if (sb_rec->ic_learned) { - /* Always update options since they may change via - * NB configuration. Also update chassis_name if - * the port has been reassigned to a different chassis. - */ - if (svc_mon->chassis_name) { - sbrec_service_monitor_set_chassis_name(sb_rec, - svc_mon->chassis_name); - } - sbrec_service_monitor_set_options(sb_rec, &db_rec->options); - refresh_sb_record_cache(&sync_data.local_sb_svcs_map, sb_rec); - } - } - - /* Delete local created records that are no longer used. */ - remove_unused_ic_records(&sync_data.local_ic_svcs_map); - remove_unused_sb_records(&sync_data.local_sb_svcs_map); - - destroy_service_monitor_data(&sync_data); -} - /* * This function implements a sequence number protocol that can be used by * the INB end user to verify that ISB is synced with all the changes that diff --git a/ic/ovn-ic.h b/ic/ovn-ic.h index 44774ba81..f98d338db 100644 --- a/ic/ovn-ic.h +++ b/ic/ovn-ic.h @@ -100,8 +100,6 @@ void port_binding_lsp_ts_map_init(struct ic_context *ctx, struct shash *map); void port_binding_collect_lsp_ts(const struct shash *lsp_ts_map, const struct nbrec_logical_switch_port *lsp, struct sset *ts_scope); -void sync_service_monitor(struct ic_context *ctx, - const struct icsbrec_availability_zone *runned_az); /* Shared IC helpers used by more than one engine node. */ uint32_t diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 6704fba4c..0adcf360b 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -996,6 +996,82 @@ OVN_CLEANUP_IC([az1]) AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- incremental processing - port binding, route, svc monitor]) + +# The en_port_binding, en_route and en_service_monitor nodes use only the AZ +# identity, not the Availability_Zone table, so a change to IC-NB Global +# options and the resulting nb_ic_cfg sequence-number bump must not recompute +# them. + +ovn_init_ic_db +net_add n1 +ovn_start az1 +as az1 +check ovn-ic-nbctl --wait=sb ts-add ts1 +OVS_WAIT_UNTIL([test "$(ovn-nbctl --bare --columns=name \ + find logical_switch name=ts1)" = ts1]) + +check ovn-ic-nbctl --wait=sb sync +check ovn-appctl -t ic/ovn-ic inc-engine/recompute +check ovn-ic-nbctl --wait=sb sync +check ovn-appctl -t ic/ovn-ic inc-engine/clear-stats + +check ovn-ic-nbctl --wait=sb set IC_NB_Global . options:foo=bar +check ovn-ic-nbctl --wait=sb sync + +AT_CHECK([test "$(ovn-appctl -t ic/ovn-ic \ + inc-engine/show-stats port_binding recompute)" = 0]) +AT_CHECK([test "$(ovn-appctl -t ic/ovn-ic \ + inc-engine/show-stats route recompute)" = 0]) +AT_CHECK([test "$(ovn-appctl -t ic/ovn-ic \ + inc-engine/show-stats service_monitor recompute)" = 0]) + +OVN_CLEANUP_IC([az1]) +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- incremental processing - service monitor SB port binding scoping]) + +# SB Port_Binding is the busiest table in the AZ, but the en_service_monitor +# sync reads only the 'up'/'chassis' of the port backing a service monitor +# targeting this AZ. A port binding that backs no such service monitor must be +# handled by the en_service_monitor SB Port_Binding handler without a full +# recompute of the node. + +ovn_init_ic_db +net_add n1 +ovn_start az1 +as az1 +check ovn-ic-nbctl --wait=sb ts-add ts1 +check ovn-nbctl ls-add sw0 +check ovn-nbctl --wait=sb lsp-add sw0 sp0 +OVS_WAIT_UNTIL([test -n "$(ovn-sbctl --bare --columns=_uuid \ + find port_binding logical_port=sp0)"]) + +# Quiesce the engine and reset stats so only the change under test counts. +check ovn-ic-nbctl --wait=sb sync +check ovn-appctl -t ic/ovn-ic inc-engine/recompute +check ovn-ic-nbctl --wait=sb sync +check ovn-appctl -t ic/ovn-ic inc-engine/clear-stats + +# Flip the SB port binding 'up' of a port that backs no service monitor, +# directly in the SB (so SB_Global is not bumped). This is a watched column, +# but the port is not a service-monitor target, so en_service_monitor must not +# recompute. +check ovn-sbctl set port_binding sp0 up=true +OVS_WAIT_UNTIL([test "$(ovn-sbctl --bare --columns=up \ + find port_binding logical_port=sp0)" = true]) +check ovn-ic-nbctl --wait=sb sync + +AT_CHECK([test "$(ovn-appctl -t ic/ovn-ic \ + inc-engine/show-stats service_monitor recompute)" = 0]) + +OVN_CLEANUP_IC([az1]) +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD([ AT_SETUP([ovn-ic -- incremental processing - route handlers]) -- 2.34.1 -- _'Esta mensagem é direcionada apenas para os endereços constantes no cabeçalho inicial. Se você não está listado nos endereços constantes no cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão imediatamente anuladas e proibidas'._ * **'Apesar do Magazine Luiza tomar todas as precauções razoáveis para assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos'.* _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
