For logical switches that are EVPN-enabled (with "dynamic-routing-vni" set), automatically learn IP MAC bindings that were advertised by remote EVPN peers as Type-2 MAC+IP routes.
As with FDB, these are learned by monitoring through NetLink the VRF linux bridge associated to the EVPN logical switch. The IP <-> MAC mappings are stored in memory in ovn-controller. The OpenFlow pipelines of the adjacent logical routers are automatically updated by ovn-controller so that the neighbor entries are handled as if they were learned as regular Southbound MAC_Bindings. NOTE: By default the EVPN learned neighbors have a higher priority than the ones learned through the SB MAC_Binding table. This can be tweaked by setting the new Logical_Switch "dynamic-routing-arp-prefer-local" option to "true" (default "false"). If set to true, SB MAC_Binding records have precedence. Reported-at: https://issues.redhat.com/browse/FDP-1388 Signed-off-by: Dumitru Ceara <[email protected]> --- NEWS | 8 ++ controller/automake.mk | 2 + controller/evpn-arp.c | 174 +++++++++++++++++++++++++ controller/evpn-arp.h | 65 +++++++++ controller/evpn-fdb.c | 2 +- controller/evpn-fdb.h | 2 +- controller/neighbor-exchange-netlink.c | 9 ++ controller/neighbor-exchange-netlink.h | 1 + controller/neighbor-exchange-stub.c | 2 +- controller/neighbor-exchange.c | 91 ++++++++----- controller/neighbor-exchange.h | 12 +- controller/neighbor-of.h | 11 +- controller/ovn-controller.c | 132 ++++++++++++++++++- controller/physical.c | 53 +++++++- controller/physical.h | 5 + northd/en-datapath-logical-switch.c | 7 + ovn-nb.xml | 23 ++++ tests/multinode.at | 104 +++++++++++++-- tests/system-ovn.at | 147 +++++++++++++++++++++ 19 files changed, 792 insertions(+), 58 deletions(-) create mode 100644 controller/evpn-arp.c create mode 100644 controller/evpn-arp.h diff --git a/NEWS b/NEWS index dc5d49a94d..f9ad8ae75e 100644 --- a/NEWS +++ b/NEWS @@ -6,6 +6,14 @@ Post v25.09.0 - Added "ic-route-deny-adv" and "ic-route-deny-learn" options to the Logical_Router/Logical_Router_Port tables to allow users to deny filter advertised/learned IC routes. + - Dynamic Routing: + * Extend the Logical Switch EVPN support to now automatically learn + IP neighbors (Type-2 MAC+IP EVPN routes) and automatically inject them + into the pipelines of the adjacent logical routers. + * Add the "other_config:dynamic-routing-arp-prefer-local" to Logical + Switches. If set to "true" ovn-controller will give preference to SB + (Static_)MAC_Bindings of adjacent logical routers over ARPs learned + through EVPN on the switch. OVN v25.09.0 - xxx xx xxxx -------------------------- diff --git a/controller/automake.mk b/controller/automake.mk index b3c6293fa9..fb27f2ae98 100644 --- a/controller/automake.mk +++ b/controller/automake.mk @@ -10,6 +10,8 @@ controller_ovn_controller_SOURCES = \ controller/chassis.h \ controller/encaps.c \ controller/encaps.h \ + controller/evpn-arp.c \ + controller/evpn-arp.h \ controller/evpn-binding.c \ controller/evpn-binding.h \ controller/evpn-fdb.c \ diff --git a/controller/evpn-arp.c b/controller/evpn-arp.c new file mode 100644 index 0000000000..f73e09504a --- /dev/null +++ b/controller/evpn-arp.c @@ -0,0 +1,174 @@ +/* Copyright (c) 2025, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include <config.h> + +#include "evpn-binding.h" +#include "neighbor-exchange.h" +#include "openvswitch/dynamic-string.h" +#include "openvswitch/vlog.h" +#include "ovn-sb-idl.h" +#include "packets.h" +#include "unixctl.h" + +#include "evpn-arp.h" + +VLOG_DEFINE_THIS_MODULE(evpn_arp); + +static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + +static struct evpn_arp *evpn_arp_add(struct hmap *evpn_arps, struct eth_addr, + const struct in6_addr *, uint32_t vni); +static struct evpn_arp *evpn_arp_find(const struct hmap *evpn_arps, + struct eth_addr, + const struct in6_addr *, + uint32_t vni); + +void +evpn_arp_run(const struct evpn_arp_ctx_in *arp_ctx_in, + struct evpn_arp_ctx_out *arp_ctx_out) +{ + struct hmapx stale_arps = HMAPX_INITIALIZER(&stale_arps); + + struct evpn_arp *arp; + HMAP_FOR_EACH (arp, hmap_node, arp_ctx_out->arps) { + hmapx_add(&stale_arps, arp); + } + + const struct evpn_static_entry *static_arp; + HMAP_FOR_EACH (static_arp, hmap_node, arp_ctx_in->static_arps) { + const struct evpn_datapath *edp = + evpn_datapath_find(arp_ctx_in->datapaths, static_arp->vni); + if (!edp) { + char addr_s[INET6_ADDRSTRLEN + 1]; + VLOG_WARN_RL(&rl, "Couldn't find EVPN datapath for ARP entry: " + "VNI: %"PRIu32" MAC: "ETH_ADDR_FMT" IP: %s.", + static_arp->vni, ETH_ADDR_ARGS(static_arp->mac), + ipv6_string_mapped(addr_s, &static_arp->ip) + ? addr_s : "(invalid)"); + continue; + } + + arp = evpn_arp_find(arp_ctx_out->arps, static_arp->mac, + &static_arp->ip, static_arp->vni); + if (!arp) { + arp = evpn_arp_add(arp_ctx_out->arps, static_arp->mac, + &static_arp->ip, static_arp->vni); + } + + bool updated = false; + if (arp->ldp != edp->ldp) { + arp->ldp = edp->ldp; + updated = true; + } + + enum neigh_of_rule_prio priority = + smap_get_bool(&arp->ldp->datapath->external_ids, + "dynamic-routing-arp-prefer-local", + false) + ? NEIGH_OF_EVPN_MAC_BINDING_LOW_PRIO + : NEIGH_OF_EVPN_MAC_BINDING_HIGH_PRIO; + if (arp->priority != priority) { + arp->priority = priority; + updated = true; + } + + if (updated) { + hmapx_add(arp_ctx_out->updated_arps, arp); + } + + hmapx_find_and_delete(&stale_arps, arp); + } + + struct hmapx_node *node; + HMAPX_FOR_EACH (node, &stale_arps) { + arp = node->data; + + uuidset_insert(arp_ctx_out->removed_arps, &arp->flow_uuid); + hmap_remove(arp_ctx_out->arps, &arp->hmap_node); + free(arp); + } + + hmapx_destroy(&stale_arps); +} + +void +evpn_arps_destroy(struct hmap *arps) +{ + struct evpn_arp *arp; + HMAP_FOR_EACH_POP (arp, hmap_node, arps) { + free(arp); + } + hmap_destroy(arps); +} + +void +evpn_arp_list(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *data_) +{ + struct hmap *arps = data_; + struct ds ds = DS_EMPTY_INITIALIZER; + + const struct evpn_arp *arp; + HMAP_FOR_EACH (arp, hmap_node, arps) { + char addr_s[INET6_ADDRSTRLEN + 1]; + ds_put_format(&ds, "UUID: "UUID_FMT", VNI: %"PRIu32", " + "MAC: "ETH_ADDR_FMT", IP: %s, " + "dp_key: %"PRId64"\n", + UUID_ARGS(&arp->flow_uuid), arp->vni, + ETH_ADDR_ARGS(arp->mac), + ipv6_string_mapped(addr_s, &arp->ip) + ? addr_s : "(invalid)", + arp->ldp->datapath->tunnel_key); + } + + unixctl_command_reply(conn, ds_cstr_ro(&ds)); + ds_destroy(&ds); +} + +static struct evpn_arp * +evpn_arp_add(struct hmap *evpn_arps, struct eth_addr mac, + const struct in6_addr *ip, uint32_t vni) +{ + struct evpn_arp *arp = xmalloc(sizeof *arp); + *arp = (struct evpn_arp) { + .flow_uuid = uuid_random(), + .mac = mac, + .ip = *ip, + .vni = vni, + }; + + uint32_t hash = hash_bytes(&mac, sizeof mac, 0); + hmap_insert(evpn_arps, &arp->hmap_node, hash); + + return arp; +} + +static struct evpn_arp * +evpn_arp_find(const struct hmap *evpn_arps, struct eth_addr mac, + const struct in6_addr *ip, uint32_t vni) +{ + uint32_t hash = hash_bytes(&mac, sizeof mac, 0); + + struct evpn_arp *arp; + HMAP_FOR_EACH_WITH_HASH (arp, hmap_node, hash, evpn_arps) { + if (arp->vni == vni && eth_addr_equals(arp->mac, mac) && + ipv6_addr_equals(&arp->ip, ip)) { + return arp; + } + } + + return NULL; +} diff --git a/controller/evpn-arp.h b/controller/evpn-arp.h new file mode 100644 index 0000000000..7f3a4c0e44 --- /dev/null +++ b/controller/evpn-arp.h @@ -0,0 +1,65 @@ +/* Copyright (c) 2025, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef EVPN_ARP_H +#define EVPN_ARP_H 1 + +#include <stdint.h> + +#include "hmapx.h" +#include "local_data.h" +#include "neighbor-of.h" +#include "openvswitch/hmap.h" +#include "uuidset.h" + +struct unixctl_conn; + +struct evpn_arp_ctx_in { + /* Contains 'struct evpn_datapath'. */ + const struct hmap *datapaths; + /* Contains 'struct evpn_static_entry' one for each ARP. */ + const struct hmap *static_arps; +}; + +struct evpn_arp_ctx_out { + /* Contains 'struct evpn_arp'. */ + struct hmap *arps; + /* Contains pointers to 'struct evpn_binding'. */ + struct hmapx *updated_arps; + /* Contains 'flow_uuid' from removed 'struct evpn_binding'. */ + struct uuidset *removed_arps; +}; + +struct evpn_arp { + struct hmap_node hmap_node; + /* UUID used to identify physical flows related to this ARP entry. */ + struct uuid flow_uuid; + /* MAC address of the remote workload. */ + struct eth_addr mac; + /* IP address of the remote workload. */ + struct in6_addr ip; + uint32_t vni; + /* Logical datapath of the switch this was learned on. */ + const struct local_datapath *ldp; + /* Priority to use for this ARP entry at OpenFlow level. */ + enum neigh_of_rule_prio priority; +}; + +void evpn_arp_run(const struct evpn_arp_ctx_in *, struct evpn_arp_ctx_out *); +void evpn_arps_destroy(struct hmap *arps); +void evpn_arp_list(struct unixctl_conn *conn, int argc, + const char *argv[], void *data_); + +#endif /* EVPN_ARP_H */ diff --git a/controller/evpn-fdb.c b/controller/evpn-fdb.c index 53312fbb2c..acef8f0a42 100644 --- a/controller/evpn-fdb.c +++ b/controller/evpn-fdb.c @@ -43,7 +43,7 @@ evpn_fdb_run(const struct evpn_fdb_ctx_in *f_ctx_in, hmapx_add(&stale_fdbs, fdb); } - const struct evpn_static_fdb *static_fdb; + const struct evpn_static_entry *static_fdb; HMAP_FOR_EACH (static_fdb, hmap_node, f_ctx_in->static_fdbs) { const struct evpn_binding *binding = evpn_binding_find(f_ctx_in->bindings, &static_fdb->ip, diff --git a/controller/evpn-fdb.h b/controller/evpn-fdb.h index a38718cd93..de58df813e 100644 --- a/controller/evpn-fdb.h +++ b/controller/evpn-fdb.h @@ -27,7 +27,7 @@ struct unixctl_conn; struct evpn_fdb_ctx_in { /* Contains 'struct evpn_binding'. */ const struct hmap *bindings; - /* Contains 'struct evpn_static_fdb'. */ + /* Contains 'struct evpn_static_entry', one for each FDB. */ const struct hmap *static_fdbs; }; diff --git a/controller/neighbor-exchange-netlink.c b/controller/neighbor-exchange-netlink.c index 66711d4980..fa2444e2f3 100644 --- a/controller/neighbor-exchange-netlink.c +++ b/controller/neighbor-exchange-netlink.c @@ -189,6 +189,15 @@ ne_is_valid_static_fdb(struct ne_nl_received_neigh *ne) ipv6_addr_is_set(&ne->addr) && ne->flags & NTF_EXT_LEARNED; } +/* OVN expects that the ARP entry has an IP address, a MAC address, + * the entry is marked as "extern learned" and "static" (noarp). */ +bool +ne_is_valid_static_arp(struct ne_nl_received_neigh *ne) +{ + return !eth_addr_is_zero(ne->lladdr) && ipv6_addr_is_set(&ne->addr) && + ne->state & NUD_NOARP && ne->flags & NTF_EXT_LEARNED; +} + static void ne_table_dump_one_ifindex(unsigned char address_family, int32_t if_index, ne_table_handle_msg_callback *handle_msg_cb, diff --git a/controller/neighbor-exchange-netlink.h b/controller/neighbor-exchange-netlink.h index ee04691ebd..6d907938eb 100644 --- a/controller/neighbor-exchange-netlink.h +++ b/controller/neighbor-exchange-netlink.h @@ -56,6 +56,7 @@ int ne_nl_sync_neigh(uint8_t family, int32_t if_index, bool ne_is_ovn_owned(const struct ne_nl_received_neigh *nd); bool ne_is_valid_remote_vtep(struct ne_nl_received_neigh *ne); bool ne_is_valid_static_fdb(struct ne_nl_received_neigh *ne); +bool ne_is_valid_static_arp(struct ne_nl_received_neigh *ne); int ne_table_parse(struct ofpbuf *, void *change); diff --git a/controller/neighbor-exchange-stub.c b/controller/neighbor-exchange-stub.c index 272c14007f..a1c89ed2ba 100644 --- a/controller/neighbor-exchange-stub.c +++ b/controller/neighbor-exchange-stub.c @@ -42,6 +42,6 @@ evpn_remote_vtep_list(struct unixctl_conn *conn OVS_UNUSED, } void -evpn_static_fdbs_clear(struct hmap *static_fdbs OVS_UNUSED) +evpn_static_entries_clear(struct hmap *static_entries OVS_UNUSED) { } diff --git a/controller/neighbor-exchange.c b/controller/neighbor-exchange.c index 2c436b4141..ea6c287b1f 100644 --- a/controller/neighbor-exchange.c +++ b/controller/neighbor-exchange.c @@ -38,13 +38,15 @@ static void evpn_remote_vtep_add(struct hmap *remote_vteps, struct in6_addr ip, static struct evpn_remote_vtep *evpn_remote_vtep_find( const struct hmap *remote_vteps, const struct in6_addr *ip, uint16_t port, uint32_t vni); -static void evpn_static_fdb_add(struct hmap *static_fdbs, struct eth_addr mac, - struct in6_addr ip, uint32_t vni); -static struct evpn_static_fdb *evpn_static_fdb_find( - const struct hmap *static_fdbs, struct eth_addr mac, +static void evpn_static_entry_add(struct hmap *static_entries, + struct eth_addr mac, struct in6_addr ip, + uint32_t vni); +static struct evpn_static_entry *evpn_static_entry_find( + const struct hmap *static_entries, struct eth_addr mac, struct in6_addr ip, uint32_t vni); -static uint32_t evpn_static_fdb_hash(const struct eth_addr *mac, - const struct in6_addr *ip, uint32_t vni); +static uint32_t evpn_static_entry_hash(const struct eth_addr *mac, + const struct in6_addr *ip, + uint32_t vni); /* Last neighbor_exchange netlink operation. */ static int neighbor_exchange_nl_status; @@ -92,8 +94,22 @@ neighbor_exchange_run(const struct neighbor_exchange_ctx_in *n_ctx_in, &received_neighbors) ); - if (nim->type == NEIGH_IFACE_VXLAN) { - struct ne_nl_received_neigh *ne; + struct ne_nl_received_neigh *ne; + switch (nim->type) { + case NEIGH_IFACE_BRIDGE: + VECTOR_FOR_EACH_PTR (&received_neighbors, ne) { + if (ne_is_valid_static_arp(ne)) { + if (!evpn_static_entry_find(n_ctx_out->static_arps, + ne->lladdr, ne->addr, + nim->vni)) { + evpn_static_entry_add(n_ctx_out->static_arps, + ne->lladdr, ne->addr, + nim->vni); + } + } + } + break; + case NEIGH_IFACE_VXLAN: VECTOR_FOR_EACH_PTR (&received_neighbors, ne) { if (ne_is_valid_remote_vtep(ne)) { uint16_t port = ne->port ? ne->port : DEFAULT_VXLAN_PORT; @@ -103,14 +119,19 @@ neighbor_exchange_run(const struct neighbor_exchange_ctx_in *n_ctx_in, port, nim->vni); } } else if (ne_is_valid_static_fdb(ne)) { - if (!evpn_static_fdb_find(n_ctx_out->static_fdbs, + if (!evpn_static_entry_find(n_ctx_out->static_fdbs, + ne->lladdr, ne->addr, + nim->vni)) { + evpn_static_entry_add(n_ctx_out->static_fdbs, ne->lladdr, ne->addr, - nim->vni)) { - evpn_static_fdb_add(n_ctx_out->static_fdbs, ne->lladdr, - ne->addr, nim->vni); + nim->vni); } } } + break; + case NEIGH_IFACE_LOOPBACK: + /* No learning from the loopback interface required. */ + break; } neighbor_table_add_watch_request(&n_ctx_out->neighbor_table_watches, @@ -154,11 +175,11 @@ evpn_remote_vtep_list(struct unixctl_conn *conn, int argc OVS_UNUSED, } void -evpn_static_fdbs_clear(struct hmap *static_fdbs) +evpn_static_entries_clear(struct hmap *static_entries) { - struct evpn_static_fdb *fdb; - HMAP_FOR_EACH_POP (fdb, hmap_node, static_fdbs) { - free(fdb); + struct evpn_static_entry *e; + HMAP_FOR_EACH_POP (e, hmap_node, static_entries) { + free(e); } } @@ -208,32 +229,32 @@ evpn_remote_vtep_hash(const struct in6_addr *ip, uint16_t port, } static void -evpn_static_fdb_add(struct hmap *static_fdbs, struct eth_addr mac, - struct in6_addr ip, uint32_t vni) +evpn_static_entry_add(struct hmap *static_entries, struct eth_addr mac, + struct in6_addr ip, uint32_t vni) { - struct evpn_static_fdb *fdb = xmalloc(sizeof *fdb); - *fdb = (struct evpn_static_fdb) { + struct evpn_static_entry *e = xmalloc(sizeof *e); + *e = (struct evpn_static_entry) { .mac = mac, .ip = ip, .vni = vni, }; - hmap_insert(static_fdbs, &fdb->hmap_node, - evpn_static_fdb_hash(&mac, &ip, vni)); + hmap_insert(static_entries, &e->hmap_node, + evpn_static_entry_hash(&mac, &ip, vni)); } -static struct evpn_static_fdb * -evpn_static_fdb_find(const struct hmap *static_fdbs, struct eth_addr mac, - struct in6_addr ip, uint32_t vni) +static struct evpn_static_entry * +evpn_static_entry_find(const struct hmap *static_entries, struct eth_addr mac, + struct in6_addr ip, uint32_t vni) { - uint32_t hash = evpn_static_fdb_hash(&mac, &ip, vni); - - struct evpn_static_fdb *fdb; - HMAP_FOR_EACH_WITH_HASH (fdb, hmap_node, hash, static_fdbs) { - if (eth_addr_equals(fdb->mac, mac) && - ipv6_addr_equals(&fdb->ip, &ip) && - fdb->vni == vni) { - return fdb; + uint32_t hash = evpn_static_entry_hash(&mac, &ip, vni); + + struct evpn_static_entry *e; + HMAP_FOR_EACH_WITH_HASH (e, hmap_node, hash, static_entries) { + if (eth_addr_equals(e->mac, mac) && + ipv6_addr_equals(&e->ip, &ip) && + e->vni == vni) { + return e; } } @@ -241,8 +262,8 @@ evpn_static_fdb_find(const struct hmap *static_fdbs, struct eth_addr mac, } static uint32_t -evpn_static_fdb_hash(const struct eth_addr *mac, const struct in6_addr *ip, - uint32_t vni) +evpn_static_entry_hash(const struct eth_addr *mac, const struct in6_addr *ip, + uint32_t vni) { uint32_t hash = 0; hash = hash_bytes(mac, sizeof *mac, hash); diff --git a/controller/neighbor-exchange.h b/controller/neighbor-exchange.h index 6122d8dee5..281733d905 100644 --- a/controller/neighbor-exchange.h +++ b/controller/neighbor-exchange.h @@ -34,8 +34,12 @@ struct neighbor_exchange_ctx_out { struct hmap neighbor_table_watches; /* Contains 'struct evpn_remote_vtep'. */ struct hmap *remote_vteps; - /* Contains 'struct evpn_static_fdb'. */ + /* Contains 'struct evpn_static_entry', remote FDB entries learnt through + * EVPN. */ struct hmap *static_fdbs; + /* Contains 'struct evpn_static_entry', remote ARP entries learnt through + * EVPN. */ + struct hmap *static_arps; }; struct evpn_remote_vtep { @@ -48,11 +52,11 @@ struct evpn_remote_vtep { uint32_t vni; }; -struct evpn_static_fdb { +struct evpn_static_entry { struct hmap_node hmap_node; /* MAC address of the remote workload. */ struct eth_addr mac; - /* Destination ip of the remote tunnel. */ + /* Destination ip of the remote tunnel or remote IP. */ struct in6_addr ip; /* VNI of the VTEP. */ uint32_t vni; @@ -64,6 +68,6 @@ int neighbor_exchange_status_run(void); void evpn_remote_vteps_clear(struct hmap *remote_vteps); void evpn_remote_vtep_list(struct unixctl_conn *, int argc, const char *argv[], void *data_); -void evpn_static_fdbs_clear(struct hmap *static_fdbs); +void evpn_static_entries_clear(struct hmap *static_entries); #endif /* NEIGHBOR_EXCHANGE_H */ diff --git a/controller/neighbor-of.h b/controller/neighbor-of.h index 874200e8be..4138d1577e 100644 --- a/controller/neighbor-of.h +++ b/controller/neighbor-of.h @@ -23,15 +23,20 @@ /* Priorities of ovn-controller generated flows for various types of MAC * Bindings in different situations. Valid preference orders, based on - * the SB.Static_MAC_Binding.override_dynamic_mac value are: + * the "dynamic-routing-arp-prefer-local" logical switch config and the + * SB.Static_MAC_Binding.override_dynamic_mac value are: * - * - static-mac-binding < dynamic-mac-binding - * - dynamic-mac-binding < static-mac-binding + * - EVPN-learned < static-mac-binding < dynamic-mac-binding + * - EVPN-learned < dynamic-mac-binding < static-mac-binding + * - static-mac-binding < dynamic-mac-binding < EVPN-learned + * - dynamic-mac-binding < static-mac-binding < EVPN-learned */ enum neigh_of_rule_prio { + NEIGH_OF_EVPN_MAC_BINDING_LOW_PRIO = 20, NEIGH_OF_STATIC_MAC_BINDING_LOW_PRIO = 50, NEIGH_OF_DYNAMIC_MAC_BINDING_PRIO = 100, NEIGH_OF_STATIC_MAC_BINDING_HIGH_PRIO = 150, + NEIGH_OF_EVPN_MAC_BINDING_HIGH_PRIO = 200, }; void diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index c8b76feb52..09bb83e3a2 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -98,6 +98,7 @@ #include "neighbor.h" #include "neighbor-exchange.h" #include "neighbor-table-notify.h" +#include "evpn-arp.h" #include "evpn-binding.h" #include "evpn-fdb.h" @@ -4589,6 +4590,15 @@ struct ed_type_evpn_fdb { struct uuidset removed_fdbs; }; +struct ed_type_evpn_arp { + /* Contains 'struct evpn_arp'. */ + struct hmap arps; + /* Contains pointers to 'struct evpn_arp'. */ + struct hmapx updated_arps; + /* Contains 'flow_uuid' from removed 'struct evpn_arps'. */ + struct uuidset removed_arps; +}; + static void init_physical_ctx(struct engine_node *node, struct ed_type_runtime_data *rt_data, struct ed_type_non_vif_data *non_vif_data, @@ -4649,6 +4659,9 @@ static void init_physical_ctx(struct engine_node *node, struct ed_type_evpn_fdb *efdb_data = engine_get_input_data("evpn_fdb", node); + struct ed_type_evpn_arp *earp_data = + engine_get_input_data("evpn_arp", node); + parse_encap_ips(ovs_table, &p_ctx->n_encap_ips, &p_ctx->encap_ips); p_ctx->sbrec_port_binding_by_name = sbrec_port_binding_by_name; p_ctx->sbrec_port_binding_by_datapath = sbrec_port_binding_by_datapath; @@ -4669,6 +4682,7 @@ static void init_physical_ctx(struct engine_node *node, p_ctx->evpn_bindings = &eb_data->bindings; p_ctx->evpn_multicast_groups = &eb_data->multicast_groups; p_ctx->evpn_fdbs = &efdb_data->fdbs; + p_ctx->evpn_arps = &earp_data->arps; struct controller_engine_ctx *ctrl_ctx = engine_get_context()->client_ctx; p_ctx->if_mgr = ctrl_ctx->if_mgr; @@ -4993,6 +5007,22 @@ pflow_output_fdb_handler(struct engine_node *node, void *data) return EN_HANDLED_UPDATED; } +static enum engine_input_handler_result +pflow_output_arp_handler(struct engine_node *node, void *data) +{ + struct ed_type_pflow_output *pfo = data; + struct ed_type_runtime_data *rt_data = + engine_get_input_data("runtime_data", node); + struct ed_type_evpn_arp *ea_data = + engine_get_input_data("evpn_arp", node); + + physical_handle_evpn_arp_changes(&rt_data->local_datapaths, + &pfo->flow_table, + &ea_data->updated_arps, + &ea_data->removed_arps); + return EN_HANDLED_UPDATED; +} + static void * en_controller_output_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -6040,8 +6070,12 @@ en_neighbor_table_notify_run(struct engine_node *node OVS_UNUSED, struct ed_type_neighbor_exchange { /* Contains 'struct evpn_remote_vtep'. */ struct hmap remote_vteps; - /* Contains 'struct evpn_static_fdb'. */ + /* Contains 'struct evpn_static_entry', remote FDB entries learnt through + * EVPN. */ struct hmap static_fdbs; + /* Contains 'struct evpn_static_entry', remote ARP entries learnt through + * EVPN. */ + struct hmap static_arps; }; static void * @@ -6052,6 +6086,7 @@ en_neighbor_exchange_init(struct engine_node *node OVS_UNUSED, *data = (struct ed_type_neighbor_exchange) { .remote_vteps = HMAP_INITIALIZER(&data->remote_vteps), .static_fdbs = HMAP_INITIALIZER(&data->static_fdbs), + .static_arps = HMAP_INITIALIZER(&data->static_arps), }; return data; @@ -6062,9 +6097,11 @@ en_neighbor_exchange_cleanup(void *data_) { struct ed_type_neighbor_exchange *data = data_; evpn_remote_vteps_clear(&data->remote_vteps); - evpn_static_fdbs_clear(&data->static_fdbs); + evpn_static_entries_clear(&data->static_fdbs); + evpn_static_entries_clear(&data->static_arps); hmap_destroy(&data->remote_vteps); hmap_destroy(&data->static_fdbs); + hmap_destroy(&data->static_arps); } static enum engine_node_state @@ -6075,7 +6112,8 @@ en_neighbor_exchange_run(struct engine_node *node, void *data_) engine_get_input_data("neighbor", node); evpn_remote_vteps_clear(&data->remote_vteps); - evpn_static_fdbs_clear(&data->static_fdbs); + evpn_static_entries_clear(&data->static_fdbs); + evpn_static_entries_clear(&data->static_arps); struct neighbor_exchange_ctx_in n_ctx_in = { .monitored_interfaces = &neighbor_data->monitored_interfaces, @@ -6085,6 +6123,7 @@ en_neighbor_exchange_run(struct engine_node *node, void *data_) HMAP_INITIALIZER(&n_ctx_out.neighbor_table_watches), .remote_vteps = &data->remote_vteps, .static_fdbs = &data->static_fdbs, + .static_arps = &data->static_arps, }; neighbor_exchange_run(&n_ctx_in, &n_ctx_out); @@ -6355,6 +6394,81 @@ evpn_fdb_vtep_binding_handler(struct engine_node *node, void *data OVS_UNUSED) return EN_UNHANDLED; } +static void * +en_evpn_arp_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_evpn_arp *data = xmalloc(sizeof *data); + *data = (struct ed_type_evpn_arp) { + .arps = HMAP_INITIALIZER(&data->arps), + .updated_arps = HMAPX_INITIALIZER(&data->updated_arps), + .removed_arps = UUIDSET_INITIALIZER(&data->removed_arps), + }; + + return data; +} + +static void +en_evpn_arp_clear_tracked_data(void *data_) +{ + struct ed_type_evpn_arp *data = data_; + hmapx_clear(&data->updated_arps); + uuidset_clear(&data->removed_arps); +} + +static void +en_evpn_arp_cleanup(void *data_) +{ + struct ed_type_evpn_arp *data = data_; + evpn_arps_destroy(&data->arps); + hmapx_destroy(&data->updated_arps); + uuidset_destroy(&data->removed_arps); +} + +static enum engine_node_state +en_evpn_arp_run(struct engine_node *node, void *data_) +{ + struct ed_type_evpn_arp *data = data_; + const struct ed_type_neighbor_exchange *ne_data = + engine_get_input_data("neighbor_exchange", node); + const struct ed_type_evpn_vtep_binding *eb_data = + engine_get_input_data("evpn_vtep_binding", node); + + struct evpn_arp_ctx_in f_ctx_in = { + .datapaths = &eb_data->datapaths, + .static_arps = &ne_data->static_arps, + }; + + struct evpn_arp_ctx_out f_ctx_out = { + .arps = &data->arps, + .updated_arps = &data->updated_arps, + .removed_arps = &data->removed_arps, + }; + + evpn_arp_run(&f_ctx_in, &f_ctx_out); + + if (hmapx_count(&data->updated_arps) || + uuidset_count(&data->removed_arps)) { + return EN_UPDATED; + } + + return EN_UNCHANGED; +} + +static enum engine_input_handler_result +evpn_arp_vtep_binding_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct ed_type_evpn_vtep_binding *eb_data = + engine_get_input_data("evpn_vtep_binding", node); + + if (hmapx_is_empty(&eb_data->updated_bindings) && + uuidset_is_empty(&eb_data->removed_bindings)) { + return EN_HANDLED_UNCHANGED; + } + + return EN_UNHANDLED; +} + /* Returns false if the northd internal version stored in SB_Global * and ovn-controller internal version don't match. */ @@ -6681,6 +6795,7 @@ main(int argc, char *argv[]) ENGINE_NODE(neighbor_exchange_status); ENGINE_NODE(evpn_vtep_binding, CLEAR_TRACKED_DATA); ENGINE_NODE(evpn_fdb, CLEAR_TRACKED_DATA); + ENGINE_NODE(evpn_arp, CLEAR_TRACKED_DATA); #define SB_NODE(NAME) ENGINE_NODE_SB(NAME); SB_NODES @@ -6938,10 +7053,16 @@ main(int argc, char *argv[]) engine_add_input(&en_evpn_fdb, &en_evpn_vtep_binding, evpn_fdb_vtep_binding_handler); + engine_add_input(&en_evpn_arp, &en_neighbor_exchange, NULL); + engine_add_input(&en_evpn_arp, &en_evpn_vtep_binding, + evpn_arp_vtep_binding_handler); + engine_add_input(&en_pflow_output, &en_evpn_vtep_binding, pflow_output_evpn_binding_handler); engine_add_input(&en_pflow_output, &en_evpn_fdb, pflow_output_fdb_handler); + engine_add_input(&en_pflow_output, &en_evpn_arp, + pflow_output_arp_handler); engine_add_input(&en_controller_output, &en_dns_cache, NULL); @@ -7025,6 +7146,8 @@ main(int argc, char *argv[]) engine_get_internal_data(&en_evpn_vtep_binding); struct ed_type_evpn_fdb *efdb_data = engine_get_internal_data(&en_evpn_fdb); + struct ed_type_evpn_arp *earp_data = + engine_get_internal_data(&en_evpn_arp); ofctrl_init(&lflow_output_data->group_table, &lflow_output_data->meter_table); @@ -7053,6 +7176,9 @@ main(int argc, char *argv[]) unixctl_command_register("evpn/vtep-fdb-list", "", 0, 0, evpn_fdb_list, &efdb_data->fdbs); + unixctl_command_register("evpn/vtep-arp-list", "", 0, 0, + evpn_arp_list, + &earp_data->arps); struct pending_pkt pending_pkt = { .conn = NULL }; unixctl_command_register("inject-pkt", "MICROFLOW", 1, 1, inject_pkt, diff --git a/controller/physical.c b/controller/physical.c index 9ca535a6cd..23a579f857 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -20,6 +20,7 @@ #include "ct-zone.h" #include "encaps.h" #include "evpn-binding.h" +#include "evpn-arp.h" #include "evpn-fdb.h" #include "flow.h" #include "ha-chassis.h" @@ -54,6 +55,7 @@ #include "util.h" #include "vswitch-idl.h" #include "hmapx.h" +#include "neighbor-of.h" VLOG_DEFINE_THIS_MODULE(physical); @@ -2778,6 +2780,28 @@ physical_consider_evpn_fdb(const struct evpn_fdb *fdb, match, ofpacts, &fdb->flow_uuid); } +static void +physical_consider_evpn_arp(const struct hmap *local_datapaths, + const struct evpn_arp *arp, + struct ovn_desired_flow_table *flow_table) +{ + /* Walk connected OVN routers and install neighbor flows for the ARPs + * learned on EVPN datapaths.*/ + const struct peer_ports *peers; + VECTOR_FOR_EACH_PTR (&arp->ldp->peer_ports, peers) { + const struct sbrec_port_binding *remote_pb = peers->remote; + struct local_datapath *peer_ld = + get_local_datapath(local_datapaths, + remote_pb->datapath->tunnel_key); + if (!peer_ld || peer_ld->is_switch) { + continue; + } + + consider_neighbor_flow(remote_pb, &arp->flow_uuid, &arp->ip, arp->mac, + flow_table, arp->priority, false); + } +} + static void physical_eval_evpn_flows(const struct physical_ctx *ctx, struct ofpbuf *ofpacts, @@ -2785,7 +2809,8 @@ physical_eval_evpn_flows(const struct physical_ctx *ctx, { if (hmap_is_empty(ctx->evpn_bindings) && hmap_is_empty(ctx->evpn_multicast_groups) && - hmap_is_empty(ctx->evpn_fdbs)) { + hmap_is_empty(ctx->evpn_fdbs) && + hmap_is_empty(ctx->evpn_arps)) { return; } @@ -2818,6 +2843,11 @@ physical_eval_evpn_flows(const struct physical_ctx *ctx, HMAP_FOR_EACH (fdb, hmap_node, ctx->evpn_fdbs) { physical_consider_evpn_fdb(fdb, ofpacts, &match, flow_table); } + + const struct evpn_arp *arp; + HMAP_FOR_EACH (arp, hmap_node, ctx->evpn_arps) { + physical_consider_evpn_arp(ctx->local_datapaths, arp, flow_table); + } } static void @@ -3001,6 +3031,27 @@ physical_handle_evpn_fdb_changes(struct ovn_desired_flow_table *flow_table, } } +void +physical_handle_evpn_arp_changes(const struct hmap *local_datapaths, + struct ovn_desired_flow_table *flow_table, + const struct hmapx *updated_arps, + const struct uuidset *removed_arps) +{ + + const struct hmapx_node *node; + HMAPX_FOR_EACH (node, updated_arps) { + const struct evpn_arp *arp = node->data; + + ofctrl_remove_flows(flow_table, &arp->flow_uuid); + physical_consider_evpn_arp(local_datapaths, arp, flow_table); + } + + const struct uuidset_node *uuidset_node; + UUIDSET_FOR_EACH (uuidset_node, removed_arps) { + ofctrl_remove_flows(flow_table, &uuidset_node->uuid); + } +} + void physical_run(struct physical_ctx *p_ctx, struct ovn_desired_flow_table *flow_table) diff --git a/controller/physical.h b/controller/physical.h index 37be46380e..0dc544823a 100644 --- a/controller/physical.h +++ b/controller/physical.h @@ -72,6 +72,7 @@ struct physical_ctx { const struct hmap *evpn_bindings; const struct hmap *evpn_multicast_groups; const struct hmap *evpn_fdbs; + const struct hmap *evpn_arps; /* Set of port binding names that have been already reprocessed during * the I-P run. */ @@ -99,4 +100,8 @@ void physical_handle_evpn_binding_changes( void physical_handle_evpn_fdb_changes(struct ovn_desired_flow_table *, const struct hmapx *updated_fdbs, const struct uuidset *removed_fdbs); +void physical_handle_evpn_arp_changes(const struct hmap *local_datapaths, + struct ovn_desired_flow_table *, + const struct hmapx *updated_arps, + const struct uuidset *removed_arps); #endif /* controller/physical.h */ diff --git a/northd/en-datapath-logical-switch.c b/northd/en-datapath-logical-switch.c index 0527239cec..c3fefd100a 100644 --- a/northd/en-datapath-logical-switch.c +++ b/northd/en-datapath-logical-switch.c @@ -98,6 +98,13 @@ gather_external_ids(const struct nbrec_logical_switch *nbs, smap_add(external_ids, "dynamic-routing-redistribute", redistribute); } + const char *prefer_evpn_arp_local = + smap_get(&nbs->other_config, "dynamic-routing-arp-prefer-local"); + if (prefer_evpn_arp_local) { + smap_add(external_ids, "dynamic-routing-arp-prefer-local", + prefer_evpn_arp_local); + } + /* For backwards-compatibility, also store the NB UUID in * external-ids:logical-switch. This is useful if ovn-controller * has not updated and expects this to be where to find the diff --git a/ovn-nb.xml b/ovn-nb.xml index ea7164e6cf..97da0177ed 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -908,6 +908,29 @@ </p> </column> + <column name="other_config" key="dynamic-routing-arp-prefer-local" + type='{"type": "boolean"}'> + <p> + This option defines the preference of ARP/ND lookup. If set to + true OVN routers connected to EVPN Logical Switches on which remote + neighbor entries have been learned (Type-2 MAC+IP EVPN routes) will + give precedence to any ARP/ND entries they might have in the SB + <code>Mac_Binding</code> table before trying to resolve the MAC + address via the <code>ovn-controller</code> local EVPN ARP/ND cache. + The option defaults to false. + </p> + + <p> + Only relevant if <ref column="other_config" key="dynamic-routing-vni" + table="Logical_switch"/> is set to valid VNI. + </p> + + <p> + NOTE: this feature is experimental and may be subject to + removal/change in the future. + </p> + </column> + <column name="other_config" key="dynamic-routing-redistribute" type='{"type": "string"}'> <p> diff --git a/tests/multinode.at b/tests/multinode.at index 959c5ba8a8..b5d3df5b33 100644 --- a/tests/multinode.at +++ b/tests/multinode.at @@ -3667,6 +3667,7 @@ check multinode_nbctl set logical_switch ls \ other_config:dynamic-routing-vni=$vni \ other_config:dynamic-routing-redistribute=fdb check multinode_nbctl --wait=hv sync +dp_key=$(m_fetch_column Datapath_Binding tunnel_key external_ids:name=ls) OVS_WAIT_UNTIL([m_as ovn-gw-1 bridge fdb | grep vxlan-10 | grep -q "00:00:00:00:00:00"]) OVS_WAIT_UNTIL([m_as ovn-gw-2 bridge fdb | grep vxlan-10 | grep -q "00:00:00:00:00:00"]) @@ -3680,23 +3681,33 @@ IP: $ext_bgp_ip_gw2, port: 4789, vni: 10 ]) AS_BOX([Check traffic to "fabric" hosts - simulate external workloads]) +check m_as ovn-gw-1 ip netns add fabric_workload +on_exit "m_as ovn-gw-1 ip netns del fabric_workload" check m_as ovn-gw-1 ip link add evpn_host type veth peer evpn_host_peer on_exit "m_as ovn-gw-1 ip link del evpn_host" check m_as ovn-gw-1 ip link set netns frr-ns evpn_host_peer check m_as ovn-gw-1 ip netns exec frr-ns ip link set evpn_host_peer master br-10 check m_as ovn-gw-1 ip netns exec frr-ns ip link set evpn_host_peer up -check m_as ovn-gw-1 ip link set evpn_host addr 00:00:00:00:01:00 -check m_as ovn-gw-1 ip addr add dev evpn_host 10.0.0.41/24 -check m_as ovn-gw-1 ip link set evpn_host up - +check m_as ovn-gw-1 ip link set netns fabric_workload evpn_host +check m_as ovn-gw-1 ip netns exec fabric_workload ip link set evpn_host addr 00:00:00:00:01:00 +check m_as ovn-gw-1 ip netns exec fabric_workload ip addr add dev evpn_host 10.0.0.41/24 +check m_as ovn-gw-1 ip netns exec fabric_workload ip link set evpn_host up +check m_as ovn-gw-1 ip netns exec fabric_workload ip r a default via 10.0.0.1 +check m_as ovn-gw-1 ip netns exec frr-ns ip a a dev br-10 10.0.0.81/24 + +check m_as ovn-gw-2 ip netns add fabric_workload +on_exit "m_as ovn-gw-2 ip netns del fabric_workload" check m_as ovn-gw-2 ip link add evpn_host type veth peer evpn_host_peer on_exit "m_as ovn-gw-2 ip link del evpn_host" check m_as ovn-gw-2 ip link set netns frr-ns evpn_host_peer check m_as ovn-gw-2 ip netns exec frr-ns ip link set evpn_host_peer master br-10 check m_as ovn-gw-2 ip netns exec frr-ns ip link set evpn_host_peer up -check m_as ovn-gw-2 ip link set evpn_host addr 00:00:00:00:02:00 -check m_as ovn-gw-2 ip addr add dev evpn_host 10.0.0.42/24 -check m_as ovn-gw-2 ip link set evpn_host up +check m_as ovn-gw-2 ip link set netns fabric_workload evpn_host +check m_as ovn-gw-2 ip netns exec fabric_workload ip link set evpn_host addr 00:00:00:00:02:00 +check m_as ovn-gw-2 ip netns exec fabric_workload ip addr add dev evpn_host 10.0.0.42/24 +check m_as ovn-gw-2 ip netns exec fabric_workload ip link set evpn_host up +check m_as ovn-gw-2 ip netns exec fabric_workload ip r a default via 10.0.0.1 +check m_as ovn-gw-2 ip netns exec frr-ns ip a a dev br-10 10.0.0.82/24 AS_BOX([Checking EVPN MACs on External BGP host]) OVS_WAIT_FOR_OUTPUT([m_as ovn-gw-1 ip netns exec frr-ns vtysh --vty_socket /run/frr/frr-ns -c 'show evpn mac vni all'], [0], [dnl @@ -3720,8 +3731,83 @@ MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s ]) AS_BOX([Check traffic to "fabric" hosts - ping from fabric]) -OVS_WAIT_UNTIL([m_as ovn-gw-1 ping -W 1 -c 1 10.0.0.11]) -OVS_WAIT_UNTIL([m_as ovn-gw-2 ping -W 1 -c 1 10.0.0.12]) +OVS_WAIT_UNTIL([m_as ovn-gw-1 ip netns exec fabric_workload ping -W 1 -c 1 10.0.0.11]) +OVS_WAIT_UNTIL([m_as ovn-gw-2 ip netns exec fabric_workload ping -W 1 -c 1 10.0.0.12]) + +AS_BOX([Check type-2 MAC+IP EVPN route advertisements]) +# Ping from the frr-ns to the fabric workload so that its IP is learned on +# the fabric EVPN peer (and advertised to OVN). +OVS_WAIT_UNTIL([m_as ovn-gw-1 ip netns exec frr-ns ip vrf exec vrf-10 ping -W 1 -c 1 10.0.0.41]) +OVS_WAIT_UNTIL([m_as ovn-gw-2 ip netns exec frr-ns ip vrf exec vrf-10 ping -W 1 -c 1 10.0.0.42]) + +# Check that OVN learned the ARPs. +OVS_WAIT_FOR_OUTPUT_UNQUOTED([m_as ovn-gw-1 ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl + VNI: 10, MAC: 00:00:00:00:01:00, IP: 10.0.0.41, dp_key: $dp_key +]) +OVS_WAIT_FOR_OUTPUT_UNQUOTED([m_as ovn-gw-2 ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl + VNI: 10, MAC: 00:00:00:00:02:00, IP: 10.0.0.42, dp_key: $dp_key +]) + +AS_BOX([Check that OVN routers used ARP entries learned through type-2 EVPN MAC+IP routes]) +# Add an OVN router with an internal switch and internal workload. +check multinode_nbctl --wait=hv \ + -- lr-add lr \ + -- lrp-add lr lr-ls 00:00:00:01:00:00 10.0.0.1/24 \ + -- lrp-add lr lr-ls-int 00:00:00:02:00:00 20.0.0.1/24 \ + -- lsp-add ls ls-lr \ + -- lsp-set-type ls-lr router \ + -- lsp-set-options ls-lr router-port=lr-ls \ + -- lsp-set-addresses ls-lr router \ + -- ls-add ls-int \ + -- lsp-add ls-int ls-int-lr \ + -- lsp-set-type ls-int-lr router \ + -- lsp-set-options ls-int-lr router-port=lr-ls-int \ + -- lsp-set-addresses ls-int-lr router \ + -- lsp-add ls-int w-int1 \ + -- lsp-set-addresses w-int1 "00:00:00:02:00:01 20.0.0.11" \ + -- lsp-add ls-int w-int2 \ + -- lsp-set-addresses w-int2 "00:00:00:02:00:02 20.0.0.12" + +rtr_dp_key=$(m_fetch_column Datapath tunnel_key external_ids:name=lr) +rtr_port_key=$(m_fetch_column Port_Binding tunnel_key logical_port=lr-ls) + +check m_as ovn-gw-1 /data/create_fake_vm.sh w-int1 w-int1 00:00:00:02:00:01 1500 20.0.0.11 24 20.0.0.1 2000::11/64 2000::1 +check m_as ovn-gw-2 /data/create_fake_vm.sh w-int2 w-int2 00:00:00:02:00:02 1500 20.0.0.12 24 20.0.0.1 2000::12/64 2000::1 +m_wait_for_ports_up + +# Check that flows are created for the type-2 EVPN MAC+IP routes, in the +# router pipeline. +AT_CHECK_UNQUOTED([m_as ovn-gw-1 ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xa00000b,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:00:01,load:0x1->NXM_NX_REG10[[6]] +priority=100,reg0=0xa00000c,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:00:02,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xa000029,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:01:00,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([m_as ovn-gw-1 ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xa00000b,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:00:01 actions=load:0x1->NXM_NX_REG10[[6]] +priority=100,arp,reg0=0xa00000c,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:00:02 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xa000029,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:01:00 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([m_as ovn-gw-2 ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xa00000b,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:00:01,load:0x1->NXM_NX_REG10[[6]] +priority=100,reg0=0xa00000c,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:00:02,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xa00002a,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:00:00:00:00:02:00,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([m_as ovn-gw-2 ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xa00000b,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:00:01 actions=load:0x1->NXM_NX_REG10[[6]] +priority=100,arp,reg0=0xa00000c,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:00:02 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xa00002a,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=00:00:00:00:02:00 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +AS_BOX([Check traffic to "fabric" hosts - ping from internal hosts]) +OVS_WAIT_UNTIL([m_as ovn-gw-1 ip netns exec w-int1 ping -W 1 -c 1 10.0.0.41]) +OVS_WAIT_UNTIL([m_as ovn-gw-2 ip netns exec w-int2 ping -W 1 -c 1 10.0.0.42]) # Remove "workloads" (VIF LSPs) on both chassis. check multinode_nbctl --wait=hv lsp-del w1 -- lsp-del w2 diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 85e432e75d..4737a804b3 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -18337,6 +18337,8 @@ on_exit "ip link del lo-$vni" check ip link set lo-$vni master br-$vni check ip link set lo-$vni up +AS_BOX([L2 EVPN VTEP and FDB learning]) + check ovn-nbctl --wait=hv set logical_switch ls-evpn other_config:dynamic-routing-vni=$vni ofport=$(ovs-vsctl --bare --columns ofport find Interface name="ovn-evpn-4789") dp_key=$(fetch_column Datapath tunnel_key external_ids:name=ls-evpn) @@ -18450,6 +18452,8 @@ check diff -q bindings_before bindings_after check diff -q mc_groups_before mc_groups_after check diff -q fdb_before fdb_after +AS_BOX([L2 EVPN FDB advertising]) + check ovn-nbctl --wait=hv set logical_switch ls-evpn other_config:dynamic-routing-redistribute=fdb OVS_WAIT_FOR_OUTPUT([bridge fdb show | grep "lo-10" | grep "f0:00:0f:16:01" | sort], [0], [dnl f0:00:0f:16:01:10 dev lo-10 master br-10 static @@ -18468,6 +18472,149 @@ check ovn-nbctl --wait=hv remove logical_switch ls-evpn other_config dynamic-rou OVS_WAIT_FOR_OUTPUT([bridge fdb show | grep "lo-10" | grep "f0:00:0f:16:01" | sort], [0], [dnl ]) +AS_BOX([L2 EVPN ARP learning]) +# Add a router connected to the EVPN logical switch. +check ovn-nbctl --wait=hv \ + -- lr-add lr \ + -- lrp-add lr lr-ls-evpn f0:00:0f:16:01:01 172.16.1.1/24 \ + -- lsp-add ls-evpn ls-evpn-lr \ + -- lsp-set-type ls-evpn-lr router \ + -- lsp-set-options ls-evpn-lr router-port=lr-ls-evpn \ + -- lsp-set-addresses ls-evpn-lr router + +rtr_dp_key=$(fetch_column Datapath tunnel_key external_ids:name=lr) +rtr_port_key=$(fetch_column Port_Binding tunnel_key logical_port=lr-ls-evpn) + +# Simulate remote workload ARPs (type-2 MAC+IP EVPN route). +# ovn-controller needs to add OF rules for ARP lookup but no rules for +# MAC_CACHE use. These entries do not age out automatically, their lifetime +# is controlled by the BGP-EVPN control plane. +check ip neigh add dev br-10 172.16.1.50 lladdr f0:00:0f:16:10:50 nud noarp extern_learn +check ip neigh add dev br-10 172.16.1.60 lladdr f0:00:0f:16:10:60 nud noarp extern_learn + +OVS_WAIT_FOR_OUTPUT_UNQUOTED([ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl + VNI: 10, MAC: f0:00:0f:16:10:50, IP: 172.16.1.50, dp_key: $dp_key + VNI: 10, MAC: f0:00:0f:16:10:60, IP: 172.16.1.60, dp_key: $dp_key +]) + +AS_BOX([Check dynamic-routing-arp-prefer-local=true]) +check ovn-nbctl --wait=hv set Logical_Switch ls-evpn other_config:dynamic-routing-arp-prefer-local=true + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xac10010a,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:01:10,load:0x1->NXM_NX_REG10[[6]] +priority=20,reg0=0xac100132,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:50,load:0x1->NXM_NX_REG10[[6]] +priority=20,reg0=0xac10013c,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:60,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xac10010a,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10 actions=load:0x1->NXM_NX_REG10[[6]] +priority=20,arp,reg0=0xac100132,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:50 actions=load:0x1->NXM_NX_REG10[[6]] +priority=20,arp,reg0=0xac10013c,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:60 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_CACHE_USE | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,arp_spa=172.16.1.10,arp_op=2 actions=drop +priority=100,ip,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,nw_src=172.16.1.10 actions=drop +]) + +AS_BOX([Check dynamic-routing-arp-prefer-local=false]) +check ovn-nbctl --wait=hv set Logical_Switch ls-evpn other_config:dynamic-routing-arp-prefer-local=false + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xac10010a,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:01:10,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xac100132,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:50,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xac10013c,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:60,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xac10010a,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xac100132,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:50 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xac10013c,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:60 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_CACHE_USE | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,arp_spa=172.16.1.10,arp_op=2 actions=drop +priority=100,ip,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,nw_src=172.16.1.10 actions=drop +]) + +# Check that the recompute won't change the UUIDs and flows. +ovn-appctl evpn/vtep-arp-list > arp_before + +check ovn-appctl inc-engine/recompute +check ovn-nbctl --wait=hv sync + +ovn-appctl evpn/vtep-arp-list > arp_after + +check diff -q arp_before arp_after + +# Remove remote workload ARP entries and check ovn-controller's state. +check ip neigh del dev br-10 172.16.1.50 +check ip neigh del dev br-10 172.16.1.60 + +OVS_WAIT_FOR_OUTPUT_UNQUOTED([ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xac10010a,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:01:10,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xac10010a,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_CACHE_USE | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,arp_spa=172.16.1.10,arp_op=2 actions=drop +priority=100,ip,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10,nw_src=172.16.1.10 actions=drop +]) + +# Re-add the remote workload ARPs, remove the router, check that flows are +# removed (vtep-arp-list should still list the ARPs as they're learned on +# the logical switch that still exists). +check ip neigh add dev br-10 172.16.1.50 lladdr f0:00:0f:16:10:50 nud noarp extern_learn +check ip neigh add dev br-10 172.16.1.60 lladdr f0:00:0f:16:10:60 nud noarp extern_learn + +OVS_WAIT_FOR_OUTPUT_UNQUOTED([ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl + VNI: 10, MAC: f0:00:0f:16:10:50, IP: 172.16.1.50, dp_key: $dp_key + VNI: 10, MAC: f0:00:0f:16:10:60, IP: 172.16.1.60, dp_key: $dp_key +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,reg0=0xac10010a,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:01:10,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xac100132,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:50,load:0x1->NXM_NX_REG10[[6]] +priority=200,reg0=0xac10013c,reg15=0x$rtr_port_key,metadata=0x$rtr_dp_key actions=mod_dl_dst:f0:00:0f:16:10:60,load:0x1->NXM_NX_REG10[[6]] +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +priority=100,arp,reg0=0xac10010a,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:01:10 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xac100132,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:50 actions=load:0x1->NXM_NX_REG10[[6]] +priority=200,arp,reg0=0xac10013c,reg14=0x$rtr_port_key,metadata=0x$rtr_dp_key,dl_src=f0:00:0f:16:10:60 actions=load:0x1->NXM_NX_REG10[[6]] +]) + +check ovn-nbctl --wait=hv lr-del lr +AT_CHECK_UNQUOTED([ovn-appctl evpn/vtep-arp-list | cut -d',' -f2- | sort], [0], [dnl + VNI: 10, MAC: f0:00:0f:16:10:50, IP: 172.16.1.50, dp_key: $dp_key + VNI: 10, MAC: f0:00:0f:16:10:60, IP: 172.16.1.60, dp_key: $dp_key +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_BINDING | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +]) + +AT_CHECK_UNQUOTED([ovs-ofctl dump-flows br-int table=OFTABLE_MAC_LOOKUP | grep priority | \ + awk '{print $7, $8}' | sort], [0], [dnl +]) + OVN_CLEANUP_CONTROLLER([hv1]) as ovn-sb -- 2.51.0 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
