On Tue, Nov 10, 2020 at 9:31 PM Ihar Hrachyshka <ihrac...@redhat.com> wrote: > > On Tue, Nov 10, 2020 at 10:25 AM Numan Siddique <num...@ovn.org> wrote: > > > > " > > > > On Tue, Nov 3, 2020 at 4:45 AM Ihar Hrachyshka <ihrac...@redhat.com> wrote: > > > > > > User stories: > > > 1) NFV: an admin wants to run two separate instances of OVN controller > > > using the same database but configuring ports on different bridges. > > > Some of these bridges may use DPDK while others may not. > > > > > > 2) Parallel OVN instances: an admin wants to run two separate > > > instances of OVN controller using different databases. The > > > instances are completely independent and serve different consumers. > > > For example, the same machine runs both OpenStack and OpenShift > > > stacks, each running its own separate OVN stack. > > > > > > To serve these use cases, several features should be added to > > > ovn-controller: > > > > > > - use different database configuration for multiple controllers; > > > - customize chassis name used by controller. > > > > > > ===== > > > > > > For each of the following database configuration options, their > > > extended chassis specific counterparts are introduced: > > > > > > external_ids:hostname > > > external_ids:ovn-bridge > > > external_ids:ovn-bridge-datapath-type > > > external_ids:ovn-bridge-mappings > > > external_ids:ovn-chassis-mac-mappings > > > external_ids:ovn-cms-options > > > external_ids:ovn-encap-csum > > > external_ids:ovn-encap-ip > > > external_ids:ovn-encap-type > > > external_ids:ovn-is-interconn > > > external_ids:ovn-monitor-all > > > external_ids:ovn-openflow-probe-interval > > > external_ids:ovn-remote > > > external_ids:ovn-remote-probe-interval > > > > > > For example, > > > > > > external_ids:ovn-bridge -> external_ids:ovn-bridge-<chassis-name>= > > > external_ids:ovn-encap-ip -> external_ids:ovn-encap-ip-<chassis-name>= > > > external_ids:ovn-remote -> external_ids:ovn-remote-<chassis-name>= > > > > > > Priority wise, <chassis-name> specific options take precedence. > > > > > > ===== > > > > > > For system-id, > > > > > > You can now pass intended chassis name via CLI argument: > > > > > > $ ovn-controller ... -n <chassis_name> > > > > > > Alternatively, you can configure a chassis name by putting it into the > > > ${ovn_sysconfdir}/system-id-override file before running the > > > controller. > > > > > > The latter option may be more useful in container environment where > > > the same image may be reused for multiple controller instances, where > > > ovs_sysconfigdir/ovn/system-id-override is a volume mounted into this > > > generic image. The override file is read once on startup. If you want > > > to apply a new chassis name to a controller instance, restart it to > > > reread the file. > > > > > > Priority wise, this is the order in which different means to configure > > > the chassis name are used: > > > > > > - ovn-controller ... -n <chassis_name> CLI argument. > > > - ${ovs_sysconfdir}/ovn/system-id-override file; > > > - external_ids:system-id= ovsdb option; > > > > > > ===== > > > > > > Concurrent chassis running on the same host may inadvertantly remove > > > patch ports that belong to their peer chassis. To avoid that, patch > > > ports are now tagged in external-ids:ovn-chassis-id with the > > > appropriate chassis name, and only patch ports that belong to the > > > chassis are touched when cleaning up. Also, now only tunnels on the > > > active integration bridge are being cleaned up. > > > > > > Note that external-ids:ovn-chassis-id key is already used for tunnel > > > ports to identify the remote tunnel endpoint. We can reuse the same > > > key for patch ports because the key usage is not overlapping. > > > > > > Alternatively, we could introduce a new key with a similar but > > > different name. This would simplify code changes needed but would > > > arguably introduce even more confusion. Since the key name is not > > > entirely self-descriptive for tunnel ports (a better name would be > > > e.g. ovn-remote-chassis or ovn-peer-chassis), the ideal scenario would > > > be to rename the key for tunnel endpoints but reuse it for patch > > > ports. This would involve additional migration steps and is probably > > > not worth the hassle. > > > > > > ===== > > > > > > This patch also expands tunnel port name to allow for long similar > > > chassis names used on the same host. > > > > > > ===== > > > > > > Note: this patch assumes that each chassis has its own unique IP. > > > Future work may consider adding support to specify custom port numbers > > > for tunneling that would allow to reuse the same IP address for > > > multiple chassis running on the same host. This work is out of scope > > > for this patch. > > > > > > Signed-off-by: Ihar Hrachyshka <ihrac...@redhat.com> > > > > > > --- > > > > > > v1: initial implementation. > > > v2: fixed test case to check ports are claimed by proper chassis. > > > v2: added NEWS entry. > > > v2: fixed some compiler warnings. > > > v2: moved file_system_id declaration inside a function that uses it. > > > v2: removed unneeded binding.h #include. > > > v2: docs: better explanation of alternatives to select chassis name. > > > v3: reverted priority order for chassis configuration: first CLI, then > > > system-id file, then ovsdb. > > > v4: introduce helpers to extract external-ids (per-chassis or global). > > > v4: introduce per-chassis config options for all keys. > > > v4: introduce -M (--concurrent) CLI argument to avoid patch ports > > > removed by concurrent chassis. > > > v5: rebased. > > > v6: switched from -M (--concurrent) to external-ids:ovn-is-concurrent. > > > v6: with ovn-is-concurrent=true, also avoid removing unknown tunnel > > > endpoints. > > > v7: don't clean up tunnel endpoints from other bridges. > > > v7: don't clean up patch ports that don't belong to the chassis. > > > v7: remove ovn-is-concurrent that is no longer needed. > > > v7: rebased. > > > v8: rename system-id -> /etc/ovn/system-id-override > > > v8: read the system-id-override file just once on startup > > > v8: free() controller_chassis (CLI arg value) on exit > > > v9: updated commit message, removed notion of ovn-is-concurrent. > > > v10: rename external-ids:owner -> ovn-chassis-id in patch ports. > > > v10: use ovn_sysconfdir for system-id-override file location. > > > v10: clean up patch ports with no ovn-chassis-id tag. > > > v10: simplify encaps_run to only iterate over br-int ports, not all > > > bridges (and then explicitly skipping them). > > > v10: added test case to validate cleanup for patch and tunnel ports. > > > v10: minor adjustment in ovn-sb.xml. > > > v11: added -n to help message. > > > v11: fixed system tests. > > > v11: expanded tunnel port name to allow long concurrent chassis names. > > > > > > Hi Ihar, > > > > Thanks for v11 and for the patience. > > > > I tested this patch out locally and I found a couple of issues. > > > > Thanks for all the testing! > > > 1. The tunnel creation needs to be handled properly. Assume there are > > 2 chassis already registered - ovn-ch-1 and ovn-gw-1 > > And we start 2 ovn-controllers - ctrl-1 (with the local ip - > > 170.168.0.4 and bridge - br-ctrl1 configured) and ctrl-2 (170.168.0.10 > > and br-ctrl2 configured) on the same node. > > > > When ctrl-1 ovn-controller is started (ovn-controller -n ctrl-1) it > > will create tunnels to ovn-ch-1 and ovn-gw-1 with the names - > > "ovn-ovn-ch-1" and "ovn-ovn-gw-1". > > When ctrl-2 ovn-controller is started, (ovn-controller -n ctrl-2) it > > will create the tunnels to ovn-ch-1 and ovn-gw-1 with the same names a > > ctrl-1 and it will. > > > > I modified your patch as - port_name = xasprintf("tmp-ovn-%s-%x", > > chassis_id, i); > > to fix this issue and started ctrl-2 ovn-controller. After that I saw > > another error - could not add network device tmp-ovn-ovn-chassis-2-0 > > to ofproto (File exists) > > > > I think this should be fixed. May be - xasprintf("%s-ovn-%s-%x", > > local_chassis_id, chassis_id, i); > > > > Yes, it makes sense to now encode both chassis names in encap > endpoints. As for the error, it comes from OVS enforcing a single port > of the same type and configuration. I removed the enforcement from OVS > and the error is gone now. Is there a reason for no two tunnel ports > of the same configuration on different bridges? > > > ****** > > ovs-vsctl show > > 9c018379-74af-4522-8168-2e401ebf45b7 > > Bridge br-ctrl1 > > fail_mode: secure > > Port ovn-controller-2-0 > > Interface ovn-controller-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.10"} > > Port ovn-ovn-gw-1-0 > > Interface ovn-ovn-gw-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.3"} > > Port ovn-ovn-chassis-2-0 > > Interface ovn-ovn-chassis-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.5"} > > Port br-ctrl1 > > Interface br-ctrl1 > > type: internal > > Bridge br-ctrl2 > > Port tmp-ovn-ovn-chassis-2-0 > > Interface tmp-ovn-ovn-chassis-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.5"} > > error: "could not add network device > > tmp-ovn-ovn-chassis-2-0 to ofproto (File exists)" > > Port tmp-ovn-ovn-gw-1-0 > > Interface tmp-ovn-ovn-gw-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.3"} > > error: "could not add network device > > tmp-ovn-ovn-gw-1-0 to ofproto (File exists)" > > Port br-ctrl2 > > Interface br-ctrl2 > > type: internal > > Port tmp-ovn-controller-1-0 > > Interface tmp-ovn-controller-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.4"} > > ovs_version: "2.14.90" > > ******************** > > > > I ran the below commands manually - > > #ovs-vsctl set interface tmp-ovn-ovn-chassis-2-0 > > options:local_ip=170.168.0.10 > > #ovs-vsctl set interface tmp-ovn-ovn-gw-1-0 options:local_ip=170.168.0.10 > > > > And after that I don't see any error > > > > ***** > > 9c018379-74af-4522-8168-2e401ebf45b7 > > Bridge br-ctrl1 > > fail_mode: secure > > Port ovn-controller-2-0 > > Interface ovn-controller-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.10"} > > Port ovn-ovn-gw-1-0 > > Interface ovn-ovn-gw-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.3"} > > Port ovn-ovn-chassis-2-0 > > Interface ovn-ovn-chassis-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.5"} > > Port patch-br-ctrl1-to-ln-public > > Interface patch-br-ctrl1-to-ln-public > > type: patch > > options: {peer=patch-ln-public-to-br-ctrl1} > > Port br-ctrl1 > > Interface br-ctrl1 > > type: internal > > Bridge br-ctrl2 > > fail_mode: secure > > Port tmp-ovn-ovn-gw-1-0 > > Interface tmp-ovn-ovn-gw-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.3"} > > error: "could not add network device > > tmp-ovn-ovn-gw-1-0 to ofproto (File exists)" > > Port tmp-ovn-controller-1-0 > > Interface tmp-ovn-controller-1-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.4"} > > Port br-ctrl2 > > Interface br-ctrl2 > > type: internal > > Port tmp-ovn-ovn-chassis-2-0 > > Interface tmp-ovn-ovn-chassis-2-0 > > type: geneve > > options: {csum="true", key=flow, remote_ip="170.168.0.5"} > > > > ***** > > > > Another issue I noticed is if an ovs port (sw0p1) on br-ctrl1 wants > > to reach to another ovs port (sw0p2) on br-ctrl2, the packet is > > tunneled from br-ctrl1 to br-ctrl2 via the > > "ovn-controller-2-0" geneve port, but the packet doesn't reach the > > br-ctrl2. From the ovs-dpctl dump-flows I see that the packet > > is dropped > > > > I'll take a look. > > > *** > > recirc_id(0),tunnel(tun_id=0x1,src=170.168.0.10,dst=170.168.0.10,geneve({class=0x102,type=0x80,len=4,0x10002/0x7fffffff}),flags(-df+csum+key)),in_port(2),eth(),eth_type(0x0800),ipv4(frag=no), > > packets:31, bytes:3038, used:0.846s, actions:drop > > recirc_id(0),in_port(1),ct_state(-new-est-rpl-trk),ct_label(0/0x1),eth(src=50:54:00:00:00:03,dst=50:54:00:00:00:04),eth_type(0x0800),ipv4(tos=0/0x3,frag=no), > > packets:31, bytes:3038, used:0.846s, > > actions:set(tunnel(tun_id=0x1,dst=170.168.0.10,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x10002}),flags(df|csum|key))),2 > > *** > > > > *** > > ovs-dpctl show > > system@ovs-system: > > lookups: hit:1453 missed:156 lost:0 > > flows: 3 > > masks: hit:3204 total:3 hit/pkt:1.99 > > port 0: ovs-system (internal) > > port 1: sw0p1 (internal) > > port 2: genev_sys_6081 (geneve: packet_type=ptap) > > port 3: br-ex (internal) > > port 4: eth2 > > port 5: br-ctrl2 (internal) > > port 6: sw0p2 (internal) > > port 7: br-ctrl1 (internal) > > **** > > > > Looks like there is some misconfiguration in the tunnel local ip and remote > > ip. > > > > Seems to me the patch needs some more testing and the user story (1) > > mentioned in the commit message is not working for the local traffic. > > If you think that it is not possible to address these issues (for > > whatever reason), maybe we can just support the user story (2) you > > mentioned in the commit message. > > > > I'll try to fix the connectivity issue you found in the next few days. > Thanks again for all the testing. >
I believe the connectivity issue mentioned above is an artifact of the tunnel port names being reused fby different virtual chassis. Once I fixed that, I couldn't reproduce the issue. I will send a patch in a second that has the tunnel port names fixed as well as a test case added for the scenario that, I believe, you describe. Please let me know if the updated version works. > > > > FYI - output of ovn-sbctl show in my local setup > > ***** > > root@ovn-central /]# ovn-sbctl show > > Chassis controller-1 > > hostname: ovn-chassis-1 > > Encap geneve > > ip: "170.168.0.4" > > options: {csum="true"} > > Port_Binding sw0-port1 > > Chassis ovn-gw-1 > > hostname: ovn-gw-1 > > Encap geneve > > ip: "170.168.0.3" > > options: {csum="true"} > > Port_Binding cr-lr0-public > > Chassis ovn-chassis-2 > > hostname: ovn-chassis-2 > > Encap geneve > > ip: "170.168.0.5" > > options: {csum="true"} > > Port_Binding sw0-port4 > > Port_Binding sw1-port1 > > Chassis controller-2 > > hostname: ovn-chassis-1 > > Encap geneve > > ip: "170.168.0.10" > > options: {csum="true"} > > Port_Binding sw0-port2 > > > > **** > > Thanks > > Numan > > > > > > > --- > > > NEWS | 5 + > > > controller/chassis.c | 77 +++++++++------ > > > controller/chassis.h | 3 +- > > > controller/encaps.c | 76 ++++++++++----- > > > controller/encaps.h | 1 - > > > controller/ovn-controller.8.xml | 17 +++- > > > controller/ovn-controller.c | 106 ++++++++++++++++---- > > > controller/ovn-controller.h | 4 + > > > controller/patch.c | 20 +++- > > > controller/physical.c | 2 +- > > > lib/ovn-util.c | 50 ++++++++++ > > > lib/ovn-util.h | 18 ++++ > > > ovn-sb.xml | 10 +- > > > tests/ovn-controller.at | 11 +-- > > > tests/ovn-macros.at | 49 ++++++++-- > > > tests/ovn.at | 167 ++++++++++++++++++++++++++++++++ > > > tests/ovs-macros.at | 1 + > > > tests/system-ovn.at | 1 + > > > 18 files changed, 514 insertions(+), 104 deletions(-) > > > > > > diff --git a/NEWS b/NEWS > > > index eaacf7340..1e8ec7309 100644 > > > --- a/NEWS > > > +++ b/NEWS > > > @@ -18,6 +18,11 @@ OVN v20.09.0 - 28 Sep 2020 > > > - Added support for external ip based NAT. Now, besides the logical > > > ip, > > > external ips will also decide if a packet will be NATed or not. > > > - Added support for VXLAN encapsulation (not just for ramp/VTEP > > > switches). > > > + - Added support for multiple ovn-controller instances on the same host > > > + (virtual chassis). Now all external-ids:* configuration options can > > > be > > > + customized for each controller instance running on the same host. > > > The only > > > + option that is not available per chassis is external-ids:system-id, > > > which > > > + stands for the chassis name and can be passed via config file or > > > CLI (-n). > > > > > > OVN v20.06.0 > > > -------------------------- > > > diff --git a/controller/chassis.c b/controller/chassis.c > > > index 7748fb94c..5d87e3301 100644 > > > --- a/controller/chassis.c > > > +++ b/controller/chassis.c > > > @@ -125,9 +125,10 @@ chassis_register_ovs_idl(struct ovsdb_idl *ovs_idl) > > > } > > > > > > static const char * > > > -get_hostname(const struct smap *ext_ids) > > > +get_hostname(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - const char *hostname = smap_get_def(ext_ids, "hostname", ""); > > > + const char *hostname = get_chassis_external_id_value( > > > + ext_ids, chassis_id, "hostname", ""); > > > > > > if (strlen(hostname) == 0) { > > > static char hostname_[HOST_NAME_MAX + 1]; > > > @@ -143,39 +144,45 @@ get_hostname(const struct smap *ext_ids) > > > } > > > > > > static const char * > > > -get_bridge_mappings(const struct smap *ext_ids) > > > +get_bridge_mappings(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-bridge-mappings", ""); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-bridge-mappings", ""); > > > } > > > > > > const char * > > > -get_chassis_mac_mappings(const struct smap *ext_ids) > > > +get_chassis_mac_mappings(const struct smap *ext_ids, const char > > > *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-chassis-mac-mappings", ""); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-chassis-mac-mappings", ""); > > > } > > > > > > static const char * > > > -get_cms_options(const struct smap *ext_ids) > > > +get_cms_options(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-cms-options", ""); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-cms-options", ""); > > > } > > > > > > static const char * > > > -get_monitor_all(const struct smap *ext_ids) > > > +get_monitor_all(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-monitor-all", "false"); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-monitor-all", "false"); > > > } > > > > > > static const char * > > > -get_enable_lflow_cache(const struct smap *ext_ids) > > > +get_enable_lflow_cache(const struct smap *ext_ids, const char > > > *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-enable-lflow-cache", "true"); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-enable-lflow-cache", "true"); > > > } > > > > > > static const char * > > > -get_encap_csum(const struct smap *ext_ids) > > > +get_encap_csum(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - return smap_get_def(ext_ids, "ovn-encap-csum", "true"); > > > + return get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-encap-csum", "true"); > > > } > > > > > > static const char * > > > @@ -189,9 +196,10 @@ get_datapath_type(const struct ovsrec_bridge *br_int) > > > } > > > > > > static bool > > > -get_is_interconn(const struct smap *ext_ids) > > > +get_is_interconn(const struct smap *ext_ids, const char *chassis_id) > > > { > > > - return smap_get_bool(ext_ids, "ovn-is-interconn", false); > > > + return get_chassis_external_id_value_bool( > > > + ext_ids, chassis_id, "ovn-is-interconn", false); > > > } > > > > > > static void > > > @@ -278,22 +286,27 @@ chassis_parse_ovs_config(const struct > > > ovsrec_open_vswitch_table *ovs_table, > > > return false; > > > } > > > > > > - const char *encap_type = smap_get(&cfg->external_ids, > > > "ovn-encap-type"); > > > - const char *encap_ips = smap_get(&cfg->external_ids, "ovn-encap-ip"); > > > + const char *chassis_id = get_ovs_chassis_id(cfg); > > > + const struct smap *ext_ids = &cfg->external_ids; > > > + > > > + const char *encap_type = get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-encap-type", NULL); > > > + const char *encap_ips = get_chassis_external_id_value( > > > + ext_ids, chassis_id, "ovn-encap-ip", NULL); > > > if (!encap_type || !encap_ips) { > > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); > > > VLOG_INFO_RL(&rl, "Need to specify an encap type and ip"); > > > return false; > > > } > > > > > > - ovs_cfg->hostname = get_hostname(&cfg->external_ids); > > > - ovs_cfg->bridge_mappings = get_bridge_mappings(&cfg->external_ids); > > > + ovs_cfg->hostname = get_hostname(ext_ids, chassis_id); > > > + ovs_cfg->bridge_mappings = get_bridge_mappings(ext_ids, chassis_id); > > > ovs_cfg->datapath_type = get_datapath_type(br_int); > > > - ovs_cfg->encap_csum = get_encap_csum(&cfg->external_ids); > > > - ovs_cfg->cms_options = get_cms_options(&cfg->external_ids); > > > - ovs_cfg->monitor_all = get_monitor_all(&cfg->external_ids); > > > - ovs_cfg->chassis_macs = get_chassis_mac_mappings(&cfg->external_ids); > > > - ovs_cfg->enable_lflow_cache = > > > get_enable_lflow_cache(&cfg->external_ids); > > > + ovs_cfg->encap_csum = get_encap_csum(ext_ids, chassis_id); > > > + ovs_cfg->cms_options = get_cms_options(ext_ids, chassis_id); > > > + ovs_cfg->monitor_all = get_monitor_all(ext_ids, chassis_id); > > > + ovs_cfg->chassis_macs = get_chassis_mac_mappings(ext_ids, > > > chassis_id); > > > + ovs_cfg->enable_lflow_cache = get_enable_lflow_cache(ext_ids, > > > chassis_id); > > > > > > if (!chassis_parse_ovs_encap_type(encap_type, > > > &ovs_cfg->encap_type_set)) { > > > return false; > > > @@ -311,7 +324,7 @@ chassis_parse_ovs_config(const struct > > > ovsrec_open_vswitch_table *ovs_table, > > > sset_destroy(&ovs_cfg->encap_ip_set); > > > } > > > > > > - ovs_cfg->is_interconn = get_is_interconn(&cfg->external_ids); > > > + ovs_cfg->is_interconn = get_is_interconn(ext_ids, chassis_id); > > > > > > return true; > > > } > > > @@ -348,7 +361,7 @@ chassis_other_config_changed(const char > > > *bridge_mappings, > > > const struct sbrec_chassis *chassis_rec) > > > { > > > const char *chassis_bridge_mappings = > > > - get_bridge_mappings(&chassis_rec->other_config); > > > + get_bridge_mappings(&chassis_rec->other_config, NULL); > > > > > > if (strcmp(bridge_mappings, chassis_bridge_mappings)) { > > > return true; > > > @@ -362,28 +375,28 @@ chassis_other_config_changed(const char > > > *bridge_mappings, > > > } > > > > > > const char *chassis_cms_options = > > > - get_cms_options(&chassis_rec->other_config); > > > + get_cms_options(&chassis_rec->other_config, NULL); > > > > > > if (strcmp(cms_options, chassis_cms_options)) { > > > return true; > > > } > > > > > > const char *chassis_monitor_all = > > > - get_monitor_all(&chassis_rec->other_config); > > > + get_monitor_all(&chassis_rec->other_config, NULL); > > > > > > if (strcmp(monitor_all, chassis_monitor_all)) { > > > return true; > > > } > > > > > > const char *chassis_enable_lflow_cache = > > > - get_enable_lflow_cache(&chassis_rec->other_config); > > > + get_enable_lflow_cache(&chassis_rec->other_config, NULL); > > > > > > if (strcmp(enable_lflow_cache, chassis_enable_lflow_cache)) { > > > return true; > > > } > > > > > > const char *chassis_mac_mappings = > > > - get_chassis_mac_mappings(&chassis_rec->other_config); > > > + get_chassis_mac_mappings(&chassis_rec->other_config, NULL); > > > if (strcmp(chassis_macs, chassis_mac_mappings)) { > > > return true; > > > } > > > @@ -802,7 +815,7 @@ chassis_get_mac(const struct sbrec_chassis > > > *chassis_rec, > > > struct eth_addr *chassis_mac) > > > { > > > const char *tokens > > > - = get_chassis_mac_mappings(&chassis_rec->other_config); > > > + = get_chassis_mac_mappings(&chassis_rec->other_config, NULL); > > > if (!tokens[0]) { > > > return false; > > > } > > > diff --git a/controller/chassis.h b/controller/chassis.h > > > index 220f726b9..c7345f0fa 100644 > > > --- a/controller/chassis.h > > > +++ b/controller/chassis.h > > > @@ -49,7 +49,8 @@ bool chassis_get_mac(const struct sbrec_chassis > > > *chassis, > > > const char *bridge_mapping, > > > struct eth_addr *chassis_mac); > > > const char *chassis_get_id(void); > > > -const char * get_chassis_mac_mappings(const struct smap *ext_ids); > > > +const char * get_chassis_mac_mappings(const struct smap *ext_ids, > > > + const char *chassis_id); > > > > > > > > > #endif /* controller/chassis.h */ > > > diff --git a/controller/encaps.c b/controller/encaps.c > > > index 7eac4bb06..f456d5370 100644 > > > --- a/controller/encaps.c > > > +++ b/controller/encaps.c > > > @@ -73,7 +73,7 @@ tunnel_create_name(struct tunnel_ctx *tc, const char > > > *chassis_id) > > > > > > for (i = 0; i < UINT16_MAX; i++) { > > > char *port_name; > > > - port_name = xasprintf("ovn-%.6s-%x", chassis_id, i); > > > + port_name = xasprintf("ovn-%s-%x", chassis_id, i); > > > > > > if (!sset_contains(&tc->port_names, port_name)) { > > > return port_name; > > > @@ -291,9 +291,31 @@ chassis_tzones_overlap(const struct sset > > > *transport_zones, > > > return false; > > > } > > > > > > +static bool > > > +is_tunnel_type(const char *port_type) > > > +{ > > > + static const char *tunnel_types[3] = { "geneve", "vxlan", "stt" }; > > > + for (size_t t = 0; t < 3; t++) { > > > + if (!strcmp(port_type, tunnel_types[t])) { > > > + return true; > > > + } > > > + } > > > + return false; > > > +} > > > + > > > +static bool > > > +is_tunnel_port(const struct ovsrec_port *port) > > > +{ > > > + for (size_t i = 0; i < port->n_interfaces; i++) { > > > + if (is_tunnel_type(port->interfaces[i]->type)) { > > > + return true; > > > + } > > > + } > > > + return false; > > > +} > > > + > > > void > > > encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > - const struct ovsrec_bridge_table *bridge_table, > > > const struct ovsrec_bridge *br_int, > > > const struct sbrec_chassis_table *chassis_table, > > > const struct sbrec_chassis *this_chassis, > > > @@ -305,7 +327,6 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > } > > > > > > const struct sbrec_chassis *chassis_rec; > > > - const struct ovsrec_bridge *br; > > > > > > struct tunnel_ctx tc = { > > > .chassis = SHASH_INITIALIZER(&tc.chassis), > > > @@ -320,28 +341,29 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > > > > /* Collect all port names into tc.port_names. > > > * > > > - * Collect all the OVN-created tunnels into tc.tunnel_hmap. */ > > > - OVSREC_BRIDGE_TABLE_FOR_EACH (br, bridge_table) { > > > - for (size_t i = 0; i < br->n_ports; i++) { > > > - const struct ovsrec_port *port = br->ports[i]; > > > - sset_add(&tc.port_names, port->name); > > > - > > > - /* > > > - * note that the id here is not just the chassis name, but > > > the > > > - * combination of <chassis_name><delim><encap_ip> > > > - */ > > > - const char *id = smap_get(&port->external_ids, > > > "ovn-chassis-id"); > > > - if (id) { > > > - if (!shash_find(&tc.chassis, id)) { > > > - struct chassis_node *chassis = xzalloc(sizeof > > > *chassis); > > > - chassis->bridge = br; > > > - chassis->port = port; > > > - shash_add_assert(&tc.chassis, id, chassis); > > > - } else { > > > - /* Duplicate port for ovn-chassis-id. Arbitrarily > > > choose > > > - * to delete this one. */ > > > - ovsrec_bridge_update_ports_delvalue(br, port); > > > - } > > > + * Collect all OVN-created tunnels of the bridge into > > > tc.tunnel_hmap. */ > > > + for (size_t i = 0; i < br_int->n_ports; i++) { > > > + const struct ovsrec_port *port = br_int->ports[i]; > > > + if (!is_tunnel_port(port)) { > > > + continue; > > > + } > > > + sset_add(&tc.port_names, port->name); > > > + > > > + /* > > > + * note that the id here is not just the chassis name, but the > > > + * combination of <chassis_name><delim><encap_ip> > > > + */ > > > + const char *id = smap_get(&port->external_ids, "ovn-chassis-id"); > > > + if (id) { > > > + if (!shash_find(&tc.chassis, id)) { > > > + struct chassis_node *chassis = xzalloc(sizeof *chassis); > > > + chassis->bridge = br_int; > > > + chassis->port = port; > > > + shash_add_assert(&tc.chassis, id, chassis); > > > + } else { > > > + /* Duplicate port for ovn-chassis-id. Arbitrarily choose > > > + * to delete this one. */ > > > + ovsrec_bridge_update_ports_delvalue(br_int, port); > > > } > > > } > > > } > > > @@ -381,6 +403,7 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > shash_delete(&tc.chassis, node); > > > free(chassis); > > > } > > > + > > > shash_destroy(&tc.chassis); > > > sset_destroy(&tc.port_names); > > > } > > > @@ -400,6 +423,9 @@ encaps_cleanup(struct ovsdb_idl_txn *ovs_idl_txn, > > > = xmalloc(sizeof *br_int->ports * br_int->n_ports); > > > size_t n = 0; > > > for (size_t i = 0; i < br_int->n_ports; i++) { > > > + if (!is_tunnel_port(br_int->ports[i])) { > > > + continue; > > > + } > > > if (!smap_get(&br_int->ports[i]->external_ids, > > > "ovn-chassis-id")) { > > > ports[n++] = br_int->ports[i]; > > > } > > > diff --git a/controller/encaps.h b/controller/encaps.h > > > index f488393c4..aff85097f 100644 > > > --- a/controller/encaps.h > > > +++ b/controller/encaps.h > > > @@ -30,7 +30,6 @@ struct sset; > > > > > > void encaps_register_ovs_idl(struct ovsdb_idl *); > > > void encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > - const struct ovsrec_bridge_table *, > > > const struct ovsrec_bridge *br_int, > > > const struct sbrec_chassis_table *, > > > const struct sbrec_chassis *, > > > diff --git a/controller/ovn-controller.8.xml > > > b/controller/ovn-controller.8.xml > > > index 16bc47b20..c181a0fa6 100644 > > > --- a/controller/ovn-controller.8.xml > > > +++ b/controller/ovn-controller.8.xml > > > @@ -235,6 +235,19 @@ > > > </dd> > > > </dl> > > > > > > + <p> > > > + Note that every <code>external_ids:*</code> key listed above has > > > its > > > + <code>external_ids:*-chassis_name</code> counterpart keys that > > > allow to > > > + configure values specific to chassis running on the same OVSDB. For > > > + example, if two chassis named <code>blue</code> and > > > <code>red</code> are > > > + available on the same host, then an admin may configure different > > > + <code>ovn-cms-options</code> for each of them by setting > > > + <code>external_ids:ovn-cms-options-blue</code> and > > > + <code>external_ids:ovn-cms-options-red</code> keys in the > > > database. The > > > + only key that is not available for per-chassis configuration is > > > + <code>external_ids:system-id</code>. > > > + </p> > > > + > > > <p> > > > <code>ovn-controller</code> reads the following values from the > > > <code>Open_vSwitch</code> database of the local OVS instance: > > > @@ -286,7 +299,9 @@ > > > The presence of this key identifies a tunnel port within the > > > integration bridge as one created by <code>ovn-controller</code> > > > to > > > reach a remote chassis. Its value is the chassis ID of the > > > remote > > > - chassis. > > > + chassis. Alternatively, for patch ports, the key identifies the > > > name of > > > + the chassis that owns it, in case of multiple virtual chassis > > > running > > > + on the same host. > > > </dd> > > > > > > <dt> > > > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c > > > index a06cae3cc..240ca3cc6 100644 > > > --- a/controller/ovn-controller.c > > > +++ b/controller/ovn-controller.c > > > @@ -18,10 +18,14 @@ > > > #include "ovn-controller.h" > > > > > > #include <errno.h> > > > +#include <fcntl.h> > > > #include <getopt.h> > > > #include <signal.h> > > > #include <stdlib.h> > > > #include <string.h> > > > +#include <sys/types.h> > > > +#include <sys/stat.h> > > > +#include <unistd.h> > > > > > > #include "bfd.h" > > > #include "binding.h" > > > @@ -46,6 +50,7 @@ > > > #include "lib/extend-table.h" > > > #include "lib/ip-mcast-index.h" > > > #include "lib/mcast-group-index.h" > > > +#include "lib/ovn-dirs.h" > > > #include "lib/ovn-sb-idl.h" > > > #include "lib/ovn-util.h" > > > #include "patch.h" > > > @@ -85,6 +90,12 @@ static unixctl_cb_func debug_delay_nb_cfg_report; > > > > > > #define CONTROLLER_LOOP_STOPWATCH_NAME "ovn-controller-flow-generation" > > > > > > +/* These variables never change after initialization and can be safely > > > used in > > > + * I-P engine. If later we decide to allow to dynamically change them, > > > I-P > > > + * machinery will need some adjustments. */ > > > +static char *controller_chassis = NULL; > > > +static char *system_id_override = NULL; > > > + > > > static char *parse_options(int argc, char *argv[]); > > > OVS_NO_RETURN static void usage(void); > > > > > > @@ -260,7 +271,9 @@ out: > > > static const char * > > > br_int_name(const struct ovsrec_open_vswitch *cfg) > > > { > > > - return smap_get_def(&cfg->external_ids, "ovn-bridge", > > > DEFAULT_BRIDGE_NAME); > > > + return get_chassis_external_id_value( > > > + &cfg->external_ids, get_ovs_chassis_id(cfg), > > > + "ovn-bridge", DEFAULT_BRIDGE_NAME); > > > } > > > > > > static const struct ovsrec_bridge * > > > @@ -361,8 +374,9 @@ process_br_int(struct ovsdb_idl_txn *ovs_idl_txn, > > > const struct ovsrec_open_vswitch *cfg; > > > cfg = ovsrec_open_vswitch_table_first(ovs_table); > > > ovs_assert(cfg); > > > - const char *datapath_type = smap_get(&cfg->external_ids, > > > - "ovn-bridge-datapath-type"); > > > + const char *datapath_type = get_chassis_external_id_value( > > > + &cfg->external_ids, get_ovs_chassis_id(cfg), > > > + "ovn-bridge-datapath-type", NULL); > > > /* Check for the datapath_type and set it only if it is defined > > > in > > > * cfg. */ > > > if (datapath_type && strcmp(br_int->datapath_type, > > > datapath_type)) { > > > @@ -372,17 +386,46 @@ process_br_int(struct ovsdb_idl_txn *ovs_idl_txn, > > > return br_int; > > > } > > > > > > -static const char * > > > -get_ovs_chassis_id(const struct ovsrec_open_vswitch_table *ovs_table) > > > +static char *get_file_system_id_override(void) > > > { > > > - const struct ovsrec_open_vswitch *cfg > > > - = ovsrec_open_vswitch_table_first(ovs_table); > > > + char *ret = NULL; > > > + char *filename = xasprintf("%s/system-id-override", > > > ovn_sysconfdir()); > > > + errno = 0; > > > + int fd = open(filename, O_RDONLY); > > > + if (fd != -1) { > > > + char file_system_id[64]; > > > + int nread = read(fd, file_system_id, sizeof file_system_id); > > > + if (nread) { > > > + file_system_id[nread] = '\0'; > > > + if (file_system_id[nread - 1] == '\n') { > > > + file_system_id[nread - 1] = '\0'; > > > + } > > > + ret = xstrdup(file_system_id); > > > + } > > > + close(fd); > > > + } > > > + > > > + free(filename); > > > + return ret; > > > +} > > > + > > > +const char * > > > +get_ovs_chassis_id(const struct ovsrec_open_vswitch *cfg) > > > +{ > > > + if (controller_chassis) { > > > + return controller_chassis; > > > + } > > > + > > > + if (system_id_override) { > > > + return system_id_override; > > > + } > > > + > > > const char *chassis_id = cfg ? smap_get(&cfg->external_ids, > > > "system-id") > > > : NULL; > > > - > > > if (!chassis_id) { > > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); > > > - VLOG_WARN_RL(&rl, "'system-id' in Open_vSwitch database is > > > missing."); > > > + VLOG_WARN_RL(&rl, "Failed to detect system-id, " > > > + "configuration not found."); > > > } > > > > > > return chassis_id; > > > @@ -477,10 +520,12 @@ static int > > > get_ofctrl_probe_interval(struct ovsdb_idl *ovs_idl) > > > { > > > const struct ovsrec_open_vswitch *cfg = > > > ovsrec_open_vswitch_first(ovs_idl); > > > - return !cfg ? OFCTRL_DEFAULT_PROBE_INTERVAL_SEC : > > > - smap_get_int(&cfg->external_ids, > > > - "ovn-openflow-probe-interval", > > > - OFCTRL_DEFAULT_PROBE_INTERVAL_SEC); > > > + if (!cfg) { > > > + return OFCTRL_DEFAULT_PROBE_INTERVAL_SEC; > > > + } > > > + return get_chassis_external_id_value_int( > > > + &cfg->external_ids, get_ovs_chassis_id(cfg), > > > + "ovn-openflow-probe-interval", > > > OFCTRL_DEFAULT_PROBE_INTERVAL_SEC); > > > } > > > > > > /* Retrieves the pointer to the OVN Southbound database from 'ovs_idl' > > > and > > > @@ -496,18 +541,21 @@ update_sb_db(struct ovsdb_idl *ovs_idl, struct > > > ovsdb_idl *ovnsb_idl, > > > } > > > > > > /* Set remote based on user configuration. */ > > > - const char *remote = smap_get(&cfg->external_ids, "ovn-remote"); > > > + const char *chassis_id = get_ovs_chassis_id(cfg); > > > + const char *remote = get_chassis_external_id_value( > > > + &cfg->external_ids, chassis_id, "ovn-remote", NULL); > > > ovsdb_idl_set_remote(ovnsb_idl, remote, true); > > > > > > /* Set probe interval, based on user configuration and the remote. */ > > > int default_interval = (remote && > > > !stream_or_pstream_needs_probes(remote) > > > ? 0 : DEFAULT_PROBE_INTERVAL_MSEC); > > > - int interval = smap_get_int(&cfg->external_ids, > > > - "ovn-remote-probe-interval", > > > default_interval); > > > + int interval = get_chassis_external_id_value_int( > > > + &cfg->external_ids, chassis_id, "ovn-remote-probe-interval", > > > + default_interval); > > > ovsdb_idl_set_probe_interval(ovnsb_idl, interval); > > > > > > - bool monitor_all = smap_get_bool(&cfg->external_ids, > > > "ovn-monitor-all", > > > - false); > > > + bool monitor_all = get_chassis_external_id_value_bool( > > > + &cfg->external_ids, chassis_id, "ovn-monitor-all", false); > > > if (monitor_all) { > > > /* Always call update_sb_monitors when monitor_all is true. > > > * Otherwise, don't call it here, because there would be > > > unnecessary > > > @@ -1166,7 +1214,9 @@ init_binding_ctx(struct engine_node *node, > > > struct ovsrec_bridge_table *bridge_table = > > > (struct ovsrec_bridge_table *)EN_OVSDB_GET( > > > engine_get_input("OVS_bridge", node)); > > > - const char *chassis_id = get_ovs_chassis_id(ovs_table); > > > + const struct ovsrec_open_vswitch *cfg = > > > + ovsrec_open_vswitch_table_first(ovs_table); > > > + const char *chassis_id = get_ovs_chassis_id(cfg); > > > const struct ovsrec_bridge *br_int = get_br_int(bridge_table, > > > ovs_table); > > > > > > ovs_assert(br_int && chassis_id); > > > @@ -2432,6 +2482,10 @@ main(int argc, char *argv[]) > > > exiting = false; > > > restart = false; > > > bool sb_monitor_all = false; > > > + > > > + /* Read from system-id-override file once on startup. */ > > > + system_id_override = get_file_system_id_override(); > > > + > > > while (!exiting) { > > > /* If we're paused just run the unixctl server and skip most of > > > the > > > * processing loop. > > > @@ -2498,7 +2552,9 @@ main(int argc, char *argv[]) > > > sbrec_chassis_private_table_get(ovnsb_idl_loop.idl); > > > const struct ovsrec_bridge *br_int = > > > process_br_int(ovs_idl_txn, bridge_table, ovs_table); > > > - const char *chassis_id = get_ovs_chassis_id(ovs_table); > > > + const struct ovsrec_open_vswitch *cfg = > > > + ovsrec_open_vswitch_table_first(ovs_table); > > > + const char *chassis_id = get_ovs_chassis_id(cfg); > > > const struct sbrec_chassis *chassis = NULL; > > > const struct sbrec_chassis_private *chassis_private = NULL; > > > if (chassis_id) { > > > @@ -2518,7 +2574,7 @@ main(int argc, char *argv[]) > > > > > > if (chassis) { > > > encaps_run(ovs_idl_txn, > > > - bridge_table, br_int, > > > + br_int, > > > > > > sbrec_chassis_table_get(ovnsb_idl_loop.idl), > > > chassis, > > > sbrec_sb_global_first(ovnsb_idl_loop.idl), > > > @@ -2779,6 +2835,8 @@ loop_done: > > > ovsdb_idl_loop_destroy(&ovs_idl_loop); > > > ovsdb_idl_loop_destroy(&ovnsb_idl_loop); > > > > > > + free(controller_chassis); > > > + free(system_id_override); > > > free(ovs_remote); > > > service_stop(); > > > > > > @@ -2804,6 +2862,7 @@ parse_options(int argc, char *argv[]) > > > STREAM_SSL_LONG_OPTIONS, > > > {"peer-ca-cert", required_argument, NULL, OPT_PEER_CA_CERT}, > > > {"bootstrap-ca-cert", required_argument, NULL, > > > OPT_BOOTSTRAP_CA_CERT}, > > > + {"chassis", required_argument, NULL, 'n'}, > > > {NULL, 0, NULL, 0} > > > }; > > > char *short_options = > > > ovs_cmdl_long_options_to_short_options(long_options); > > > @@ -2836,6 +2895,10 @@ parse_options(int argc, char *argv[]) > > > stream_ssl_set_ca_cert_file(optarg, true); > > > break; > > > > > > + case 'n': > > > + controller_chassis = xstrdup(optarg); > > > + break; > > > + > > > case '?': > > > exit(EXIT_FAILURE); > > > > > > @@ -2871,6 +2934,7 @@ usage(void) > > > daemon_usage(); > > > vlog_usage(); > > > printf("\nOther options:\n" > > > + " -n custom chassis name\n" > > > " -h, --help display this help message\n" > > > " -V, --version display version information\n"); > > > exit(EXIT_SUCCESS); > > > diff --git a/controller/ovn-controller.h b/controller/ovn-controller.h > > > index 5d9466880..9994dd777 100644 > > > --- a/controller/ovn-controller.h > > > +++ b/controller/ovn-controller.h > > > @@ -21,6 +21,7 @@ > > > #include "lib/ovn-sb-idl.h" > > > > > > struct ovsrec_bridge_table; > > > +struct ovsrec_open_vswitch; > > > > > > /* Linux supports a maximum of 64K zones, which seems like a fine > > > default. */ > > > #define MAX_CT_ZONES 65535 > > > @@ -87,4 +88,7 @@ enum chassis_tunnel_type { > > > > > > uint32_t get_tunnel_type(const char *name); > > > > > > +const char *get_ovs_chassis_id(const struct ovsrec_open_vswitch *cfg); > > > +bool is_concurrent_chassis(const struct ovsrec_open_vswitch *cfg); > > > + > > > #endif /* controller/ovn-controller.h */ > > > diff --git a/controller/patch.c b/controller/patch.c > > > index a2a7bcd79..3b3df278c 100644 > > > --- a/controller/patch.c > > > +++ b/controller/patch.c > > > @@ -76,6 +76,7 @@ create_patch_port(struct ovsdb_idl_txn *ovs_idl_txn, > > > const char *key, const char *value, > > > const struct ovsrec_bridge *src, const char *src_name, > > > const struct ovsrec_bridge *dst, const char *dst_name, > > > + const char *chassis_name, > > > struct shash *existing_ports) > > > { > > > for (size_t i = 0; i < src->n_ports; i++) { > > > @@ -101,7 +102,8 @@ create_patch_port(struct ovsdb_idl_txn *ovs_idl_txn, > > > port = ovsrec_port_insert(ovs_idl_txn); > > > ovsrec_port_set_name(port, src_name); > > > ovsrec_port_set_interfaces(port, &iface, 1); > > > - const struct smap ids = SMAP_CONST1(&ids, key, value); > > > + const struct smap ids = SMAP_CONST2(&ids, key, value, > > > + "ovn-chassis-id", chassis_name); > > > ovsrec_port_set_external_ids(port, &ids); > > > > > > struct ovsrec_port **ports; > > > @@ -157,7 +159,9 @@ add_ovs_bridge_mappings(const struct > > > ovsrec_open_vswitch_table *ovs_table, > > > const char *mappings_cfg; > > > char *cur, *next, *start; > > > > > > - mappings_cfg = smap_get(&cfg->external_ids, > > > "ovn-bridge-mappings"); > > > + mappings_cfg = get_chassis_external_id_value( > > > + &cfg->external_ids, get_ovs_chassis_id(cfg), > > > + "ovn-bridge-mappings", NULL); > > > if (!mappings_cfg || !mappings_cfg[0]) { > > > return; > > > } > > > @@ -269,9 +273,11 @@ add_bridge_mappings(struct ovsdb_idl_txn > > > *ovs_idl_txn, > > > char *name1 = patch_port_name(br_int->name, > > > binding->logical_port); > > > char *name2 = patch_port_name(binding->logical_port, > > > br_int->name); > > > create_patch_port(ovs_idl_txn, patch_port_id, > > > binding->logical_port, > > > - br_int, name1, br_ln, name2, existing_ports); > > > + br_int, name1, br_ln, name2, chassis->name, > > > + existing_ports); > > > create_patch_port(ovs_idl_txn, patch_port_id, > > > binding->logical_port, > > > - br_ln, name2, br_int, name1, existing_ports); > > > + br_ln, name2, br_int, name1, chassis->name, > > > + existing_ports); > > > free(name1); > > > free(name2); > > > } > > > @@ -323,6 +329,12 @@ patch_run(struct ovsdb_idl_txn *ovs_idl_txn, > > > SHASH_FOR_EACH_SAFE (port_node, port_next_node, &existing_ports) { > > > port = port_node->data; > > > shash_delete(&existing_ports, port_node); > > > + > > > + const char *port_chassis = smap_get(&port->external_ids, > > > + "ovn-chassis-id"); > > > + if (port_chassis && strcmp(port_chassis, chassis->name)) { > > > + continue; > > > + } > > > remove_port(bridge_table, port); > > > } > > > shash_destroy(&existing_ports); > > > diff --git a/controller/physical.c b/controller/physical.c > > > index a7f3efd2f..306f7f912 100644 > > > --- a/controller/physical.c > > > +++ b/controller/physical.c > > > @@ -432,7 +432,7 @@ populate_remote_chassis_macs(const struct > > > sbrec_chassis *my_chassis, > > > } > > > > > > const char *tokens > > > - = get_chassis_mac_mappings(&chassis->other_config); > > > + = get_chassis_mac_mappings(&chassis->other_config, NULL); > > > > > > if (!strlen(tokens)) { > > > continue; > > > diff --git a/lib/ovn-util.c b/lib/ovn-util.c > > > index 1daf66503..93057af60 100644 > > > --- a/lib/ovn-util.c > > > +++ b/lib/ovn-util.c > > > @@ -667,3 +667,53 @@ ovn_smap_get_uint(const struct smap *smap, const > > > char *key, unsigned int def) > > > > > > return u_value; > > > } > > > + > > > +const char * > > > +get_chassis_external_id_value(const struct smap *external_ids, > > > + const char *chassis_id, const char > > > *option_key, > > > + const char *def) > > > +{ > > > + const char *option_value = NULL; > > > + if (chassis_id != NULL) { > > > + char *chassis_option_key = xasprintf("%s-%s", option_key, > > > chassis_id); > > > + option_value = smap_get(external_ids, chassis_option_key); > > > + free(chassis_option_key); > > > + } > > > + if (!option_value) { > > > + option_value = smap_get_def(external_ids, option_key, def); > > > + } > > > + return option_value; > > > +} > > > + > > > +int > > > +get_chassis_external_id_value_int(const struct smap *external_ids, > > > + const char *chassis_id, > > > + const char *option_key, > > > + int def) > > > +{ > > > + const char *value = get_chassis_external_id_value( > > > + external_ids, chassis_id, option_key, NULL); > > > + > > > + int i_value; > > > + if (!value || !str_to_int(value, 10, &i_value)) { > > > + return def; > > > + } > > > + > > > + return i_value; > > > +} > > > + > > > +bool > > > +get_chassis_external_id_value_bool(const struct smap *external_ids, > > > + const char *chassis_id, > > > + const char *option_key, > > > + bool def) > > > +{ > > > + const char *value = get_chassis_external_id_value( > > > + external_ids, chassis_id, option_key, ""); > > > + > > > + if (def) { > > > + return strcasecmp("false", value) != 0; > > > + } else { > > > + return !strcasecmp("true", value); > > > + } > > > +} > > > diff --git a/lib/ovn-util.h b/lib/ovn-util.h > > > index 1d22a691f..143b42248 100644 > > > --- a/lib/ovn-util.h > > > +++ b/lib/ovn-util.h > > > @@ -18,6 +18,7 @@ > > > > > > #include "lib/packets.h" > > > #include "include/ovn/version.h" > > > +#include "smap.h" > > > > > > #define ovn_set_program_name(name) \ > > > ovs_set_program_name(name, OVN_PACKAGE_VERSION) > > > @@ -157,6 +158,23 @@ char *normalize_v46_prefix(const struct v46_ip > > > *prefix, unsigned int plen); > > > unsigned int ovn_smap_get_uint(const struct smap *smap, const char *key, > > > unsigned int def); > > > > > > +const char * > > > +get_chassis_external_id_value(const struct smap *external_ids, > > > + const char *chassis_id, const char > > > *option_key, > > > + const char *def); > > > + > > > +int > > > +get_chassis_external_id_value_int(const struct smap *external_ids, > > > + const char *chassis_id, > > > + const char *option_key, > > > + int def); > > > + > > > +bool > > > +get_chassis_external_id_value_bool(const struct smap *external_ids, > > > + const char *chassis_id, > > > + const char *option_key, > > > + bool def); > > > + > > > /* Returns a lowercase copy of orig. > > > * Caller must free the returned string. > > > */ > > > diff --git a/ovn-sb.xml b/ovn-sb.xml > > > index 182ff0a8a..bcc2ca35a 100644 > > > --- a/ovn-sb.xml > > > +++ b/ovn-sb.xml > > > @@ -240,10 +240,12 @@ > > > > > > <column name="name"> > > > OVN does not prescribe a particular format for chassis names. > > > - ovn-controller populates this column using <ref key="system-id" > > > - table="Open_vSwitch" column="external_ids" db="Open_vSwitch"/> > > > - in the Open_vSwitch database's <ref table="Open_vSwitch" > > > - db="Open_vSwitch"/> table. ovn-controller-vtep populates this > > > + ovn-controller populates this column using the <code>-n</code> > > > + CLI argument, or <code>system-id-override</code> configuration > > > file, or > > > + <ref key="system-id" table="Open_vSwitch" column="external_ids" > > > + db="Open_vSwitch"/> in the Open_vSwitch database's > > > + <ref table="Open_vSwitch" db="Open_vSwitch"/> table. > > > + ovn-controller-vtep populates this > > > column with <ref table="Physical_Switch" column="name" > > > db="hardware_vtep"/> in the hardware_vtep database's > > > <ref table="Physical_Switch" db="hardware_vtep"/> table. > > > diff --git a/tests/ovn-controller.at b/tests/ovn-controller.at > > > index d8061345f..0ab00a842 100644 > > > --- a/tests/ovn-controller.at > > > +++ b/tests/ovn-controller.at > > > @@ -50,8 +50,7 @@ patch > > > # is mirrored into the Chassis record in the OVN_Southbound db. > > > check_bridge_mappings () { > > > local_mappings=$1 > > > - sysid=$(ovs-vsctl get Open_vSwitch . external_ids:system-id) > > > - OVS_WAIT_UNTIL([test x"${local_mappings}" = x$(ovn-sbctl get Chassis > > > ${sysid} other_config:ovn-bridge-mappings | sed -e 's/\"//g')]) > > > + OVS_WAIT_UNTIL([test x"${local_mappings}" = x$(ovn-sbctl get Chassis > > > ${sandbox} other_config:ovn-bridge-mappings | sed -e 's/\"//g')]) > > > } > > > > > > # Initially there should be no patch ports. > > > @@ -133,13 +132,13 @@ ovs-vsctl \ > > > -- add-br br-eth2 > > > ovn_attach n1 br-phys 192.168.0.1 > > > > > > -sysid=$(ovs-vsctl get Open_vSwitch . external_ids:system-id) > > > +sysid=${sandbox} > > > > > > # Make sure that the datapath_type set in the Bridge table > > > # is mirrored into the Chassis record in the OVN_Southbound db. > > > check_datapath_type () { > > > datapath_type=$1 > > > - chassis_datapath_type=$(ovn-sbctl get Chassis ${sysid} > > > other_config:datapath-type | sed -e 's/"//g') #" > > > + chassis_datapath_type=$(ovn-sbctl get Chassis ${sandbox} > > > other_config:datapath-type | sed -e 's/"//g') #" > > > test "${datapath_type}" = "${chassis_datapath_type}" > > > } > > > > > > @@ -187,7 +186,7 @@ OVS_WAIT_UNTIL([ > > > test "${expected_iface_types}" = "${chassis_iface_types}" > > > ]) > > > > > > -# Change the value of external_ids:system-id and make sure it's mirrored > > > +# Set the value of external_ids:system-id and make sure it's mirrored > > > # in the Chassis record in the OVN_Southbound database. > > > sysid=${sysid}-foo > > > ovs-vsctl set Open_vSwitch . external-ids:system-id="${sysid}" > > > @@ -256,7 +255,7 @@ ovs-vsctl \ > > > ovn_attach n1 br-phys 192.168.0.1 > > > > > > check_tunnel_property () { > > > - test "`ovs-vsctl get interface ovn-fakech-0 $1`" = "$2" > > > + test "`ovs-vsctl get interface ovn-fakechassis-0 $1`" = "$2" > > > } > > > > > > # Start off with a remote chassis supporting STT > > > diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at > > > index a6719be83..f846c6336 100644 > > > --- a/tests/ovn-macros.at > > > +++ b/tests/ovn-macros.at > > > @@ -215,7 +215,7 @@ net_attach () { > > > > > > # ovn_az_attach AZ NETWORK BRIDGE IP [MASKLEN] > > > ovn_az_attach() { > > > - local az=$1 net=$2 bridge=$3 ip=$4 masklen=${5-24} > > > encap=${6-geneve,vxlan} > > > + local az=$1 net=$2 bridge=$3 ip=$4 masklen=${5-24} > > > encap=${6-geneve,vxlan} intbr=${7-br-int} chassis=$8 > > > net_attach $net $bridge || return 1 > > > > > > mac=`ovs-vsctl get Interface $bridge mac_in_use | sed s/\"//g` > > > @@ -229,15 +229,48 @@ ovn_az_attach() { > > > else > > > ovn_remote=unix:$ovs_base/$az/ovn-sb/ovn-sb.sock > > > fi > > > + > > > + if [[ -n "${chassis}" ]]; then > > > + bridge_key=ovn-bridge-${chassis} > > > + remote_key=ovn-remote-${chassis} > > > + encap_type_key=ovn-encap-type-${chassis} > > > + encap_ip_key=ovn-encap-ip-${chassis} > > > + chassis_args="-n $chassis" > > > + chassis_vsctl_args= > > > + else > > > + bridge_key=ovn-bridge > > > + remote_key=ovn-remote > > > + encap_type_key=ovn-encap-type > > > + encap_ip_key=ovn-encap-ip > > > + chassis=$sandbox > > > + chassis_args= > > > + chassis_vsctl_args="-- set Open_vSwitch . > > > external-ids:system-id=$chassis" > > > + fi > > > + > > > ovs-vsctl \ > > > - -- set Open_vSwitch . external-ids:system-id=$sandbox \ > > > - -- set Open_vSwitch . external-ids:ovn-remote=$ovn_remote \ > > > - -- set Open_vSwitch . external-ids:ovn-encap-type=$encap \ > > > - -- set Open_vSwitch . external-ids:ovn-encap-ip=$ip \ > > > - -- --may-exist add-br br-int \ > > > - -- set bridge br-int fail-mode=secure > > > other-config:disable-in-band=true \ > > > + $chassis_vsctl_args \ > > > + -- set Open_vSwitch . external-ids:$bridge_key=$intbr \ > > > + -- set Open_vSwitch . external-ids:$remote_key=$ovn_remote \ > > > + -- set Open_vSwitch . external-ids:$encap_type_key=$encap \ > > > + -- set Open_vSwitch . external-ids:$encap_ip_key=$ip \ > > > + -- --may-exist add-br ${intbr} \ > > > + -- set bridge ${intbr} fail-mode=secure > > > other-config:disable-in-band=true \ > > > || return 1 > > > - start_daemon ovn-controller || return 1 > > > + > > > + if [[ "${intbr}" = br-int ]]; then > > > + pidfile="${OVS_RUNDIR}/ovn-controller.pid" > > > + logfile="${OVS_LOGDIR}/ovn-controller.log" > > > + else > > > + pidfile="${OVS_RUNDIR}/ovn-controller-${intbr}.pid" > > > + logfile="${OVS_LOGDIR}/ovn-controller-${chassis}.log" > > > + fi > > > + > > > + ovn-controller \ > > > + ${chassis_args} \ > > > + -vconsole:off --detach --no-chdir \ > > > + --pidfile=${pidfile} \ > > > + --log-file=${logfile} || return 1 > > > + on_exit "test -e \"$pidfile\" && kill \`cat \"$pidfile\"\`" > > > } > > > > > > # ovn_attach NETWORK BRIDGE IP [MASKLEN] > > > diff --git a/tests/ovn.at b/tests/ovn.at > > > index 27a94e64f..aa291da4b 100644 > > > --- a/tests/ovn.at > > > +++ b/tests/ovn.at > > > @@ -1735,6 +1735,107 @@ AT_CLEANUP > > > > > > AT_BANNER([OVN end-to-end tests]) > > > > > > +AT_SETUP([ovn -- 3 virtual hosts, same node]) > > > +AT_KEYWORDS([ovn]) > > > +ovn_start > > > +ovn-nbctl ls-add lsw0 > > > +net_add n1 > > > +sim_add hv > > > + > > > +as hv > > > +for i in 1 2 3; do > > > + chassis=host-$i > > > + ovs-vsctl add-br br-phys-$i > > > + ovn_attach n1 br-phys-$i 192.168.0.$i 24 geneve br-int-$i $chassis > > > + > > > + for j in 1 2 3; do > > > + lpname=lp$i$j > > > + ovn-nbctl lsp-add lsw0 $lpname > > > + ovn-nbctl --wait=hv --timeout=3 lsp-set-options $lpname > > > requested-chassis=$chassis > > > + ovs-vsctl add-port br-int-$i vif$i$j -- set Interface vif$i$j > > > external-ids:iface-id=$lpname > > > + OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up $lpname` = xup]) > > > + > > > + pb_chassis_id=$(ovn-sbctl --bare --columns chassis list > > > port_binding $lpname) > > > + pb_chassis_name=$(ovn-sbctl get chassis $pb_chassis_id name) > > > + AT_FAIL_IF([test x$pb_chassis_name != x$chassis]) > > > + done > > > +done > > > + > > > +for i in 1 2 3; do > > > + > expout > > > + for vif in 1 2 3; do > > > + echo vif$i$vif >> expout > > > + done > > > + AT_CHECK([ovs-vsctl list-ports br-int-$i | grep vif], [0], [expout]) > > > +done > > > + > > > +AT_CLEANUP > > > + > > > +AT_SETUP([ovn -- system-id in file]) > > > +AT_KEYWORDS([ovn]) > > > + > > > +ovn_start > > > +net_add n1 > > > +sim_add hv > > > + > > > +as hv > > > + > > > +echo otherid > ${OVN_SYSCONFDIR}/system-id-override > > > +ovs-vsctl add-br br-phys > > > +ovn_attach n1 br-phys 192.168.0.1 > > > + > > > +# system-id-override file overrides chassis name selected via cli > > > +echo otherid > expout > > > +AT_CHECK([ovn-sbctl --bare --columns name list chassis], [0], [expout]) > > > + > > > +AT_CLEANUP > > > + > > > +AT_SETUP([ovn -- concurrent controllers avoid fighting for each others' > > > resources]) > > > +AT_KEYWORDS([ovn]) > > > + > > > +ovn_start > > > +sim_add hv > > > + > > > +for i in 1 2; do > > > + net_add n-$i > > > +done > > > + > > > +as hv > > > +for i in 1 2; do > > > + AT_CHECK([ovn-nbctl ls-add ls-$i]) > > > + AT_CHECK([ovn-nbctl lsp-add ls-$i ln_port-$i]) > > > + AT_CHECK([ovn-nbctl lsp-set-addresses ln_port-$i unknown]) > > > + AT_CHECK([ovn-nbctl lsp-set-type ln_port-$i localnet]) > > > + AT_CHECK([ovn-nbctl --wait=hv lsp-set-options ln_port-$i > > > network_name=phys-$i]) > > > +done > > > + > > > +for i in 1 2; do > > > + as hv > > > + ovs-vsctl add-br br-phys-$i > > > + ovs-vsctl set open . > > > external-ids:ovn-bridge-mappings-controller-$i=phys-$i:br-phys-$i > > > + ovn_attach n-$i br-phys-$i 192.168.0.$i 24 geneve br-int-$i > > > controller-$i > > > + > > > + ovs-vsctl add-port br-int-$i vif-$i -- set Interface vif-$i > > > external-ids:iface-id=lp-$i > > > + ovn-nbctl lsp-add ls-$i lp-$i > > > + OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp-$i` = xup]) > > > +done > > > + > > > +# check that both patch ports are present > > > +AT_CHECK([ovs-vsctl --bare --columns=name find interface type="patch" | > > > awk NF | sort], [0], > > > +[[patch-br-int-1-to-ln_port-1 > > > +patch-br-int-2-to-ln_port-2 > > > +patch-ln_port-1-to-br-int-1 > > > +patch-ln_port-2-to-br-int-2 > > > +]]) > > > + > > > +# check that both tunnel endpoints are present > > > +AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" | > > > awk NF | sort], [0], > > > +[[ovn-controller-1-0 > > > +ovn-controller-2-0 > > > +]]) > > > + > > > +AT_CLEANUP > > > + > > > # 3 hypervisors, one logical switch, 3 logical ports per hypervisor > > > AT_SETUP([ovn -- 3 HVs, 1 LS, 3 lports/HV]) > > > AT_KEYWORDS([ovnarp]) > > > @@ -6975,6 +7076,72 @@ OVN_CLEANUP([hv1]) > > > > > > AT_CLEANUP > > > > > > +AT_SETUP([ovn -- obsolete patch ports and tunnel endpoints removed]) > > > +AT_KEYWORDS([cleanup-test]) > > > +ovn_start > > > + > > > +net_add n1 > > > +net_add n2 > > > + > > > +for i in 1 2; do > > > + ovs-vsctl add-br br-phys$i > > > + ovs-vsctl set open . > > > external-ids:ovn-bridge-mappings-hv$i=physnet$i:br-phys$i > > > + ovn_attach n$i br-phys$i 192.168.0.$i 24 geneve br-int$i hv$i > > > +done > > > + > > > +# create irrelevant patch and tunnel ports > > > +for i in 1 2; do > > > + # patch without chassis owner set > > > + ovs-vsctl add-port br-int$i fakepatch$i > > > external-ids:ovn-logical-patch-port=fakeport$i -- \ > > > + set Interface fakepatch$i type=patch > > > + > > > + # patch marked as owned by the chassis > > > + ovs-vsctl add-port br-int$i owned_fakepatch$i > > > external-ids:ovn-logical-patch-port=owned_fakeport$i \ > > > + > > > external-ids:ovn-chassis-id=hv$i -- \ > > > + set Interface owned_fakepatch$i type=patch > > > + > > > + # patch marked as owned by some other chassis > > > + ovs-vsctl add-port br-int$i alien_fakepatch$i > > > external-ids:ovn-logical-patch-port=alien_fakeport$i \ > > > + > > > external-ids:ovn-chassis-id=alien_hv$i -- \ > > > + set Interface alien_fakepatch$i type=patch > > > + > > > + # OVN tunnel endpoint on a bridge owned by a controller > > > + ovs-vsctl add-port br-int$i faketunnel$i > > > external-ids:ovn-chassis-id=fakechassis -- \ > > > + set Interface faketunnel$i type=geneve > > > +done > > > + > > > +# tunnel endpoint on a bridge NOT owned by a controller > > > +ovs-vsctl add-br alien_br > > > +ovs-vsctl add-port alien_br alien_tunnel > > > external-ids:ovn-chassis-id=fakechassis -- \ > > > + set Interface alien_tunnel type=geneve > > > + > > > +AT_CHECK([ovn-nbctl ls-add lsw0]) > > > +AT_CHECK([ovn-nbctl lsp-add lsw0 lnport]) > > > +AT_CHECK([ovn-nbctl lsp-set-addresses lnport unknown]) > > > +AT_CHECK([ovn-nbctl lsp-set-type lnport localnet]) > > > +AT_CHECK([ovn-nbctl --wait=hv lsp-set-options lnport > > > network_name=physnet1]) > > > + > > > +ovs-vsctl add-port br-int1 vif -- set Interface vif > > > external-ids:iface-id=lp0 > > > +ovn-nbctl lsp-add lsw0 lp0 > > > +OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp0` = xup]) > > > + > > > +# check that only patch ports that belong to some other chassis and > > > those for localnet ports are present > > > +AT_CHECK([ovs-vsctl --bare --columns=name find interface type="patch" | > > > awk NF | sort], [0], > > > +[[alien_fakepatch1 > > > +alien_fakepatch2 > > > +patch-br-int1-to-lnport > > > +patch-lnport-to-br-int1 > > > +]]) > > > + > > > +# check that only controller tunnel endpoints and a tunnel on another > > > bridge are present > > > +AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" | > > > awk NF | sort], [0], > > > +[[alien_tunnel > > > +ovn-hv1-0 > > > +ovn-hv2-0 > > > +]]) > > > + > > > +AT_CLEANUP > > > + > > > AT_SETUP([ovn -- nd_na ]) > > > ovn_start > > > > > > diff --git a/tests/ovs-macros.at b/tests/ovs-macros.at > > > index 3dcf8f96d..7b812e5dd 100644 > > > --- a/tests/ovs-macros.at > > > +++ b/tests/ovs-macros.at > > > @@ -53,6 +53,7 @@ ovs_setenv() { > > > OVS_LOGDIR=$ovs_dir; export OVS_LOGDIR > > > OVS_DBDIR=$ovs_dir; export OVS_DBDIR > > > OVS_SYSCONFDIR=$ovs_dir; export OVS_SYSCONFDIR > > > + OVN_SYSCONFDIR=$ovs_dir; export OVN_SYSCONFDIR > > > OVS_PKGDATADIR=$ovs_dir; export OVS_PKGDATADIR > > > } > > > > > > diff --git a/tests/system-ovn.at b/tests/system-ovn.at > > > index 091b61f91..fb27cca34 100644 > > > --- a/tests/system-ovn.at > > > +++ b/tests/system-ovn.at > > > @@ -5443,6 +5443,7 @@ OVS_APP_EXIT_AND_WAIT([ovn-northd]) > > > > > > as > > > OVS_TRAFFIC_VSWITCHD_STOP(["/.*error receiving.*/d > > > +/failed to query port patch-.*/d > > > /.*terminating with signal 15.*/d"]) > > > AT_CLEANUP > > > > > > -- > > > 2.28.0 > > > > > > _______________________________________________ > > > dev mailing list > > > d...@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > > > > _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev