Hi Ales, The problem is with the rarp/garp sent by ovn-controller before the VM (on AZ2) is ready. This leads to traffic being switched to the VM on AZ2 resulting in traffic loss.
Even with the vif set to "unknown" address on the migrating VM, we are unable to avoid this. Let me know if I am missing something obvious. Thanks, Shibir From: Ales Musil <amu...@redhat.com> Date: Tuesday, 16 July 2024 at 11:45 AM To: Shibir Basak <shibir.ba...@nutanix.com> Cc: d...@openvswitch.org <d...@openvswitch.org> Subject: Re: [ovs-dev] [PATCH ovn] controller: Add lsp option disable_garp_rarp. On Wed, Jul 10, 2024 at 1: 06 PM Shibir Basak <shibir. basak@ nutanix. com> wrote: Hi, Yes, that’s right. Let’s say, the VM is getting copied to the destination AZ or say it is not yet ready to process any traffic, yet. In such cases we may ZjQcmQRYFpfptBannerStart CAUTION: External Email ZjQcmQRYFpfptBannerEnd On Wed, Jul 10, 2024 at 1:06 PM Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> wrote: Hi, Yes, that’s right. Let’s say, the VM is getting copied to the destination AZ or say it is not yet ready to process any traffic, yet. In such cases we may want OVN to avoid sending RARP/GARP early and lead to traffic outage. Thanks, Shibir Hi, sorry for the late reply, would it work if you'll use an "unknown" address initially on the second interface and change it when the VM is done with migration? From: Ales Musil <amu...@redhat.com<mailto:amu...@redhat.com>> Date: Wednesday, 10 July 2024 at 4:20 PM To: Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> Cc: d...@openvswitch.org<mailto:d...@openvswitch.org> <d...@openvswitch.org<mailto:d...@openvswitch.org>> Subject: Re: [ovs-dev] [PATCH ovn] controller: Add lsp option disable_garp_rarp. On Wed, Jul 10, 2024 at 12: 26 PM Shibir Basak <shibir. basak@ nutanix. com> wrote: Hi Ales, The use case we are trying to address is not limited to migration within a local AZ. Here, the VM can also be migrated across AZs and it is supposed ZjQcmQRYFpfptBannerStart CAUTION: External Email ZjQcmQRYFpfptBannerEnd On Wed, Jul 10, 2024 at 12:26 PM Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> wrote: Hi Ales, The use case we are trying to address is not limited to migration within a local AZ. Here, the VM can also be migrated across AZs and it is supposed to retain the same MAC & IP addresses. Hence, requested-chassis along with LSP option "activation-strategy" is not applicable for such a case. Thanks, Shibir Hi, so to make sure I understand the use case, you are migrating across AZ, and to make it faster you are creating an LSP with the same IP + MAC on the second AZ in advance. But because the VM is not there yet you don't want to announce that to the ToR switch is that assumption correct? From: Ales Musil <amu...@redhat.com<mailto:amu...@redhat.com>> Date: Wednesday, 10 July 2024 at 12:40 PM To: Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> Cc: d...@openvswitch.org<mailto:d...@openvswitch.org> <d...@openvswitch.org<mailto:d...@openvswitch.org>> Subject: Re: [ovs-dev] [PATCH ovn] controller: Add lsp option disable_garp_rarp. On Tue, Jul 9, 2024 at 7: 57 AM Shibir Basak <shibir. basak@ nutanix. com> wrote: If the lsp option 'disable_garp_rarp' is set to true, GARP and RARP announcements are not sent by ovn when a VIF port is created on a bridged logical ZjQcmQRYFpfptBannerStart CAUTION: External Email ZjQcmQRYFpfptBannerEnd On Tue, Jul 9, 2024 at 7:57 AM Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> wrote: If the lsp option 'disable_garp_rarp' is set to true, GARP and RARP announcements are not sent by ovn when a VIF port is created on a bridged logical switch. Usecase ======== This option is useful during VM migration to let the hypervisor/VM send the RARP/GARP once VM is ready to process the packets post migration. This helps to reduce downtime during VM migration. Signed-off-by: Shibir Basak <shibir.ba...@nutanix.com<mailto:shibir.ba...@nutanix.com>> Acked-by: Naveen Yerramneni <naveen.yerramn...@nutanix.com<mailto:naveen.yerramn...@nutanix.com>> Hello, I would like to avoid yet another config option, have you considered using "activation-strategy" LSP option along with setting multiple requested-chassis during the migration process? This should ensure that the downtime is actually as low as possible. See presentation from Ihar about this specific option [0]. --- controller/pinctrl.c | 4 +-- ovn-nb.xml | 7 +++++ tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=kO4kh_rderyzcFXnAd_3EpWrdxVQFPeXwBImdB7V5yg&e=> | 68 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+), 2 deletions(-) diff --git a/controller/pinctrl.c b/controller/pinctrl.c index 9a2f3f5b3..800c85d21 100644 --- a/controller/pinctrl.c +++ b/controller/pinctrl.c @@ -6618,7 +6618,7 @@ send_garp_rarp_prepare(struct ovsdb_idl_txn *ovnsb_idl_txn, SSET_FOR_EACH (iface_id, &localnet_vifs) { const struct sbrec_port_binding *pb = lport_lookup_by_name( sbrec_port_binding_by_name, iface_id); - if (pb) { + if (pb && !smap_get_bool(&pb->options, "disable_garp_rarp", false)) { send_garp_rarp_update(ovnsb_idl_txn, sbrec_mac_binding_by_lport_ip, local_datapaths, pb, &nat_addresses, @@ -6631,7 +6631,7 @@ send_garp_rarp_prepare(struct ovsdb_idl_txn *ovnsb_idl_txn, SSET_FOR_EACH (gw_port, &local_l3gw_ports) { const struct sbrec_port_binding *pb = lport_lookup_by_name(sbrec_port_binding_by_name, gw_port); - if (pb) { + if (pb && !smap_get_bool(&pb->options, "disable_garp_rarp", false)) { send_garp_rarp_update(ovnsb_idl_txn, sbrec_mac_binding_by_lport_ip, local_datapaths, pb, &nat_addresses, garp_max_timeout, garp_continuous); diff --git a/ovn-nb.xml b/ovn-nb.xml index 9552534f6..3a0f0e34d 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -1290,6 +1290,13 @@ The default value is <code>false</code>. </column> + <column name="options" key="disable_garp_rarp" + type='{"type": "boolean"}'> + If set to <code>true</code>, GARP and RARP announcements are not + sent when a VIF port is created on a bridged logical switch. + The default value is <code>false</code>. + </column> + <column name="options" key="pkt_clone_type" type='{"type": "string", "enum": ["set", ["mc_unknown"]]}'> If set to mc_unknown, packets going to this VIF get cloned to all diff --git a/tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=kO4kh_rderyzcFXnAd_3EpWrdxVQFPeXwBImdB7V5yg&e=> b/tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=kO4kh_rderyzcFXnAd_3EpWrdxVQFPeXwBImdB7V5yg&e=> index 87a64499f..a71a83394 100644 --- a/tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=kO4kh_rderyzcFXnAd_3EpWrdxVQFPeXwBImdB7V5yg&e=> +++ b/tests/ovn.at [ovn.at]<https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=kO4kh_rderyzcFXnAd_3EpWrdxVQFPeXwBImdB7V5yg&e=> @@ -25013,6 +25013,74 @@ OVN_CLEANUP([hv1]) AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD([ +AT_SETUP([Disabling RARP/GARP announcements]) +ovn_start + +# In this test case we create 1 switch and bring up 4 VIFs on it. +# Two VIFs will be assigned MAC addresses only (i.e. without ips) +# and two VIFs will be assigned IP addresses along with MAC addresses. +# VIFs with IPs are supposed to send GARPs and VIFs with only MAC +# addresses are supposed to send RARP. However, we test the lsp +# option disable_garp_rarp, which when set to true for an lsp does +# not send the GARP/RARP announcements. + +ovn-nbctl ls-add ls1 +ovn-nbctl lsp-add ls1 ln1 "" 101 +ovn-nbctl lsp-set-addresses ln1 unknown +ovn-nbctl lsp-set-type ln1 localnet +ovn-nbctl lsp-set-options ln1 network_name=phys + +ovn-nbctl lsp-add ls1 lp11 +ovn-nbctl lsp-set-addresses lp11 "f0:00:00:00:00:11" + +ovn-nbctl lsp-add ls1 lp12 +ovn-nbctl lsp-set-addresses lp12 "f0:00:00:00:00:12" +ovn-nbctl set logical_switch_port lp12 options:disable_garp_rarp=true + +ovn-nbctl lsp-add ls1 lp13 +ovn-nbctl lsp-set-addresses lp13 "f0:00:00:00:00:13 192.168.1.3" + +ovn-nbctl lsp-add ls1 lp14 +ovn-nbctl lsp-set-addresses lp14 "f0:00:00:00:00:14 192.168.1.4" +ovn-nbctl set logical_switch_port lp14 options:disable_garp_rarp=true + +net_add n1 + +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys +ovn_attach n1 br-phys 192.168.0.1 + +AT_CHECK([ovs-vsctl add-port br-phys snoopvif -- set Interface snoopvif options:tx_pcap=hv1/snoopvif-tx.pcap options:rxq_pcap=hv1/snoopvif-rx.pcap]) + +ovs-vsctl add-port br-int vif11 -- \ + set Interface vif11 external-ids:iface-id=lp11 + +ovs-vsctl add-port br-int vif12 -- \ + set Interface vif12 external-ids:iface-id=lp12 + +ovs-vsctl add-port br-int vif13 -- \ + set Interface vif13 external-ids:iface-id=lp13 + +ovs-vsctl add-port br-int vif14 -- \ + set Interface vif14 external-ids:iface-id=lp14 + +wait_for_ports_up +ovn-nbctl --wait=sb sync + +# RARP packet for lp11 +echo "fffffffffffff000000000118100006580350001080006040003f0000000001100000000f0000000001100000000" > expected +# GARP packet for lp13 +echo "fffffffffffff000000000138100006508060001080006040001f00000000013c0a80103000000000000c0a80103" >> expected +OVN_CHECK_PACKETS_UNIQ([hv1/snoopvif-tx.pcap], [expected]) + +OVN_CLEANUP([hv1]) + +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD([ AT_SETUP([Stateless Floating IP]) ovn_start -- 2.22.3 _______________________________________________ dev mailing list d...@openvswitch.org<mailto:d...@openvswitch.org> https://mail.openvswitch.org/mailman/listinfo/ovs-dev [mail.openvswitch.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=RxWPATormrWDKlK8Hw_58xqlJy8RuGGO-nQbR0laENo&e=> Thanks, Ales [0] https://www.youtube.com/watch?v=ijZTMXAg-eI&t=780s [youtube.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3DijZTMXAg-2DeI-26t-3D780s&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=wMe7dkLzEskberAdngb7nZU-nF--0TKuJP8A5w0V2aM&e=> -- Ales Musil Senior Software Engineer - OVN Core Red Hat EMEA [redhat.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=9Es0tZFVNRcrCHC3-yHufRKznxxWatLX_W3r19p6KFw&e=> amu...@redhat.com<mailto:amu...@redhat.com> Error! Filename not specified.[red.ht]<https://urldefense.proofpoint.com/v2/url?u=https-3A__red.ht_sig&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=LvH3Q_0l3c2ZVGXqyDjrnRiIancWq5NZVQ6Kf-4RwIxjOB2kd7hmwUjz0n9N8H-C&s=0ulSMiKaHDPCVwaOI-J0pNq0mu6_57w4gh_LAYNaJ3E&e=> Thanks, Ales -- Ales Musil Senior Software Engineer - OVN Core Red Hat EMEA [redhat.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=qGcZkOTO7KU-WPypcKtQOT5nBSrQ_C9h3KhMboc9h9alQsldn17cM3-ha4mJbe1X&s=T4b5B3AwYT6lrhYJDErqLRdIImSMcxjE-X01OfTkgA0&e=> amu...@redhat.com<mailto:amu...@redhat.com> Error! Filename not specified.[red.ht]<https://urldefense.proofpoint.com/v2/url?u=https-3A__red.ht_sig&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=qGcZkOTO7KU-WPypcKtQOT5nBSrQ_C9h3KhMboc9h9alQsldn17cM3-ha4mJbe1X&s=_ZrTH6GbjhR-tmd0C3qalggfCD-gLVB8SUbrxGckUJ0&e=> Thanks, Ales -- Ales Musil Senior Software Engineer - OVN Core Red Hat EMEA [redhat.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=z6N0lEMRy1guaVKW-5eErOzhvBkS8R6lbzs0t31b4qZKfpTDO6Pdapm3bRbPmyj-&s=KVHtbTCyZHPGpsi0jaf_EmahIr_tKPHAcTtskfWAMfU&e=> amu...@redhat.com<mailto:amu...@redhat.com> [Image removed by sender.][red.ht]<https://urldefense.proofpoint.com/v2/url?u=https-3A__red.ht_sig&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=awcqDnvm2rXKiid0_aQK6D2Z4I6GarSRmeHuqVFlgU4&m=z6N0lEMRy1guaVKW-5eErOzhvBkS8R6lbzs0t31b4qZKfpTDO6Pdapm3bRbPmyj-&s=sQmaurWoLQRpLlOqtIGNkI1TVXAZBBw-_6ANXhHxov4&e=> _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
