[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
As discussed in https://bugs.launchpad.net/ubuntu/+source/keepalived/+bug/1819074 and pointed out in https://bugs.launchpad.net/ubuntu/bionic/+source/keepalived/+bug/1815101/comments/65, as KeepConfiguration= is part of systemd nowadays, and since the keepalived fix is available since 2.x, This should be fixed nowadays, and I am marking the keepalived task as such. ** Tags removed: server-todo ** Changed in: keepalived (Ubuntu) Status: In Progress => Fix Released ** Changed in: keepalived (Ubuntu) Assignee: Athos Ribeiro (athos-ribeiro) => (unassigned) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: Fix Released Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Won't Fix Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Closing bionic as per https://bugs.launchpad.net/netplan/+bug/1815101/comments/59 (it is also on EOSS). ** Changed in: keepalived (Ubuntu Bionic) Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: Fix Released Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Won't Fix Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: keepalived (Ubuntu Xenial) Assignee: (unassigned) => Athos Ribeiro (athos-ribeiro) ** Changed in: keepalived (Ubuntu Bionic) Assignee: (unassigned) => Athos Ribeiro (athos-ribeiro) ** No longer affects: keepalived (Ubuntu Xenial) ** Changed in: keepalived (Ubuntu Focal) Assignee: (unassigned) => Athos Ribeiro (athos-ribeiro) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 #
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: keepalived (Ubuntu) Assignee: (unassigned) => Athos Ribeiro (athos-ribeiro) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Marking todo to recheck how the situation is today. ** Tags removed: server-triage-discuss ** Tags added: server-todo -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
What's the status of keepalived in Mantic on this bug? ** Tags added: server-triage-discuss -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Alex - Can you please open a new bug report about the crash you are seeing? Please include any details you have about how to reproduce the crash. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
We are seeing SIGSEGV related to this in Bionic. lp1815101-0006-network- make-KeepConfiguration-static-drop-DHCP-addr.patch is missing network checks in link_drop_foreign_config Would it be possible to incorporate this patch from upstream to prevent this? https://github.com/systemd/systemd/commit/b1b0b42e48303134731e017a108c6c334ef5f4c8 Core was generated by `/lib/systemd/systemd-networkd'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 2741../src/network/networkd-link.c: No such file or directory. (gdb) bt #0 0x555962aa1c34 in link_drop_foreign_config (link=link@entry=0x555963583f30) at ../src/network/networkd-link.c:2741 #1 0x555962aa233d in link_carrier_lost.lto_priv.328 (link=, link=) at ../src/network/networkd-link.c:3462 #2 0x555962a8e9b2 in link_update (m=0x5559635702c0, link=) at ../src/network/networkd-link.c:3698 #3 manager_rtnl_process_link (rtnl=, message=0x5559635702c0, userdata=) at ../src/network/networkd-manager.c:713 #4 0x555962a48a16 in process_match (m=0x5559635702c0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:388 #5 process_running (ret=0x0, rtnl=0x55596355d990) at ../src/libsystemd/sd-netlink/sd-netlink.c:418 #6 sd_netlink_process (rtnl=0x55596355d990, ret=ret@entry=0x0) at ../src/libsystemd/sd-netlink/sd-netlink.c:452 #7 0x555962a48cb3 in time_callback (s=, usec=, userdata=) at ../src/libsystemd/sd-netlink/sd-netlink.c:759 #8 0x555962a4dbae in source_dispatch (s=s@entry=0x55596355dce0) at ../src/libsystemd/sd-event/sd-event.c:2311 #9 0x555962a4de2a in sd_event_dispatch (e=, e@entry=0x55596355bf70) at ../src/libsystemd/sd-event/sd-event.c:2663 #10 0x555962a4dfb9 in sd_event_run (e=, e@entry=0x55596355bf70, timeout=timeout@entry=18446744073709551615) at ../src/libsystemd/sd-event/sd-event.c:2723 #11 0x555962a4e1fb in sd_event_loop (e=) at ../src/libsystemd/sd-event/sd-event.c:2744 #12 0x555962a223d6 in main (argc=, argv=) at ../src/network/networkd.c:158 (gdb) p link->network $2 = (struct Network *) 0x0 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I think fix will be released in systemd 253. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Well, no weird thing, yes, probably it just restart interface, this is wrong if other interface config is changed... -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I did report to systemd: https://github.com/systemd/systemd/issues/25067 As for timestamps, systemd-networkd does not do any weird things when timestamps change on non-bridge interfaces. ** Bug watch added: github.com/systemd/systemd/issues #25067 https://github.com/systemd/systemd/issues/25067 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 #
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
If problem is in timestamp, then this is netplan problem- why it regenerates files with no changes? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
In bug #1992241 it is suggested that this problem is due to sd-networkd relying on the file's timestamp to decide if the interface needs to be re-started. Netplan generate/apply would re-create the (identical) files in /run/systemd/network/10-netplan-*.network but those would have a newer timestmp... -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 #
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Thank you for the minimal reproducer! Using that, I can reconstruct the issue in a Kinetic LXD container and I can observe the logs as you do, especially the "eth0: Link DOWN" part. Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Reconfiguring with /run/systemd/network/10-netplan-eth0.network. Oct 20 13:43:47 test systemd-networkd[1276]: bond_all: Reconfiguring with /run/systemd/network/10-netplan-bond_all.network. Oct 20 13:43:47 test systemd-networkd[1276]: bond_all: DHCPv6 lease lost Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Link DOWN Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Lost carrier Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Configuring with /run/systemd/network/10-netplan-eth0.network. Oct 20 13:43:47 test systemd-networkd[1276]: bond_all: Configuring with /run/systemd/network/10-netplan-bond_all.network. Oct 20 13:43:47 test systemd-networkd[1276]: bond_all: DHCPv6 lease lost Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Link UP Oct 20 13:43:47 test systemd-networkd[1276]: eth0: Gained carrier I tried using override configs, but that didn't change anything. # cat /etc/systemd/network/10-netplan-{eth0,bond_all}.network.d/override.conf [Network] KeepConfiguration=true The same issue happens when calling "netplan apply", "networkctl reconfigure eth0/bond_all" (which netplan apply is calling under the hood) or "systemctl restart systemd-networkd.service". I think there isn't a lot that netplan could do to avoid this, if sd- networkd keeps on insisting to restart the interface. Would you mind opening a bug report with the upstream systemd developers about this? https://github.com/systemd/systemd/issues ** Changed in: netplan Importance: Medium => Low -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: netplan Status: Incomplete => Triaged -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Triaged Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
It is reproducible on a VM with a simple netplan file with only one ethernet and one bond interface: network: ethernets: n1p1: match: name: ens192 bonds: bond_all: interfaces: - n1p1 parameters: mode: balance-rr addresses: - 10.1.1.206/24 nameservers: addresses: - 10.1.1.1 routes: - metric: 150 to: 0.0.0.0/0 via: 10.1.1.1 This always produces following logs, together with a brief network loss: Oct 18 09:03:17 test01 systemd[1]: Reloading. Oct 18 09:03:18 test01 systemd-networkd[738]: bond_all: Re-configuring with /run/systemd/network/10-netplan-bond_all.network Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Re-configuring with /run/systemd/network/10-netplan-n1p1.network Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Link DOWN Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Lost carrier Oct 18 09:03:18 test01 systemd-networkd[738]: bond_all: Re-configuring with /run/systemd/network/10-netplan-bond_all.network Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Re-configuring with /run/systemd/network/10-netplan-n1p1.network Oct 18 09:03:18 test01 kernel: vmxnet3 :0b:00.0 ens192: intr type 3, mode 0, 5 vectors allocated Oct 18 09:03:18 test01 kernel: vmxnet3 :0b:00.0 ens192: NIC Link is Up 1 Mbps Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Link UP Oct 18 09:03:18 test01 systemd-networkd[738]: ens192: Gained carrier -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Incomplete Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I do not see any difference. However I am not convinced that this is not a networkd issue anyway... here are journal logs from `netplan apply`, where n1p1, n1p2 are first and second port of first network card, n2p2, n2p2 for second network card: systemd[1]: Reloading. systemd-networkd[2352]: enp161s0f1np1: Re-configuring with /run/systemd/network/10-netplan-n2p2.network systemd-networkd[2352]: lan_si: Re-configuring with /run/systemd/network/10-netplan-lan_si.network systemd-networkd[2352]: eno1: Re-configuring with /run/systemd/network/10-netplan-zzz-all-en.network systemd-networkd[2352]: wan: Re-configuring with /run/systemd/network/10-netplan-wan.network systemd-networkd[2352]: bond_wan: Re-configuring with /run/systemd/network/10-netplan-bond_wan.network systemd-networkd[2352]: eno2: Re-configuring with /run/systemd/network/10-netplan-zzz-all-en.network systemd-networkd[2352]: lan_mgmnt: Re-configuring with /run/systemd/network/10-netplan-lan_mgmnt.network systemd-networkd[2352]: eno33np0: Re-configuring with /run/systemd/network/10-netplan-n1p1.network systemd-networkd[2352]: eno34np1: Re-configuring with /run/systemd/network/10-netplan-n1p2.network systemd-networkd[2352]: bond_lan: Re-configuring with /run/systemd/network/10-netplan-bond_lan.network systemd-networkd[2352]: enp161s0f0np0: Re-configuring with /run/systemd/network/10-netplan-n2p1.network systemd-networkd[2352]: enp161s0f1np1: Link DOWN systemd-networkd[2352]: enp161s0f1np1: Lost carrier systemd-networkd[2352]: enp161s0f1np1: Re-configuring with /run/systemd/network/10-netplan-n2p2.network systemd-networkd[2352]: eno2: Re-configuring with /run/systemd/network/10-netplan-zzz-all-en.network systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped. chronyd[17235]: Source 10.160.4.2 offline chronyd[17235]: Source 10.160.4.1 offline kernel: bond_wan: (slave enp161s0f0np0): link status definitely down, disabling slave kernel: bond_wan: now running without any active interface! kernel: bond_wan: (slave eno33np0): link status definitely down, disabling slave kernel: bond_lan: (slave enp161s0f1np1): link status definitely down, disabling slave kernel: bond_lan: (slave eno34np1): link status definitely down, disabling slave kernel: bond_lan: now running without any active interface! systemd-networkd[2352]: enp161s0f0np0: Re-configuring with /run/systemd/network/10-netplan-n2p1.network systemd-networkd[2352]: eno34np1: Re-configuring with /run/systemd/network/10-netplan-n1p2.network systemd-networkd[2352]: eno33np0: Re-configuring with /run/systemd/network/10-netplan-n1p1.network systemd-networkd[2352]: eno33np0: Link DOWN systemd-networkd[2352]: eno33np0: Lost carrier systemd-networkd[2352]: eno1: Re-configuring with /run/systemd/network/10-netplan-zzz-all-en.network systemd-networkd[2352]: eno34np1: Link DOWN systemd-networkd[2352]: eno34np1: Lost carrier systemd-networkd[2352]: bond_wan: Re-configuring with /run/systemd/network/10-netplan-bond_wan.network systemd-networkd[2352]: bond_lan: Re-configuring with /run/systemd/network/10-netplan-bond_lan.network systemd-networkd[2352]: enp161s0f0np0: Link DOWN systemd-networkd[2352]: enp161s0f0np0: Lost carrier systemd-networkd[2352]: lan_si: Re-configuring with /run/systemd/network/10-netplan-lan_si.network systemd-networkd[2352]: wan: Re-configuring with /run/systemd/network/10-netplan-wan.network systemd-networkd[2352]: lan_mgmnt: Re-configuring with /run/systemd/network/10-netplan-lan_mgmnt.network systemd-networkd[2352]: bond_wan: Lost carrier systemd-networkd[2352]: wan: Lost carrier systemd-networkd[2352]: bond_lan: Lost carrier systemd-networkd[2352]: lan_mgmnt: Lost carrier systemd-networkd[2352]: lan_si: Lost carrier kernel: mlx5_core :a1:00.1 enp161s0f1np1: Link up kernel: 8021q: adding VLAN 0 to HW filter on device enp161s0f1np1 kernel: mlx5_core :63:00.0 eno33np0: Link up kernel: 8021q: adding VLAN 0 to HW filter on device eno33np0 kernel: mlx5_core :63:00.1 eno34np1: Link up kernel: 8021q: adding VLAN 0 to HW filter on device eno34np1 kernel: mlx5_core :a1:00.0 enp161s0f0np0: Link up kernel: 8021q: adding VLAN 0 to HW filter on device enp161s0f0np0 systemd-networkd[2352]: enp161s0f1np1: Link UP kernel: bond_lan: (slave enp161s0f1np1): link status definitely up, 1 Mbps full duplex kernel: bond_lan: active interface up! kernel: bond_lan: (slave eno34np1): link status definitely up, 1 Mbps full duplex kernel: bond_wan: (slave enp161s0f0np0): link status definitely up, 1 Mbps full duplex kernel: bond_wan: active interface up! kernel: bond_wan: (slave eno33np0): link status definitely up, 1 Mbps full duplex systemd-networkd[2352]: enp161s0f1np1: Gained carrier systemd-networkd[2352]: eno33np0: Link UP systemd-networkd[2352]: eno33np0: Gained carrier systemd-networkd[2352]: eno34np1: Link UP systemd-networkd[2352]: eno34np1: Gained carrier systemd-networkd[2352]: enp161s0f0np0: Link UP
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
The KeepConfiguration= setting is part of systemd nowadays, so I'm closing the systemd component. Netplan does not currently make use of "KeepConfiguration=" [0] though. Marcin, could you try to place a systemd-networkd override file, e.g. in /etc/systemd/network/10-netplan-eth0.network.d/override.conf (depending on your interface name), that contains a corresponding KeepConfiguration setting and check if that makes any difference? [0] https://www.freedesktop.org/software/systemd/man/systemd.network.html#KeepConfiguration= ** Changed in: netplan Importance: Undecided => Medium ** Changed in: netplan Status: Fix Released => Incomplete ** Changed in: systemd (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Incomplete Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: Fix Released Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Hi! I am having trouble making this work in graceful manner. Fresh Ubuntu 22.04, systemd 249.11-0ubuntu3.6, netplan 0.104-0ubuntu2.1. Host has multiple interfaces (6 physical, 2 bonds, 3 vlans on those bonds) and multiple (20 to 30) IP addresses on those vlan interfaces. EVERY time i issue `netplan apply` everything goes down and up again. Even with no changes to netplan configuration. systemd-networkd logs interfaces going down and up again. Changing anything in netplan config (like adding or removing one of IP addresses from single interface), doing `netplan generate` && `networkctl reload` restarts everything as well. How I am supposed to simply add or remove IP address to existing interface without causing delay of about 5 seconds in network traffic, when all interfaces are being restarted? Am I doing something incorrectly here? Regards, Marcin. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Fix Released Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com]
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
netplan uses the "networkctl reload/reconfigure" commands nowadays, instead of hard restarting systemd-networkd: https://github.com/canonical/netplan/pull/200 This change was activated in v0.104 in the Distro, which landed in Jammy and is currently being SRUed to Focal and Impish (LP: #1964481). Please re-open if you can still observe this issue with netplan 0.104. ** Changed in: netplan Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Fix Released Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
This bug was fixed in the package systemd - 237-3ubuntu10.51 --- systemd (237-3ubuntu10.51) bionic; urgency=medium * Add support to keepconfiguration (LP: #1815101) - lp1815101-0001-add-macro-if-flags-are-set.patch - lp1815101-0002-networkd-add-support-to-keepconfiguration.patch - lp1815101-0003-network-use-hashmap_steal_first-rather-than-hashmap_.patch - lp1815101-0004-networkd-stop-clients-when-networkd-shuts-down.patch - lp1815101-0005-network-add-KeepConfiguration-dhcp-on-stop.patch - lp1815101-0006-network-make-KeepConfiguration-static-drop-DHCP-addr.patch - lp1815101-0007-man-add-documentation-about-KeepConfiguration.patch -- Eric Desrochers Mon, 26 Jul 2021 11:31:02 -0400 ** Changed in: systemd (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
systemd reaches day 7 wo/ autopkgtest failure nor negative outcome/feedbacks. I have asked 'sil2100' to promote the package into bionic-updates. Stay tuned ... - Eric -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Released Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Thanks Maanus for all the testing. Much appreciated. I'll take care of the rest in a couple of days with Lukasz. Package needs to stay in proposed for a couple more days (minimum 7 days) - Eric ** Tags removed: verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Committed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I installed the package from bionic-proposed and performed the test cases using keepalived - VIP is not lost. This resolves it. Thank you! -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Committed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps }
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Hi Maanus, Would you mind perform the 'test cases' now against the systemd's bionic-proposed package ? And report any outcome. This is one of the last steps for Lukasz to approve the proposed package after the baking minimum aging period (7 days) in verification phase. - Eric -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Committed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Hello Leroy, or anyone else affected, Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.51 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: systemd (Ubuntu Bionic) Status: In Progress => Fix Committed ** Tags removed: verification-done ** Tags added: verification-needed verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Committed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com,
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Since the changeset is composed from multiple patches, please be sure to execute both the test cases mentioned in the Test Case section. Thanks! -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Fix Committed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Thank you Dan and Eric! Summarizing howto with bionic for myself: Upgrade systemd to the version with "keepconfiguration" fix. Look at /run/systemd/network/*.network filename, in my example it is "10-netplan-ens160.network" Make a directory adding ".d" to this name: /etc/systemd/network/10-netplan-ens160.network.d Add a file "override.conf" into this directory, with content: --- [Network] KeepConfiguration=static --- -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Uploaded into bionic upload queue, now waiting for SRU approval. - Eric ** Changed in: systemd (Ubuntu Bionic) Status: Incomplete => In Progress ** Changed in: systemd (Ubuntu Bionic) Assignee: (unassigned) => Eric Desrochers (slashd) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I've tested 237-3ubuntu10.50+testpkg20210802b3 the ppa and it resolves the issue. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
> The "default bionic install" uses only netplan yaml and /etc/systemd/network/ is empty. yes, but netplan creates .network config files named in a deterministic way so you know what the name of the networkd file it creates will be. I don't think netplan currently has a mechanism to include keep- configuration options in the networkd config it creates. > Using '/etc/systemd/network/10-netplan-ens160.network.d/override.conf' with KeepConfiguration works perfectly the same. exactly, which is why you should use the drop-in instead of manually duplicating and editing the entire netplan-created .network config file, as you complained about when you said: > Now I need to have IP and gateway in two files since, no, you don't. Or, drop netplan and just directly configure systemd-networkd. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
The "default bionic install" uses only netplan yaml and /etc/systemd/network/ is empty. If I switch to systemd.network style, I have to create config files from scratch anyway and then this KeepConfiguration helps. (Of course I do not need to keep the netplan yaml after enabling systemd-networkd, my remark for having the IP configuration in two files was wrong) Using '/etc/systemd/network/10-netplan-ens160.network.d/override.conf' with KeepConfiguration works perfectly the same. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Good catch Dan. Maanus could you repeat the testing with what Dan brought up ? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
> Now I need to have IP and gateway in two files - /etc/netplan/01-netcfg.yaml > and > /etc/systemd/network/10-netplan-ens160.network to clarify, don't do that, you should create a systemd-networkd 'drop- in' instead of copying/modifying the file. For example, if the networkd config filename for your interface is '10-netplan-ens160.network' (regardless of which directory it's located in), you should create a new file '/etc/systemd/network/10-netplan- ens160.network.d/override.conf' with only the content you want to add/modify, specifically in this case: --- [Network] KeepConfiguration=static --- For more detail on systemd-networkd 'drop-in' files, see 'man systemd.network' in the first paragraph. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Thanks @Maanus. So the outcome is positive here then. - Eric -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I am using netplan - default in bionic. I added the repo, made update and reboot. 'systemctl restart systemd-networkd' resulted with keepalived VIP lost. I added following file and rebooted: /etc/systemd/network/10-netplan-ens160.network : --- [Match] Name=ens160 [Network] KeepConfiguration=static --- Network did not come up after reboot. changed /etc/systemd/network/10-netplan-ens160.network and rebooted: --- [Match] Name=ens160 [Network] Address=10.1.1.233/29 Gateway=10.1.1.238 KeepConfiguration=static --- 'systemctl restart systemd-networkd' did not have any bad results - keepalived VIP remained as expected. Now I need to have IP and gateway in two files - /etc/netplan/01-netcfg.yaml and /etc/systemd/network/10-netplan- ens160.network Maanus -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
@maanus, That is a first iteration. sudo add-apt-repository ppa:slashd/keepconfiguration sudo apt-get update Let me know the outcome. - Eric -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps }
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I want to test it in Bionic. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Any volunteer to test a package in Bionic in the attempt to support keepconfiguration ? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
> > i'm inclined to also mark this as wont-fix for bionic, unless there are > > still people affected by this problem using bionic without some other workaround. > What kind of workarounds for binoic does this refer to? I have not found any > reliable workarounds yet at least. +1. I did not find any other solution either. Please help backward fix binoic as well. Thanks -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine,
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
> i'm inclined to also mark this as wont-fix for bionic, unless there are still people affected by this problem using bionic without some other workaround. What kind of workarounds for binoic does this refer to? I have not found any reliable workarounds yet at least. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
For the people on Focal that want to use netplan and keepalived together: you can just put in an override for the network unit file, to keep systemd-networkd from touching your interface! ``` $ cat /etc/systemd/network/10-netplan-eno1.network.d/override.conf [Network] KeepConfiguration=static $ ``` This might be a good enough workaround until this is really fixed. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, #
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
i'm marking this as wont-fix for xenial. i'm inclined to also mark this as wont-fix for bionic, unless there are still people affected by this problem using bionic without some other workaround. ** Changed in: systemd (Ubuntu Xenial) Status: Confirmed => Won't Fix ** Changed in: systemd (Ubuntu Bionic) Status: In Progress => Incomplete -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Won't Fix Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Incomplete Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: systemd (Ubuntu Bionic) Assignee: Eric Desrochers (slashd) => (unassigned) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Dear Eric Desrochers, I add the PPA into my ubuntu 18.04 corosync/pacemaker service node. Then I upgrade the following libnss-systemd libpam-systemd libsystemd0 libudev1 systemd systemd-sysv udev package. But it still failed after re-connect the notwork. The crmsh show as below: ERROR: status: crm_mon (rc=107): Connection to cluster failed: Transport endpoint is not connected. I also check the dmesg but it seem ok. e1000: enp0s8 NIC Link is Down e1000: enp0s8 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX Could you describe more detail to help me figure out it is my mistake operation or something wrong in my environment? Thank you. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
I have a first iteration of a package: It's not a final solution nor a long term solution. It is only made to determine if its fix the problem before considering an SRU: (Ideally one would test this package in non-production area) Adding this PPA to your system sudo add-apt-repository ppa:slashd/sf263217 sudo apt-get update Please report back any feedbacks in this bug. - Eric -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine,
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
The above test package has been made for 'systemd' in bionic ^ -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Ubuntu HA wise: I'm recommending all HA clusters to have the cluster interfaces configured with systemd-networkd DIRECTLY instead of wrapping it through netplan.io. At least until we're sure that HA has no issues with netplan.io, having it configured directly will allow us to isolate possible issues. I see that this has been assigned to @slashd. Eric, Important thing here is to have netplan fix in focal (as it is the latest LTS) in order for HA to be supported with it. KeepConfiguration= is good enough, for now, if using systemd-networkd only. Thank you! ** Tags removed: server-next -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: systemd (Ubuntu Bionic) Assignee: Jorge Niedbalski (niedbalski) => Eric Desrochers (slashd) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Changed in: systemd (Ubuntu Bionic) Assignee: (unassigned) => Jorge Niedbalski (niedbalski) ** Changed in: systemd (Ubuntu Bionic) Status: Confirmed => In Progress -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: In Progress Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
@napsty: the "workaround" (from your blog) is actually to use: - ifupdown/bridge-utils/vlan/resolvconf for network setup OR - use systemd-networkd DIRECTLY with the KeepConfiguration= option in .network file Just highlighting it here. @ddstreet, you said you would try to come up with the netplan change for KeepConfiguration. Did you have time to check on this ? (just checking). Cheers o/ ** Changed in: keepalived (Ubuntu) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: keepalived (Ubuntu Xenial) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: keepalived (Ubuntu Bionic) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: keepalived (Ubuntu Disco) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: keepalived (Ubuntu Eoan) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: systemd (Ubuntu) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: systemd (Ubuntu Xenial) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: systemd (Ubuntu Bionic) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: systemd (Ubuntu Disco) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: systemd (Ubuntu Eoan) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: netplan Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** No longer affects: keepalived (Ubuntu Eoan) ** No longer affects: keepalived (Ubuntu Disco) ** Also affects: heartbeat (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: keepalived (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: systemd (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: systemd (Ubuntu Focal) Status: New => Fix Released ** Changed in: keepalived (Ubuntu Focal) Status: New => Confirmed ** No longer affects: heartbeat (Ubuntu Focal) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
TL;DR TODO SUMMARY: - netplan change to support KeepConfiguration= for systemd-networkd backend (Groovy) - backport this change: netplan for Ubuntu Focal (SRU) - backport this change: netplan for Ubuntu Eoan (SRU, WontFix due to EOL ?) - backport this change: netplan for Ubuntu Bionic (SRU) - backport this change: netplan for Ubuntu Xenial (SRU, WontFix ?) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in systemd source package in Disco: Won't Fix Status in systemd source package in Eoan: Fix Released Status in keepalived source package in Focal: Confirmed Status in systemd source package in Focal: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
FYI I stumbled on this problem after a system update (which broke production!) and collected data in a troubleshooting session and documented this here https://www.claudiokuenzler.com/blog/959 /keepalived-virtual-ip-addresses-gone-lost-after-systemd-update. Once I found out the restart of systemd-networkd causes the keepalived vips to be gone, I finally came across this bug. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, #
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Removing charm-keepalived since I believe no changes are needed there. It should pick up fixes once they are available on apt archives. ** No longer affects: charm-keepalived -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Tags added: ddstreet -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan
Re: [Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Balint, based on your input... > thanks for the fixes in Eoan. Unfortunately we have a product based on > disco and cannot move forward at this time. Being a networking shop, > this issue has a serious effect on us and we would like to avoid moving > to something like ifupdown2 within our stable branch. So, Disco is EOL as it is not a LTS version, that is why it did not get a fix (as the fix is very close to the one done in Eoan). Since its unsupported by the community, it's up to you backport the Eoan fixes to Disco if you'd like... you can even create a PPA for your product and distribute along. > For our users the real impact of the bug is not that that the interface > that we are currently reconfiguring is suffering a downtime, but the > fact that _all_ interfaces have their aliases removed if networkd is > restarted. The proposed KeepConfiguration solution kind of beats the > purpose of reconfiguring the interfaces, as old addresses are kept and > need to be handled manually. Also it interferes with how DHCP works. I > believe this might be an issue for others as well. We are following systemd-networkd upstream decisions here. The option "dhcp" only exists for CERTAIN scenarios (when root disk depends on that connection, for iSCSI and/or NFS/ROOT for example). It is explicitly said in the documentation: """ Takes a boolean or one of "static", "dhcp-on-stop", "dhcp". When "static", systemd-networkd will not drop static addresses and routes on starting up process. When set to "dhcp-on-stop", systemd-networkd will not drop addresses and routes on stopping the daemon. When "dhcp", the addresses and routes provided by a DHCP server will never be dropped even if the DHCP lease expires. This is contrary to the DHCP specification, but may be the best choice if, e.g., the root filesystem relies on this connection. The setting "dhcp" implies "dhcp-on-stop", and "yes" implies "dhcp" and "static". Defaults to "no". """ and it is a question of choice: to have a window of opportunity for duplicate IPs - in cases where there is no dynamic IP mapping to that mac address - but possibly maintain the connection instead of causing uninterruptable I/Os trying to shutdown a machine, for example. I particularly don't like this option but it is not the default one and was meant for a specific purpose. > > >From our point of view the ideal solution would be a combination of the > keepalived patch that detects VIP removal and systemd version 244 that > already supports "networkctl reconfigure" and "networkctl reload". networkctl reconfigure/reload is a new functionality and won't be added to previous already released versions as this is against SRU guidelines. Systemd 244.2-1ubuntu1 is being included in 20.04, our NEXT LTS version. Like said before, you can try backporting systemd 244 to disco, or bionic, if you are willing to support it on your own as it was already EOL for community support. You should follow: https://packaging.ubuntu.com/html/backports.html if you would like to do that. For the keepalived patches, they could be backported to Eoan, maybe Bionic and Xenial depending on the amount of work. But then I would need a practical example of why the systemd-networkd fix is no good in most used scenarios. > Is there any chance that v244 is backported to bionic? It is already > included in focal and debian stable backports, but unfortunately I am > not familiar enough with systemd development to tell what the impact of > this would be. Problem with backports is that they are unsupported even on supported releases. I wouldn't be able to guarantee functionalities or fix it in a constant basis. You can do it on your own and have it in a PPA of your product, for example. As systemd nowadays include networkd, udev management, sysV runtime generators, tmpfiles creation, sockets creation, cgroups integration for the process slices, etc etc... it is very very risky to backport systemd to have "just" those 2 functionalities. > > As for keepalived, in bug #1819074 there was an ongoing investigation on > the patch, that implements the keepalived transition on removing the > VIP. We have traced back this functionality to this patch: > > https://github.com/acassen/keepalived/commit/0b1528c76d3fe8d1c5765841df86c59570a036da > > It was born before v1.3.6 was released, so we hope that it is self- > contained enough for a backport if v2.0 of keepalived is not included in > bionic-backports. Let me check keepalived fix more closely and see what can be done for the previous releases. As we are close to freeze date for our next LTS release, it is unlikely that I do it before 2 weeks from now (as our focus is in the development version entirely and I still need to fix netplan to support the networkd KeepConfiguration functionality). Lets keep talking.. I'll first patch netplan and go back with other releases to check what can be done for them. For now I would *strongly* recommend that in previous releases, whoever wants to
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Also affects: heartbeat (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: keepalived (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: systemd (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: keepalived (Ubuntu Xenial) Importance: Undecided => Medium ** Changed in: keepalived (Ubuntu Xenial) Status: New => Confirmed ** Changed in: keepalived (Ubuntu Xenial) Assignee: (unassigned) => Rafael David Tinoco (rafaeldtinoco) ** No longer affects: heartbeat (Ubuntu Xenial) ** Changed in: systemd (Ubuntu Xenial) Importance: Undecided => Medium ** Changed in: systemd (Ubuntu Xenial) Status: New => Confirmed ** Changed in: systemd (Ubuntu Xenial) Assignee: (unassigned) => Rafael David Tinoco (rafaeldtinoco) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Xenial: Confirmed Status in systemd source package in Xenial: Confirmed Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Hi all, thanks for the fixes in Eoan. Unfortunately we have a product based on disco and cannot move forward at this time. Being a networking shop, this issue has a serious effect on us and we would like to avoid moving to something like ifupdown2 within our stable branch. For our users the real impact of the bug is not that that the interface that we are currently reconfiguring is suffering a downtime, but the fact that _all_ interfaces have their aliases removed if networkd is restarted. The proposed KeepConfiguration solution kind of beats the purpose of reconfiguring the interfaces, as old addresses are kept and need to be handled manually. Also it interferes with how DHCP works. I believe this might be an issue for others as well. >From our point of view the ideal solution would be a combination of the keepalived patch that detects VIP removal and systemd version 244 that already supports "networkctl reconfigure" and "networkctl reload". Is there any chance that v244 is backported to bionic? It is already included in focal and debian stable backports, but unfortunately I am not familiar enough with systemd development to tell what the impact of this would be. As for keepalived, in bug #1819074 there was an ongoing investigation on the patch, that implements the keepalived transition on removing the VIP. We have traced back this functionality to this patch: https://github.com/acassen/keepalived/commit/0b1528c76d3fe8d1c5765841df86c59570a036da It was born before v1.3.6 was released, so we hope that it is self- contained enough for a backport if v2.0 of keepalived is not included in bionic-backports. Best, Balint -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Can we backport the fixes to Bionic? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
as disco reaches EOL next week, marking this as wontfix for disco. ** Changed in: systemd (Ubuntu Disco) Status: Confirmed => Won't Fix ** Changed in: keepalived (Ubuntu Disco) Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Won't Fix Status in systemd source package in Disco: Won't Fix Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps }
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
This bug was fixed in the package systemd - 242-7ubuntu3.2 --- systemd (242-7ubuntu3.2) eoan; urgency=medium [ Dan Streetman ] * d/extra/dhclient-enter-resolved-hook: - Replace use of bash-only &> with > and 2> (LP: #1849608) * d/p/lp1849658-resolved-set-stream-type-during-DnsStream-creation.patch: - Fix bug in refcounting TCP stream types (LP: #1849658) * d/extra/dhclient-enter-resolved-hook: cleanup temp $newstate file [ Rafael David Tinoco ] * Add support to KeepConfiguration= fixing behaviour for HA (LP: #1815101) - d/p/lp1815101-01-networkd-add-support-to-keep-configuration.patch - d/p/lp1815101-02-networkd-stop-clients-when-networkd-shuts-down.patch - d/p/lp1815101-03-network-add-KeepConfiguration-dhcp-on-stop.patch - d/p/lp1815101-04-network-make-KeepConfiguration-static-drop-DHCP-addr.patch - d/p/lp1815101-05-man-add-documentation-about-KeepConfiguration.patch systemd (242-7ubuntu3.1) eoan; urgency=medium [ Balint Reczey ] * Fix shutdown and related actions from the login screen (LP: #1847896) File: debian/patches/logind-consider-greeter-sessions-suitable-as-display-sess.patch https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=b407dfd8c9dc81594553c27467c35b38d74c * debian/gbp.conf: Set debian-branch to ubuntu-eoan File: debian/gbp.conf https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=f399ce2cf4701a2dbb4b3505d2dd17a210d62f5c [ Dan Streetman ] * Fix bogus routes after DHCP lease change (LP: #1831787) Files: - debian/patches/lp1831787/0001-networkd-Add-back-static-routes-after-DHCPv4-lease-e.patch - debian/patches/lp1831787/0002-network-set-preferred-source-in-removing-route-entry.patch - debian/patches/lp1831787/0003-network-lower-log-level-about-critical-connection.patch - debian/patches/lp1831787/0004-network-reset-Link-dhcp4_configured-flag-earlier.patch - debian/patches/lp1831787/0005-network-split-dhcp_lease_lost-into-small-pieces.patch https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ced3f5c2f619083f7beb164d94d4ccfe5fe8 * Set src address for dhcp 'classless' routes (LP: #1835581) File: debian/patches/lp1835581-src-network-networkd-dhcp4.c-set-prefsrc-for-classle.patch https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=6a7ef370fb1335548448920be4ae6176b67044a8 * Allows cache=no-negative option to be set, ignoring negative answers to be cached (LP: #1668771) File: debian/patches/lp1668771-resolved-switch-cache-option-to-a-tri-state-option-s.patch https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=27def26f5b1d1b8ba314c4a925fc1b7c43837f86 -- Dan Streetman Fri, 01 Nov 2019 16:33:08 -0400 ** Changed in: systemd (Ubuntu Eoan) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Released Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Flagging this as wont fix as heartbeat is already being kept just for historical reasons (and systemd-networkd can workaround that by the fix we're backporting to it: KeepConfiguration .service file stanza). ** Changed in: heartbeat (Ubuntu Eoan) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: heartbeat (Ubuntu Disco) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: heartbeat (Ubuntu Bionic) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** Changed in: heartbeat (Ubuntu) Assignee: Rafael David Tinoco (rafaeldtinoco) => (unassigned) ** No longer affects: heartbeat (Ubuntu Eoan) ** No longer affects: heartbeat (Ubuntu Disco) ** No longer affects: heartbeat (Ubuntu Bionic) ** Changed in: heartbeat (Ubuntu) Status: Triaged => Won't Fix -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Won't Fix Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Committed Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
(k)rafaeldtinoco@kcluster01:~$ dpkg -l | grep "ii systemd " ii systemd243-3ubuntu1amd64system and service manager k)rafaeldtinoco@kcluster01:~$ for name in kcluster01 kcluster02 kcluster03; do ssh $name "dpkg -l | grep systemd "; done | grep "ii systemd " ii systemd243-3ubuntu1amd64system and service manager ii systemd243-3ubuntu1amd64system and service manager ii systemd243-3ubuntu1amd64system and service manager (k)rafaeldtinoco@kcluster01:~$ for name in kcluster01 kcluster02 kcluster03; do ssh $name "cat /etc/systemd/network/10-netplan-eth3.network"; done [Match] Name=eth3 [Network] LinkLocalAddressing=ipv6 Address=10.0.3.2/24 KeepConfiguration=static [Match] Name=eth3 [Network] LinkLocalAddressing=ipv6 Address=10.0.3.3/24 KeepConfiguration=static [Match] Name=eth3 [Network] LinkLocalAddressing=ipv6 Address=10.0.3.4/24 KeepConfiguration=static (k)rafaeldtinoco@kcluster01:~$ crm status Stack: corosync Current DC: kcluster01 (version 2.0.1-9e909a5bdd) - partition with quorum Last updated: Tue Nov 19 16:38:15 2019 Last change: Mon Nov 18 12:41:14 2019 by root via crm_resource on kcluster01 3 nodes configured 5 resources configured Online: [ kcluster01 kcluster02 kcluster03 ] Full list of resources: fence_kcluster01 (stonith:fence_virsh): Started kcluster02 fence_kcluster02 (stonith:fence_virsh): Started kcluster01 fence_kcluster03 (stonith:fence_virsh): Started kcluster01 Resource Group: webserver_virtual_ip webserver (systemd:lighttpd): Started kcluster01 virtual_ip (ocf::heartbeat:IPaddr2): Started kcluster01 (k)rafaeldtinoco@kcluster01:~$ for name in kcluster01 kcluster02 kcluster03; do ssh $name "hostname ; ip addr show eth3"; done kcluster01 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:11:a0:03 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe11:a003/64 scope link valid_lft forever preferred_lft forever kcluster02 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:1d:1a:cc brd ff:ff:ff:ff:ff:ff inet 10.0.3.3/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe1d:1acc/64 scope link valid_lft forever preferred_lft forever kcluster03 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b0:13:16 brd ff:ff:ff:ff:ff:ff inet 10.0.3.4/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feb0:1316/64 scope link valid_lft forever preferred_lft forever in parallel: (k)rafaeldtinoco@kcluster01:~$ journalctl -f -u pacemaker and check if events are generated (vip monitor detects changes) (k)rafaeldtinoco@kcluster01:~$ systemctl restart systemd-networkd No VIP changes: (k)rafaeldtinoco@kcluster01:~$ for name in kcluster01 kcluster02 kcluster03; do ssh $name "hostname ; ip addr show eth3"; done kcluster01 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:11:a0:03 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe11:a003/64 scope link valid_lft forever preferred_lft forever kcluster02 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:1d:1a:cc brd ff:ff:ff:ff:ff:ff inet 10.0.3.3/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe1d:1acc/64 scope link valid_lft forever preferred_lft forever kcluster03 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b0:13:16 brd ff:ff:ff:ff:ff:ff inet 10.0.3.4/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feb0:1316/64 scope link valid_lft forever preferred_lft forever and no events generated! verification-done ** Tags removed: verification-needed verification-needed-eoan ** Tags added: verification-done verification-done-eoan -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Hello Leroy, or anyone else affected, Accepted systemd into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/242-7ubuntu3.2 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: systemd (Ubuntu Eoan) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-eoan -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: Fix Committed Bug description: [impact] - ALL related HA software has a small problem if interfaces are being managed by systemd-networkd: nic restarts/reconfigs are always going to wipe all interfaces aliases when HA software is not expecting it to (no coordination between them. - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is smarter in this case because it has a service monitor that will restart the virtual IP resource, in affected node & nic, before considering a real failure, but other HA service might consider a real failure when it is not. [test case] - comment #14 is a full test case: to have 3 node pacemaker, in that example, and cause a networkd service restart: it will trigger a failure for the virtual IP resource monitor. - other example is given in the original description for keepalived. both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant complexity to networkd's configuration and behavior, which could lead to regressions in correctly configuring the network at networkd start, or incorrectly maintaining configuration at networkd restart, or losing network state at networkd stop. - Any regressions are most likely to occur during networkd start, restart, or stop, and most likely to involve missing or incorrect ip address(es). - the change is based in upstream patches adding the exact feature we needed to fix this issue & it will be integrated with a netplan change to add the needed stanza to systemd nic configuration file (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com,
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Description changed: [impact] - ip addresses managed by keepalived are lost across networkd restarts + - ALL related HA software has a small problem if interfaces are being + managed by systemd-networkd: nic restarts/reconfigs are always going to + wipe all interfaces aliases when HA software is not expecting it to (no + coordination between them. + + - keepalived, smb ctdb, pacemaker, all suffer from this. Pacemaker is + smarter in this case because it has a service monitor that will restart + the virtual IP resource, in affected node & nic, before considering a + real failure, but other HA service might consider a real failure when it + is not. [test case] - see original description below + - comment #14 is a full test case: to have 3 node pacemaker, in that + example, and cause a networkd service restart: it will trigger a failure + for the virtual IP resource monitor. + + - other example is given in the original description for keepalived. + both suffer from the same issue (and other HA softwares as well). [regression potential] - this backports KeepConfiguration parameter, which adds some significant - complexity to networkd's configuration and behavior, which could lead to - regressions in correctly configuring the network at networkd start, or - incorrectly maintaining configuration at networkd restart, or losing - network state at networkd stop. Any regressions are most likely to - occur during networkd start, restart, or stop, and most likely to - involve missing or incorrect ip address(es). + - this backports KeepConfiguration parameter, which adds some + significant complexity to networkd's configuration and behavior, which + could lead to regressions in correctly configuring the network at + networkd start, or incorrectly maintaining configuration at networkd + restart, or losing network state at networkd stop. + + - Any regressions are most likely to occur during networkd start, + restart, or stop, and most likely to involve missing or incorrect ip + address(es). + + - the change is based in upstream patches adding the exact feature we + needed to fix this issue & it will be integrated with a netplan change + to add the needed stanza to systemd nic configuration file + (KeepConfiguration=) [other info] original description: --- Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } }
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Description changed: + [impact] + + ip addresses managed by keepalived are lost across networkd restarts + + [test case] + + see original description below + + [regression potential] + + this backports KeepConfiguration parameter, which adds some significant + complexity to networkd's configuration and behavior, which could lead to + regressions in correctly configuring the network at networkd start, or + incorrectly maintaining configuration at networkd restart, or losing + network state at networkd stop. Any regressions are most likely to + occur during networkd start, restart, or stop, and most likely to + involve missing or incorrect ip address(es). + + [other info] + + original description: + --- + Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: - ethernets: - eth0: - addresses: [192.168.0.5/24] - dhcp4: false - nameservers: - search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] - addresses: [10.22.11.1] - eth2: - addresses: - - 12.13.14.18/29 - - 12.13.14.19/29 - gateway4: 12.13.14.17 - dhcp4: false - nameservers: - search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] - addresses: [10.22.11.1] - eth3: - addresses: [10.22.11.6/24] - dhcp4: false - nameservers: - search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] - addresses: [10.22.11.1] - eth4: - addresses: [10.22.14.6/24] - dhcp4: false - nameservers: - search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] - addresses: [10.22.11.1] - eth7: - addresses: [9.5.17.34/29] - dhcp4: false - optional: true - nameservers: - search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] - addresses: [10.22.11.1] - version: 2 + ethernets: + eth0: + addresses: [192.168.0.5/24] + dhcp4: false + nameservers: + search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] + addresses: [10.22.11.1] + eth2: + addresses: + - 12.13.14.18/29 + - 12.13.14.19/29 + gateway4: 12.13.14.17 + dhcp4: false + nameservers: + search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] + addresses: [10.22.11.1] + eth3: + addresses: [10.22.11.6/24] + dhcp4: false + nameservers: + search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] + addresses: [10.22.11.1] + eth4: + addresses: [10.22.14.6/24] + dhcp4: false + nameservers: + search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] + addresses: [10.22.11.1] + eth7: + addresses: [9.5.17.34/29] + dhcp4: false + optional: true + nameservers: + search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] + addresses: [10.22.11.1] + version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { - sysadm...@blah.com + sysadm...@blah.com } - notification_email_from keepali...@system3.hq.blah.com - smtp_server 10.22.11.7 # IP - smtp_connect_timeout 30 # integer, seconds - router_id system3 # string identifying the machine, - # (doesn't have to be hostname). - vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 - vrrp_mcast_group6 ff02::12 # optional, default ff02::12 - enable_traps # enable SNMP traps + notification_email_from keepali...@system3.hq.blah.com + smtp_server 10.22.11.7 # IP + smtp_connect_timeout 30 # integer, seconds + router_id system3 # string identifying the machine, + # (doesn't have to be hostname). + vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 + vrrp_mcast_group6 ff02::12 # optional, default ff02::12 + enable_traps # enable SNMP traps } vrrp_sync_group collection { - group { - wan - lan - phone - } + group { + wan
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether ab:cd:ef:90:c0:e3 brd ff:ff:ff:ff:ff:ff inet 10.22.14.6/24 brd 10.22.14.255 scope global eth4 valid_lft forever preferred_lft forever inet 10.22.14.3/24 scope global secondary eth4 valid_lft forever preferred_lft forever inet6 fe80::ae1f:6bff:fe90:c0e3/64 scope link valid_lft forever preferred_lft forever 7: eth3: mtu 1500 qdisc mq state UP group default qlen 1000
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Eoan SRU: MR: https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/systemd/+git/systemd/+merge/374027 PPA: https://launchpad.net/~rafaeldtinoco/+archive/ubuntu/lp1815101 Add: KeepConfiguration=static to .network file to keep aliases to. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether ab:cd:ef:90:c0:e3 brd ff:ff:ff:ff:ff:ff inet 10.22.14.6/24 brd 10.22.14.255 scope global eth4 valid_lft forever preferred_lft forever inet 10.22.14.3/24 scope global secondary eth4
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Merge proposal linked: https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/systemd/+git/systemd/+merge/374027 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether ab:cd:ef:90:c0:e3 brd ff:ff:ff:ff:ff:ff inet 10.22.14.6/24 brd 10.22.14.255 scope global eth4 valid_lft forever preferred_lft forever inet 10.22.14.3/24 scope global secondary eth4 valid_lft forever preferred_lft forever inet6 fe80::ae1f:6bff:fe90:c0e3/64 scope link valid_lft
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
(k)rafaeldtinoco@kcluster01:~$ sudo vi /etc/systemd/network/10-netplan- eth3.network (k)rafaeldtinoco@kcluster01:~$ systemctl restart systemd-networkd (k)rafaeldtinoco@kcluster01:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:11:f0:03 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe11:f003/64 scope link valid_lft forever preferred_lft forever (k)rafaeldtinoco@kcluster01:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:11:f0:03 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe11:f003/64 scope link valid_lft forever preferred_lft forever (k)rafaeldtinoco@kcluster01:~$ systemctl restart systemd-networkd (k)rafaeldtinoco@kcluster01:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:11:f0:03 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe11:f003/64 scope link valid_lft forever preferred_lft forever Voila. Needs better testing with KeepConfiguration=dhcp. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Test Case: (k)rafaeldtinoco@kcluster03:~$ crm status Stack: corosync Current DC: kcluster02 (version 2.0.1-9e909a5bdd) - partition with quorum Last updated: Thu Oct 10 17:13:19 2019 Last change: Thu Oct 10 17:11:48 2019 by root via cibadmin on kcluster01 3 nodes configured 5 resources configured Online: [ kcluster01 kcluster02 kcluster03 ] Full list of resources: fence_kcluster01 (stonith:fence_virsh): Started kcluster02 fence_kcluster02 (stonith:fence_virsh): Started kcluster01 fence_kcluster03 (stonith:fence_virsh): Started kcluster01 Resource Group: webserver_virtual_ip webserver (systemd:lighttpd): Started kcluster03 virtual_ip (ocf::heartbeat:IPaddr2): Started kcluster03 (k)rafaeldtinoco@kcluster03:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b0:c3:06 brd ff:ff:ff:ff:ff:ff inet 10.0.3.4/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feb0:c306/64 scope link valid_lft forever preferred_lft forever (k)rafaeldtinoco@kcluster03:~$ systemctl restart systemd-networkd (k)rafaeldtinoco@kcluster03:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b0:c3:06 brd ff:ff:ff:ff:ff:ff inet 10.0.3.4/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feb0:c306/64 scope link valid_lft forever preferred_lft forever (k)rafaeldtinoco@kcluster03:~$ ip addr show eth3 5: eth3: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b0:c3:06 brd ff:ff:ff:ff:ff:ff inet 10.0.3.4/24 brd 10.0.3.255 scope global eth3 valid_lft forever preferred_lft forever inet 10.0.3.1/24 brd 10.0.3.255 scope global secondary eth3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:feb0:c306/64 scope link valid_lft forever preferred_lft forever (k)rafaeldtinoco@kcluster03:~$ Pacemaker logs: Oct 10 17:14:37 kcluster03 IPaddr2(virtual_ip)[6901]: INFO: IP status = no, IP_CIP= Oct 10 17:14:37 kcluster03 pacemaker-controld[1266]: notice: Result of stop operation for virtual_ip on kcluster03: 0 (ok) Oct 10 17:14:37 kcluster03 IPaddr2(virtual_ip)[6951]: INFO: Adding inet address 10.0.3.1/24 with broadcast address 10.0.3.255 to device eth3 Oct 10 17:14:37 kcluster03 IPaddr2(virtual_ip)[6956]: INFO: Bringing device eth3 up Oct 10 17:14:37 kcluster03 IPaddr2(virtual_ip)[6961]: INFO: /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /run/resource-agents/send_arp-10.0.3.1 eth3 10.0.3.1 auto not_used not_used Oct 10 17:14:37 kcluster03 pacemaker-controld[1266]: notice: Result of start operation for virtual_ip on kcluster03: 0 (ok) for the operation. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4:
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
@ed, I just finished the backport to Eoan it was straightforward, I'll finish tests tomorrow with HA related software and networkd enabled HA clusters. After that I'll give you a better estimation about Disco and Bionic. This is the total size of changes (systemd-networkd-tests.py is not so great to backport, will review that): $ cat *.patch | diffstat man/systemd.network.xml| 27 +- src/network/networkd-dhcp4.c | 8 src/network/networkd-link.c| 57 +- src/network/networkd-link.h| 2 src/network/networkd-manager.c | 2 src/network/networkd-network-gperf.gperf | 3 src/network/networkd-network.c | 44 src/network/networkd-network.h | 26 ++ test/fuzz/fuzz-network-parser/directives.network | 1 test/test-network/conf/24-keep-configuration-static.network| 5 test/test-network/conf/dhcp-client-keep-configuration-dhcp-on-stop.network | 4 test/test-network/conf/dhcp-client-keep-configuration-dhcp.network | 7 test/test-network/systemd-networkd-tests.py| 94 +- 13 files changed, 235 insertions(+), 45 deletions(-) Good thing is that the logic is not drastically changed for this feature to exist. Sorry for the delay here, because of freeze we were running to close out some urgent issues for Eoan. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection {
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Also affects: charm-keepalived Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in Keepalived Charm: New Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether ab:cd:ef:90:c0:e3 brd ff:ff:ff:ff:ff:ff inet 10.22.14.6/24 brd 10.22.14.255 scope global eth4 valid_lft forever preferred_lft forever inet 10.22.14.3/24 scope global secondary eth4 valid_lft forever preferred_lft forever inet6 fe80::ae1f:6bff:fe90:c0e3/64 scope link valid_lft forever preferred_lft forever 7: eth3: mtu
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
Thanks Rafael/Christian, I see that all those patches are in 243 and Eoan is currently on 242 (albeit -6 but i dont think any are already backported) so we'll need to get this backported all the way down to Bionic. max@power:~/git/systemd$ _c=( 7da377e 95355a2 db51778 c98d78d 1e49885 ) max@power:~/git/systemd$ for c in ${_c[@]}; do git tag --contains $c| egrep -v "\-rc"; done| sort -u v243 Do we have a feel for if/when the keepalived fix(es) will be backportable to B (1.x) as well? Since those fixes already exist in Discco (2.0.10) it might be easier to start with those? I will add the charm-keepalived to this LP since it will need support for the networkd/netplan fix once that is available. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
If you are lazy to look for these commits, feel free to use these links https://github.com/systemd/systemd/commit/7da377ef16a2112a673247b39041a180b07e973a https://github.com/systemd/systemd/commit/95355a281c06c5970b7355c38b066910c3be4958 https://github.com/systemd/systemd/commit/db51778f85cb076e9ed1fe7f7e29cc740365c245 https://github.com/systemd/systemd/commit/c98d78d32abba6aadbe89eece7acf0742f59047c https://github.com/systemd/systemd/commit/1e498853a39b46155cb89b5c9e74ecb27aaba3ed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
The commits bellow implement support to "keep configuration": commit 1e498853a39b46155cb89b5c9e74ecb27aaba3ed Author: Yu Watanabe Date: Mon Jun 3 01:21:13 2019 test-network: add tests for KeepConfiguration= commit c98d78d32abba6aadbe89eece7acf0742f59047c Author: Yu Watanabe Date: Mon Jun 3 03:37:25 2019 man: add documentation about KeepConfiguration commit db51778f85cb076e9ed1fe7f7e29cc740365c245 Author: Yu Watanabe Date: Mon Jun 3 00:33:13 2019 network: make KeepConfiguration=static drop DHCP addresses and routes Also, KeepConfiguration=dhcp drops static foreign addresses and routes. commit 95355a281c06c5970b7355c38b066910c3be4958 Author: Yu Watanabe Date: Mon Jun 3 14:05:26 2019 network: add KeepConfiguration=dhcp-on-stop The option prevents to drop lease address on stop. By setting this, we can safely restart networkd. commit 7da377ef16a2112a673247b39041a180b07e973a Author: Susant Sahani Date: Mon Jun 3 00:31:13 2019 networkd: add support to keep configuration for systemd-networkd. IMO, we should rely in setting the keep configuration flag for the interfaes to be managed by 3rd part software (adding/removing aliases for virtual networks, VRRP interfaces, etc). -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface
[Touch-packages] [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)
** Summary changed: - [master] Restarting systemd-networkd breaks keepalived clusters + [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1815101 Title: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted) Status in netplan: Confirmed Status in heartbeat package in Ubuntu: Triaged Status in keepalived package in Ubuntu: In Progress Status in systemd package in Ubuntu: In Progress Status in heartbeat source package in Bionic: Triaged Status in keepalived source package in Bionic: Confirmed Status in systemd source package in Bionic: Confirmed Status in heartbeat source package in Disco: Triaged Status in keepalived source package in Disco: Confirmed Status in systemd source package in Disco: Confirmed Status in heartbeat source package in Eoan: Triaged Status in keepalived source package in Eoan: In Progress Status in systemd source package in Eoan: In Progress Bug description: Configure netplan for interfaces, for example (a working config with IP addresses obfuscated) network: ethernets: eth0: addresses: [192.168.0.5/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth2: addresses: - 12.13.14.18/29 - 12.13.14.19/29 gateway4: 12.13.14.17 dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth3: addresses: [10.22.11.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth4: addresses: [10.22.14.6/24] dhcp4: false nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] eth7: addresses: [9.5.17.34/29] dhcp4: false optional: true nameservers: search: [blah.com, other.blah.com, hq.blah.com, cust.blah.com, phone.blah.com] addresses: [10.22.11.1] version: 2 Configure keepalived (again, a working config with IP addresses obfuscated) global_defs # Block id { notification_email { sysadm...@blah.com } notification_email_from keepali...@system3.hq.blah.com smtp_server 10.22.11.7 # IP smtp_connect_timeout 30 # integer, seconds router_id system3 # string identifying the machine, # (doesn't have to be hostname). vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18 vrrp_mcast_group6 ff02::12 # optional, default ff02::12 enable_traps # enable SNMP traps } vrrp_sync_group collection { group { wan lan phone } vrrp_instance wan { state MASTER interface eth2 virtual_router_id 77 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass BlahBlah } virtual_ipaddress { 12.13.14.20 } } vrrp_instance lan { state MASTER interface eth3 virtual_router_id 78 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MoreBlah } virtual_ipaddress { 10.22.11.13/24 } } vrrp_instance phone { state MASTER interface eth4 virtual_router_id 79 priority 150 advert_int 1 smtp_alert authentication { auth_type PASS auth_pass MostBlah } virtual_ipaddress { 10.22.14.3/24 } } At boot the affected interfaces have: 5: eth4: mtu 1500 qdisc mq state UP group default qlen 1000 link/ether ab:cd:ef:90:c0:e3 brd ff:ff:ff:ff:ff:ff inet 10.22.14.6/24 brd 10.22.14.255 scope global eth4 valid_lft forever preferred_lft forever inet 10.22.14.3/24 scope global secondary eth4 valid_lft forever preferred_lft forever inet6