** Description changed: [ Impact ] + On Ubuntu Core 26 (and potentially Ubuntu hybrid 26.04 with TPM/FDE), systemd calls `socket(AF_NETLINK, ..., NETLINK_NETFILTER)` multiple times per unit during daemon-reload, even when no NFTSet= is configured. When the `nfnetlink` kernel module is not loaded, this call is very slow and fails. Since daemon-reload processes each unit twice, this significantly slows down boot and snap operations that trigger daemon-reload. - * Systemd calls socket(AF_NETLINK, ..., NETLINK_NETFILTER) multiple times even - when not needed. When the nfnetlink module is not available, this call is - very slow. - - * On daemon-reload this is done twice per unit. - - * The boot of UC26 is then much slower than UC24. It probably affects also - Ubuntu hybrid 26.04 (TPM/FDE). - - * We added nfnetlink kernel module to work-around it. But this module - does not belong in initrd because nothing actually uses it. Apart - from opening the socket. + The snapd fix (PR https://github.com/canonical/snapd/pull/17014) adds + the `nfnetlink` kernel module to the initrd so the socket call succeeds + quickly. The root cause fix is in systemd upstream + (https://github.com/systemd/systemd/pull/41930) which skips the socket + open when no NFT sets are configured. [ Test Plan ] - - * Build an initrd with https://github.com/canonical/snapd/pull/17014 - reverted. - - * Run Ubuntu Core 26 with bootchart enabled. Compare with and without - fix. The time for initrd should be shorter with the fix. This is - specially noticable around daemon-reload calls. + 1. Build an initrd with https://github.com/canonical/snapd/pull/17014 + reverted. + 2. Build an Ubuntu Core 26 image with snapd 2.76 (which includes the nfnetlink module in initrd). + 3. Enable bootchart and measure boot time, particularly around daemon-reload calls in initrd. + 4. Compare against an image without the fix (snapd < 2.76, nfnetlink not in initrd). + 5. The time for initrd phase should be noticeably shorter with the fix. [ Where problems could occur ] + The change adds the `nfnetlink` kernel module to the initrd. This affects NFTSet= from systemd.resource‐control(5). The change is simple enough that there is high confidence that nothing will break. But if something breaks, it is for that field. - * This affects NFTSet= from systemd.resource‐control(5). - The change is simple enough that there is high confidence that - nothing will break. But if something breaks, it is for that field. + This increases initrd size slightly. The module itself is inert unless + something actively uses netfilter netlink sockets. If the module has + dependencies not satisfied in the initrd environment, it could fail to + load (harmless — falls back to current slow behaviour). The risk is very + low. - [ Other Info ] + [ Applicability ] + Ubuntu Core 26 and Ubuntu hybrid 26.04 (TPM/FDE) only. Does not affect Classic Ubuntu or Ubuntu Core 24 and earlier. Fix was merged upstream https://github.com/systemd/systemd/pull/41930 - * Fix was merged upstream https://github.com/systemd/systemd/pull/41930 - - [ Original report ] + ---original--- It has been detected that daemon-reload requests take more time on UC26 than on UC24. The root cause is a new call to "sd_nfnl_socket_open()" in unit_modify_user_nft_set() that takes some time to run and finally fails. An optimization for this is https://pastebin.canonical.com/p/P6QFgQy5cH/ (when nft_set is empty, the call is prevented).
** Description changed: + [SRU] 2.76 + (https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/2154498) + [ Impact ] On Ubuntu Core 26 (and potentially Ubuntu hybrid 26.04 with TPM/FDE), systemd calls `socket(AF_NETLINK, ..., NETLINK_NETFILTER)` multiple times per unit during daemon-reload, even when no NFTSet= is configured. When the `nfnetlink` kernel module is not loaded, this call is very slow and fails. Since daemon-reload processes each unit twice, this significantly slows down boot and snap operations that trigger daemon-reload. The snapd fix (PR https://github.com/canonical/snapd/pull/17014) adds the `nfnetlink` kernel module to the initrd so the socket call succeeds quickly. The root cause fix is in systemd upstream (https://github.com/systemd/systemd/pull/41930) which skips the socket open when no NFT sets are configured. [ Test Plan ] 1. Build an initrd with https://github.com/canonical/snapd/pull/17014 reverted. 2. Build an Ubuntu Core 26 image with snapd 2.76 (which includes the nfnetlink module in initrd). 3. Enable bootchart and measure boot time, particularly around daemon-reload calls in initrd. 4. Compare against an image without the fix (snapd < 2.76, nfnetlink not in initrd). 5. The time for initrd phase should be noticeably shorter with the fix. [ Where problems could occur ] The change adds the `nfnetlink` kernel module to the initrd. This affects NFTSet= from systemd.resource‐control(5). The change is simple enough that there is high confidence that nothing will break. But if something breaks, it is for that field. This increases initrd size slightly. The module itself is inert unless something actively uses netfilter netlink sockets. If the module has dependencies not satisfied in the initrd environment, it could fail to load (harmless — falls back to current slow behaviour). The risk is very low. [ Applicability ] Ubuntu Core 26 and Ubuntu hybrid 26.04 (TPM/FDE) only. Does not affect Classic Ubuntu or Ubuntu Core 24 and earlier. Fix was merged upstream https://github.com/systemd/systemd/pull/41930 ---original--- It has been detected that daemon-reload requests take more time on UC26 than on UC24. The root cause is a new call to "sd_nfnl_socket_open()" in unit_modify_user_nft_set() that takes some time to run and finally fails. An optimization for this is https://pastebin.canonical.com/p/P6QFgQy5cH/ (when nft_set is empty, the call is prevented). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2150773 Title: Slowness on UC26 on daemon-reload requests To manage notifications about this bug go to: https://bugs.launchpad.net/snapd/+bug/2150773/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
