I have now identified the lower-level trigger for the generator timeout.

During a controlled reproduction, `systemd-fstab-generator` was already
hanging before the package upgrade. Tracing showed that it blocks while
accessing an existing NFS mount:

    fass-v:/pool -> /mnt/fass/pool
    NFSv4.1, hard, sec=krb5

The underlying NFS client is stuck in NFSv4.1 session recovery against
Microsoft Server for NFS.

On affected clients we observed:

    nfsv4 SEQUENCE ... q:xprt_pending

and, on one client, the NFS state-manager thread is blocked in:

    nfs4_drain_slot_tbl
    nfs4_reset_session
    nfs4_state_manager
    nfs4_run_state_manager

New requests wait on the ForeChannel slot table. TCP connectivity to the
server is still alive, but a 135-second packet capture showed no NFS RPC
progress.

This also reproduces independently on another Ubuntu 24.04 client with
systemd 255.4-1ubuntu8.17 and kernel 6.8.

During the fatal systemd reproduction the sequence is therefore now
clear:

    hung NFSv4.1 mount
      -> systemd-fstab-generator blocks
      -> daemon-reexec
      -> generator infrastructure times out after ~90 seconds
      -> "Failed to fork off sandboxing environment for executing generators: 
Protocol error"
      -> "Freezing execution."

So the NFS problem is the trigger, but the systemd issue remains that a
blocked generator during `daemon-reexec` causes PID 1 to enter the fatal
"Freezing execution" state instead of recovering or failing the
generator operation gracefully.

I have also reported the underlying NFSv4.1 recovery hang to the
existing linux-nfs upstream thread:

https://www.spinics.net/lists/kernel/msg6188959.html

I can provide the generator strace, SUNRPC debugfs state, mountstats and
packet capture if useful.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2165295

Title:
  systemd freezes after daemon-reexec triggered by unattended-upgrades /
  needrestart on Ubuntu 24.04

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2165295/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to