[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
I can confirm this, by installing network-manager, nfs-kernel-server, and setserial into a clean VM. setserial's init.d script is very demanding as it both wants to start in run level S (sysinit.target in systemd's mapping) *and* requires $remote_fs. But we can't start remote file systems that early if we need NetworkManager for them, this only works with very simple networking like ifupdown. I suppose /etc/init.d/setserial wants $remote_fs as it reads stuff from /var/lib/setserial/ which might potentially be on NFS. Having setserial on an NFS /var with NetworkManager is certainly a very special case, and I suppose we could just ignore it. But I think we can relax the early startup instead. As far as I can see, the most appropriate course here is to provide a systemd unit for setserial with more fine-grained ordering, or just a drop-in to relax the overly harsh Default-Start: S (as it really just Before=system-getty.slice). I don't want to completely replace the init.d script as it has lots of code (which really doesn't belong there, but I leave that to the Debian maintainer, and this is too intrusive for an SRU). ** Package changed: nfs-utils (Ubuntu Wily) = setserial (Ubuntu Wily) ** Changed in: setserial (Ubuntu Wily) Status: Triaged = In Progress ** Changed in: setserial (Ubuntu Wily) Assignee: (unassigned) = Martin Pitt (pitti) ** Summary changed: - vivid NFSd does not start due to systemd ordering cycle + NFSd does not start due to systemd ordering cycle with setserial ** Description changed: NFSd does not start due to systemd ordering cycle. Manually starting with systemctl start nfs-kernel-server after boot works fine. here systemd lines from dmesg: - [2.559819] random: systemd-udevd urandom read with 8 bits of entropy available - [ 43.181921] systemd[1]: Inserted module 'autofs4' - [ 43.210140] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN) - [ 43.234042] systemd[1]: Detected architecture x86-64. - [ 43.328158] systemd[1]: Set hostname to xpc. - [ 43.434248] systemd-sysv-generator[337]: Overwriting existing symlink /run/systemd/generator.late/mdadm-waitidle.service with real service [ 43.569341] systemd[1]: Cannot add dependency job for unit gssproxy.service, ignoring: Unit gssproxy.service failed to load: No such file or directory. [ 43.595080] systemd[1]: Found ordering cycle on nfs-server.service/start [ 43.620748] systemd[1]: Found dependency on network.target/start [ 43.646476] systemd[1]: Found dependency on NetworkManager.service/start [ 43.672292] systemd[1]: Found dependency on basic.target/start [ 43.697952] systemd[1]: Found dependency on sockets.target/start [ 43.723381] systemd[1]: Found dependency on cups.socket/start [ 43.748530] systemd[1]: Found dependency on sysinit.target/start [ 43.773533] systemd[1]: Found dependency on setserial.service/start [ 43.798185] systemd[1]: Found dependency on remote-fs.target/start [ 43.822739] systemd[1]: Found dependency on remote-fs-pre.target/start [ 43.871768] systemd[1]: Found dependency on nfs-server.service/start [ 43.896442] systemd[1]: Breaking ordering cycle by deleting job network.target/start [ 43.921641] systemd[1]: Job network.target/start deleted to break ordering cycle starting with nfs-server.service/start - [ 44.109024] systemd[1]: Reached target Encrypted Volumes. - [ 44.135558] systemd[1]: Starting Encrypted Volumes. - [ 44.189110] systemd[1]: Created slice Root Slice. - [ 44.215900] systemd[1]: Starting Root Slice. - [ 44.269513] systemd[1]: Listening on /dev/initctl Compatibility Named Pipe. - [ 44.296806] systemd[1]: Starting /dev/initctl Compatibility Named Pipe. - [ 44.351462] systemd[1]: Listening on LVM2 metadata daemon socket. - [ 44.378792] systemd[1]: Starting LVM2 metadata daemon socket. - [ 44.459356] systemd[1]: Dependency failed for pNFS block layout mapping daemon. - [ 44.486177] systemd[1]: Job nfs-blkmap.service/start failed with result 'dependency'. - [ 44.539816] systemd[1]: Listening on Delayed Shutdown Socket. - [ 44.566481] systemd[1]: Starting Delayed Shutdown Socket. - [ 44.619658] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. - [ 44.647063] systemd[1]: Starting Arbitrary Executable File Formats File System Automount Point. - [ 44.702240] systemd[1]: Listening on Journal Socket (/dev/log). - [ 44.730201] systemd[1]: Starting Journal Socket (/dev/log). - [ 44.786161] systemd[1]: Listening on fsck to fsckd communication Socket. - [ 44.814391] systemd[1]: Starting fsck to fsckd communication Socket. - [ 44.870869] systemd[1]: Listening on udev Control Socket. - [ 44.899276] systemd[1]: Starting udev Control Socket. - [ 44.927774] systemd[1]: Started Forward
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
I can confirm this, by installing network-manager, nfs-kernel-server, and setserial into a clean VM. setserial's init.d script is very demanding as it both wants to start in run level S (sysinit.target in systemd's mapping) *and* requires $remote_fs. But we can't start remote file systems that early if we need NetworkManager for them, this only works with very simple networking like ifupdown. I suppose /etc/init.d/setserial wants $remote_fs as it reads stuff from /var/lib/setserial/ which might potentially be on NFS. Having setserial on an NFS /var with NetworkManager is certainly a very special case, and I suppose we could just ignore it. But I think we can relax the early startup instead. As far as I can see, the most appropriate course here is to provide a systemd unit for setserial with more fine-grained ordering, or just a drop-in to relax the overly harsh Default-Start: S (as it really just Before=system-getty.slice). I don't want to completely replace the init.d script as it has lots of code (which really doesn't belong there, but I leave that to the Debian maintainer, and this is too intrusive for an SRU). ** Package changed: nfs-utils (Ubuntu Wily) = setserial (Ubuntu Wily) ** Changed in: setserial (Ubuntu Wily) Status: Triaged = In Progress ** Changed in: setserial (Ubuntu Wily) Assignee: (unassigned) = Martin Pitt (pitti) ** Summary changed: - vivid NFSd does not start due to systemd ordering cycle + NFSd does not start due to systemd ordering cycle with setserial ** Description changed: NFSd does not start due to systemd ordering cycle. Manually starting with systemctl start nfs-kernel-server after boot works fine. here systemd lines from dmesg: - [2.559819] random: systemd-udevd urandom read with 8 bits of entropy available - [ 43.181921] systemd[1]: Inserted module 'autofs4' - [ 43.210140] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN) - [ 43.234042] systemd[1]: Detected architecture x86-64. - [ 43.328158] systemd[1]: Set hostname to xpc. - [ 43.434248] systemd-sysv-generator[337]: Overwriting existing symlink /run/systemd/generator.late/mdadm-waitidle.service with real service [ 43.569341] systemd[1]: Cannot add dependency job for unit gssproxy.service, ignoring: Unit gssproxy.service failed to load: No such file or directory. [ 43.595080] systemd[1]: Found ordering cycle on nfs-server.service/start [ 43.620748] systemd[1]: Found dependency on network.target/start [ 43.646476] systemd[1]: Found dependency on NetworkManager.service/start [ 43.672292] systemd[1]: Found dependency on basic.target/start [ 43.697952] systemd[1]: Found dependency on sockets.target/start [ 43.723381] systemd[1]: Found dependency on cups.socket/start [ 43.748530] systemd[1]: Found dependency on sysinit.target/start [ 43.773533] systemd[1]: Found dependency on setserial.service/start [ 43.798185] systemd[1]: Found dependency on remote-fs.target/start [ 43.822739] systemd[1]: Found dependency on remote-fs-pre.target/start [ 43.871768] systemd[1]: Found dependency on nfs-server.service/start [ 43.896442] systemd[1]: Breaking ordering cycle by deleting job network.target/start [ 43.921641] systemd[1]: Job network.target/start deleted to break ordering cycle starting with nfs-server.service/start - [ 44.109024] systemd[1]: Reached target Encrypted Volumes. - [ 44.135558] systemd[1]: Starting Encrypted Volumes. - [ 44.189110] systemd[1]: Created slice Root Slice. - [ 44.215900] systemd[1]: Starting Root Slice. - [ 44.269513] systemd[1]: Listening on /dev/initctl Compatibility Named Pipe. - [ 44.296806] systemd[1]: Starting /dev/initctl Compatibility Named Pipe. - [ 44.351462] systemd[1]: Listening on LVM2 metadata daemon socket. - [ 44.378792] systemd[1]: Starting LVM2 metadata daemon socket. - [ 44.459356] systemd[1]: Dependency failed for pNFS block layout mapping daemon. - [ 44.486177] systemd[1]: Job nfs-blkmap.service/start failed with result 'dependency'. - [ 44.539816] systemd[1]: Listening on Delayed Shutdown Socket. - [ 44.566481] systemd[1]: Starting Delayed Shutdown Socket. - [ 44.619658] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. - [ 44.647063] systemd[1]: Starting Arbitrary Executable File Formats File System Automount Point. - [ 44.702240] systemd[1]: Listening on Journal Socket (/dev/log). - [ 44.730201] systemd[1]: Starting Journal Socket (/dev/log). - [ 44.786161] systemd[1]: Listening on fsck to fsckd communication Socket. - [ 44.814391] systemd[1]: Starting fsck to fsckd communication Socket. - [ 44.870869] systemd[1]: Listening on udev Control Socket. - [ 44.899276] systemd[1]: Starting udev Control Socket. - [ 44.927774] systemd[1]: Started Forward
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
I am getting the same problem here, but using the open-iscsi initiator (which - of course - depends on the network being initialized): Found ordering cycle on sysinit.target/start [3.657825] systemd[1]: Found dependency on open-iscsi.service/start [3.657827] systemd[1]: Found dependency on network-online.target/start [3.657828] systemd[1]: Found dependency on NetworkManager-wait-online.service/start [3.657830] systemd[1]: Found dependency on basic.target/start [3.657831] systemd[1]: Found dependency on sockets.target/start [3.657833] systemd[1]: Found dependency on acpid.socket/start [3.657834] systemd[1]: Found dependency on sysinit.target/start [3.657836] systemd[1]: Breaking ordering cycle by deleting job open-iscsi.service/start [3.657837] systemd[1]: Job open-iscsi.service/start deleted to break ordering cycle starting with sysinit.target/start So, it is quite a systematic problem with the System-V init scripts and their dependency setup in systemd. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
I wonder if some update arrived between the bug reporting... After reboot: [ 43.777385] systemd[1]: Found dependency on nfs-server.service/start [ 43.803123] systemd[1]: Found dependency on network.target/start [ 43.828678] systemd[1]: Breaking ordering cycle by deleting job NetworkManager.service/start the rpcinfo shows it started OK. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
There is something weird going on, since every boot behaves a bit differently: [ 44.264445] systemd[1]: Found ordering cycle on nfs-server.service/start [ 44.264447] systemd[1]: Found dependency on nfs-mountd.service/start [ 44.264448] systemd[1]: Found dependency on network.target/start [ 44.264449] systemd[1]: Found dependency on NetworkManager.service/start [ 44.264450] systemd[1]: Found dependency on basic.target/start [ 44.264451] systemd[1]: Found dependency on sockets.target/start [ 44.264452] systemd[1]: Found dependency on acpid.socket/start [ 44.264453] systemd[1]: Found dependency on sysinit.target/start [ 44.264455] systemd[1]: Found dependency on setserial.service/start [ 44.264457] systemd[1]: Found dependency on remote-fs.target/start [ 44.264460] systemd[1]: Found dependency on remote-fs-pre.target/start [ 44.264461] systemd[1]: Found dependency on nfs-server.service/start [ 44.264462] systemd[1]: Breaking ordering cycle by deleting job nfs-mountd.service/start [ 44.264463] systemd[1]: Job nfs-mountd.service/start deleted to break ordering cycle starting with nfs-server.service/start This time NFS is not working. rpcinfo program version netid addressserviceowner 104tcp6 ::.0.111 portmapper superuser 103tcp6 ::.0.111 portmapper superuser 104udp6 ::.0.111 portmapper superuser 103udp6 ::.0.111 portmapper superuser 104tcp 0.0.0.0.0.111 portmapper superuser 103tcp 0.0.0.0.0.111 portmapper superuser 102tcp 0.0.0.0.0.111 portmapper superuser 104udp 0.0.0.0.0.111 portmapper superuser 103udp 0.0.0.0.0.111 portmapper superuser 102udp 0.0.0.0.0.111 portmapper superuser 104local /run/rpcbind.sock portmapper superuser 103local /run/rpcbind.sock portmapper superuser But sudo systemctl start nfs-kernel-server starts nfs server beautifully -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
** Tags added: systemd-boot -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
The nfs-server unit has a wrong dependency on 'network.target'. The only services that should depend on this are those which implement the network management service itself. ** Changed in: nfs-utils (Ubuntu) Importance: Undecided = High ** Changed in: nfs-utils (Ubuntu) Status: New = Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
** Branch linked: lp:ubuntu/nfs-utils -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1452644] Re: vivid NFSd does not start due to systemd ordering cycle
I assumed that this would be fixed by removing the dependency on network.target (which is certainly wrong), but I'm not sure that it does, because nfs-server is Before=remote-fs-pre.target. More investigation is needed. ** Also affects: nfs-utils (Ubuntu Wily) Importance: High Status: Triaged ** Also affects: nfs-utils (Ubuntu Vivid) Importance: Undecided Status: New ** Changed in: nfs-utils (Ubuntu Vivid) Status: New = Triaged ** Changed in: nfs-utils (Ubuntu Vivid) Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1452644 Title: vivid NFSd does not start due to systemd ordering cycle To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1452644/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
