Hi Ludo, Thankyou for taking a look at this issue!
I think I didn't make it clear in the original bug report what the actual problem was. I understand that in the beginning of the boot process the "device-mapping-streetkid_VG_storage-streetkid_LV_home" shepherd service is failing and this is causing shepherd services that depend on it to fail to start as a consequence. Later in the boot process the "device-mapping-streetkid_VG_storage-streetkid_LV_home" does successfully start though. What happens is that after the "device-mapping-streetkid_VG_storage-streetkid_LV_home" does start (after a couple of tries), other shepherd services will then start including the "user-processes" shepherd service. The problem is that the "guix-daemon" shepherd service doesn't always start after that. Sometimes it will, but sometimes it won't (see my first message for examples). Shouldn't the "guix-daemon" shepherd service ALWAYS start after the "user-processes" shepherd service starts? I also found that the "nscd" shepherd service is failing to start even though the "user-processes" shepherd service has started. The following terminal output was taken after my streetkid server had finished booting up. This should make things clearer: 1 mark@streetkid ~$ sudo herd status 2 Started: 3 + /proc/fs/nfsd 4 + console-font-tty1 5 + console-font-tty2 6 + console-font-tty3 7 + console-font-tty4 8 + console-font-tty5 9 + console-font-tty6 10 + device-mapping-streetkid_VG_storage-streetkid_LV_home 11 + file-system-/dev/pts 12 + file-system-/dev/shm 13 + file-system-/gnu/store 14 + file-system-/home 15 + file-system-/sys/firmware/efi/efivars 16 + file-system-/sys/kernel/debug 17 + file-systems 18 + idmap-daemon 19 + loopback 20 + mcron 21 + networking 22 + nfs 23 + ntpd 24 + root 25 + root-file-system 26 + rpc-pipefs 27 + rpc.mountd 28 + rpc.nfsd 29 + rpc.statd 30 + rpcbind-daemon 31 + ssh-daemon 32 + sshd-1 33 + syslogd 34 + term-tty1 35 + term-tty2 36 + term-tty3 37 + term-tty4 38 + term-tty5 39 + term-tty6 40 + udev 41 + urandom-seed 42 + user-file-systems 43 + user-processes 44 + virtual-terminal 45 Stopped: 46 - guix-daemon 47 - nscd 48 - term-console 49 One-shot: 50 * host-name 51 * sysctl 52 * user-homes As you can see on line 10 "device-mapping-streetkid_VG_storage-streetkid_LV_home" has been started and on line 43 "user-processes" has been started. The problem is that "guix-daemon" hasn't been started (line 46) and "nscd" also hasn't been started (line 47). Shouldn't Shepherd start these two services once "user-processes" has been started? I checked other services that depend on "user-processes" (see the attached shepherd dependency graph for my system). The "mcron" service and "ntpd" successfully start (lines 20 and 23). This still leaves the mystery of why "guix-daemon" and "nscd" won't start even though "user-processes" HAS been started. I hope this makes things clearer. Cheers Markus
digraph "Guix shepherd-service" { "user-file-systems" [label = "user-file-systems", shape = box, fontname = sans]; "file-systems" -> "user-file-systems" [color = darkviolet]; "file-systems" [label = "file-systems", shape = box, fontname = sans]; "user-processes" -> "file-systems" [color = dimgrey]; "user-homes" -> "file-systems" [color = darkgoldenrod]; "urandom-seed" -> "file-systems" [color = magenta]; "user-processes" [label = "user-processes", shape = box, fontname = sans]; "nscd" -> "user-processes" [color = darkseagreen]; "guix-daemon" -> "user-processes" [color = dimgrey]; "term-tty6" -> "user-processes" [color = peachpuff4]; "term-tty5" -> "user-processes" [color = red]; "term-tty4" -> "user-processes" [color = darkseagreen]; "term-tty3" -> "user-processes" [color = darkgoldenrod]; "term-tty2" -> "user-processes" [color = dimgrey]; "term-tty1" -> "user-processes" [color = magenta]; "term-console" -> "user-processes" [color = magenta]; "syslogd" -> "user-processes" [color = darkseagreen]; "ntpd" -> "user-processes" [color = red]; "mcron" -> "user-processes" [color = darkgoldenrod]; "nscd" [label = "nscd", shape = box, fontname = sans]; "guix-daemon" [label = "guix-daemon", shape = box, fontname = sans]; "term-tty6" [label = "term-tty6", shape = box, fontname = sans]; "console-font-tty6" -> "term-tty6" [color = darkviolet]; "console-font-tty6" [label = "console-font-tty6", shape = box, fontname = sans]; "term-tty5" [label = "term-tty5", shape = box, fontname = sans]; "console-font-tty5" -> "term-tty5" [color = peachpuff4]; "console-font-tty5" [label = "console-font-tty5", shape = box, fontname = sans]; "term-tty4" [label = "term-tty4", shape = box, fontname = sans]; "console-font-tty4" -> "term-tty4" [color = red]; "console-font-tty4" [label = "console-font-tty4", shape = box, fontname = sans]; "term-tty3" [label = "term-tty3", shape = box, fontname = sans]; "console-font-tty3" -> "term-tty3" [color = cyan3]; "console-font-tty3" [label = "console-font-tty3", shape = box, fontname = sans]; "term-tty2" [label = "term-tty2", shape = box, fontname = sans]; "console-font-tty2" -> "term-tty2" [color = blue]; "console-font-tty2" [label = "console-font-tty2", shape = box, fontname = sans]; "term-tty1" [label = "term-tty1", shape = box, fontname = sans]; "console-font-tty1" -> "term-tty1" [color = darkseagreen]; "console-font-tty1" [label = "console-font-tty1", shape = box, fontname = sans]; "term-console" [label = "term-console", shape = box, fontname = sans]; "syslogd" [label = "syslogd", shape = box, fontname = sans]; "term-console" -> "syslogd" [color = magenta]; "ssh-daemon ssh sshd" -> "syslogd" [color = cyan3]; "ssh-daemon ssh sshd" [label = "ssh-daemon ssh sshd", shape = box, fontname = sans]; "ntpd" [label = "ntpd", shape = box, fontname = sans]; "mcron" [label = "mcron", shape = box, fontname = sans]; "user-homes" [label = "user-homes", shape = box, fontname = sans]; "user-processes" -> "user-homes" [color = dimgrey]; "urandom-seed" [label = "urandom-seed", shape = box, fontname = sans]; "user-processes" -> "urandom-seed" [color = dimgrey]; "root-file-system" [label = "root-file-system", shape = box, fontname = sans]; "file-systems" -> "root-file-system" [color = darkviolet]; "file-system-/home" -> "root-file-system" [color = darkseagreen]; "file-system-/dev/pts" -> "root-file-system" [color = red]; "file-system-/sys/kernel/debug" -> "root-file-system" [color = cyan3]; "file-system-/dev/shm" -> "root-file-system" [color = peachpuff4]; "file-system-/sys/firmware/efi/efivars" -> "root-file-system" [color = red]; "file-system-/gnu/store" -> "root-file-system" [color = magenta]; "udev" -> "root-file-system" [color = cyan3]; "virtual-terminal" -> "root-file-system" [color = peachpuff4]; "file-system-/home" [label = "file-system-/home", shape = box, fontname = sans]; "file-systems" -> "file-system-/home" [color = darkviolet]; "file-system-/dev/pts" [label = "file-system-/dev/pts", shape = box, fontname = sans]; "file-systems" -> "file-system-/dev/pts" [color = darkviolet]; "file-system-/sys/kernel/debug" [label = "file-system-/sys/kernel/debug", shape = box, fontname = sans]; "file-systems" -> "file-system-/sys/kernel/debug" [color = darkviolet]; "file-system-/dev/shm" [label = "file-system-/dev/shm", shape = box, fontname = sans]; "file-systems" -> "file-system-/dev/shm" [color = darkviolet]; "file-system-/sys/firmware/efi/efivars" [label = "file-system-/sys/firmware/efi/efivars", shape = box, fontname = sans]; "file-systems" -> "file-system-/sys/firmware/efi/efivars" [color = darkviolet]; "file-system-/gnu/store" [label = "file-system-/gnu/store", shape = box, fontname = sans]; "file-systems" -> "file-system-/gnu/store" [color = darkviolet]; "udev" [label = "udev", shape = box, fontname = sans]; "device-mapping-streetkid_VG_storage-streetkid_LV_home" -> "udev" [color = blue]; "file-system-/home" -> "udev" [color = darkseagreen]; "file-system-/dev/pts" -> "udev" [color = red]; "file-system-/sys/kernel/debug" -> "udev" [color = cyan3]; "file-system-/dev/shm" -> "udev" [color = peachpuff4]; "file-system-/sys/firmware/efi/efivars" -> "udev" [color = red]; "file-system-/gnu/store" -> "udev" [color = magenta]; "urandom-seed" -> "udev" [color = magenta]; "term-tty6" -> "udev" [color = peachpuff4]; "term-tty5" -> "udev" [color = red]; "term-tty4" -> "udev" [color = darkseagreen]; "term-tty3" -> "udev" [color = darkgoldenrod]; "term-tty2" -> "udev" [color = dimgrey]; "term-tty1" -> "udev" [color = magenta]; "term-console" -> "udev" [color = magenta]; "networking" -> "udev" [color = darkviolet]; "device-mapping-streetkid_VG_storage-streetkid_LV_home" [label = "device-mapping-streetkid_VG_storage-streetkid_LV_home", shape = box, fontname = sans]; "file-system-/home" -> "device-mapping-streetkid_VG_storage-streetkid_LV_home" [color = darkseagreen]; "networking" [label = "networking", shape = box, fontname = sans]; "rpc.nfsd" -> "networking" [color = darkseagreen]; "ntpd" -> "networking" [color = red]; "rpcbind-daemon" -> "networking" [color = dimgrey]; "rpc.nfsd" [label = "rpc.nfsd", shape = box, fontname = sans]; "nfs" -> "rpc.nfsd" [color = darkseagreen]; "nfs" [label = "nfs", shape = box, fontname = sans]; "rpcbind-daemon" [label = "rpcbind-daemon", shape = box, fontname = sans]; "rpc.statd" -> "rpcbind-daemon" [color = cyan3]; "nfs" -> "rpcbind-daemon" [color = darkseagreen]; "idmap-daemon" -> "rpcbind-daemon" [color = dimgrey]; "rpc.statd" [label = "rpc.statd", shape = box, fontname = sans]; "rpc.mountd" -> "rpc.statd" [color = peachpuff4]; "rpc.nfsd" -> "rpc.statd" [color = darkseagreen]; "nfs" -> "rpc.statd" [color = darkseagreen]; "rpc.mountd" [label = "rpc.mountd", shape = box, fontname = sans]; "nfs" -> "rpc.mountd" [color = darkseagreen]; "idmap-daemon" [label = "idmap-daemon", shape = box, fontname = sans]; "virtual-terminal" [label = "virtual-terminal", shape = box, fontname = sans]; "term-tty6" -> "virtual-terminal" [color = peachpuff4]; "term-tty5" -> "virtual-terminal" [color = red]; "term-tty4" -> "virtual-terminal" [color = darkseagreen]; "term-tty3" -> "virtual-terminal" [color = darkgoldenrod]; "term-tty2" -> "virtual-terminal" [color = dimgrey]; "term-tty1" -> "virtual-terminal" [color = magenta]; "host-name" [label = "host-name", shape = box, fontname = sans]; "term-tty6" -> "host-name" [color = peachpuff4]; "term-tty5" -> "host-name" [color = red]; "term-tty4" -> "host-name" [color = darkseagreen]; "term-tty3" -> "host-name" [color = darkgoldenrod]; "term-tty2" -> "host-name" [color = dimgrey]; "term-tty1" -> "host-name" [color = magenta]; "term-console" -> "host-name" [color = magenta]; "sysctl" [label = "sysctl", shape = box, fontname = sans]; "loopback" [label = "loopback", shape = box, fontname = sans]; "ssh-daemon ssh sshd" -> "loopback" [color = cyan3]; "/proc/fs/nfsd" [label = "/proc/fs/nfsd", shape = box, fontname = sans]; "rpc.statd" -> "/proc/fs/nfsd" [color = cyan3]; "rpc.mountd" -> "/proc/fs/nfsd" [color = peachpuff4]; "rpc.nfsd" -> "/proc/fs/nfsd" [color = darkseagreen]; "nfs" -> "/proc/fs/nfsd" [color = darkseagreen]; "rpc-pipefs" [label = "rpc-pipefs", shape = box, fontname = sans]; "idmap-daemon" -> "rpc-pipefs" [color = dimgrey]; }