Dear Debian community, I have noticed that all failed services were missing some directories under /run directory. I checked the service which is supposed to create them:
* systemd-tmpfiles-setup.service - Create Volatile Files and Directories Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: enabled) Active: inactive (dead) Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) systemd[1]: sysinit.target: Found ordering cycle on systemd-tmpfiles-setup.service/start systemd[1]: sysinit.target: Found dependency on local-fs.target/start systemd[1]: sysinit.target: Found dependency on zram-setup@zram1.service/start systemd[1]: sysinit.target: Found dependency on sysinit.target/start systemd[1]: sysinit.target: Job systemd-tmpfiles-setup.service/start deleted to break ordering cycle starting with sysinit.target/start and it looks like there is problem in zram-setup@zram1.service which I configured like that: [Unit] Requires=dev-%i.device After=dev-%i.device Before=local-fs.target [Install] WantedBy=local-fs.target # systemctl show -p Requires,Wants,Requisite,BindsTo,PartOf,Before,After local-fs.target Requires=home.mount -.mount var.mount Requisite= Wants=systemd-fsck-root.service zram-setup@zram0.service zram-setup@zram1.service systemd-remount-fs.service BindsTo= PartOf= Before=binfmt-support.service sysinit.target systemd-machine-id-commit.service systemd-tmpfiles-setup.service networking.service systemd-tmpfiles-clean.service console-setup.service After=run-user-1000.mount zram-setup@zram0.service root.mount -.mount tmp.mount zram-setup@zram1.service systemd-fsck-root.service systemd-remount-fs.service home.mount var.mount local-fs-pre.target Even though I don't see any conflict with dependencies, it is confusing systemd. Any idea concerning how to fix that is welcomed. On 2020-06-29 01:37, Dmitry Katsubo wrote: > Dear Debian community, > > I hit the similar problem but this time with /run folder. Few services have > failed to start: > > # systemctl status php7.0-fpm.service > Jun 24 23:09:48 debian php-fpm7.0[893]: [24-Jun-2020 23:09:48] ERROR: unable > to bind listening socket for address '/run/php/php7.0-fpm.sock': No such file > or directory (2) > Jun 24 23:09:48 debian php-fpm7.0[893]: [24-Jun-2020 23:09:48] ERROR: FPM > initialization failed > Jun 24 23:09:48 debian systemd[1]: php7.0-fpm.service: Main process exited, > code=exited, status=78/CONFIG > Jun 24 23:09:48 debian systemd[1]: php7.0-fpm.service: Failed with result > 'exit-code'. > Jun 24 23:09:48 debian systemd[1]: Failed to start The PHP 7.0 FastCGI > Process Manager. > > # systemctl status motioneye.service > Jun 24 23:09:47 debian systemd[1]: Started motionEye Server. > Jun 24 23:09:48 debian meyectl[895]: INFO: hello! this is motionEye > server 0.41 > Jun 24 23:09:48 debian meyectl[895]: CRITICAL: pid directory "/run/motioneye" > does not exist or is not writable > Jun 24 23:09:48 debian systemd[1]: motioneye.service: Main process exited, > code=exited, status=255/EXCEPTION > Jun 24 23:09:48 debian systemd[1]: motioneye.service: Failed with result > 'exit-code'. > > # cat /etc/tmpfiles.d/php.conf > d /run/php/sessions 1733 root root > > # cat /etc/tmpfiles.d/motioneye.conf > d /run/motioneye 0750 motion motion > > Just after the boot I have inspected /run folder. It was created/mounted > correctly and there have been a lot of files/directories there. I suspect that > all services that have created the necessary directory under /run were able to > start normally. Few of them which relied on existence of specific directory, > have failed to started. After I have replayed the corresponding instructions > for > tmpfiles.d, the services have started normally. > > I have a feeling that systemd-tmpfiles was executed before /run was mounted. > > Needless to note that the problem is not persistent: sometimes OS boots > without > a single failed service. > > How can I debug the problem? > > Thank you! > > On 2020-05-18 02:39, Dmitry Katsubo wrote: >> On 2020-05-11 20:11, Darac Marjal wrote: >>> On 11/05/2020 08:40, Reco wrote: >>>> Hi. >>>> >>>> On Mon, May 11, 2020 at 09:33:59AM +0200, Dmitry Katsubo wrote: >>>> >>>>> root@debian:~ # systemctl status binfmt-support >>>>> * binfmt-support.service - Enable support for additional executable >>>>> binary formats >>>>> Loaded: loaded (/lib/systemd/system/binfmt-support.service; enabled; >>>>> vendor preset: enabled) >>>>> Active: failed (Result: exit-code) since Sun 2020-05-10 21:54:27 CEST; >>>>> 10h ago >>>>> Docs: man:update-binfmts(8) >>>>> Process: 353 ExecStart=/usr/sbin/update-binfmts --enable (code=exited, >>>>> status=2) >>>>> Main PID: 353 (code=exited, status=2) >>>>> >>>>> May 10 21:54:27 debian update-binfmts[353]: update-binfmts: unable to >>>>> open /var/lib/binfmts: No such file or directory >>>>> >>>>> Any help is appreciated. >>>> This should help your problem: >>>> >>>> mkdir /etc/systemd/system/binfmt-support.service.d >>>> >>>> cat > /etc/systemd/system/binfmt-support.service.d/override.conf << EOF >>>> [Unit] >>>> RequiresMountsFor=/var >>>> EOF >>> >>> As another alternative, one can run "systemctl edit >>> binfmt-support.service", which will create the intervening folders and >>> files for you, and reload the daemon if the editor exits with success. >> >> Thanks for suggestion! I have tried the advise and it actually worked >> (I will keep an eye on that because one reboot may not be representative). >> I wonder nevertheless what is the problem with this specific unit? It has >> dependency on local-fs.target which in turn should mount /var. So what >> exactly went wrong? -- With best regards, Dmitry