>> /var/run directories are not created properly
>what specifically do you mean by this? just that you see log errors?
Following directories were not created automatically, I had to create
them manually in order to enable services to start:
/var/run/fail2ban
/var/run/screen
/var/run/sshd
/var/run/zabbix
Following directory was created, but its ownership was not proper:
/var/run/redis
> What's the output of:
> $ sudo find /run -ls
Please see attached file: sudo_find_run_ls.txt (working situation) and
find_run_ls_after_reboot.txt (after the system was rebooted i.e. not working
situation).
> $ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/simfs 52428800 7818704 44610096 15% /
devtmpfs 1572864 0 1572864 0% /dev
tmpfs 1572864 0 1572864 0% /dev/shm
tmpfs 1572864 9488 1563376 1% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 1572864 0 1572864 0% /sys/fs/cgroup
none 1572864 0 1572864 0% /run/shm
To make this a bit more complete, I removed the 0000var.conf and restarted the
system. When I checked the md5sum of files /usr/lib/tmpfiles.d, they matched to
my other Ubuntu 16.04 server.
After reboot, syslog relevant parts showed following:
---8<---8<---
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: [/usr/lib/tmpfiles.d/var.conf:14]
Duplicate line for path "/var/log", ignoring.
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of /run/named failed:
Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/fail2ban: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of /run/redis failed:
Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/screen: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/sshd: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/sudo: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/sudo/ts: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of /run/utmp failed:
Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of /run/systemd/netif
failed: Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of
/run/systemd/netif/links failed: Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of
/run/systemd/netif/leases failed: Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: Failed to validate path
/var/run/zabbix: Too many levels of symbolic links
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of /run/log/journal
failed: Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of
/run/log/journal/6d9c7cc322ee4c48af7c0ec3b492b5cc failed: Invalid argument
Apr 11 08:44:50 vspk systemd-tmpfiles[85]: fchownat() of
/run/log/journal/6d9c7cc322ee4c48af7c0ec3b492b5cc/system.journal failed:
Invalid argument
Apr 11 08:44:50 vspk systemd[1]: systemd-tmpfiles-setup.service: Main process
exited, code=exited, status=1/FAILURE
Apr 11 08:44:50 vspk systemd[1]: Failed to start Create Volatile Files and
Directories.
Apr 11 08:44:50 vspk systemd[1]: systemd-tmpfiles-setup.service: Unit entered
failed state.
Apr 11 08:44:50 vspk systemd[1]: systemd-tmpfiles-setup.service: Failed with
result 'exit-code'.
---8<---8<---
This caused, that sshd, redis, fail2ban and zabbix agent were not able
to start.
In order for me to make the system usable, I needed to perform following
steps:
1) Log in to system using the serial console from the VPS control panel.
root@vspk:/var/run# ls -la /var/run
lrwxrwxrwx 1 root root 4 May 14 2018 /var/run -> /run
Contents of /run directory:
root@vspk:/var/run# find /run -ls > /home/kalle/find_run_ls_after_reboot.txt
(attached)
2) Give following commands:
root@vspk:/var/run# mkdir sshd
root@vspk:/var/run# /etc/init.d/ssh start
Starting ssh (via systemctl): ssh.service.
--> Now the system can be again accessed through SSH.
After that:
kalle@vspk:~$ sudo mkdir /var/run/zabbix
kalle@vspk:~$ sudo chown zabbix:zabbix /var/run/zabbix
kalle@vspk:~$ sudo chown redis:redis /var/run/redis
kalle@vspk:~$ sudo mkdir /var/run/fail2ban
kalle@vspk:~$ sudo mkdir /var/run/screen
kalle@vspk:~$ sudo chmod 777 /var/run/screen
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1818814
Title:
systemd-tmpfiles-setup.services fails to create /var/run directories
Status in systemd package in Ubuntu:
New
Bug description:
1) The release of Ubuntu you are using, via 'lsb_release -rd' or System ->
About Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
2) The version of the package you are using, via 'apt-cache policy pkgname'
or by checking in Software Center
systemd:
Installed: 229-4ubuntu21.16
Candidate: 229-4ubuntu21.16
Version table:
*** 229-4ubuntu21.16 500
500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
Packages
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64
Packages
100 /var/lib/dpkg/status
229-4ubuntu4 500
500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
3) What you expected to happen
4) What happened instead
Ubuntu server (running in OpenVZ VPS farm, thus the old kernel
version) has been up and running happily, until I performed apt-get
upgrade and rebooted the server. After reboot, I could not establish
SSH connection to server, port 22 connection was refused.
I opened a HTML console to my server instance and checked logs. From
the logs, it was shown, that SSH server could not start, as it did not
have the /var/run/sshd directory. After scrolling back the
/var/log/syslog, I noticed that there were lots of other /var/run
subdirectories, which were not created. Here is cut&paste from
/var/log/syslog, related to systemd-tmpfiles:
---8<---8<---
Mar 6 12:32:54 vspk systemd-tmpfiles[81]:
[/usr/lib/tmpfiles.d/00rsyslog.conf:6] Duplicate line for path "/v
ar/log", ignoring.
Mar 6 12:32:54 vspk systemd[1]: Starting Raise network interfaces...
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of /run/named failed:
Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/fail2ban: Too many levels of symb
olic links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/screen: Too many levels of symbol
ic links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/sshd: Too many levels of symbolic
links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/sudo: Too many levels of symbolic
links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/sudo/ts: Too many levels of symbo
lic links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of /run/utmp failed:
Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of /run/systemd/netif
failed: Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of
/run/systemd/netif/links failed: Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of
/run/systemd/netif/leases failed: Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: Failed to validate path
/var/run/zabbix: Too many levels of symbol
ic links
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of /run/log/journal
failed: Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of
/run/log/journal/6d9c7cc322ee4c48af7c0ec3b492b5cc fa
iled: Invalid argument
Mar 6 12:32:54 vspk systemd-tmpfiles[81]: fchownat() of
/run/log/journal/6d9c7cc322ee4c48af7c0ec3b492b5cc/sy
stem.journal failed: Invalid argument
Mar 6 12:32:54 vspk systemd[1]: systemd-tmpfiles-setup.service: Main process
exited, code=exited, status=1/F
AILURE
Mar 6 12:32:54 vspk systemd[1]: Failed to start Create Volatile Files and
Directories.
Mar 6 12:32:54 vspk systemd[1]: systemd-tmpfiles-setup.service: Unit entered
failed state.
Mar 6 12:32:54 vspk systemd[1]: systemd-tmpfiles-setup.service: Failed with
result 'exit-code'.
---8<---8<---
My first idea was, that for some reason, systemd-tmpfiles was not able
to create the /var directory properly, so I renamed
/usr/lib/tmpfiles.d/var.conf to 0000var.conf, but it was no help. The
only difference was the first line on the above log, complaining about
duplicate line for path /var/log.
As I created this error using the "ubuntu-bug" command, I assume
necessary background information is automatically attached. I'll check
those and add more, if necessary.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: systemd 229-4ubuntu21.16
Uname: Linux 2.6.32-042stab128.2 x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
CurrentDmesg:
Date: Wed Mar 6 12:43:51 2019
Lsusb: Error: command ['lsusb'] failed with exit code 1:
ProcEnviron:
TERM=vt220
PATH=(custom, no user)
ProcInterrupts: Error: [Errno 2] No such file or directory: '/proc/interrupts'
ProcKernelCmdLine: quiet
ProcModules:
SourcePackage: systemd
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1818814/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
