This problem is not new and it appears again with the new systemd
"229-4ubuntu21.15" released.

See also these bug reports which somehow relate to the issue:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804847
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804603

As I'm no Linux pro I can't tell how these issues relate to each other
but they definitely do. I don't quite understand why this issue was
strictly focused on OpenVZ and then neglected because it definitely is
not only OpenVZ related.

I didn't face the issue in systemd versions <systemd-229-4ubuntu21.9
with systemd-229-4ubuntu21.10 the problem was fixed again via a
regression and now it's back again in 229-4ubuntu21.15. I'm on:

Linux xxx 3.4.113-sun8i #2 SMP PREEMPT Sat Jan 12 15:54:26 CET 2019 armv7l 
armv7l armv7l GNU/Linux
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.5 LTS
Release:        16.04
Codename:       xenial

Armbian and I suspect this has nothing to do with OpenVZ. See more here
in the following link concerning Armbian based distributions with the
same issue https://forum.armbian.com/topic/8852-ssh-doesnt-work-on-
orange-pi-zero/

Moreover it doesn't only affect SSH but rather many other services
because from what I understand some directories aren't created on
startup, because of some system-tmpfiles changes. Here is the output of
journalctl -b 0 -u systemd-tmpfiles-setup.service

Jän 14 11:01:51 xxx systemd[1]: Starting Create Volatile Files and 
Directories...
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: [/usr/lib/tmpfiles.d/var.conf:14] 
Duplicate line for path "/var/log", ignoring.
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var: Bad 
file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/log: 
Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/lib: 
Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path 
/run/sendsigs.omit.d: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /home: Bad 
file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /srv: Bad 
file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path 
/run/lock/subsys: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path 
/var/run/lighttpd: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/cache: 
Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path 
/var/cache/man: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path 
/run/openvpn: Bad file descriptor
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Main process 
exited, code=exited, status=1/FAILURE
Jän 14 11:01:51 xxx systemd[1]: Failed to start Create Volatile Files and 
Directories.
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Unit entered 
failed state.
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Failed with 
result 'exit-code'.

In my case the following service won't start because of the systemd
update. I'm no facing the exact same issue / behaviour like in
systemd-229-4ubuntu21.9. Same service are affected, same errors.


# dnsmasq.service                    loaded failed failed dnsmasq - A 
lightweight DHCP and caching DNS server
# lighttpd.service                   loaded failed failed Lighttpd Daemon
# openvpn@server.service             loaded failed failed OpenVPN connection to 
server
# ssh.service                        loaded failed failed OpenBSD Secure Shell 
server
# systemd-tmpfiles-setup-dev.service loaded failed failed Create Static Device 
Nodes in /dev
# systemd-tmpfiles-setup.service     loaded failed failed Create Volatile Files 
and Directories


I can't tell why this happens and this is everything on information I can 
provide. I would also provide you more info if you tell me what you need. I can 
rollback to a version <21.9 or >21.9 and <21.15 and everything will work again. 
But TBH I don't want to mark systemd on hold for too long. Maybe some sort of 
temporary fix other then rolling back would be fantastic...

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1811580

Title:
  systemd fails to start sshd at reboot

Status in systemd package in Ubuntu:
  New

Bug description:
  Ubuntu 16.04.5, systemd 229-4ubuntu21.15

  The latest systemd update has somehow changed the method it uses to
  start 'ssh.service' i.e. 'sshd'. systemd fails to start sshd if
  /etc/ssh/sshd_config contains "UsePrivilegeSeparation yes" and
  /var/run/sshd/ does not already exist. Being as this is the default,
  virtually EVERY Ubuntu 16.04 server in the world has
  UsePrivilegeSeparation set to yes. Furthermore, at the time when the
  user performs 'apt upgrade' and receives the newest version of
  systemd, /var/run/sshd/ already exists, so sshd successfully reloads
  for as long as the server doesn't get rebooted. BUT, as soon as the
  server is rebooted for any reason, /var/run/sshd/ gets cleaned away,
  and sshd fails to start, causing the remote user to be completely
  locked out of his system. This is a MAJOR issue for millions of VPS
  servers worldwide, as they are all about to get locked out of their
  servers and potentially lose data. The next reboot is a ticking time
  bomb waiting to spring. The bomb can be defused by implicitly setting
  'UsePrivilegeSeparation no' in /etc/ssh/sshd_config, however
  unsuspecting administrators are bound to be caught out by the
  millions. I got caught by it in the middle of setting up a new server
  yesterday, and it took a whole day to find the source.

  The appropriate fix would be to ensure that systemd can successfully
  'start ssh.service' even when 'UsePrivilegeSeparation yes' is set.
  systemd needs to test that /var/run/sshd/ exists before starting sshd,
  just as the init.d script for sshd does. openssl could also be patched
  so that UsePrivilegeSeparation is no longer enabled by default,
  however that is not going to solve the problem for millions of pre-
  existing config files. Only an update to openssl to force-override
  that flag to 'no' would solve the problem. Thus systemd still needs to
  be responsible for ensuring that it inits sshd properly by ensuring
  that /var/run/sshd/ exists before it sends the 'start' command.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1811580/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to     : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to