Bug#1002591: misdetects socket activated ssh
On Sun, Jan 02, 2022 at 09:25:49PM +0100, Thomas Liske wrote: > Looks like a systemd/cgroup related change in bullseye, buster seems > not to be affected. That sounds correct, I had attributed this behavior to some ssh updates in the past. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1002591: misdetects socket activated ssh
Hi Marc, On Sat, 2022-01-01 at 20:55 +0100, Marc Haber wrote: > Sure: > 1 [1/4996]mh@torres:~ $ pgrep ssh > 315675 > 315738 > [2/4997]mh@torres:~ $ sudo cat /proc/315675/cgroup > [sudo] password for mh on torres: > 0::/user.slice/user-1001.slice/session-296.scope > [3/4998]mh@torres:~ $ sudo cat /proc/315738/cgroup > 0::/user.slice/user-1001.slice/session-296.scope > [4/4999]mh@torres:~ $ > thanks! Needrestart should ignore those ssh instances since there is a user slice cgroup. It does not work due to this check[1] in needrestart. [1] https://github.com/liske/needrestart/blob/v3.5/needrestart#L637 Looks like a systemd/cgroup related change in bullseye, buster seems not to be affected. Regards, Thomas > > As a workaround you might blacklist sshd in needrestart but I think > > a > > generic approach handling socket activation services in needrestart > > would be better. Therefore needrestart need a way to detect if the > > process belongs to a socket activated service. > > It is also possible to mask ssh.service entirely in systemd. But of > couse having the heuristic fixed would be better. > > Greetings > Marc >
Bug#1002591: misdetects socket activated ssh
Hi Thomas, On Sat, Jan 01, 2022 at 02:31:04PM +0100, Thomas Liske wrote: > could you please provide the content of /proc/$PID/cgroup for an socket > activated sshd instance? Sure: 1 [1/4996]mh@torres:~ $ pgrep ssh 315675 315738 [2/4997]mh@torres:~ $ sudo cat /proc/315675/cgroup [sudo] password for mh on torres: 0::/user.slice/user-1001.slice/session-296.scope [3/4998]mh@torres:~ $ sudo cat /proc/315738/cgroup 0::/user.slice/user-1001.slice/session-296.scope [4/4999]mh@torres:~ $ > As a workaround you might blacklist sshd in needrestart but I think a > generic approach handling socket activation services in needrestart > would be better. Therefore needrestart need a way to detect if the > process belongs to a socket activated service. It is also possible to mask ssh.service entirely in systemd. But of couse having the heuristic fixed would be better. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Bug#1002591: misdetects socket activated ssh
Hi Marc, could you please provide the content of /proc/$PID/cgroup for an socket activated sshd instance? As a workaround you might blacklist sshd in needrestart but I think a generic approach handling socket activation services in needrestart would be better. Therefore needrestart need a way to detect if the process belongs to a socket activated service. TIA & HTH, Thomas On Fri, 2021-12-24 at 22:25 +0100, Marc Haber wrote: > Package: needrestart > Version: 3.5-5 > Severity: normal > > Hi, > > when using ssh as a socket activated service (systemctl stop/disable > ssh.service, systemctl enable/start ssh.socket), after a library > update > needrestart will offer to restart ssh.service. This fails since port > 22 > is occupied by the instance services and causes the machine to be > without listening process after logging out. > > A possible workaround is masking ssh.service, see #1001320. > > Restarting services... > systemctl restart console-log.service cron.service exim4.service > haveged.service ippl.service ntp.service rsyslog.service > serial-getty@ttyS0.service ssh.service systemd-journald.service > systemd-networkd.service systemd-resolved.service systemd- > udevd.service > Job for ssh.service failed because the control process exited with > error code. > See "systemctl status ssh.service" and "journalctl -xeu ssh.service" > for details. > Service restarts being deferred: > /etc/needrestart/restart.d/dbus.service > systemctl restart getty@tty1.service > systemctl restart systemd-logind.service > systemctl restart user@1001.service > > and the following log entries: > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: Puts a > logfile pager on virtual consoles... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Regular > background program processing daemon... > Dec 8 12:58:26 emptybookworm82 systemd[1]: cron.service: Deactivated > successfully. > Dec 8 12:58:26 emptybookworm82 cron[429258]: (CRON) INFO (pidfile fd > = 3) > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Regular > background program processing daemon. > Dec 8 12:58:26 emptybookworm82 systemd[1]: cron.service: Consumed > 15min 4.856s CPU time. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Started Regular > background program processing daemon. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: exim Mail > Transport Agent... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Entropy Daemon > based on the HAVEGE algorithm... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: IP > protocols logger... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Network Time > Service... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping System Logging > Service... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Serial Getty on > ttyS0... > Dec 8 12:58:26 emptybookworm82 systemd[1]: > serial-getty@ttyS0.service: Deactivated successfully. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Serial Getty on > ttyS0. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Started Serial Getty on > ttyS0. > Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.socket: Deactivated > successfully. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Closed OpenBSD Secure > Shell server socket. > Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.socket: Consumed > 10.571s CPU time. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Starting OpenBSD Secure > Shell server... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Flush Journal to > Persistent Storage... > Dec 8 12:58:26 emptybookworm82 systemd[1]: systemd-networkd-wait- > online.service: Deactivated successfully. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Wait for Network > to be Configured. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Wait for Network > to be Configured... > Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Network Name > Resolution... > Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.service: Main process > exited, code=exited, status=255/EXCEPTION > Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.service: Failed with > result 'exit-code'. > Dec 8 12:58:26 emptybookworm82 systemd[1]: Failed to start OpenBSD > Secure Shell server. > Dec 8 12:58:26 emptybookworm82 ntpd[298]: ntpd exiting on signal 15 > (Terminated) > Dec 8 12:58:26 emptybookworm82 ntpd[298]: 2a01:4f8:140:246a::2 local > addr 2a01:4f8:140:246a::52:100 -> > Dec 8 12:58:26 emptybookworm82 haveged[220]: haveged: Stopping due > to signal 15 > Dec 8 12:58:27 emptybookworm82 cron[429258]: (CRON) INFO (Skipping > @reboot jobs -- not system startup) > Dec 8 12:58:27 emptybookworm82 systemd[1]: systemd-journal- > flush.service: Deactivated successfully. > Dec 8 12:58:27 emptybookworm82 systemd[1]: Stopped Flush Journal to > Persistent Storage. > Dec 8 12:58:27 emptybookworm82 exim4[429259]: exim4_listener. > > Here is what Timo Weingärtner found out in relation to my bug report > against sshd: > > > To me it looks like a problem in
Bug#1002591: misdetects socket activated ssh
Package: needrestart Version: 3.5-5 Severity: normal Hi, when using ssh as a socket activated service (systemctl stop/disable ssh.service, systemctl enable/start ssh.socket), after a library update needrestart will offer to restart ssh.service. This fails since port 22 is occupied by the instance services and causes the machine to be without listening process after logging out. A possible workaround is masking ssh.service, see #1001320. Restarting services... systemctl restart console-log.service cron.service exim4.service haveged.service ippl.service ntp.service rsyslog.service serial-getty@ttyS0.service ssh.service systemd-journald.service systemd-networkd.service systemd-resolved.service systemd-udevd.service Job for ssh.service failed because the control process exited with error code. See "systemctl status ssh.service" and "journalctl -xeu ssh.service" for details. Service restarts being deferred: /etc/needrestart/restart.d/dbus.service systemctl restart getty@tty1.service systemctl restart systemd-logind.service systemctl restart user@1001.service and the following log entries: Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: Puts a logfile pager on virtual consoles... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Regular background program processing daemon... Dec 8 12:58:26 emptybookworm82 systemd[1]: cron.service: Deactivated successfully. Dec 8 12:58:26 emptybookworm82 cron[429258]: (CRON) INFO (pidfile fd = 3) Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Regular background program processing daemon. Dec 8 12:58:26 emptybookworm82 systemd[1]: cron.service: Consumed 15min 4.856s CPU time. Dec 8 12:58:26 emptybookworm82 systemd[1]: Started Regular background program processing daemon. Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: exim Mail Transport Agent... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Entropy Daemon based on the HAVEGE algorithm... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping LSB: IP protocols logger... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Network Time Service... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping System Logging Service... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Serial Getty on ttyS0... Dec 8 12:58:26 emptybookworm82 systemd[1]: serial-getty@ttyS0.service: Deactivated successfully. Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Serial Getty on ttyS0. Dec 8 12:58:26 emptybookworm82 systemd[1]: Started Serial Getty on ttyS0. Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.socket: Deactivated successfully. Dec 8 12:58:26 emptybookworm82 systemd[1]: Closed OpenBSD Secure Shell server socket. Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.socket: Consumed 10.571s CPU time. Dec 8 12:58:26 emptybookworm82 systemd[1]: Starting OpenBSD Secure Shell server... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Flush Journal to Persistent Storage... Dec 8 12:58:26 emptybookworm82 systemd[1]: systemd-networkd-wait-online.service: Deactivated successfully. Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopped Wait for Network to be Configured. Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Wait for Network to be Configured... Dec 8 12:58:26 emptybookworm82 systemd[1]: Stopping Network Name Resolution... Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.service: Main process exited, code=exited, status=255/EXCEPTION Dec 8 12:58:26 emptybookworm82 systemd[1]: ssh.service: Failed with result 'exit-code'. Dec 8 12:58:26 emptybookworm82 systemd[1]: Failed to start OpenBSD Secure Shell server. Dec 8 12:58:26 emptybookworm82 ntpd[298]: ntpd exiting on signal 15 (Terminated) Dec 8 12:58:26 emptybookworm82 ntpd[298]: 2a01:4f8:140:246a::2 local addr 2a01:4f8:140:246a::52:100 -> Dec 8 12:58:26 emptybookworm82 haveged[220]: haveged: Stopping due to signal 15 Dec 8 12:58:27 emptybookworm82 cron[429258]: (CRON) INFO (Skipping @reboot jobs -- not system startup) Dec 8 12:58:27 emptybookworm82 systemd[1]: systemd-journal-flush.service: Deactivated successfully. Dec 8 12:58:27 emptybookworm82 systemd[1]: Stopped Flush Journal to Persistent Storage. Dec 8 12:58:27 emptybookworm82 exim4[429259]: exim4_listener. Here is what Timo Weingärtner found out in relation to my bug report against sshd: | To me it looks like a problem in needrestart. The (forked off) sshd process | handling your client connection belongs to cgroup session-NN.scope, no matter | if it was started by systemd socket activation or regular sshd. | | needrestart (invoked with "-vlp" here) detects a process with outdated libs: | | [main] #2111961 uses deleted /lib/x86_64-linux-gnu/libnss_files-2.32.so | [main] #2111961 is a child of #2111904 | | Then it figures out the binary and the cgroup: | | [main] #2111961 exe => /usr/sbin/sshd | [main] trying systemctl status | | cgroup detection didn't work, so: | | [main] #2111961 running /etc/needrestart/hook.d/10-dpkg | [main]
