Re: [systemd-devel] systemd behavior during shutdown
Hi, Any insights on below error which I am seeing. Is it something expected with "DefaultDependencies=False"? Thanks, Hari. -Original Message- From: Tiwari, Hari Sahaya Sent: Thursday, September 20, 2018 12:15 AM To: 'Zbigniew Jędrzejewski-Szmek' Cc: systemd-devel@lists.freedesktop.org Subject: RE: [systemd-devel] systemd behavior during shutdown HI, Many thanks for the reply. I tried putting DefaultDependencies=false in both .socket & .service files. I was able to verify that socket was still in "listening" state when my other systemd service tried to start a new connection with socket. Also the "Suppressing connection request since unit stop is scheduled" message is no more seen. Now I am getting below error when the new connection is requested. Sep 19 23:31:33 jara1 systemd[1]: hacl-cfg.socket: Incoming traffic Sep 19 23:31:33 jara1 systemd[1]: hacl-cfg@6-127.0.0.1:5302-127.0.0.1:63714.service: Trying to enqueue job hacl-cfg@6-127.0.0.1:5302-127.0.0.1:63714.service/start/replace Sep 19 23:31:33 jara1 systemd[1]: Requested transaction contradicts existing jobs: Transaction is destructive. Sep 19 23:31:33 jara1 systemd[1]: hacl-cfg.socket: One connection closed, 1 left. Sep 19 23:31:33 jara1 systemd[1]: hacl-cfg.socket: Failed to queue service startup job (Maybe the service file is missing or not a non-template unit?): Transaction is destructive. Sep 19 23:31:33 jara1 systemd[1]: hacl-cfg.socket: Changed listening -> failed Do I have to set some other parameter in the systemd unit files ? Following are the contents of systemd files, Service File --- # cat hacl-cfg@.service [Unit] Description=config TCP service Requires=hacl-cfg.socket DefaultDependencies=false [Service] ExecStart=/opt/cmcluster/bin/cmclconfd -c StandardInput=socket KillMode=process [Install] WantedBy=multi-user.target Socket File - # cat hacl-cfg.socket [Unit] Description= config TCP socket DefaultDependencies=false [Socket] ListenStream=5302 Accept=true MaxConnections=900 [Install] WantedBy=sockets.target Service file which tries to open a new connection with hacl-cfg.socket during shutdown # cat test.service [Unit] Description=test Service [Service] ExecStart=/bin/test start ExecStop=/bin/test stop Type=oneshot RemainAfterExit=yes [Install] WantedBy=multi-user.target Thanks, Hari. -Original Message- From: Zbigniew Jędrzejewski-Szmek [mailto:zbys...@in.waw.pl] Sent: Wednesday, September 19, 2018 12:11 PM To: Tiwari, Hari Sahaya Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] systemd behavior during shutdown On Wed, Sep 19, 2018 at 05:01:33AM +, Tiwari, Hari Sahaya wrote: > Hi, > > I am facing one issue with systemd where systems socket is not opening a new > connection during shutdown. > I get below logs, > > Sep 12 20:01:32 jara2 systemd[1]: mytestX.socket: Incoming traffic Sep > 12 20:01:32 jara2 systemd[1]: mytestX.socket: Suppressing connection request > since unit stop is scheduled. > > I have one systemd service which is trying to open a new connection during > shutdown sequence but looks like systemd sockets stop accepting new > connections as soon as they are marked for stop. > I tried putting various combination of dependencies but that didn't help. > Everytime getting the above message. > > Is there any parameter which can be set in unit files to resolve this issue? > Any pointers will be appreciated. Hi, yes, this is intentional. It was added to avoid the situation where services are stop and subsequently restarted during shutdown because something opens a connection, leading to loops. If you absolutely need to open a connection to a socket activated unit, then you could try making the .socket and .service units have DefaultDependencies=false, so that they will not conflict with shutdown.target and the start jobs will not be created for them. But then you need to make sure that they actually *are* stopped at the right time, but issuing a 'systemctl stop' request for them. This can be done, but will be messy, so I'd use a different approach if possible. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] exim4 only queues mails sent by systemd service
It is something strange with sending mails from systemd system service: assume we have service file /etc/systemd/system/mailtest.service: --8<---cut here---start->8--- [Unit] Description="Test maili" [Service] #User=kjonca NoNewPrivileges=false Type=oneshot ExecStart=-zsh -c 'echo xxx|mail news' ExecStart=-zsh -c 'echo xxx|mutt -F /dev/null -s subject -e \'set copy=no\' kjonca' --8<---cut here---end--->8--- When I call sudo systemctl start mailtest.service Two messages are put in exim queue, but not deliveried immediately. Why? What am I missing? Morevoer this service run as "--user" service works as expected - mails are delivered at once. Does systemd interfere with processes (suid/sgid? file access limitations?) KJ -- http://wolnelektury.pl/wesprzyj/teraz/ Hold the MAYO & pass the COSMIC AWARENESS ... ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] bpfilter blocks root unmount during shutdown
Dracut /shutdown script first tries to kill all processes still running
off old root. Unfortunately this fails for special user process that
runs bpfilter because it does not include reference to /oldroot in
places where dracut looks for in kilall_proc_mountpoint()
10:~ # ps -ef | fgrep '[none]'
root 984 2 0 09:46 ?00:00:00 [none]
/proc/984:
total 0
dr-xr-xr-x 2 root 0 0 Sep 23 10:11 attr
-r 1 root 0 0 Sep 23 10:11 auxv
-r--r--r-- 1 root 0 0 Sep 23 10:11 cgroup
--w--- 1 root 0 0 Sep 23 10:11 clear_refs
-r--r--r-- 1 root 0 0 Sep 23 10:10 cmdline
-rw-r--r-- 1 root 0 0 Sep 23 10:11 comm
-rw-r--r-- 1 root 0 0 Sep 23 10:11 coredump_filter
-r--r--r-- 1 root 0 0 Sep 23 10:11 cpuset
lrwxrwxrwx 1 root 0 0 Sep 23 10:11 cwd -> /
-r 1 root 0 0 Sep 23 10:11 environ
lrwxrwxrwx 1 root 0 0 Sep 23 10:11 exe -> / (deleted)
-rw-r--r-- 1 root 0 0 Sep 23 10:11 fail-nth
dr-x-- 2 root 0 0 Sep 23 10:11 fd
dr-x-- 2 root 0 0 Sep 23 10:11 fdinfo
-rw-r--r-- 1 root 0 0 Sep 23 10:11 gid_map
-r 1 root 0 0 Sep 23 10:11 io
-r--r--r-- 1 root 0 0 Sep 23 10:11 latency
-r--r--r-- 1 root 0 0 Sep 23 10:11 limits
-rw-r--r-- 1 root 0 0 Sep 23 10:11 loginuid
-rw-r--r-- 1 root 0 0 Sep 23 10:11 make-it-fail
dr-x-- 2 root 0 0 Sep 23 10:11 map_files
-r--r--r-- 1 root 0 0 Sep 23 10:10 maps
-rw--- 1 root 0 0 Sep 23 10:11 mem
-r--r--r-- 1 root 0 0 Sep 23 10:11 mountinfo
-r--r--r-- 1 root 0 0 Sep 23 10:11 mounts
-r 1 root 0 0 Sep 23 10:11 mountstats
dr-xr-xr-x 6 root 0 0 Sep 23 10:11 net
dr-x--x--x 2 root 0 0 Sep 23 10:11 ns
-r--r--r-- 1 root 0 0 Sep 23 10:11 numa_maps
-rw-r--r-- 1 root 0 0 Sep 23 10:11 oom_adj
-r--r--r-- 1 root 0 0 Sep 23 10:11 oom_score
-rw-r--r-- 1 root 0 0 Sep 23 10:11 oom_score_adj
-r 1 root 0 0 Sep 23 10:11 pagemap
-r 1 root 0 0 Sep 23 10:11 patch_state
-r 1 root 0 0 Sep 23 10:11 personality
-rw-r--r-- 1 root 0 0 Sep 23 10:11 projid_map
lrwxrwxrwx 1 root 0 0 Sep 23 10:11 root -> /
-rw-r--r-- 1 root 0 0 Sep 23 10:11 sched
-r--r--r-- 1 root 0 0 Sep 23 10:11 schedstat
-r--r--r-- 1 root 0 0 Sep 23 10:11 sessionid
-rw-r--r-- 1 root 0 0 Sep 23 10:11 setgroups
-r--r--r-- 1 root 0 0 Sep 23 10:11 smaps
-r--r--r-- 1 root 0 0 Sep 23 10:11 smaps_rollup
-r 1 root 0 0 Sep 23 10:11 stack
-r--r--r-- 1 root 0 0 Sep 23 10:10 stat
-r--r--r-- 1 root 0 0 Sep 23 10:11 statm
-r--r--r-- 1 root 0 0 Sep 23 10:10 status
-r 1 root 0 0 Sep 23 10:11 syscall
dr-xr-xr-x 3 root 0 0 Sep 23 10:11 task
-r--r--r-- 1 root 0 0 Sep 23 10:11 timers
-rw-rw-rw- 1 root 0 0 Sep 23 10:11 timerslack_ns
-rw-r--r-- 1 root 0 0 Sep 23 10:11 uid_map
-r--r--r-- 1 root 0 0 Sep 23 10:11 wchan
/proc/984/fd:
total 0
lr-x-- 1 root 0 64 Sep 23 10:11 0 -> pipe:[19409]
l-wx-- 1 root 0 64 Sep 23 10:11 1 -> pipe:[19410]
lrwx-- 1 root 0 64 Sep 23 10:11 2 -> /oldsys/dev/console
But it does contain reference to /oldroot in its mapped libraries list
(/proc/984/maps):
563b63002000-563b63003000 r--p 00:05 19404
/ (deleted)
563b63003000-563b63004000 r-xp 1000 00:05 19404
/ (deleted)
563b63004000-563b63005000 r--p 2000 00:05 19404
/ (deleted)
563b63005000-563b63006000 r--p 2000 00:05 19404
/ (deleted)
563b63006000-563b63007000 rw-p 3000 00:05 19404
/ (deleted)
563b63fb4000-563b63fd5000 rw-p 00:00 0
[heap]
7fa3a46cc000-7fa3a4882000 r-xp 00:2a 7728
/oldroot/lib64/libc-2.27.so
7fa3a4882000-7fa3a4a82000 ---p 001b6000 00:2a 7728
/oldroot/lib64/libc-2.27.so
7fa3a4a82000-7fa3a4a86000 r--p 001b6000 00:2a 7728
/oldroot/lib64/libc-2.27.so
7fa3a4a86000-7fa3a4a88000 rw-p 001ba000 00:2a 7728
/oldroot/lib64/libc-2.27.so
7fa3a4a88000-7fa3a4a8c000 rw-p 00:00 0
7fa3a4a8c000-7fa3a4ab1000 r-xp 00:2a 7720
/oldroot/lib64/ld-2.27.so
7fa3a4ca7000-7fa3a4ca9000 rw-p 00:00 0
7fa3a4cb1000-7fa3a4cb2000 r--p 00025000 00:2a 7720
/oldroot/lib64/ld-2.27.so
7fa3a4cb2000-7fa3a4cb3000 rw-p 00026000 00:2a 7720
/oldroot/lib64/ld-2.27.so
7fa3a4cb3000-7fa3a4cb4000 rw-p 00:00 0
7ffea03b4000-7ffea03d5000 rw-p 00:00 0
[stack]
7ffea03df000-7ffea03e2000 r--p 00:00 0
[vvar]
7ffea03e2000-7ffea03e4000 r-xp 00:00 0
[vdso]
ff60-ff601000 r-xp 00:00 0
[vsyscall]
So the quick fix would be to extend check for root references to also
look into /proc/$PID/maps. Something like (verified):
--- dracut-lib.sh.orig 2018-09-18 13:24:49.0 +0300
+++ dracut-lib.sh 2018-09-23 10:31:13.300054544 +0300
@@ -118,7 +118,7 @@ killall_proc_mountpoint() {
esac
[ -e "/proc/$_pid/exe" ] || continue
[ -e "/proc/$_pid/root" ] || continue
-strstr "$(ls -l -- "/proc/$_pid" "/proc/$_pid/fd" 2>/dev/null)"
"$1" && kill -9 "$_pid"
+strstr "$(ls -l -- "/proc/$_pid" "/proc/$_pid/fd" 2>/dev/null;
cat "/proc/$_pid/maps" 2> /dev/null)" "$1" && kill -9 "$_pid"
done
}
Note that there are also other places that use similar check (most
obvious being /shutdown script
