Re: [systemd-devel] Concurrent login / daemon-reload produces abandoned sessions
On Fri, Jul 09, 2021 at 11:42:22AM +0200, Nicolas Bock wrote: > I have already opened an issue [1] but it was closed. Maybe > we can just re-open it? FWIW, https://github.com/systemd/systemd/pull/20199 Michal signature.asc Description: Digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Concurrent login / daemon-reload produces abandoned sessions
On Thu, Jul 08 2021, Michal Koutný wrote:
> Hello Nicolas.
>
> On Wed, Jul 07, 2021 at 02:12:51PM +0200, Nicolas Bock
> wrote:
>> Using systemd-248.3-1ubuntu1 on Ubuntu Impish the following
>> script produces multiple abandoned sessions:
>>
>> $ for i in {1..100}; do sleep 0.2; ssh localhost sudo systemctl
>> daemon-reload & ssh localhost sleep 1 & done
>> $ sleep 2
>> $ jobs -p | xargs --verbose --no-run-if-empty kill -KILL
>> $ systemctl | grep abandoned
>>session-174.scopeloaded active abandoned
>> Session 174 of user ubuntu
>>session-175.scopeloaded active abandoned
>> Session 175 of user ubuntu
>>session-176.scopeloaded active abandoned
>> Session 176 of user ubuntu
>>session-25.scope loaded active abandoned
>> Session 25 of user ubuntu
>>
>> I would like to debug this behavior further and understand
>> why this is happening but don't know where to look next.
>
> It might be a bit challenging :)
>
>> Is there any information in particular I should look at?
>
> I assume you use hybrid or unified cgroup setup and that the abandoned
> scopes are empty (no processes in their cgroups), correct?
Yes, you are correct. Ubuntu builds their systemd with
`-Ddefault-hierarchy=hybrid`. And yes, the abandoned scopes
have no processes.
> My hypothesis is following
>
> // race between scope abandonement, emptiness notification -> abandon comes
> first
> manager_reload
> manager_clear_jobs_and_units
> unit_release_cgroup
> inotify_rm_watch(u->manager->cgroup_inotify_fd,
> u->cgroup_control_inotify_wd)
> [...]
> // last process terminates somewhere here but we're not watching emptiness yet
> scope_coldplug()
> // scope should be checked for emptiness here
>
> I _think_ this could be fixed with the patch
> --- a/src/core/scope.c
> +++ b/src/core/scope.c
> @@ -243,8 +243,8 @@ static int scope_coldplug(Unit *u) {
> if (r < 0 && r != -EEXIST)
> return r;
> }
> -} else
> -(void) unit_enqueue_rewatch_pids(u);
> +}
> +(void) unit_enqueue_rewatch_pids(u);
> }
>
> bus_scope_track_controller(s);
>
> Can you file a Github issue to track this (and possibly try if this
> works for you)?
I have already opened an issue [1] but it was closed. Maybe
we can just re-open it?
I patched the impish systemd package but the issue is still
present.
Thanks,
Nick
[1] https://github.com/systemd/systemd/issues/20136
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Concurrent login / daemon-reload produces abandoned sessions
Hello Nicolas.
On Wed, Jul 07, 2021 at 02:12:51PM +0200, Nicolas Bock
wrote:
> Using systemd-248.3-1ubuntu1 on Ubuntu Impish the following
> script produces multiple abandoned sessions:
>
> $ for i in {1..100}; do sleep 0.2; ssh localhost sudo systemctl
> daemon-reload & ssh localhost sleep 1 & done
> $ sleep 2
> $ jobs -p | xargs --verbose --no-run-if-empty kill -KILL
> $ systemctl | grep abandoned
> session-174.scopeloaded active abandoned
> Session 174 of user ubuntu
> session-175.scopeloaded active abandoned
> Session 175 of user ubuntu
> session-176.scopeloaded active abandoned
> Session 176 of user ubuntu
> session-25.scope loaded active abandoned
> Session 25 of user ubuntu
>
> I would like to debug this behavior further and understand
> why this is happening but don't know where to look next.
It might be a bit challenging :)
> Is there any information in particular I should look at?
I assume you use hybrid or unified cgroup setup and that the abandoned
scopes are empty (no processes in their cgroups), correct?
My hypothesis is following
// race between scope abandonement, emptiness notification -> abandon comes
first
manager_reload
manager_clear_jobs_and_units
unit_release_cgroup
inotify_rm_watch(u->manager->cgroup_inotify_fd,
u->cgroup_control_inotify_wd)
[...]
// last process terminates somewhere here but we're not watching emptiness yet
scope_coldplug()
// scope should be checked for emptiness here
I _think_ this could be fixed with the patch
--- a/src/core/scope.c
+++ b/src/core/scope.c
@@ -243,8 +243,8 @@ static int scope_coldplug(Unit *u) {
if (r < 0 && r != -EEXIST)
return r;
}
-} else
-(void) unit_enqueue_rewatch_pids(u);
+}
+(void) unit_enqueue_rewatch_pids(u);
}
bus_scope_track_controller(s);
Can you file a Github issue to track this (and possibly try if this
works for you)?
Thanks,
Michal
signature.asc
Description: Digital signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Concurrent login / daemon-reload produces abandoned sessions
Hi,
Using systemd-248.3-1ubuntu1 on Ubuntu Impish the following
script produces multiple abandoned sessions:
$ for i in {1..100}; do sleep 0.2; ssh localhost sudo systemctl
daemon-reload & ssh localhost sleep 1 & done
$ sleep 2
$ jobs -p | xargs --verbose --no-run-if-empty kill -KILL
$ systemctl | grep abandoned
session-174.scopeloaded active abandoned
Session 174 of user ubuntu
session-175.scopeloaded active abandoned
Session 175 of user ubuntu
session-176.scopeloaded active abandoned
Session 176 of user ubuntu
session-25.scope loaded active abandoned
Session 25 of user ubuntu
I would like to debug this behavior further and understand
why this is happening but don't know where to look next. Is
there any information in particular I should look at?
Thanks!
Nick
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
