Bug#777601: systemd: Loosing LXC memory cgroups after service install
On Tue, 25 Aug 2015 19:10:50 +0200 Michael Biebl wrote: > Am 25.08.2015 um 18:41 schrieb Luca Bruno: > > > I've tried this patch and looks like adding another bug to me. Please > > confirm what I'm experiencing. It's true, it does not remove cgroups > > created by systemd, but then it doesn't cleanup cgroups that systemd > > created either. > > > > Example: > > > > 1) set MemoryLimit to a service, the memory limit will appear in the cgroups > > 2) unset MemoryLimit to the same service, reload daemon, restart, > > disable, re-enable, whatever... but the memory limit will NOT disappear > > from the cgroups > > > > Seems wrong and possibly worse to me. > > Please raise your issue upstream The issue I've mentioned is caused by the patch applied in the debian package. It fixes an issue, and introduces another one. It's not an upstream issue.
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Am 25.08.2015 um 18:41 schrieb Luca Bruno: > I've tried this patch and looks like adding another bug to me. Please > confirm what I'm experiencing. It's true, it does not remove cgroups > created by systemd, but then it doesn't cleanup cgroups that systemd > created either. > > Example: > > 1) set MemoryLimit to a service, the memory limit will appear in the cgroups > 2) unset MemoryLimit to the same service, reload daemon, restart, > disable, re-enable, whatever... but the memory limit will NOT disappear > from the cgroups > > Seems wrong and possibly worse to me. Please raise your issue upstream -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#777601: systemd: Loosing LXC memory cgroups after service install
On Tue, 25 Aug 2015 18:41:59 +0200 Luca Bruno wrote: > I've tried this patch and looks like adding another bug to me. Please > confirm what I'm experiencing. It's true, it does not remove cgroups > created by systemd, but then it doesn't cleanup cgroups that systemd > created either. Correction, sorry for the confusing wording: "It's true, it does not remove cgroups *not* created by systemd, but then it doesn't cleanup cgroups that systemd created either."
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Control: unarchive -1 On Thu, 12 Feb 2015 15:43:30 +0100 Martin Pitt wrote: > Hello again, > > so the patch that got proposed at > > http://lists.freedesktop.org/archives/systemd-devel/2014-September/023276.html > > actually makes a lot of sense: This makes systemd only clean up > cgroups that it created by itself, and thus won't clean up empty ones > in other controllers that LXC created. I tested this and committed > this for the experimental branch for now: > > http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=experimental&id=286ef78fd > > I'd like to see this out in the wild for some time before applying it > to jessie, though. I'm also still interested in what the actual impact > of that is -- critical seems rather inflated? Losing empty cgroups > doesn't sound that dangerous after all, aside from the LXC warnings > when shutting down a container? > > Thanks, I've tried this patch and looks like adding another bug to me. Please confirm what I'm experiencing. It's true, it does not remove cgroups created by systemd, but then it doesn't cleanup cgroups that systemd created either. Example: 1) set MemoryLimit to a service, the memory limit will appear in the cgroups 2) unset MemoryLimit to the same service, reload daemon, restart, disable, re-enable, whatever... but the memory limit will NOT disappear from the cgroups Seems wrong and possibly worse to me. Best regards,
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Control: tag -1 -moreinfo Pierre Mavro [2015-02-12 16:11 +0100]: > I think this may not be a critical bug, however as this is the only way > (as far as I know) to know/monitor the used resources of a container, I > think it's important. Not just a cosmetical problem. Ah, ok. The patch is simple enough (well, at least for me..) so I backported it to master for jessie: http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?id=255ae60 We'll upload 215-12 in the next days, would be nice if you could confirm that this fixes things! Thanks, Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Hi Martin, So many questions...thanks for answers ! I think this may not be a critical bug, however as this is the only way (as far as I know) to know/monitor the used resources of a container, I think it's important. Not just a cosmetical problem. To my level, it's hard to say if this issue is linked or not (https://bugzilla.redhat.com/show_bug.cgi?id=1139223). But, what I can see is, swap and memory are both options that requires to be activated from grub to work compared to others which work out of the box. This issue really looks like the same (http://lists.freedesktop.org/archives/systemd-devel/2014-September/023276.html). It may be a good idea to integrate this patch. I also tested something which may be another problem (so another ticket) but the memory and swap limitation doesn't work at all :-(...with or without this bug. I can't tell if it's linked or not. I tested cpu pinning and it works like a charm. The issue seems related to memory and swap only. Thanks Pierre Le 02/12/2015 03:43 PM, Martin Pitt a écrit : > Control: tag -1 pending > > Hello again, > > so the patch that got proposed at > > > http://lists.freedesktop.org/archives/systemd-devel/2014-September/023276.html > > actually makes a lot of sense: This makes systemd only clean up > cgroups that it created by itself, and thus won't clean up empty ones > in other controllers that LXC created. I tested this and committed > this for the experimental branch for now: > > > http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=experimental&id=286ef78fd > > I'd like to see this out in the wild for some time before applying it > to jessie, though. I'm also still interested in what the actual impact > of that is -- critical seems rather inflated? Losing empty cgroups > doesn't sound that dangerous after all, aside from the LXC warnings > when shutting down a container? > > Thanks, > > Martin -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Control: tag -1 pending Hello again, so the patch that got proposed at http://lists.freedesktop.org/archives/systemd-devel/2014-September/023276.html actually makes a lot of sense: This makes systemd only clean up cgroups that it created by itself, and thus won't clean up empty ones in other controllers that LXC created. I tested this and committed this for the experimental branch for now: http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?h=experimental&id=286ef78fd I'd like to see this out in the wild for some time before applying it to jessie, though. I'm also still interested in what the actual impact of that is -- critical seems rather inflated? Losing empty cgroups doesn't sound that dangerous after all, aside from the LXC warnings when shutting down a container? Thanks, Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Control: notfixed -1 218-1 Control: tag -1 moreinfo Hello again, Martin Pitt [2015-02-12 13:24 +0100]: > I remember a similar problem when I worked on Ubuntu's "user LXC > container" support/patch, but this issue got fixed with 217 or 218. > Indeed I can't reproduce this report with the experimental version, > marking as fixed there; can you confirm? Sorry, I noticed that sometimes restarting a service isn't enough (sometimes it is..), one really needs to do daemon-reload *and* restart. With that I still get the issue with 218-1, so there's no upstream fix for this yet. So https://bugzilla.redhat.com/show_bug.cgi?id=1139223 might be related, but that wouldn't actually help us, will it? As nothign sets the "Delegate" flag for LXC, this is only done for nspawn. But that bug also talked about moving pids from existing cgroups (i. e. containers), which I cannot reproduce -- i. e. at least as far as I can see this is just the cleanup of empty cgroups, not modifying existing non-empty ones. Can you confirm this? Thanks, Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Control: tag -1 confirmed Control: fixed -1 218-1 Hey Pierre, Pierre Mavro [2015-02-10 15:45 +0100]: > Let's say I've an LXC container called jessie running on jessie. The > container is started and I can see the memory is correctly visible in > the cgroup fs: > > $ ls /sys/fs/cgroup/memory/lxc/jessie/ > [...] > Then let's say I install samba (works with other services): > > $ apt-get -y install samba > > Then the cgroup fs for the container disappeared : > > $ ls /sys/fs/cgroup/memory/lxc/jessie/ > ls: cannot access /sys/fs/cgroup/memory/lxc/jessie/: No such file or > directory I can reproduce this with systemd 215 from jessie/unstable. For the record, it's not necessary to install samba, "systemctl restart haveged" worked for me too. I remember a similar problem when I worked on Ubuntu's "user LXC container" support/patch, but this issue got fixed with 217 or 218. Indeed I can't reproduce this report with the experimental version, marking as fixed there; can you confirm? How much of an actual issue is that in practice? At least for me reloading a service only triggers the cleanup of empty cgroups. Indeed LXC creates the cgroups in all controllers, but only actually puts the processes in the container in some of them (e. g. perf_event and freezer, but not devices and memory), and those which are non-empty stay alive for me. So isn't that just a rather cosmetical problem at this point? Thanks, Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: Digital signature
Bug#777601: systemd: Loosing LXC memory cgroups after service install
Am 10.02.2015 um 15:45 schrieb Pierre Mavro: > Package: systemd > Version: 215-10 > Severity: critical > Justification: breaks unrelated software > > Dear Maintainer, > > I encounter a problem with LXC running with systemd. Following the > documentation (https://wiki.debian.org/LXC), I updated grub to manage > memory (cgroup_enable=memory) a,d lxc-checkconfig returns all green. > > I got the issue on 2 different servers running Jessie with systemd and > as you can see it is reproductible. > > Let's say I've an LXC container called jessie running on jessie. The > container is started and I can see the memory is correctly visible in > the cgroup fs: > > $ ls /sys/fs/cgroup/memory/lxc/jessie/ > cgroup.clone_children memory.max_usage_in_bytesmemory.numa_stat > memory.usage_in_bytes > cgroup.event_control memory.memsw.failcnt memory.oom_control > memory.use_hierarchy > cgroup.procs memory.memsw.limit_in_bytes > memory.pressure_level notify_on_release > memory.failcnt memory.memsw.max_usage_in_bytes > memory.soft_limit_in_bytes tasks > memory.force_empty memory.memsw.usage_in_bytes memory.stat > memory.limit_in_bytes memory.move_charge_at_immigrate > memory.swappiness > > Then let's say I install samba (works with other services): > > $ apt-get -y install samba > > Then the cgroup fs for the container disappeared : > > $ ls /sys/fs/cgroup/memory/lxc/jessie/ > ls: cannot access /sys/fs/cgroup/memory/lxc/jessie/: No such file or > directory > > Hope this will help to reproduce. > That seems to be https://bugzilla.redhat.com/show_bug.cgi?id=1139223 Can you confirm that -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
