Re: [systemd-devel] systemd-machined sigterm, cgroups trimmed
On Nov 6, 2014, at 11:24 AM, Lennart Poettering lenn...@poettering.net wrote: Right after the machine was started, but before machined dies, do you see the machine's scope registered in systemctl list-units? Do you see the machine show up in machinectl's output? Hey Lennart, I can see the details about the machine via systemctl list-units and machinectl before systemd-machined exits. Also, even after systemd-machined exits, running machinectl will start it up again. If I keep machinectl running in a loop in tmux every 5-10 seconds, systemd-machined stays up indefinitely. I'll also take a look at Zbigniew's suggestion in the thread about disabling watchdog for systemd-machined temporarily. — Major Hayden ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd-machined sigterm, cgroups trimmed
On Tue, 04.11.14 08:38, Major Hayden (ma...@mhtx.net) wrote: Hello there, I'm currently running systemd 216 on Fedora 21 and I've found an issue where systemd-machined stops running and cgroups are trimmed from the scope of running qemu virtual machines. The series of events looks like this: 1) OpenStack nova creates a KVM virtual machine via libvirt 2) Libvirt registers the VM with systemd-machined, cgroups appear 3) 0-300 seconds pass 4) systemd-machined gets a sigterm and stops 5) cgroups are trimmed for running virtual machines The cgroup trimming is a disaster because it removes devices.allow and devices.deny ACL's for the running virtual machine. Here is a snippet of the systemd journal with two virtual machines running. You'll see that systemd-machined gets a sigterm and stops: https://gist.github.com/major/2d76cbf0d0de0d62ce3f Shortly after systemd-machined goes offline, the cgroups are removed. Rebooting the hypervisor, rebooting a virtual machine, or building a new virtual machine will cause systemd-machined to start again but it will stop shortly afterwards. I figure machined terminates because it's idle. This would suggest the registration of the machine didn't work, or that the unit that libvirt registered went away. Right after the machine was started, but before machined dies, do you see the machine's scope registered in systemctl list-units? Do you see the machine show up in machinectl's output? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd-machined sigterm, cgroups trimmed
Apologize for the self-reply, but upgrading to dbus-1.8.6 seems to have fixed the cgroup trimming issue. However, systemd-machined still gets a sigterm within a minute or two after a VM starts. For what it's worth, throwing a loop in bash that calls machinectl every 10 seconds will keep systemd-machined running indefinitely and avoid any cgroup trimming of active virtual machines: while true; do machinectl; sleep 10; done Should systemd-machined remain running whenever a VM or container is running? -- Major Hayden On Nov 4, 2014, at 08:38, Major Hayden ma...@mhtx.net wrote: I'm currently running systemd 216 on Fedora 21 and I've found an issue where systemd-machined stops running and cgroups are trimmed from the scope of running qemu virtual machines. The series of events looks like this: 1) OpenStack nova creates a KVM virtual machine via libvirt 2) Libvirt registers the VM with systemd-machined, cgroups appear 3) 0-300 seconds pass 4) systemd-machined gets a sigterm and stops 5) cgroups are trimmed for running virtual machines The cgroup trimming is a disaster because it removes devices.allow and devices.deny ACL's for the running virtual machine. Here is a snippet of the systemd journal with two virtual machines running. You'll see that systemd-machined gets a sigterm and stops: https://gist.github.com/major/2d76cbf0d0de0d62ce3f Shortly after systemd-machined goes offline, the cgroups are removed. Rebooting the hypervisor, rebooting a virtual machine, or building a new virtual machine will cause systemd-machined to start again but it will stop shortly afterwards. I'm running libvirt 1.2.9 and systemd 216 on Linux 3.16.7. Thanks in advance for your help. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd-machined sigterm, cgroups trimmed
On Tue, Nov 04, 2014 at 08:38:51AM -0600, Major Hayden wrote: Hello there, I'm currently running systemd 216 on Fedora 21 and I've found an issue where systemd-machined stops running and cgroups are trimmed from the scope of running qemu virtual machines. The series of events looks like this: 1) OpenStack nova creates a KVM virtual machine via libvirt 2) Libvirt registers the VM with systemd-machined, cgroups appear 3) 0-300 seconds pass 4) systemd-machined gets a sigterm and stops I do not see in the trace, I think that systemd-machined stops on it's own after a period of inactivity. 5) cgroups are trimmed for running virtual machines I don't see this either... (the log is fairly long and hard to read, so maybe I missed it). I only see systemd-machined exiting. DEFAULT_EXIT_USEC is 30 s, and it exits 37 s after startup, consistent with that timeout. Can you post the full cgroup tree before systemd-machined exists and after (systemd-cgls output)? You might want to disable the watchdog for systemd-machined (by adding [Service] WatchdogSec=0 to a drop-in file for systemd-machined), and then stop it with sigstop, and see if your machines still get killed. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd-machined sigterm, cgroups trimmed
Hello there, I'm currently running systemd 216 on Fedora 21 and I've found an issue where systemd-machined stops running and cgroups are trimmed from the scope of running qemu virtual machines. The series of events looks like this: 1) OpenStack nova creates a KVM virtual machine via libvirt 2) Libvirt registers the VM with systemd-machined, cgroups appear 3) 0-300 seconds pass 4) systemd-machined gets a sigterm and stops 5) cgroups are trimmed for running virtual machines The cgroup trimming is a disaster because it removes devices.allow and devices.deny ACL's for the running virtual machine. Here is a snippet of the systemd journal with two virtual machines running. You'll see that systemd-machined gets a sigterm and stops: https://gist.github.com/major/2d76cbf0d0de0d62ce3f Shortly after systemd-machined goes offline, the cgroups are removed. Rebooting the hypervisor, rebooting a virtual machine, or building a new virtual machine will cause systemd-machined to start again but it will stop shortly afterwards. I'm running libvirt 1.2.9 and systemd 216 on Linux 3.16.7. Thanks in advance for your help. -- Major Hayden ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
