Re: [systemd-devel] systemd default target
Am Sat, 13 Feb 2016 14:34:59 -0800 schrieb Pathangi Janardhanan : > Hi, > > I have some services configured with the above, Restart=on-failure > and StartLimitInterval/StartLimitBurst and also StartLimitAction with > reboot. > > The problem I am trying to look at is, how to get out of a continuous > reboot cycle, if one of the services is failing. That is why I was > looking for any options where systemd can have its default target > changed to recover or something equivalent, if the system is > undergoing repeated reboots within a certain interval? > > I may be able to do this by writing a separate unit/service for > this, but was wondering if there was any already inbuilt mechanism to > prevent continuous reboot cycle from a mis-behaving service/unit, > which is configured for reload on failure? If you're using dracut-generated initrd you could simply add "emergency" or "rescue" to the kernel cmdline. I think you can also explicitly set a systemd.target=... variable at the cmdline, tho I'm not sure about it. > On Sat, Feb 13, 2016 at 2:48 AM, Lennart Poettering > wrote: > > > On Fri, 12.02.16 20:58, Pathangi Janardhanan (path.j...@gmail.com) > > wrote: > > > > > Hi All, > > > > > > The default target is usually set to multi-user or someother > > > equivalent target. Is there any way in systemd that I can say > > > something like > > > > > > " If the system is reloaded n number of times within the last x > > > second", than set the default target to recover or emergency mode > > > etc.? > > > > > > Basically I am looking for recovering from a failing > > > unit/service that > > is > > > forcing the system to reboot, and that is going in a cycle? > > > > You may configure whether a unit shall be automatically restarted > > with Restart=on-failure. You may put a limit on it with > > StartLimitInterval= and StartLimitBurst=. You may configure what > > shall happen if the limit is hit with StartLimitAction=, which > > includes making the system reboot. > > > > See the systemd.service and systemd.unit man pages. > > > > Lennart > > > > -- > > Lennart Poettering, Red Hat > > > -- Regards, Kai Replies to list-only preferred. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd default target
Hi, I have some services configured with the above, Restart=on-failure and StartLimitInterval/StartLimitBurst and also StartLimitAction with reboot. The problem I am trying to look at is, how to get out of a continuous reboot cycle, if one of the services is failing. That is why I was looking for any options where systemd can have its default target changed to recover or something equivalent, if the system is undergoing repeated reboots within a certain interval? I may be able to do this by writing a separate unit/service for this, but was wondering if there was any already inbuilt mechanism to prevent continuous reboot cycle from a mis-behaving service/unit, which is configured for reload on failure? Thanks Jana On Sat, Feb 13, 2016 at 2:48 AM, Lennart Poettering wrote: > On Fri, 12.02.16 20:58, Pathangi Janardhanan (path.j...@gmail.com) wrote: > > > Hi All, > > > > The default target is usually set to multi-user or someother equivalent > > target. Is there any way in systemd that I can say something like > > > > " If the system is reloaded n number of times within the last x second", > > than set the default target to recover or emergency mode etc.? > > > > Basically I am looking for recovering from a failing unit/service that > is > > forcing the system to reboot, and that is going in a cycle? > > You may configure whether a unit shall be automatically restarted with > Restart=on-failure. You may put a limit on it with StartLimitInterval= > and StartLimitBurst=. You may configure what shall happen if the limit > is hit with StartLimitAction=, which includes making the system reboot. > > See the systemd.service and systemd.unit man pages. > > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] systemd shutdown behavior
Hi, I have a general question concerning systemd's behavior during systemctl reboot, halt or poweroff. As the system is in the process of shutting down, I am wondering if a service fails during the shutdown process does systemd still honor the restart/reboot behavior defined in that programs service file. For example if a service has defined a restart sequence, will systemd try to restart that service even though it is in the process of shutting down? Same question for a service that has defined a FailureAction=reboot? I am running systemd 213 so maybe this is not the case now, but I have observed behavior that makes me think the answer to both questions is yes, the defined actions are still honored. Thanks in advance, -Jay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
Am 13.02.2016 um 14:17 schrieb Christian Seiler: On 02/13/2016 01:44 PM, Reindl Harald wrote: [root@mail-gw:~]$ cat /etc/fstab | grep log UUID=a8fc620d-8046-47a3-8b63-102c525ce5e8 /var/log ext4 defaults, noexec 0 1 To be fair to Lennart: your specific use case, if /var/log is just a separate partition, will work perfectly fine even with Lennart's change, because mounting a partition on a local storage device using the UUID is already supported by initramfs implementations (see the /usr case) that's what i wanted to have finally confirmed signature.asc Description: OpenPGP digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
On Thu, Feb 11, 2016 at 7:06 PM, Lennart Poettering wrote: > 5) Here's the controversial one I think: support for booting up >without /var. We have kludges at quite a few places because we >cannot access /var early during boot. I am tempted to stop >supporting this altogether. Of course, this does *not* mean that >people with split off /var would be left in the cold. It just means >that they have to mount /var from the initrd, exactly like this is >already handled from /usr. > I didn't care about /usr, I always thought mounting it from the initrd is fine. (Especially since most people I've seen only have the separate filesystem directly at /usr, not separate ones at /usr/bin, /usr/lib, etc.) But doing that with more and more filesystems, especially /var, somewhat feels like going backwards from what systemd itself was trying to push once, doesn't it? It used to be "sysv's mount -a is inflexible and you should specify correct dependencies and use RequiresMountsFor" and all that. Now we're going back to "specifying correct dependencies is a pain and you should mount everything at once before starting any services"... It's also unclear what precisely needs to be present, since people *do* often have mountpoints for individual subdirectories under /var, e.g. I've seen /var/log and /var/lib/mysql. So, for example, will systemd require /var/lib/mysql to be mounted by my initramfs, even if mysql.service already takes care of it? Or will it only require /var/lib/systemd or /var/log? How is one supposed to know? -- Mantas Mikulėnas ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
On 02/13/2016 01:44 PM, Reindl Harald wrote: > [root@mail-gw:~]$ cat /etc/fstab | grep log > UUID=a8fc620d-8046-47a3-8b63-102c525ce5e8 /var/log ext4 defaults, noexec > 0 1 To be fair to Lennart: your specific use case, if /var/log is just a separate partition, will work perfectly fine even with Lennart's change, because mounting a partition on a local storage device using the UUID is already supported by initramfs implementations (see the /usr case). My objection to the change is about more complicated storage setups, where one would need to put in a lot of work to be rid of limitations we all have accepted for / and /usr. Regards, Christian signature.asc Description: OpenPGP digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
On 02/13/2016 01:01 PM, Lennart Poettering wrote: > On Sat, 13.02.16 00:10, Christian Seiler (christ...@iwakd.de) wrote: > >> On 02/12/2016 10:34 PM, Lennart Poettering wrote: >>> On Fri, 12.02.16 17:49, Simon McVittie (simon.mcvit...@collabora.co.uk) >>> wrote: >>> On 11/02/16 17:06, Lennart Poettering wrote: > 5) Here's the controversial one I think: support for booting up >without /var. We have kludges at quite a few places because we >cannot access /var early during boot. I don't think /var is really the same thing as /usr: for a start, it has to be read/write, whereas /usr and / can be read-only for at least the early stages of boot. On stateless systems with a read-only / and /etc, requiring /var to be mounted from the initramfs would mean that the mechanism for setting up /var (NFS or tmpfs or whatever) would have to move into the initramfs. >>> >>> Since initrds tend to cover root-on-nfs, root-on-iscsi and so on >>> anyway, that sounds like no change in behaviour really.. >> >> Well, kind-of. The root-on-nfs and root-on-iscsi are dumbed-down >> versions of what's possible once a system is booted. > > Well, to my understanding dracut and stuff makes pretty much all > storage tech that is available during the normal system also available > in the initrd, with the same software, to make sure testing stays > managable. Ok, so I just checked. So the very last version of dracut (released 3 months ago) supports starting iscsid in the initramfs, older versions do not - and only if systemd is installed in the initramfs image. If there's no systemd in the initramfs or an older version is used, dracut uses the iscsistart binary as I explained. So support for this is very, very new. NFS (latest dracut git master): idmapping for NFSv4 is supported to some extent, but there are a lot of assumptions going in: it assumes that nsswitch is used for idmapping (which is probably the most common case on clients, but it need not be the only possibility), it tries to install all nsswitch modules found in /etc/nsswitch.conf, but doesn't know anything about their configuration. What's not supported at all: NFS with Kerberos support. I should note note that NFS w/ Kerberos doesn't work out of the box for /var on a running system either, because you still need to fetch a Kerberos ticket for all the system users that access /var, but it's relatively easy to set something like that up, because you just need to write a simple systemd service for that and run it at boot - which most admins will be able to do - modifying the initramfs is a lot more complicated (especially because the documentation is sparse and the criticism of many people that initramfs are black magic is there for a reason). So basically: it's probably going to work in many cases as long as you don't want a Kerberized setup, but not everything will work out of the box, even if you don't use Kerberos. NBD: dracut (current git master) currently doesn't even support /usr on nbd if / isn't on it (and even then only if on the same device). And then there are still other setups that I had mentioned. DRBD with cluster filesystem: not supported in initramfs. sshfs: doesn't work in initramfs. FUSE filesystems in general: I only know that some people have experimented with zfs-fuse for the rootfs, and it was very fiddly at best from what I remember. And while most people who want to use ZFS on Linux nowadays use the kernel module (where binaries can't be distributed for legal reasons), other FUSE filesystems still have the same issue. The problem is that storage is complicated. You need to special-case each different storage type and add specific code to make that work in the initramfs. If you look at the dracut source code, every different storage solution is special-cased. You need to make sure that the programs required for that storage work in an initramfs environment and can either terminated before switch_root and then restarted in the running system, or alternatively support being kept around during switch_root. You need to have glue code that copies the right files into the initramfs. For rootfs there are certain limitations that have been widely accepted, but if you now start to say that lots of common mount points, especially some parts of /var, to be required to be mounted in the initramfs, you'll make life a lot more complicated for a lot of people. A _lot_ of different projects will have to make sure that their software now works in an initramfs context, even if it was never intended to be used for the root filesystem. That's a lot of work for other people, especially since coordination between different projects is required, just to save yourself from a bit of complexity in systemd. > I think you are extrapolating from limitations of one specific initrd > implementation, no? No. dracut has had some improvements in this regard very recently, but the main point that more compl
Re: [systemd-devel] [RFC] the chopping block
Am 13.02.2016 um 13:01 schrieb Lennart Poettering: My maingoal here is really about having read-access to things, and being able to schedule stuff based on configuration and existance of things, even if that stuff cannot be written to yet. If /var is mounted this late it makes things a lot more complex. it's still unclear *what* that chnage means for setups like over the long and it makes a lot of sense to have /var/log not on the sysroot for as example inbound spamfilters logging a lot to prevent fill the rootfs [root@mail-gw:~]$ cat /etc/fstab | grep log UUID=a8fc620d-8046-47a3-8b63-102c525ce5e8 /var/log ext4 defaults, noexec 0 1 signature.asc Description: OpenPGP digital signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
On Sat, 13.02.16 00:10, Christian Seiler (christ...@iwakd.de) wrote: > On 02/12/2016 10:34 PM, Lennart Poettering wrote: > > On Fri, 12.02.16 17:49, Simon McVittie (simon.mcvit...@collabora.co.uk) > > wrote: > > > >> On 11/02/16 17:06, Lennart Poettering wrote: > >>> 5) Here's the controversial one I think: support for booting up > >>>without /var. We have kludges at quite a few places because we > >>>cannot access /var early during boot. > >> > >> I don't think /var is really the same thing as /usr: for a start, it has > >> to be read/write, whereas /usr and / can be read-only for at least the > >> early stages of boot. > >> > >> On stateless systems with a read-only / and /etc, requiring /var to be > >> mounted from the initramfs would mean that the mechanism for setting up > >> /var (NFS or tmpfs or whatever) would have to move into the > >> initramfs. > > > > Since initrds tend to cover root-on-nfs, root-on-iscsi and so on > > anyway, that sounds like no change in behaviour really.. > > Well, kind-of. The root-on-nfs and root-on-iscsi are dumbed-down > versions of what's possible once a system is booted. Well, to my understanding dracut and stuff makes pretty much all storage tech that is available during the normal system also available in the initrd, with the same software, to make sure testing stays managable. It waits for the root disk and for /usr with the same machanics (i.e. the systemd logic of .device units, fsck services and .mount units). And quite frankly that's really how an initrd should work really: use the same logic and the same software as the host would... I think you are extrapolating from limitations of one specific initrd implementation, no? It sounds really as if that implementation should be fixed though... > PS: Btw. if you do run journald already in initramfs (which I > think is a good thing to have), then it still needs to have > code to flush /run/log/journal to /var/log/journal. So in that > case you don't actually gain anything. The journal isn't really the issue, as /var is generally read-only initially anyway. (and yeah, I am pretty sure initrd should run systemd – which implies running journald too – for reasons described above: you want to keep the differences minimal) My maingoal here is really about having read-access to things, and being able to schedule stuff based on configuration and existance of things, even if that stuff cannot be written to yet. If /var is mounted this late it makes things a lot more complex. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] systemd default target
On Fri, 12.02.16 20:58, Pathangi Janardhanan (path.j...@gmail.com) wrote: > Hi All, > > The default target is usually set to multi-user or someother equivalent > target. Is there any way in systemd that I can say something like > > " If the system is reloaded n number of times within the last x second", > than set the default target to recover or emergency mode etc.? > > Basically I am looking for recovering from a failing unit/service that is > forcing the system to reboot, and that is going in a cycle? You may configure whether a unit shall be automatically restarted with Restart=on-failure. You may put a limit on it with StartLimitInterval= and StartLimitBurst=. You may configure what shall happen if the limit is hit with StartLimitAction=, which includes making the system reboot. See the systemd.service and systemd.unit man pages. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC] the chopping block
On Fri, 12.02.16 23:37, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote: > > > > As long as it's only one package I am happy to break this I must say... > > > I check this now, and samba compiles fine with systemd-compat-libs.rpm > > > removed. So no problem here. > > > > > > Our compat support consists of two parts: > > > libsystemd-{journal,daemon,...}.pc and the .so libraries. I think we > > > should still keep the .pc files for now. > > > > Why? do you know any package that still needs it? > > Not everything is packaged, people might have other code. > There is no maintenance cost, it's just four tiny text files. > So if we remove something that was documented as an official > interface, I think there should be a clear saving. Well, but we really want people to fix up their stuff eventually. I am fine with maintaining compat for a while, and we did for 2y, but I am very sure we should be strict after the transition phase and cut things off. There *must* always be a strategy how to finish the transition, and they only viable strategy I see is to make a hard cut after a long transition phase, and I think that's now. In general, I think ABI compatibility is a ton more relevant than mere API compatibility anyway. Maintaining ABI compat means being nice to users, and I think that's worth a lot. But providing API compat that requires only a rebuild to fix ABI is just a service to developers, and that's far less of a priority. Moreover, the .so files we shipped for the last 2y generated linker warnings anyway, so people should really have noticed. And fixing their build is still trivially easy, it just means changing the pkg-config line in their configure script a tiny bit. I think we should cut this off fully. Let's please get rid of this bitrotting crap now! Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [ANNOUNCE] systemd v229
On Fri, 12.02.16 23:51, Mikhail Kasimov (mikhail.kasi...@gmail.com) wrote: > > RuntimeMaxSec= just says "abort this shit if it takes longer than > > this". The usecase is to use it for stuff which is not supposed to > > take this long, and where it's better to abort it, and complain than > > to leave it running unbounded. > > Ok, got it. Thanks! And one more question -- will 'systemctl status' and > 'journalctl' contain the explanation about RuntimeMaxSec= limit overflow > in logs? E.g. "[FAILED] bla-bla-bla.service. Forced exit, because > RuntimeMaxSec limit is hit". Or something around this. So, "systemctl status" first shows you the general unit status and then shows you recent log data from and about the unit. In the general unit status you find a short field explaining the cause for the failure, and for timeouts it will say "timeout" (but not be any more specific). In the log data you'll find a longer, human readable explanation. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel