Re: [systemd-devel] From dbus notification, how to know service entered failed state and will not start without admin action
Hi, Can you please help ? I see that in OnFailure behaviour is changed and these units in OnFailure gets triggerd when service is failed-and-will-not-restart-automatically. https://lists.freedesktop.org/archives/systemd-devel/2018-June/040879.html In such case, does systemd also send dbus signal whenever it triggers OnFailure ? Thanks On Fri, Feb 18, 2022 at 8:55 PM Prashantkumar dhotre wrote: > Hi, > In my app, I am monitoring a few services and would like to do some > cleanup in my app, when these services fail in such a way that they > don't restart back up without admin action. > From dbus-monitor , I see that 'Result' dbus PropertiesChanged signal > comes first and then 'ActiveState' PropertiesChanged signal comes next. > I was hoping that to use 'ActiveState' = "failed" and "SubState" = > "failed" but that does not satisfy my requirement as > when I send sigkill i get "ActiveState" and "SubState" as "failed" in > DBus notification but service starts back up automatically (due to > 'Restart' & 'ActiveState'settings ) > Also when app exits with non-zero code also , I get "ActiveState" > and "SubState" as "failed" in DBus notification. > So I can not use only "ActiveState" and "SubState" values , as app > restarts after these states report "failed" (in which case my app should > not do cleanup) > Using dbus notification, how can determine that service is down and not > going to start automatically so that in my app i can do cleanup ? > Is there any parameter in PropertiesChanged signal that tells that app is > down and will not start back up automatically ? > I am using systemd 243. > Thanks for your help. > Regards > > > >
Re: [systemd-devel] [systemd‑devel] Q: Perform action for reboots happen too frequently?
Am Freitag, dem 18.02.2022 um 09:52 +0100 schrieb Ulrich Windl: > > > > Thanks Silvio! > > That looks like I was after. But admitted: I had expected that > systemd could > do some of the "shell scripting magic" (like counting and comparing). > > Regards, > Ulrich Well, counting and comparing goes into the area of turing completness, doesn't it? So not having this inside systemd makes the whole system just easier.
[systemd-devel] From dbus notification, how to know service entered failed state and will not start without admin action
Hi, In my app, I am monitoring a few services and would like to do some cleanup in my app, when these services fail in such a way that they don't restart back up without admin action. >From dbus-monitor , I see that 'Result' dbus PropertiesChanged signal comes first and then 'ActiveState' PropertiesChanged signal comes next. I was hoping that to use 'ActiveState' = "failed" and "SubState" = "failed" but that does not satisfy my requirement as when I send sigkill i get "ActiveState" and "SubState" as "failed" in DBus notification but service starts back up automatically (due to 'Restart' & 'ActiveState'settings ) Also when app exits with non-zero code also , I get "ActiveState" and "SubState" as "failed" in DBus notification. So I can not use only "ActiveState" and "SubState" values , as app restarts after these states report "failed" (in which case my app should not do cleanup) Using dbus notification, how can determine that service is down and not going to start automatically so that in my app i can do cleanup ? Is there any parameter in PropertiesChanged signal that tells that app is down and will not start back up automatically ? I am using systemd 243. Thanks for your help. Regards
[systemd-devel] Antw: Re: Antw: [EXT] Re: [systemd‑devel] Q: Perform action for reboots happen too frequently?
>>> Silvio Knizek schrieb am 18.02.2022 um 09:37 in Nachricht <7d547aa30624f2dc8d22ea7f84507fa1d8e9c678.ca...@gmx.net>: > Am Donnerstag, dem 17.02.2022 um 10:50 +0100 schrieb Ulrich Windl: >> > > > Lennart Poettering schrieb am >> > > > 16.02.2022 um 18:17 >> in >> Nachricht : >> > On Mi, 16.02.22 14:09, Ulrich Windl >> > (ulrich.wi...@rz.uni‑regensburg.de) >> wrote: >> > >> > > Hi! >> > > >> > > I wonder: Is it possible with systemd to detect multiple reboots >> > > within a specific time interval, and react, like executing some >> > > systemctl command (that is expected to "improve" things)? With >> > > "reboots" I'm mainly thinking of unexpected boots, not so much >> > > the >> > > "shutdown ‑r" commands, but things like external resets, kernel >> > > panics, watchdog timeouts, etc. >> > >> > The information why a system was reset is hard to get. Some >> > watchdog >> > hw will tell you if it was recently triggered, and some expensive >> > hw >> > might log serious errors to the acpi pstore stuff, but it's all >> > icky >> > and lacks integration. >> >> Lennart, >> >> you got me wrong: I don't want to filter on the reasons of the >> (re)boot, but I >> just wanted to rule out any possible solution that stores information >> during an >> orderly shutdown/reboot ;-) >> >> > >> > A safe approach would probably to instead track boots where the >> > root >> > fs or so is in dirty state. Pretty much all of today's file systems >> > track that. >> >> In the past (when there are a few logins only) I would have been >> tempted to >> use "last" to grep for boots, but if there are many (automated) >> logins, those >> boots may be "rolled out" and thus won't be counted possibly. >> >> > >> > It sounds like an OK feature to add to the systemd root fs mount >> > logic >> > to check for the dirty flag before doing that and then using that >> > is >> > hint to boot into a target other than default.target that could >> > then >> > apply further policy (and maybe then go on to enter >> > default.target). >> > >> > TLDR: nope, this does not exist yet, but parts of it sound like >> > worthwile feature additions to systemd. >> >> Something very primitive: >> a) Increment some counter on every boot, and reset that counter after >> some >> time. >> b) Check the counter and if it is >= some threshold, execute a one- >> shot >> command >> >> Could systemd help with a) and b)? >> >> Regards, >> Ulrich > > Hi Ulrich, > > so, incrementing a counter and executing stuff should be easy. > Something like > > --- boot-counter.service > [Unit] > Description=Incrementing the boot counter > RequiresMountsFor=/var/lib > DefaultDependencies=no > > [Service] > Type=oneshot > Environment="T=5" > ExecStart=/usr/bin/bash -c 'declare -i c="$(< /var/lib/boot.counter)"; c+=1; > echo "$c" > /var/lib/boot.counter' > ExecStart=/usr/bin/bash -c 'if (($(< /var/lib/boot.counter) > T)); then > /clean-script.sh; fi' > > [Install] > WantedBy=multi-user.target > --- > > For the reset you can employ a .timer and a .service > > --- reset-boot-counter.timer > [Unit] > Description=Reset boot counter after 30 minutes > > [Timer] > OnBoot=30m > AccuracySec=1us > > [Install] > WantedBy=timer.target > --- > > --- reset-boot-counter.service > [Unit] > Description=Set /var/lib/boot.counter to zero > RequiresMountsFor=/var/lib > > [Service] > Type=oneshot > ExecStart=/usr/bin/bash -c 'echo "0" > /var/lib/boot.counter > --- Thanks Silvio! That looks like I was after. But admitted: I had expected that systemd could do some of the "shell scripting magic" (like counting and comparing). Regards, Ulrich > > HTH > Silvio
Re: [systemd-devel] Antw: [EXT] Re: [systemd‑devel] Q: Perform action for reboots happen too frequently?
Am Donnerstag, dem 17.02.2022 um 10:50 +0100 schrieb Ulrich Windl: > > > > Lennart Poettering schrieb am > > > > 16.02.2022 um 18:17 > in > Nachricht : > > On Mi, 16.02.22 14:09, Ulrich Windl > > (ulrich.wi...@rz.uni‑regensburg.de) > wrote: > > > > > Hi! > > > > > > I wonder: Is it possible with systemd to detect multiple reboots > > > within a specific time interval, and react, like executing some > > > systemctl command (that is expected to "improve" things)? With > > > "reboots" I'm mainly thinking of unexpected boots, not so much > > > the > > > "shutdown ‑r" commands, but things like external resets, kernel > > > panics, watchdog timeouts, etc. > > > > The information why a system was reset is hard to get. Some > > watchdog > > hw will tell you if it was recently triggered, and some expensive > > hw > > might log serious errors to the acpi pstore stuff, but it's all > > icky > > and lacks integration. > > Lennart, > > you got me wrong: I don't want to filter on the reasons of the > (re)boot, but I > just wanted to rule out any possible solution that stores information > during an > orderly shutdown/reboot ;-) > > > > > A safe approach would probably to instead track boots where the > > root > > fs or so is in dirty state. Pretty much all of today's file systems > > track that. > > In the past (when there are a few logins only) I would have been > tempted to > use "last" to grep for boots, but if there are many (automated) > logins, those > boots may be "rolled out" and thus won't be counted possibly. > > > > > It sounds like an OK feature to add to the systemd root fs mount > > logic > > to check for the dirty flag before doing that and then using that > > is > > hint to boot into a target other than default.target that could > > then > > apply further policy (and maybe then go on to enter > > default.target). > > > > TLDR: nope, this does not exist yet, but parts of it sound like > > worthwile feature additions to systemd. > > Something very primitive: > a) Increment some counter on every boot, and reset that counter after > some > time. > b) Check the counter and if it is >= some threshold, execute a one- > shot > command > > Could systemd help with a) and b)? > > Regards, > Ulrich Hi Ulrich, so, incrementing a counter and executing stuff should be easy. Something like --- boot-counter.service [Unit] Description=Incrementing the boot counter RequiresMountsFor=/var/lib DefaultDependencies=no [Service] Type=oneshot Environment="T=5" ExecStart=/usr/bin/bash -c 'declare -i c="$(< /var/lib/boot.counter)"; c+=1; echo "$c" > /var/lib/boot.counter' ExecStart=/usr/bin/bash -c 'if (($(< /var/lib/boot.counter) > T)); then /clean-script.sh; fi' [Install] WantedBy=multi-user.target --- For the reset you can employ a .timer and a .service --- reset-boot-counter.timer [Unit] Description=Reset boot counter after 30 minutes [Timer] OnBoot=30m AccuracySec=1us [Install] WantedBy=timer.target --- --- reset-boot-counter.service [Unit] Description=Set /var/lib/boot.counter to zero RequiresMountsFor=/var/lib [Service] Type=oneshot ExecStart=/usr/bin/bash -c 'echo "0" > /var/lib/boot.counter --- HTH Silvio
