Re: [systemd-devel] Power aware units
On Mon, 2013-11-04 at 16:52 +0100, Lennart Poettering wrote: On Mon, 04.11.13 16:10, Bastien Nocera (had...@hadess.net) wrote: And I figure other ones to add here are: - mandb cache cleanups - rotation of legacy log files (are there more usecases you can come up with?) All these are exclusively cronjobs, i.e. services triggered only by time, and nothing else. Those _were_ all cron jobs. I don't think that any of those should be cron jobs, especially the 4 AM update-db or man-db runs. Anacron is just patching over the fact that most machines aren't running at that time of day. Well, but if not cronjob, what should they be? For me, a cron job is something executed by cron. This should be executed, once a day, when the screen is locked, on AC battery, and not under any resource pressure (especially RAM). They're non-essential functions, and I don't think they should be running if I'm in front of my computer if I'm using it, even at 4AM. Automatically installing application updates is another use case, or tracker indexing files. Basically, anything that runs in the background without direct user interaction. And all of these strike me as things where we really should run them on battery too, as not doing them at all will sooner or later have a negative effect on the stability and safety of the system. I don't think that update-db or man-db should be updated when on battery, and even less when on low battery. Well, let's say you have a tablet or so which is never attached to AC, but where people simply replace the battery when it runs empty with a new charged one. There's still hardware like that? If that were the case, I wouldn't make it run under low battery then. You certainly wan't these cronjobs to eventually run. I mean, if you installed man-db/update-db you probably want them to work correctly, no? They're installed by default on a lot of distributions (they are, at least on Fedora), so yes, you want them to work correctly, but not at the detriment of other tasks or your battery life. It also shouldn't wake up the hard disk, or be run when I have interactive tasks on-going (eg. it should run when the screen lock is on, not in the middle of streaming a film). Hmm, well, is the policy here that clear? I mean, if our kernel IO/CPU schedulers were any good then scheduling these things while you watch a movie should actually have no drawbacks, and even more be a good thing since the CPU is awake anyway and it's always good to do things when the CPU is awake anyway. We should be *very* careful in basically adding a second-level scheduler here that executes jobs when the CPU load is low, because we don't trust the first-level scheduler in the scheduler. A scheduler that watches another scheduler, and tries to detect when it is idle sounds like trouble. Major trouble. At this point I am quite sure we don't even make use of all the functionality the existing IO/CPU scheduler exposes, and hence there's really no reason to add a second-level scheduler on top. For example, do we actually set those cron jobs to SCHED_IDLE and IOPRIO_CLASS_IDLE? We already have a way to detect idle, the screensaver being on, and yes, we should be setting those jobs to have extremely low priority. But they also shouldn't be running when on low battery. (And also possible on the other end, do we set the bits necessariy for video playback to something rt'ish?) No, not right now, because we'd need to make X rt'ish, etc. That's probably another discussion. So, would this make sense to you? Is there anything this logic wouldn't cover? You want backups to run when on AC and the screen is locked (or even better, when the machine is supposed to be sleeping, with the lid closed for laptops). But why exactly do you want this? Because our current IO/CPU scheduler setup causes these jobs to interfere too much with your foreground work? Or is there more to it? Because they might interfere with your work. This isn't all about schedulers, it's also about network usage, and battery usage. (Oh, and also note that there actually already is ConditionACPower= which allows you to skip execution of certain services if AC is plugged in/is not plugged in. See systemd.unit(5) for details). It doesn't include UPS status, it doesn't know when the hard disk is sleeping, it doesn't know when the screen lock is on. Hmm, so correct me if I am wrong: UPS stuff is only last-resort kind of stuff: i.e. when we run off UPS we are in the process of shutting down anyway, right? I mean, UPSes are not something that is used for continous operation, it's just the fallback in the case shit happens? UPSes are continuously plugged in, you need to know its state to be able to act upon it. So for every mention of low battery above, think of the UPS discharging as critical battery. You have 10/15 minutes of battery
Re: [systemd-devel] Power aware units
On 11/03/2013 02:36 PM, Bastien Nocera wrote: In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? in Fedora we already have started that migration process for relevant units ( not all packages that contain ron jobs should be/will be migrated ). I've just been waiting ( since July/August ) for the first step of that migration to be completed as in those packages that will not be migrated fix their dependency on crontabs. Ones those packagers apply the patches I sent them in that regard I will start the actual cron to timer units migration. JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Power aware units
On Sun, 03.11.13 15:36, Bastien Nocera (had...@hadess.net) wrote: Heya, systemd already allows launching specific tasks based on a timer, and intervals, and I was wondering whether power awareness was something planned for launching and stopping units. MacOS X 10.9 has some additional metadata for units that allows launchd to stop and start particular tasks based on power levels: http://arstechnica.com/apple/2013/10/os-x-10-9/16/ So, hmm. So far our policy on Linux was mostly If something good for battery cases, then it is good for AC cases too, i.e. the dichotomy between services that may run on battery and that may not be run on on battery is not necessarily a good one. I am really wondering hence whether policies like this one really should exist at all? And to figure that out we probably should look closely at the usecases. So, what are the use cases? You listed: - backups - tmpfiles/cache ageing (including trash expulsion, thumbnail cleanup) - update-db And I figure other ones to add here are: - mandb cache cleanups - rotation of legacy log files (are there more usecases you can come up with?) All these are exclusively cronjobs, i.e. services triggered only by time, and nothing else. And all of these strike me as things where we really should run them on battery too, as not doing them at all will sooner or later have a negative effect on the stability and safety of the system. So, thinking about this I have the suspicion all that we really can do is modulate the timers a bit: i.e. if the system is on battery power simply lower the frequency how often we dispatch timer events. More specifically: what about adding a new option for .timer units maybe called BatterySlackSec= or so, where you can specify some extra time the execution of the job can be delayed for at maximum if the machine is on battery. If it is not on battery we'd ignore this value. If the machine changes from on battery to on ac we'd immediately execute all services which are already in their battery slack phase. This would then allow people to easily express things like Clean-up /tmp every day, but if I am on battery delay it for another day at max, and possibly exeucte it immediately if I replug power. Or something similar. So, would this make sense to you? Is there anything this logic wouldn't cover? (Oh, and also note that there actually already is ConditionACPower= which allows you to skip execution of certain services if AC is plugged in/is not plugged in. See systemd.unit(5) for details). We could implement this in 2 ways: - systemd itself speaks over D-Bus to UPower (using the new DisplayDevice to merge UPS and batteries) and stop/starts the units. I am a bit afraid of making systemd depend on external daemons for things like this. Making systemd a client to another service that is started by systemd, is always problematic, due to potential deadlocks and cyclic deps. We are currently working on getting rid of a dependency like this in regards to dbus-daemon where kdbus makes this all nice and clean for us, and I'd really prefer not to add another case like this... I don't even think this would be necessary really. The power source kernel device class is sufficiently simple for us to use directly, going via upower doesn't appear necessary (I mean, for this logic we don't need battery states of mice and suchlike, hence this shouldn't be a problem.) In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? It certainly would make sense, but my plan was to first add a couple of things to our timer logic implementation that is substantially better than classic cron so that we have strong case to make why people should adopt it. However, never found the time to work on this. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Power aware units
On Mon, 2013-11-04 at 15:27 +0100, Lennart Poettering wrote: On Sun, 03.11.13 15:36, Bastien Nocera (had...@hadess.net) wrote: Heya, systemd already allows launching specific tasks based on a timer, and intervals, and I was wondering whether power awareness was something planned for launching and stopping units. MacOS X 10.9 has some additional metadata for units that allows launchd to stop and start particular tasks based on power levels: http://arstechnica.com/apple/2013/10/os-x-10-9/16/ So, hmm. So far our policy on Linux was mostly If something good for battery cases, then it is good for AC cases too, i.e. the dichotomy between services that may run on battery and that may not be run on on battery is not necessarily a good one. I am really wondering hence whether policies like this one really should exist at all? And to figure that out we probably should look closely at the usecases. So, what are the use cases? You listed: - backups - tmpfiles/cache ageing (including trash expulsion, thumbnail cleanup) - update-db And I figure other ones to add here are: - mandb cache cleanups - rotation of legacy log files (are there more usecases you can come up with?) All these are exclusively cronjobs, i.e. services triggered only by time, and nothing else. Those _were_ all cron jobs. I don't think that any of those should be cron jobs, especially the 4 AM update-db or man-db runs. Anacron is just patching over the fact that most machines aren't running at that time of day. Automatically installing application updates is another use case, or tracker indexing files. Basically, anything that runs in the background without direct user interaction. And all of these strike me as things where we really should run them on battery too, as not doing them at all will sooner or later have a negative effect on the stability and safety of the system. I don't think that update-db or man-db should be updated when on battery, and even less when on low battery. It also shouldn't wake up the hard disk, or be run when I have interactive tasks on-going (eg. it should run when the screen lock is on, not in the middle of streaming a film). So, thinking about this I have the suspicion all that we really can do is modulate the timers a bit: i.e. if the system is on battery power simply lower the frequency how often we dispatch timer events. More specifically: what about adding a new option for .timer units maybe called BatterySlackSec= or so, where you can specify some extra time the execution of the job can be delayed for at maximum if the machine is on battery. If it is not on battery we'd ignore this value. If the machine changes from on battery to on ac we'd immediately execute all services which are already in their battery slack phase. This would then allow people to easily express things like Clean-up /tmp every day, but if I am on battery delay it for another day at max, and possibly exeucte it immediately if I replug power. Or something similar. So, would this make sense to you? Is there anything this logic wouldn't cover? You want backups to run when on AC and the screen is locked (or even better, when the machine is supposed to be sleeping, with the lid closed for laptops). (Oh, and also note that there actually already is ConditionACPower= which allows you to skip execution of certain services if AC is plugged in/is not plugged in. See systemd.unit(5) for details). It doesn't include UPS status, it doesn't know when the hard disk is sleeping, it doesn't know when the screen lock is on. We could implement this in 2 ways: - systemd itself speaks over D-Bus to UPower (using the new DisplayDevice to merge UPS and batteries) and stop/starts the units. I am a bit afraid of making systemd depend on external daemons for things like this. Making systemd a client to another service that is started by systemd, is always problematic, due to potential deadlocks and cyclic deps. We are currently working on getting rid of a dependency like this in regards to dbus-daemon where kdbus makes this all nice and clean for us, and I'd really prefer not to add another case like this... I don't even think this would be necessary really. The power source kernel device class is sufficiently simple for us to use directly, going via upower doesn't appear necessary (I mean, for this logic we don't need battery states of mice and suchlike, hence this shouldn't be a problem.) It's not, because you're missing information about low batteries (eg. that's a policy in UPower, which I don't think you are or should be replicating in systemd), and about UPSes. In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? It certainly would make sense, but my plan was to first add a couple of things to our timer logic implementation that is substantially better than classic cron so that we have
Re: [systemd-devel] Power aware units
On Mon, 04.11.13 16:10, Bastien Nocera (had...@hadess.net) wrote: And I figure other ones to add here are: - mandb cache cleanups - rotation of legacy log files (are there more usecases you can come up with?) All these are exclusively cronjobs, i.e. services triggered only by time, and nothing else. Those _were_ all cron jobs. I don't think that any of those should be cron jobs, especially the 4 AM update-db or man-db runs. Anacron is just patching over the fact that most machines aren't running at that time of day. Well, but if not cronjob, what should they be? Automatically installing application updates is another use case, or tracker indexing files. Basically, anything that runs in the background without direct user interaction. And all of these strike me as things where we really should run them on battery too, as not doing them at all will sooner or later have a negative effect on the stability and safety of the system. I don't think that update-db or man-db should be updated when on battery, and even less when on low battery. Well, let's say you have a tablet or so which is never attached to AC, but where people simply replace the battery when it runs empty with a new charged one. You certainly wan't these cronjobs to eventually run. I mean, if you installed man-db/update-db you probably want them to work correctly, no? It also shouldn't wake up the hard disk, or be run when I have interactive tasks on-going (eg. it should run when the screen lock is on, not in the middle of streaming a film). Hmm, well, is the policy here that clear? I mean, if our kernel IO/CPU schedulers were any good then scheduling these things while you watch a movie should actually have no drawbacks, and even more be a good thing since the CPU is awake anyway and it's always good to do things when the CPU is awake anyway. We should be *very* careful in basically adding a second-level scheduler here that executes jobs when the CPU load is low, because we don't trust the first-level scheduler in the scheduler. A scheduler that watches another scheduler, and tries to detect when it is idle sounds like trouble. Major trouble. At this point I am quite sure we don't even make use of all the functionality the existing IO/CPU scheduler exposes, and hence there's really no reason to add a second-level scheduler on top. For example, do we actually set those cron jobs to SCHED_IDLE and IOPRIO_CLASS_IDLE? (And also possible on the other end, do we set the bits necessariy for video playback to something rt'ish?) So, would this make sense to you? Is there anything this logic wouldn't cover? You want backups to run when on AC and the screen is locked (or even better, when the machine is supposed to be sleeping, with the lid closed for laptops). But why exactly do you want this? Because our current IO/CPU scheduler setup causes these jobs to interfere too much with your foreground work? Or is there more to it? (Oh, and also note that there actually already is ConditionACPower= which allows you to skip execution of certain services if AC is plugged in/is not plugged in. See systemd.unit(5) for details). It doesn't include UPS status, it doesn't know when the hard disk is sleeping, it doesn't know when the screen lock is on. Hmm, so correct me if I am wrong: UPS stuff is only last-resort kind of stuff: i.e. when we run off UPS we are in the process of shutting down anyway, right? I mean, UPSes are not something that is used for continous operation, it's just the fallback in the case shit happens? Upower doesn't know when the disk is sleeping either, does it? I am not sure even the kernel knows this? It certainly would make sense, but my plan was to first add a couple of things to our timer logic implementation that is substantially better than classic cron so that we have strong case to make why people should adopt it. However, never found the time to work on this. Do you have a list of what's missing? The things I had on my list: - anacron-like persistency - adjust timers to coalesce them once in each second (similar to what sd-event is doing or glib) - timer jitter for cluster setups (related to the cluster thing above) - timer events that cause the system to resume - a list-timers command in systemctl (similar to list-sockets) - timer evets on DST changes or when the user alters the system time - syntax for specifying dates from the end of the month - modulate timer frequency based on battery state The first five or so I guess are the interesting ones. The benefit in adding usage classes (as Apple did), rather than tweaking each background task manually is that it makes it easier to change the policy for running those, and affect all the uses of that class. I have the suspicion that the Apple thing is too opaque for people to grok. And I also have the suspicion that in the end the number of services that need this is relatively limited
Re: [systemd-devel] Power aware units
On Monday 2013-11-04 16:52, Lennart Poettering wrote: Those _were_ all cron jobs. I don't think that any of those should be cron jobs, especially the 4 AM update-db or man-db runs. Anacron is just patching over the fact that most machines aren't running at that time of day. Well, but if not cronjob, what should they be? Some wishful thinking: daemons. locate would be a lot more useful if it was constantly adding new entries to its database -- perhaps fsnotify, or whatever became of efficiently watching directory changes recursively-- would be of use. mandb itself seems to be for keyword searches à la whatis/apropos, but not for plain man operation where you look up by page/filename, like man systemctl. It will (or should) know to just glob in /usr/share/man/ for systemctl.*. Hmm, so correct me if I am wrong: UPS stuff is only last-resort kind of stuff: i.e. when we run off UPS we are in the process of shutting down anyway, right? I mean, UPSes are not something that is used for continous operation, it's just the fallback in the case shit happens? Shutdowns are not triggered when power is lost first, but after perhaps 2–10 minutes (take your pick). Not all servers are terribly overloaded with VMs, enterprise databases and sysvinit scripts that take ages to shutdown. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Power aware units
Heya, systemd already allows launching specific tasks based on a timer, and intervals, and I was wondering whether power awareness was something planned for launching and stopping units. MacOS X 10.9 has some additional metadata for units that allows launchd to stop and start particular tasks based on power levels: http://arstechnica.com/apple/2013/10/os-x-10-9/16/ We could implement this in 2 ways: - systemd itself speaks over D-Bus to UPower (using the new DisplayDevice to merge UPS and batteries) and stop/starts the units. - systemd ships a set of units that UPower will launch/stop based on battery status. This would require UPower to know more about some other subsystems as well such as the lock screen status, or the hard drive state. This would be useful for things like backups, housekeeping (emptying old files from the trash, old thumbnails, etc.), launching update-db, etc. in addition to the simple timer intervals. I think the first option is the best one, as in addition to UPower, we'd need to talk to the kernel/udev (HDD spinning state), and logind (lock screen status). In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? Cheers ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Power aware units
On Sun, Nov 3, 2013 at 3:36 PM, Bastien Nocera had...@hadess.net wrote: systemd already allows launching specific tasks based on a timer, and intervals, and I was wondering whether power awareness was something planned for launching and stopping units. MacOS X 10.9 has some additional metadata for units that allows launchd to stop and start particular tasks based on power levels: http://arstechnica.com/apple/2013/10/os-x-10-9/16/ We could implement this in 2 ways: - systemd itself speaks over D-Bus to UPower (using the new DisplayDevice to merge UPS and batteries) and stop/starts the units. - systemd ships a set of units that UPower will launch/stop based on battery status. This would require UPower to know more about some other subsystems as well such as the lock screen status, or the hard drive state. This would be useful for things like backups, housekeeping (emptying old files from the trash, old thumbnails, etc.), launching update-db, etc. in addition to the simple timer intervals. I think the first option is the best one, as in addition to UPower, we'd need to talk to the kernel/udev (HDD spinning state), and logind (lock screen status). Systemd should not get any direct or indirect dependency on upower for primary service management tasks. It just doesn't sound right to do dependencies in this direction. If systemd needs more information than the current on-battery vs. AC, we would need to add that to systemd itself, or even add support for a virtual/composite battery to the kernel. In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? This all sounds nice to have, and what we want to have in the longer run. We could do most of that already today, with just the battery/AC information, I think. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Power aware units
On Sun, 2013-11-03 at 16:06 +0100, Kay Sievers wrote: On Sun, Nov 3, 2013 at 3:36 PM, Bastien Nocera had...@hadess.net wrote: systemd already allows launching specific tasks based on a timer, and intervals, and I was wondering whether power awareness was something planned for launching and stopping units. MacOS X 10.9 has some additional metadata for units that allows launchd to stop and start particular tasks based on power levels: http://arstechnica.com/apple/2013/10/os-x-10-9/16/ We could implement this in 2 ways: - systemd itself speaks over D-Bus to UPower (using the new DisplayDevice to merge UPS and batteries) and stop/starts the units. - systemd ships a set of units that UPower will launch/stop based on battery status. This would require UPower to know more about some other subsystems as well such as the lock screen status, or the hard drive state. This would be useful for things like backups, housekeeping (emptying old files from the trash, old thumbnails, etc.), launching update-db, etc. in addition to the simple timer intervals. I think the first option is the best one, as in addition to UPower, we'd need to talk to the kernel/udev (HDD spinning state), and logind (lock screen status). Systemd should not get any direct or indirect dependency on upower for primary service management tasks. It just doesn't sound right to do dependencies in this direction. That doesn't strike me as as crazy as it may seem. systemd would only monitor a couple of properties on a remote D-Bus name. If the name isn't present, use defaults (not on battery, not discharging, HDD spinning). If systemd needs more information than the current on-battery vs. AC, we would need to add that to systemd itself, or even add support for a virtual/composite battery to the kernel. That would leave policy handling (eg. what's low-battery?), and UPS support on the wayside. I also don't think you want to start polling batteries in systemd. In addition to that, would it make sense for distributions to start porting their cron jobs to use systemd? This all sounds nice to have, and what we want to have in the longer run. We could do most of that already today, with just the battery/AC information, I think. I think it's all or nothing. Merging batteries is a crappy enough job (with work-arounds for bad firmwares) that you really don't want it in the kernel or systemd itself. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Power aware units
On Sun, 2013-11-03 at 17:07 +0100, Bastien Nocera wrote: snip Systemd should not get any direct or indirect dependency on upower for primary service management tasks. It just doesn't sound right to do dependencies in this direction. That doesn't strike me as as crazy as it may seem. systemd would only monitor a couple of properties on a remote D-Bus name. If the name isn't present, use defaults (not on battery, not discharging, HDD spinning). FWIW, systemd would also need to pull in a dependency on logind. Maybe you're actually asking for UPower to be merged in systemd? That's probably do-able :) ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
