Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Wed, Jan 28, 2015 at 02:38:52AM +0100, Lennart Poettering wrote: To clarify this: if people do this, then this pulls in systemd-udev-settle.service, which slows down boot. Every service that does that is hence a majour source of slowness. It's a hack to use this, not a solution. Well, yes. There aren't really any good solutions with our event driven model - we never finish booting, we just get to a point where nothing has been happening for a while. I have been thinking that we need to just admit that properly and do something timer based - have a timer that gets reset every time we instantiate something, then do all our end of boot actions when nothing happened for a while. It's not elegant but I don't think elegant is a realistic goal here. signature.asc Description: Digital signature
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, 17.01.15 17:03, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote: On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. I think you can just create a unit like: # disable-new-hardware.service [Unit] After=systemd-udev-settle.service systemd-modules-load.service Wants=systemd-udev-settle.service [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/ping-the-kernel So the question to you is would you be willing to include such functionality in udev? I don't think udevd has enough knowledge. But a systemd unit like the one above should work. To clarify this: if people do this, then this pulls in systemd-udev-settle.service, which slows down boot. Every service that does that is hence a majour source of slowness. It's a hack to use this, not a solution. Lennart -- Lennart Poettering, Red Hat -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: Dear udev developers, We (me and some kernel devs mostly) would like to add support to the kernel for userspace telling the kernel that it is done with the *initial* loading of modules, with the purpose of cleaning up (disabling) unused harware resources like e.g. regulators and clocks. But you don't know when that happens. Especially with discoverable busses (PCI, USB, etc.), you know this :) Currently the kernel does this cleanup just before it starts init (which may very well be init from a ramdisk). In some cases this is too early really, because later on a module may get loaded which needs this resources, these resources will then get turned on again by the loaded driver, and most of the time this is not an issue, but sometimes it is. I realize very well that there is no magic moment where udev is really ever done loading modules, but the case which we want to support only involves devices which are *already enumerated*, but may not yet have a driver loaded, when udev starts. We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. The kernel doesn't even know when this type of thing is, how can udev know this? So the question to you is would you be willing to include such functionality in udev ? Note this signal would need to be emitted when udev from the real rootfs is done with the initial module loading, as the real rootfs may very well have more modules available then the initrd. ### With the generic story above told let me also give the concrete example / problem which has let to me asking this (note this has been brought up before on various kernel lists, it is a re-occuring theme, this is just an example really) : The problem at hand is a sata connector which also has a sata-power connector on an embedded (ish) board where the sata-power is controlled through a gpio. The sata-power connector is modeled in devicetree as a power-supply and this supply gets controlled by the ahci_platform driver. The disk power may very well have already been turned on by the bootloader, so we add a regulator-boot-on property to the regulator node in devicetree to make sure that it is left untouched when the regulator driver loads. If the ahci_platform driver is build into the kernel, it will then take control of the regulator and everything works well. If however the ahci_platform driver is a module, then as soon as the kernel is ready to start init, unused regulators are turned off and the disk looses its power while spinning and ends up doing an emergency heads park. What turns off the power in this situation? The kernel? Or userspace? Don't you have control of this? Have you tried to even create a patch that could do this type of thing to udev to see if it is even possible? thanks, greg k-h -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, Jan 17, 2015 at 05:56:30AM -0800, Greg KH wrote: On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: If however the ahci_platform driver is a module, then as soon as the kernel is ready to start init, unused regulators are turned off and the disk looses its power while spinning and ends up doing an emergency heads park. What turns off the power in this situation? The kernel? Or userspace? Don't you have control of this? The kernel turns off the power in late_initcall because it noticed that nothing is using the regulator, it does this because otherwise we're just left with whatever the bootloader had which will may well be wasting power on unused rails due to inflexible PMICs (we do similar things for other subsystems like clocks). This works fine for most use cases since in embedded systems anything critical tends to end up built in. There is configurability to allow things to be flagged as always on so they never get turned off. signature.asc Description: Digital signature
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
Hi, On 17-01-15 14:56, Greg KH wrote: On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: Dear udev developers, We (me and some kernel devs mostly) would like to add support to the kernel for userspace telling the kernel that it is done with the *initial* loading of modules, with the purpose of cleaning up (disabling) unused harware resources like e.g. regulators and clocks. But you don't know when that happens. Especially with discoverable busses (PCI, USB, etc.), you know this :) Right, I'm just bringing this up because it keeps coming back as a possible solution to hardware resources getting turned off too early during various kernel discussions, so I thought that as this is a re-occuring theme, at a minimum someone should discuss this with the udev people. IOW I'm merely the manager, and you know what they say about messengers ... :) Currently the kernel does this cleanup just before it starts init (which may very well be init from a ramdisk). In some cases this is too early really, because later on a module may get loaded which needs this resources, these resources will then get turned on again by the loaded driver, and most of the time this is not an issue, but sometimes it is. I realize very well that there is no magic moment where udev is really ever done loading modules, but the case which we want to support only involves devices which are *already enumerated*, but may not yet have a driver loaded, when udev starts. We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. The kernel doesn't even know when this type of thing is, how can udev know this? This is why I clearly limited this to *already enumerated* devices, udev will know when it has exhausted whatever the kernel has enumerated before it started, because at one point it will stop getting uevents when replaying. This does not mean enumeration is really complete in any way, it just means that modprobe has been run for any *already enumerated* devices. Most of these problems are on embedded (ish) systems, and there it is a good bet that the troublesome device is enumerated already, since it likely is devicetree instantiated. So the question to you is would you be willing to include such functionality in udev ? Note this signal would need to be emitted when udev from the real rootfs is done with the initial module loading, as the real rootfs may very well have more modules available then the initrd. ### With the generic story above told let me also give the concrete example / problem which has let to me asking this (note this has been brought up before on various kernel lists, it is a re-occuring theme, this is just an example really) : The problem at hand is a sata connector which also has a sata-power connector on an embedded (ish) board where the sata-power is controlled through a gpio. The sata-power connector is modeled in devicetree as a power-supply and this supply gets controlled by the ahci_platform driver. The disk power may very well have already been turned on by the bootloader, so we add a regulator-boot-on property to the regulator node in devicetree to make sure that it is left untouched when the regulator driver loads. If the ahci_platform driver is build into the kernel, it will then take control of the regulator and everything works well. If however the ahci_platform driver is a module, then as soon as the kernel is ready to start init, unused regulators are turned off and the disk looses its power while spinning and ends up doing an emergency heads park. What turns off the power in this situation? The kernel? Or userspace? The kernel. Don't you have control of this? To some degree, this can be controller by a driver, and a driver can stop this turning off, but atm this means that the driver MUST be builtin, that is what we would like to fix. Have you tried to even create a patch that could do this type of thing to udev to see if it is even possible? No, because as you indicated at the start of your mail I know this area pretty well and I know this a somewhat controversial proposal, so IMHO there is no use in spending time on this without at least some buy-in of involved people. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, Jan 17, 2015 at 8:56 AM, Greg KH g...@kroah.com wrote: On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: Dear udev developers, We (me and some kernel devs mostly) would like to add support to the kernel for userspace telling the kernel that it is done with the *initial* loading of modules, with the purpose of cleaning up (disabling) unused harware resources like e.g. regulators and clocks. But you don't know when that happens. Especially with discoverable busses (PCI, USB, etc.), you know this :) Consider the ARM multi-arch kernels where everything is a loadable device driver. 1) BIOS does basic system setup 2) Core kernel boots with very few drivers. 3) Late-init calls runs - does power/clock clean up and turns everything off --everything in system blinks--- 4) Now the modules start loading and devices all get turned back on. These are loadable modules for hardware that is known to be present since it is listed in the device tree. -- better place to be doing power/clock clean up 5) Doesn't matter now if you load a module an hour later, you wanted the clock/power turned off for that hour. I know of two ways to fix this interval where the system blinks 1) An enumerated protection scheme. Device tree lists out all of the clock/regulators/etc... that it doesn't want the power cleanup turning off. Then it hopes that a driver gets loaded that uses these devices. If not those clock/regulators stay on forever. This is fragile and requires coordination with the BIOS. 2) in my opinion power clean is happening too early. In needs to moved after the initial phase of loading modules for device tree enumerated devices. Power/clock clean up does not need to be at an exact point in system boot. It is just a clean up to reduce power consumption from things the BIOS turned on that kernel does not use. It is usually not fatal if it runs at the wrong point, but it makes everything blink when it runs too early. For example the automatic clean up could be removed from late_init calls and turned into something that could be triggered from a script. Then runs that script action late in the boot. Or if udev is smart enough it could trigger cleanup after it has loaded modules for all devices it is aware of. But any way you look at it, cleaning up in late-init call for a multi-arch kernel is the too early. Currently the kernel does this cleanup just before it starts init (which may very well be init from a ramdisk). In some cases this is too early really, because later on a module may get loaded which needs this resources, these resources will then get turned on again by the loaded driver, and most of the time this is not an issue, but sometimes it is. I realize very well that there is no magic moment where udev is really ever done loading modules, but the case which we want to support only involves devices which are *already enumerated*, but may not yet have a driver loaded, when udev starts. We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. The kernel doesn't even know when this type of thing is, how can udev know this? So the question to you is would you be willing to include such functionality in udev ? Note this signal would need to be emitted when udev from the real rootfs is done with the initial module loading, as the real rootfs may very well have more modules available then the initrd. ### With the generic story above told let me also give the concrete example / problem which has let to me asking this (note this has been brought up before on various kernel lists, it is a re-occuring theme, this is just an example really) : The problem at hand is a sata connector which also has a sata-power connector on an embedded (ish) board where the sata-power is controlled through a gpio. The sata-power connector is modeled in devicetree as a power-supply and this supply gets controlled by the ahci_platform driver. The disk power may very well have already been turned on by the bootloader, so we add a regulator-boot-on property to the regulator node in devicetree to make sure that it is left untouched when the regulator driver loads. If the ahci_platform driver is build into the kernel, it will then take control of the regulator and everything works well. If however the ahci_platform driver is a module, then as soon as the kernel is ready to start init, unused regulators are turned off and the disk looses its power while spinning and ends up doing an emergency heads park. What turns off the power in this situation? The kernel? Or userspace? Don't you have control of this? Have you tried to even create a patch that could do this type of thing to udev to see if it is even possible? thanks, greg k-h
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
Hi On Sat, Jan 17, 2015 at 9:44 AM, Hans de Goede hdego...@redhat.com wrote: Dear udev developers, We (me and some kernel devs mostly) would like to add support to the kernel for userspace telling the kernel that it is done with the *initial* loading of modules, with the purpose of cleaning up (disabling) unused harware resources like e.g. regulators and clocks. Currently the kernel does this cleanup just before it starts init (which may very well be init from a ramdisk). In some cases this is too early really, because later on a module may get loaded which needs this resources, these resources will then get turned on again by the loaded driver, and most of the time this is not an issue, but sometimes it is. I realize very well that there is no magic moment where udev is really ever done loading modules, but the case which we want to support only involves devices which are *already enumerated*, but may not yet have a driver loaded, when udev starts. We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. So the question to you is would you be willing to include such functionality in udev ? Note this signal would need to be emitted when udev from the real rootfs is done with the initial module loading, as the real rootfs may very well have more modules available then the initrd. We spent quite some time making module loading 'on demand'. This means, udev is by no means the only one loading modules. Furthermore, udev might broadcast events which is reacted on by external listeners. We cannot rely on them *not* loading modules. A notion of settled would be handy in a lot of situations, but I doubt that we can enforce it in udev. Thanks David -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. I think you can just create a unit like: # disable-new-hardware.service [Unit] After=systemd-udev-settle.service systemd-modules-load.service Wants=systemd-udev-settle.service [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/ping-the-kernel So the question to you is would you be willing to include such functionality in udev? I don't think udevd has enough knowledge. But a systemd unit like the one above should work. Zbyszek -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [systemd-devel] Making udev emit a signal when it is done loading modules
On Sat, Jan 17, 2015 at 5:03 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Sat, Jan 17, 2015 at 09:44:00AM +0100, Hans de Goede wrote: We would like udev to emit a signal (ABI to be discussed) when it is done trying to load modules for everything which was already enumerated when it starts, iow when there are no new device events pending anymore when udev does its initial hotplug replay. I think you can just create a unit like: # disable-new-hardware.service [Unit] After=systemd-udev-settle.service systemd-modules-load.service Wants=systemd-udev-settle.service [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/local/bin/ping-the-kernel So the question to you is would you be willing to include such functionality in udev? I don't think udevd has enough knowledge. But a systemd unit like the one above should work. No, nothing new, maintained or properly working tool should use or rely on udevadm settle. Settle is just a dirty hack to make broken legacy tools work until they disappear. Kay -- To unsubscribe from this list: send the line unsubscribe devicetree in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
