Re: [systemd-devel] device enumeration by systemd user instances
On Do, 25.07.19 17:51, Pawel Szewczyk (p.szewc...@samsung.com) wrote: > On 7/23/19 18:00, Lennart Poettering wrote: > > > > Do you have some profiler results about this? i.e. what exactly is the > > time spent on? > > I will probably try to do some 'real' study of this problem by using > perf or other tool. So far all I know is that the device enumeration > (i.e. device_enumerate() function in src/core/device) takes around 100ms > on my arm devices. As I noted before, around 80 devices are > enumerated. 80 devices sounds like a lot? On my beefy laptop here i have like 50, how come it's so many on some ARM device? I'd expect like 10 or so... normally it should just be a handful of block devices, ttys and network devices. You got so many of them? > > Hmm, so some people appear to use this, since we recently fixed a bug > > in this area that people noticed while making us of this... > > Of course, someone can use it and it seems reasonable to have the > ability for device units in user session. I would argue, however, that > having the same set of devices processed for both 'worlds' is sub-optimal. > Why not have separate tag in udev for 'user' devices, that would be > enumerated by 'systemd --user'? Hmm, we recently went the other way, and made the .device handling in the --user instance more like the one in the system instance. But you do have a point, it might be worth adding a more resricted tag there (after all, unpriv userspace should never need to know anything about block devices, as one example). But also: why is this so slow and why are so many devices tagged for you? to keep things simple i'd add a separate tag only as last resort and rather see slow stuff improved... Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] device enumeration by systemd user instances
On 7/23/19 18:00, Lennart Poettering wrote: > > Do you have some profiler results about this? i.e. what exactly is the > time spent on? I will probably try to do some 'real' study of this problem by using perf or other tool. So far all I know is that the device enumeration (i.e. device_enumerate() function in src/core/device) takes around 100ms on my arm devices. As I noted before, around 80 devices are enumerated. > > [...] > > Hmm, so some people appear to use this, since we recently fixed a bug > in this area that people noticed while making us of this... Of course, someone can use it and it seems reasonable to have the ability for device units in user session. I would argue, however, that having the same set of devices processed for both 'worlds' is sub-optimal. Why not have separate tag in udev for 'user' devices, that would be enumerated by 'systemd --user'? > >> So here is the question: is this a safe thing to do? I wonder if trying >> to start units without having device units created would break something >> in systemd. > > Well, the work needs to be done anyway, so what do you gain if you do > it a bit later? The same thing that we already get for starting units: the ability to do that in parallel with other things. > > What specifically do you try to optimize? latency until the first > process is started by systemd --user? I think the main goal is starting all units in user session as fast as possible. Thanks, Paweł Szewczyk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] device enumeration by systemd user instances
On Thu, Jul 25, 2019 at 05:29:33PM +0200, Pawel Szewczyk wrote: > On 7/17/19 23:14, Greg KH wrote: > > > > 100ms seems like a really long time, what exactly is it doing during > > that time? Is the kernel spending too much time regenerating the > > uevents? > > > > How many devices are you talking about? Any chance to see what really > > is happening here by running perf to see where the hot-spots are? > > There are ~80 devices being enumerated and 1ms per device still seem a > little too long. Note, that we are running this on arm architecture. > > Maybe using perf or other tools to find out what exactly is taking this > time is a good idea (never used perf to be honest). For now just by > adding some verbose logs it seems that processing of each device > actually takes around 1ms. Run perf and see what is going on, that seems like way too long per device. thanks, greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] device enumeration by systemd user instances
On 7/17/19 23:14, Greg KH wrote: > > 100ms seems like a really long time, what exactly is it doing during > that time? Is the kernel spending too much time regenerating the > uevents? > > How many devices are you talking about? Any chance to see what really > is happening here by running perf to see where the hot-spots are? There are ~80 devices being enumerated and 1ms per device still seem a little too long. Note, that we are running this on arm architecture. Maybe using perf or other tools to find out what exactly is taking this time is a good idea (never used perf to be honest). For now just by adding some verbose logs it seems that processing of each device actually takes around 1ms. Regards, Paweł Szewczyk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] device enumeration by systemd user instances
On Mi, 17.07.19 17:46, Pawel Szewczyk (p.szewc...@samsung.com) wrote: > Dear all > > I was recently working on reducing booting time and I see one issue that > I think is worth discussing. > > The problem is the device units enumeration seems to take a long time > (>100ms in our case) before any units are beginning to start. I think > there is some potential for improvement there. Do you have some profiler results about this? i.e. what exactly is the time spent on? Note that systemd generally uses udev "tags" for filtering, which means we should only processes that are actually supposed to show up as .device units, and nothing else. Before optimizing this, pleas find out though what the time is actually spent on. > I see that on the system I use for testing the user instance of > systemd is doing an enumeration of devices as well. Since no matter > if the systemd is pid 1 or user instance, it loads the same set of > units which are tagged 'systemd' in udev, if I understand > correctly. In user session, however, no unit seems to depend on > device units, so the whole effort of creating them is for > nothing. My guess is that for many use cases the device units are > not needed by session services. Hmm, so some people appear to use this, since we recently fixed a bug in this area that people noticed while making us of this... > So here is the question: is this a safe thing to do? I wonder if trying > to start units without having device units created would break something > in systemd. Well, the work needs to be done anyway, so what do you gain if you do it a bit later? What specifically do you try to optimize? latency until the first process is started by systemd --user? Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] device enumeration by systemd user instances
On Wed, Jul 17, 2019 at 05:46:58PM +0200, Pawel Szewczyk wrote: > Dear all > > I was recently working on reducing booting time and I see one issue that > I think is worth discussing. > > The problem is the device units enumeration seems to take a long time > (>100ms in our case) before any units are beginning to start. 100ms seems like a really long time, what exactly is it doing during that time? Is the kernel spending too much time regenerating the uevents? How many devices are you talking about? Any chance to see what really is happening here by running perf to see where the hot-spots are? thanks, greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] device enumeration by systemd user instances
Dear all I was recently working on reducing booting time and I see one issue that I think is worth discussing. The problem is the device units enumeration seems to take a long time (>100ms in our case) before any units are beginning to start. I think there is some potential for improvement there. I see that on the system I use for testing the user instance of systemd is doing an enumeration of devices as well. Since no matter if the systemd is pid 1 or user instance, it loads the same set of units which are tagged 'systemd' in udev, if I understand correctly. In user session, however, no unit seems to depend on device units, so the whole effort of creating them is for nothing. My guess is that for many use cases the device units are not needed by session services. One of the options that I can think of is to delay the coldplug a bit and do it in paralel with all the units loading. As for now, the device enumeration happens at the very beginning of systemd lifecycle, along with other unit loading. But comparing to loading unit files this looks like a time-consuming operation. So here is the question: is this a safe thing to do? I wonder if trying to start units without having device units created would break something in systemd. And did anyone considered this issue before? I would be happy to hear your ideas. Best Regards, Paweł Szewczyk ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
