Re: [PATCH 0/3] mdev: add daemon mode
If mdev is used as kernel hotplug helper and the system generates many hotplug events it will quickly consume considerable resources because a process is forked for each event and, if the mdev.seq feature is used, they must also coordinate among each other. While the uevent applet mitigates some of the resource problems you still have to pay the cost for a fork/exec and parsing of mdev.conf for each event. Doing it that way spares the fork/exec indeed, but you're still parsing mdev.conf for every event. The parsing is done in make_device(), which is called at least once per event. And there's no way to prevent that - that's just how mdev is structured. To parse mdev.conf only once and have a really fast code path on event reception, you'd basically need to rewrite mdev entirely. Are you aware of https://skarnet.org/software/mdevd/ ? :) -- Laurent ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 3/3] mdev: add support to run as daemon
On Sat, May 04, 2019 at 08:06:44AM +0200, Peter Korsgaard wrote: > > "Jan" == Jan Klötzke writes: > > > Adds the -s option to run mdev in daemon mode handling hotplug events > > from the kernel like udev. If the system generates many hotplug events > > this mode of operation will consume less resources than registering > > mdev as hotplug helper or using the uevent applet. > > -s is already scan. Did you mean -d? m( sure. Will be fixed in v2... Regards, Jan ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 0/3] mdev: add daemon mode
On Fri, May 03, 2019 at 11:16:34PM +0200, Markus Gothe wrote: > Err, zram ofc :) ... > > Or even better you could turn on zswap on your embedded device and avoid the > OOM-kiler which I guess is the real reason you fixed this. Actually RAM was not the problem at all. It really was the time to spawn all the processes and the overhead to serialize them. > Actually I want to know when this happens that the kernel creates that many > hotplug events; seems like the issue is somewhere else. We're building an IVI (In Vehicle Infotainment) system that needs to boot really fast. For example we have to show a rear view camera in 2s from cold boot. To achive this the kernel is kept very small and most of the drivers are loaded as modules when they are actually needed. This alone creates so many events that the impact is measurable. Regards, Jan > > //M > > Sent from my BlackBerry — the most secure mobile device > > > Original Message > > > > From: j...@kloetzke.net > Sent: 3 May 2019 23:03 > To: busybox@busybox.net > Subject: [PATCH 0/3] mdev: add daemon mode > > > If mdev is used as kernel hotplug helper and the system generates many > hotplug events it will quickly consume considerable resources because a > process is forked for each event and, if the mdev.seq feature is used, > they must also coordinate among each other. While the uevent applet > mitigates some of the resource problems you still have to pay the cost > for a fork/exec and parsing of mdev.conf for each event. > > The following patches add an optional daemon mode to mdev that let's > mdev listen on a netlink socket for kernel hotplug events. This mode > works much like udev and avoids the fork/exec/parse overhead on many > events. > > > ___ > busybox mailing list > busybox@busybox.net > http://lists.busybox.net/mailman/listinfo/busybox ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 0/3] mdev: add daemon mode
On Sat, May 04, 2019 at 07:42:06AM +, Laurent Bercot wrote: > > > If mdev is used as kernel hotplug helper and the system generates many > > hotplug events it will quickly consume considerable resources because a > > process is forked for each event and, if the mdev.seq feature is used, > > they must also coordinate among each other. While the uevent applet > > mitigates some of the resource problems you still have to pay the cost > > for a fork/exec and parsing of mdev.conf for each event. > > Doing it that way spares the fork/exec indeed, but you're still > parsing mdev.conf for every event. The parsing is done in > make_device(), which is called at least once per event. And there's > no way to prevent that - that's just how mdev is structured. > To parse mdev.conf only once and have a really fast code path on > event reception, you'd basically need to rewrite mdev entirely. Actually that is not the case anymore. There was already code for the '-s' case that parsed mdev.conf only once and stored the rules in G.rule_vec. The daemon mode just reuses this. Maybe it could be optimized further but it was sufficient for us. > Are you aware of https://skarnet.org/software/mdevd/ ? :) Oh, actually not. Looks indeed interesting. I'll have a look... Regards, Jan ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH 0/3] mdev: add daemon mode
Ouch, then fork()'ing is indeed an issue. And Linux not being an RTOS is another issue, but I hope you are using an RTOS (like QNX or FreeRTOS) and not Linux. As pointed out the whole Linux hotplugging system is the real issue here and something like mdevd would be the answer. //M Sent from my BlackBerry — the most secure mobile device Original Message From: j...@kloetzke.net Sent: 4 May 2019 10:49 To: nietzs...@lysator.liu.se Cc: busybox@busybox.net Subject: Re: [PATCH 0/3] mdev: add daemon mode On Fri, May 03, 2019 at 11:16:34PM +0200, Markus Gothe wrote: > Err, zram ofc :) ... > > Or even better you could turn on zswap on your embedded device and avoid the > OOM-kiler which I guess is the real reason you fixed this. Actually RAM was not the problem at all. It really was the time to spawn all the processes and the overhead to serialize them. > Actually I want to know when this happens that the kernel creates that many > hotplug events; seems like the issue is somewhere else. We're building an IVI (In Vehicle Infotainment) system that needs to boot really fast. For example we have to show a rear view camera in 2s from cold boot. To achive this the kernel is kept very small and most of the drivers are loaded as modules when they are actually needed. This alone creates so many events that the impact is measurable. Regards, Jan > > //M > > Sent from my BlackBerry — the most secure mobile device > > > Original Message > > > > From: j...@kloetzke.net > Sent: 3 May 2019 23:03 > To: busybox@busybox.net > Subject: [PATCH 0/3] mdev: add daemon mode > > > If mdev is used as kernel hotplug helper and the system generates many > hotplug events it will quickly consume considerable resources because a > process is forked for each event and, if the mdev.seq feature is used, > they must also coordinate among each other. While the uevent applet > mitigates some of the resource problems you still have to pay the cost > for a fork/exec and parsing of mdev.conf for each event. > > The following patches add an optional daemon mode to mdev that let's > mdev listen on a netlink socket for kernel hotplug events. This mode > works much like udev and avoids the fork/exec/parse overhead on many > events. > > > ___ > busybox mailing list > busybox@busybox.net > http://lists.busybox.net/mailman/listinfo/busybox ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
