open_transformer fork, depmod

2014-11-25 Thread Lauri Kasanen
Hi,

Can anyone fill me in on the reason why open_transformer always forks,
instead of straight decompressing?

I found the original commit from 2003:
http://lists.busybox.net/pipermail/busybox-cvs/2003-November/019040.html
But it doesn't explain why the move was done.

I ask because doing a depmod -a with gzipped modules results in over
800 forks, which is surely not optimal neither from memory nor from
performance points of view.

In the case of gzip, the uncompressed size is available, so there would
be no RAM waste in allocating the maximum specified 64mb first. For
compression formats that do not include the uncompressed size, the
fork+pipe approach does save RAM, but for gzip it seems to be needless
overhead.

- Lauri

-- 
http://www.fastmail.com - Send your email first class

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox


Re: open_transformer fork, depmod

2014-11-25 Thread walter harms


Am 25.11.2014 18:25, schrieb Lauri Kasanen:
 Hi,
 
 Can anyone fill me in on the reason why open_transformer always forks,
 instead of straight decompressing?
 
 I found the original commit from 2003:
 http://lists.busybox.net/pipermail/busybox-cvs/2003-November/019040.html
 But it doesn't explain why the move was done.
 
 I ask because doing a depmod -a with gzipped modules results in over
 800 forks, which is surely not optimal neither from memory nor from
 performance points of view.
 
 In the case of gzip, the uncompressed size is available, so there would
 be no RAM waste in allocating the maximum specified 64mb first. For
 compression formats that do not include the uncompressed size, the
 fork+pipe approach does save RAM, but for gzip it seems to be needless
 overhead.
 
 - Lauri
 

hi Lauri,
so far i understand is the code idea to pipe the daten before doing the work,
fine for tar (like: gzcat file | tar -xzf-)
i guess that was done to avoid creating an gunzip function call.
In the moddep case this will result in calling the gzcat everytime a compressed 
module is
found. Maybe you can make a special function that extracts only the required 
information
and the drops the rest.

re,
 wh


___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox