Re: [xz-devel] Can "xz -T0 SMALLFILE1 SMALLFILE2 ... FILE99" be made multi-threaded?

2018-06-24 Thread James Cloos
Modern gnu xargs(1) supports --max-procs/-P to run parallel invocations of the target command. Or, gnu parallel(1) is designed from the start to run its target command in parallel. Modern bsd xargs(1) also supports -P. See their man pages for details. -JimC -- James Cloos OpenPGP: 0x9

Re: [xz-devel] Can "xz -T0 SMALLFILE1 SMALLFILE2 ... FILE99" be made multi-threaded?

2018-06-15 Thread Lasse Collin
On 2018-06-15 Denys Vlasenko wrote: > I noticed that one of my build scripts has this: > > find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs xz > > A rather typical usage, right? > > In my case, this takes several minutes to complete, > as there are 4000+ modules. The quick check co

Re: [xz-devel] Can "xz -T0 SMALLFILE1 SMALLFILE2 ... FILE99" be made multi-threaded?

2018-06-15 Thread John Reiser
find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs xz Shorten each piped pathname so that fewer invocations of xz are required: ( cd $RPM_BUILD_ROOT/lib/modules/; find . -type f -name '*.ko' | xargs xz ) --

Re: [xz-devel] Can "xz -T0 SMALLFILE1 SMALLFILE2 ... FILE99" be made multi-threaded?

2018-06-15 Thread John Reiser
find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs xz The quick check confirms that xz runs single-threaded [even with -T0]. A shell script can implement parallelism for cases such as this. A similar task scheme appears in dracut. At one time [~2014] these files: 90kernel-mod