Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-05 Thread Vincent Danjean
Josselin Mouette wrote: Le lundi 05 janvier 2009 à 00:58 +0100, Samuel Thibault a écrit : You mean Scheduler Activations? There's a patch against linux 2.4 ;) We're definitely diving into OS research :) Well it would be nice if things that was research at the time of Linux 2.4 could have

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-05 Thread Josselin Mouette
Le lundi 05 janvier 2009 à 00:58 +0100, Samuel Thibault a écrit : You mean Scheduler Activations? There's a patch against linux 2.4 ;) We're definitely diving into OS research :) Well it would be nice if things that was research at the time of Linux 2.4 could have turned into usable code now

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Guus Sliepen
On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote: PS: I plan to hack it a little bit and use syssconf function on Debian systems to determine the real number of CPU cores (#x) since pigz's default value is 8 which is much more than home systems have nowadays, and the performance

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Eduard Bloch
#include hallo.h * Guus Sliepen [Sun, Jan 04 2009, 10:45:23AM]: On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote: PS: I plan to hack it a little bit and use syssconf function on Debian systems to determine the real number of CPU cores (#x) since pigz's default value is 8 which

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le dimanche 04 janvier 2009 à 15:49 +0100, Eduard Bloch a écrit : Sounds like a plan, but I don't feel very comfortable to do that in the Debian package. Let me explain why: - sched_setaffinity method seems to be Linux specific How is that a problem? You only need to use it in Linux builds.

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Sun 04 Jan 2009 16:07:25 +0100, a écrit : Le dimanche 04 janvier 2009 à 15:49 +0100, Eduard Bloch a écrit : Sounds like a plan, but I don't feel very comfortable to do that in the Debian package. Let me explain why: - sched_setaffinity method seems to be Linux

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le dimanche 04 janvier 2009 à 23:45 +0100, Samuel Thibault a écrit : It’s already the case in HPC environments, and CPU pinning is certainly going to be used more widely as the number of cores increases. And that's a shame. Linux shouldn't be so happy to move tasks between CPUs...

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Mon 05 Jan 2009 00:20:42 +0100, a écrit : Samuel Thibault, le Sun 04 Jan 2009 23:45:22 +0100, a écrit : It’s already the case in HPC environments, and CPU pinning is certainly going to be used more widely as the number of cores increases. And that's a shame. Linux

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Ron Johnson
On 01/04/09 17:20, Josselin Mouette wrote: Le dimanche 04 janvier 2009 à 23:45 +0100, Samuel Thibault a écrit : It’s already the case in HPC environments, and CPU pinning is certainly going to be used more widely as the number of cores increases. And that's a shame. Linux shouldn't be so

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le lundi 05 janvier 2009 à 00:38 +0100, Samuel Thibault a écrit : Sure, but that should be only a user-explicitely-wanting thing. I would really not like to see pigz systematically bind threads. What if I e.g. want to run several pigz processes at the same time because I have a lot of cores

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Ron Johnson, le Sun 04 Jan 2009 17:40:08 -0600, a écrit : On 01/04/09 17:20, Josselin Mouette wrote: Still, it is better to use CPU pinning since you often want finer control than that, and that’s especially true in multi-user environments where resources can be sub-host. Wouldn't it be

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Mon 05 Jan 2009 00:47:02 +0100, a écrit : There is probably a missing piece here. If you start several pigz processes, the kernel only sees processes starting a lot of threads, and processes only see a given number of cores. There is no interface that allows a process to

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Paul Wise
On Mon, Jan 5, 2009 at 8:49 AM, Samuel Thibault samuel.thiba...@ens-lyon.org wrote: That's precisely the kind of thing that makes it better to just leave it up to Linux. The case of HPC is quite particular in that you usually really precisely control your computation. In the case of

Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-03 Thread Eduard Bloch
Package: wnpp Severity: wishlist Owner: Eduard Bloch bl...@debian.org * Package name: pigz Version : 2.1.4 Upstream Author : Mark Adler mad...@alumni.caltech.edu * URL : http://www.example.org/ * License : ZLib license Programming Lang: C Description :