Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-05 Thread Vincent Danjean
Josselin Mouette wrote: > Le lundi 05 janvier 2009 à 00:58 +0100, Samuel Thibault a écrit : >> You mean Scheduler Activations? There's a patch against linux 2.4 ;) >> We're definitely diving into OS research :) > > Well it would be nice if things that was research at the time of Linux > 2.4 could

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-05 Thread Josselin Mouette
Le lundi 05 janvier 2009 à 00:58 +0100, Samuel Thibault a écrit : > You mean Scheduler Activations? There's a patch against linux 2.4 ;) > We're definitely diving into OS research :) Well it would be nice if things that was research at the time of Linux 2.4 could have turned into usable code now

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Paul Wise
On Mon, Jan 5, 2009 at 8:49 AM, Samuel Thibault wrote: > That's precisely the kind of thing that makes it better to just leave it > up to Linux. The case of HPC is quite particular in that you usually > really precisely control your computation. In the case of > general-purpose tools, I would r

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Mon 05 Jan 2009 00:47:02 +0100, a écrit : > There is probably a missing piece here. If you start several pigz > processes, the kernel only sees processes starting a lot of threads, and > processes only see a given number of cores. There is no interface that > allows a process t

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Ron Johnson, le Sun 04 Jan 2009 17:40:08 -0600, a écrit : > On 01/04/09 17:20, Josselin Mouette wrote: > >Still, it is better to use CPU pinning since you often want finer > >control than that, and that’s especially true in multi-user environments > >where resources can be sub-host. > > Wouldn't i

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le lundi 05 janvier 2009 à 00:38 +0100, Samuel Thibault a écrit : > Sure, but that should be only a user-explicitely-wanting thing. I would > really not like to see pigz systematically bind threads. What if I e.g. > want to run several pigz processes at the same time because I have a lot > of cor

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Ron Johnson
On 01/04/09 17:20, Josselin Mouette wrote: Le dimanche 04 janvier 2009 à 23:45 +0100, Samuel Thibault a écrit : It’s already the case in HPC environments, and CPU pinning is certainly going to be used more widely as the number of cores increases. And that's a shame. Linux shouldn't be so happy

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Mon 05 Jan 2009 00:20:42 +0100, a écrit : > Samuel Thibault, le Sun 04 Jan 2009 23:45:22 +0100, a écrit : > > > It’s already the case in HPC environments, and CPU pinning is certainly > > > going to be used more widely as the number of cores increases. > > > > And that's a sha

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le dimanche 04 janvier 2009 à 23:45 +0100, Samuel Thibault a écrit : > > It’s already the case in HPC environments, and CPU pinning is certainly > > going to be used more widely as the number of cores increases. > > And that's a shame. Linux shouldn't be so happy to move tasks between > CPUs...

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Samuel Thibault
Josselin Mouette, le Sun 04 Jan 2009 16:07:25 +0100, a écrit : > Le dimanche 04 janvier 2009 à 15:49 +0100, Eduard Bloch a écrit : > > Sounds like a plan, but I don't feel very comfortable to do that in the > > Debian package. Let me explain why: > > > > - sched_setaffinity method seems to be Lin

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Josselin Mouette
Le dimanche 04 janvier 2009 à 15:49 +0100, Eduard Bloch a écrit : > Sounds like a plan, but I don't feel very comfortable to do that in the > Debian package. Let me explain why: > > - sched_setaffinity method seems to be Linux specific How is that a problem? You only need to use it in Linux buil

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Eduard Bloch
#include * Guus Sliepen [Sun, Jan 04 2009, 10:45:23AM]: > On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote: > > > PS: I plan to hack it a little bit and use syssconf function on Debian > > systems to determine the real number of CPU cores (#x) since pigz's > > default value is 8 which

Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-04 Thread Guus Sliepen
On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote: > PS: I plan to hack it a little bit and use syssconf function on Debian > systems to determine the real number of CPU cores (#x) since pigz's > default value is 8 which is much more than home systems have nowadays, > and the performanc

Bug#510624: ITP: pigz -- Parallel Implementation of GZip

2009-01-03 Thread Eduard Bloch
Package: wnpp Severity: wishlist Owner: Eduard Bloch * Package name: pigz Version : 2.1.4 Upstream Author : Mark Adler * URL : http://www.example.org/ * License : ZLib license Programming Lang: C Description : Parallel Implementation of GZip pigz, wh