Re: parallel builds revisited

2007-04-14 Thread Ulrich Spoerlein
Pav Lucistnik wrote: > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > > Some time ago, after buying a Core 2 Duo system, I've become interested > > in doing something about the inherent single-threadedness of the ports. > > Even though I have a dualcore machine, ports builds only ever u

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Sat, Apr 14, 2007 at 07:12:21AM +0200, [LoN]Kamikaze wrote: > Kris Kennaway wrote: > > On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > > > >>> Btw, do you think it's possible that a port can only be built with, n > >>> parallel make jobs, but will fail with n+1? > >> No. I do

Re: parallel builds revisited

2007-04-13 Thread [LoN]Kamikaze
Kris Kennaway wrote: > On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > >>> Btw, do you think it's possible that a port can only be built with, n >>> parallel make jobs, but will fail with n+1? >> No. I do not think this can be the case. > > It certainly is the case. If a makefi

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Fri, Apr 13, 2007 at 04:51:33PM -0700, [EMAIL PROTECTED] wrote: > My opinion is that there should be a threshold value empirically derived by > the developer / retrieved by bug reports, as well as a knob, to specify the > maximum number of parallel jobs to be used for a particular port, that

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > > Btw, do you think it's possible that a port can only be built with, n > > parallel make jobs, but will fail with n+1? > > No. I do not think this can be the case. It certainly is the case. If a makefile has incorrectly specifie

Re: parallel builds revisited

2007-04-13 Thread youshi10
On Fri, 13 Apr 2007, David Nečas (Yeti) wrote: On Fri, Apr 13, 2007 at 05:10:47PM +0200, Pav Lucistnik wrote: Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > > I was thinking about having it embedded in every port's Makefile > directly, instead. Something like > > USE_MAKE_JOBS= 2

Re: parallel builds revisited

2007-04-13 Thread [LoN]Kamikaze
Peter Jeremy wrote: > Can anyone come up with either examples of ports that fall into the > second category above or counter-examples to my first paragraph? I have used 'make -j' for quite some time, and to my experience it either works or doesn't. I've never seen a port that works sometimes. __

Re: parallel builds revisited

2007-04-13 Thread Peter Jeremy
On 2007-Apr-13 17:43:54 +0200, "David Ne?as (Yeti)" <[EMAIL PROTECTED]> wrote: >implements it. How many ports exist that can fail with N+1 >jobs yet cannot break with N jobs (for N > 1)? As far as I can see, for a port to be safely built with N>1 jobs then it needs to have proper dependency relat

Re: parallel builds revisited

2007-04-13 Thread Yeti
On Fri, Apr 13, 2007 at 05:10:47PM +0200, Pav Lucistnik wrote: > Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > > > > > > I was thinking about having it embedded in every port's Makefile > > > directly, instead. Something like > > > > > > USE_MAKE_JOBS=2 > > > > IMHO, hardcoding the

Re: parallel builds revisited

2007-04-13 Thread Pav Lucistnik
Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > On Tue, Apr 10, 2007 at 07:44:47PM +0200, Pav Lucistnik wrote: > > Benjamin Lutz p??e v ?t 10. 04. 2007 v 04:52 +0200: > [snip] > > > 3) Save this to /usr/local/etc/parallel_builds.conf: > > > http://www.maxlor.com/temp/parallel_builds.c

Re: parallel builds revisited

2007-04-13 Thread Peter Pentchev
On Tue, Apr 10, 2007 at 07:44:47PM +0200, Pav Lucistnik wrote: > Benjamin Lutz p??e v ?t 10. 04. 2007 v 04:52 +0200: [snip] > > 3) Save this to /usr/local/etc/parallel_builds.conf: > > http://www.maxlor.com/temp/parallel_builds.conf . > > This is a list of ports as stored in PKGORIGIN,

Re: parallel builds revisited

2007-04-12 Thread Coleman Kane
On Thu, 2007-04-12 at 21:07 +0200, Benjamin Lutz wrote: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > I dunno how you want to approach this, but gmake does recommend 2 > > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > > dual core machines. > > -Garrett > > S

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, Benjamin Lutz wrote: On Thursday 12 April 2007 22:20, Pav Lucistnik wrote: Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > I dunno how you want to approach this, but gmake does recommend 2 > jobs be run in parall

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, [LoN]Kamikaze wrote: Benjamin Lutz wrote: On Thursday 12 April 2007 11:06, Garrett Cooper wrote: I dunno how you want to approach this, but gmake does recommend 2 jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a dual core machines. -Garrett So far the

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 22:20, Pav Lucistnik wrote: > Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: > > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > > I dunno how you want to approach this, but gmake does recommend 2 > > > jobs be run in parallel for HTT enabled chips, and 3 o

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, Benjamin Lutz wrote: On Thursday 12 April 2007 18:32, [LoN]Kamikaze wrote: Robert Noland wrote: Have any of you looked at sysutils/bsdadminscripts, it's buildflags options allow for parallel builds as well as ccache / distcc use. I have a reasonable list of ports that m

Re: parallel builds revisited

2007-04-12 Thread Pav Lucistnik
Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > I dunno how you want to approach this, but gmake does recommend 2 > > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > > dual core machines. > > -Garrett > > So fa

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Benjamin Lutz wrote: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: >> I dunno how you want to approach this, but gmake does recommend 2 >> jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a >> dual core machines. >> -Garrett > > So far the approach is one job per CPU.

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > I dunno how you want to approach this, but gmake does recommend 2 > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > dual core machines. > -Garrett So far the approach is one job per CPU. I'll do some benchmarks lateron

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Benjamin Lutz wrote: > I've looked at your patches and programs, and I'm starting to have a > fairly clear idea of solution should look like. I would like to see: > > * Integration into the existing ports framework. No new scripts or files > should be required. The whitelist file needs to go.

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 18:32, [LoN]Kamikaze wrote: > Robert Noland wrote: > > Have any of you looked at sysutils/bsdadminscripts, it's buildflags > > options allow for parallel builds as well as ccache / distcc use. > > I have a reasonable list of ports that must have some or all of > > these o

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Robert Noland wrote: > On Thu, 2007-04-12 at 05:43 +0200, Benjamin Lutz wrote: >> Hello Pav, >> >> On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: >>> Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: Some time ago, after buying a Core 2 Duo system, I've become interested in doing

Re: parallel builds revisited

2007-04-12 Thread Robert Noland
On Thu, 2007-04-12 at 05:43 +0200, Benjamin Lutz wrote: > Hello Pav, > > On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: > > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > > Some time ago, after buying a Core 2 Duo system, I've become > > > interested in doing something about the inh

Re: parallel builds revisited

2007-04-12 Thread Garrett Cooper
Pav Lucistnik wrote: Benjamin Lutz píše v čt 12. 04. 2007 v 05:43 +0200: Is there any detailed information available on what's planned here that isn't in your description on SoC page? I don't know if I can forward you the proposal text, they might not be public. But it's basically co

Re: parallel builds revisited

2007-04-12 Thread Pav Lucistnik
Benjamin Lutz píše v čt 12. 04. 2007 v 05:43 +0200: > Is there any detailed information available on what's planned here > that > isn't in your description on SoC page? I don't know if I can forward you the proposal text, they might not be public. But it's basically copy&paste from the Ideas pag

Re: parallel builds revisited

2007-04-11 Thread Benjamin Lutz
Hello Pav, On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > Some time ago, after buying a Core 2 Duo system, I've become > > interested in doing something about the inherent > > single-threadedness of the ports. Even though I have a dua

Re: parallel builds revisited

2007-04-10 Thread Pav Lucistnik
Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > Some time ago, after buying a Core 2 Duo system, I've become interested > in doing something about the inherent single-threadedness of the ports. > Even though I have a dualcore machine, ports builds only ever use one > core. I started think

parallel builds revisited

2007-04-09 Thread Benjamin Lutz
Hello, Some time ago, after buying a Core 2 Duo system, I've become interested in doing something about the inherent single-threadedness of the ports. Even though I have a dualcore machine, ports builds only ever use one core. I started thinking about various approaches to introduce parallelis