Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-30 Thread Stephen Montgomery-Smith
On Wed, 30 May 2007, Bakul Shah wrote: Peter Jeremy <[EMAIL PROTECTED]> wrote: On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: Given the size and complexity of the port system I have long felt that rather than do everything via more and more complex Mk/*.mk what is is ne

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-30 Thread Bakul Shah
Peter Jeremy <[EMAIL PROTECTED]> wrote: > On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: > >Given the size and complexity of the port system I have long > >felt that rather than do everything via more and more complex > >Mk/*.mk what is is needed is a ports server and a thin C

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Jeremy Chadwick
On Tue, May 29, 2007 at 08:34:29PM +1000, Peter Jeremy wrote: > On 2007-May-27 15:30:48 -0700, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: > >This sounds like a good solution. In fact, I'm lead to believe that > >heavy reliance on /bin/sh is part of why the ports collection is slow. > > Someone ne

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Joan Picanyol i Puig
* Hartmut Brandt <[EMAIL PROTECTED]> [20070529 08:21]: > I've seen no numbers WHAT actually makes the ports stuff so slow. To > make my point a last time: until there are numbers, there is no guess > around what to do. It just occured to me that DTrace could be a big help with this task. I sugge

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread soralx
Ivan Voras <[EMAIL PROTECTED]> wrote: > > Because the information is not a constant. For example, the mpg123 > > port changes its PKGNAME as soon as esound is installed. > > Maybe the time has come to give up on some of the flexibility the > ports tree has (and this particular one is confusing t

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 15:30:48 -0700, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: >This sounds like a good solution. In fact, I'm lead to believe that >heavy reliance on /bin/sh is part of why the ports collection is slow. Someone needs to enable accounting on a recent -current (with the high-resolution

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 15:52:16 -0500, Stephen Montgomery-Smith <[EMAIL PROTECTED]> wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. No

Re: Looking for speed increases in "make index"

2007-05-29 Thread Peter Jeremy
On 2007-May-28 00:15:28 +0200, Michel Talon <[EMAIL PROTECTED]> wrote: >Of course a lot of people have thinked about it, and quickly realized >that it was not going to work. In the bsd.ports.mk, evaluation of one >variable may be dependent on some conditional, which may itself be >dependant on some

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: >Given the size and complexity of the port system I have long >felt that rather than do everything via more and more complex >Mk/*.mk what is is needed is a ports server and a thin CLI >frontend to it. I don't believe this is pra

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsin

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Rick C. Petty
On Sun, May 27, 2007 at 03:30:48PM -0700, Jeremy Chadwick wrote: > > That said, I'll ask this out in the open: am I the only one who sees the > benefit of GNU make in regards to this? There's a lot of built-in > functions in GNU make which could help in regards to ports. I have no > qualms with

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Stephen Montgomery-Smith wrote: Roman Divacky wrote: On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "mak

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Roman Divacky wrote: On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and thin

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Alexander Nedotsukov
Correct me if I wrong. Don't you missed the fact that chdir(2) changes process wide attribute? Though it's easy to fix with -C option. Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot abo

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Mike Meyer
In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > Mike Meyer wrote: > > In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > >> 1. make and its sub-makes for a) reading the file; b) parsing the file > >> (note that .if and .for processing is done while parsing); c)

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Roman Divacky
On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: > Jeremy Chadwick wrote: > >On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: > >> I have been thinking a lot about looking for speed increases for "make > >> index" and pkg_version and things like th

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone over already. See http:/

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Ivan Voras
Stephen Montgomery-Smith wrote: > Ivan Voras wrote: >> As long as far-out ideas are being discussed, how about caching such >> information (including dependenices) in a file (I'd call it a database >> but then I'd had to start a holy war :) ) so it's calculated only once, >> preferably on the port

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Garrett Cooper wrote: Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone o

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone over already. See http:/

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Mike Meyer
In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > 1. make and its sub-makes for a) reading the file; b) parsing the file > (note that .if and .for processing is done while parsing); c) processing > targets. Make and submakes have been gone over already. See http://miller.emu.id.a

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
On Mon, 28 May 2007, David Naylor wrote: On Monday 28 May 2007 03:43, you wrote: Maybe I should look at the inner workings of cmake and gmake. Maybe they have some good ideas. However having looked through the source code of make, and also looking at the cvs logs, it does seem to be well wr

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Matthew Seaman
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ivan Voras wrote: > Stephen Montgomery-Smith wrote: >> I have been thinking a lot about looking for speed increases for "make >> index" and pkg_version and things like that. So for example, in >> pkg_version, it calls "make -V PKGNAME" for every ins

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread David Naylor
On Monday 28 May 2007 03:43, you wrote: > Maybe I should look at the inner workings of cmake and gmake. Maybe > they have some good ideas. However having looked through the source > code of make, and also looking at the cvs logs, it does seem to be well > written. The only possibility I see of m

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Ivan Voras wrote: Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Ivan Voras
Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "make -V PKGNAME" should be a speedy operation, bu

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Stephen Montgomery-Smith wrote: Hartmut Brandt wrote: Having done a great deal of rewriting of make some two years ago I can tell you that even a small change to make is a tough job testing-wise: run all the combinations of !-j and -j on all architectures and run the change through the port-bu

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Hartmut Brandt wrote: Having done a great deal of rewriting of make some two years ago I can tell you that even a small change to make is a tough job testing-wise: run all the combinations of !-j and -j on all architectures and run the change through the port-building cluster. That's a warning

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy operation, but th

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Matthew Seaman wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every install

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Matthew Seaman
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "

Re: Looking for speed increases in "make index"

2007-05-27 Thread Edwin Groothuis
On Sun, May 27, 2007 at 10:51:56PM -0400, Mike Meyer wrote: > In <[EMAIL PROTECTED]>, Edwin Groothuis <[EMAIL PROTECTED]> typed: > > On Sun, May 27, 2007 at 08:44:56PM -0500, Mark Linimon wrote: > > > In summary, the ports infrastructure is really complicated because it's > > > trying to deal with

Re: Looking for speed increases in "make index"

2007-05-27 Thread Mike Meyer
In <[EMAIL PROTECTED]>, Edwin Groothuis <[EMAIL PROTECTED]> typed: > On Sun, May 27, 2007 at 08:44:56PM -0500, Mark Linimon wrote: > > In summary, the ports infrastructure is really complicated because it's > > trying to deal with all kinds of constraints and conditions. I challenge > Reading this

Re: Looking for speed increases in "make index"

2007-05-27 Thread Mark Linimon
On Mon, May 28, 2007 at 12:15:28AM +0200, Michel Talon wrote: > To gain some performance, a first idea would be to simplify > bsd.ports.mk. I am convinced that a substantial part of the 4000 lines > are historical crap which serve no useful purpose. 11272 of LOC in bsd.*.mk, but who's counting. >

Re: Looking for speed increases in "make index"

2007-05-27 Thread Edwin Groothuis
On Sun, May 27, 2007 at 08:44:56PM -0500, Mark Linimon wrote: > In summary, the ports infrastructure is really complicated because it's > trying to deal with all kinds of constraints and conditions. I challenge Reading this, I was wondering what the ports infrastructure has ever done for us? See

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
I'm looking for something that will work with the existing framework. But yes, I get the feeling that maybe using "make" to process the ports might be the source of the problem. Make is a program primarily designed for figuring out which was made first, the target or the source, but in the por

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed

Re: Looking for speed increases in "make index"

2007-05-27 Thread Stephen Montgomery-Smith
On Mon, 28 May 2007, Michel Talon wrote: Stephen Montgomery-Smith said: I suggest rewriting "make" so that variables are only evaluated on a "need to know" basis. or "I have tried to do this." Of course a lot of people have thinked about it, and quickly realized that it was not going

Re: Looking for speed increases in "make index"

2007-05-27 Thread Michel Talon
Stephen Montgomery-Smith said: > I suggest rewriting "make" so that variables are only evaluated on a > "need to know" basis. > or "I have tried to do this." Of course a lot of people have thinked about it, and quickly realized that it was not going to work. In the bsd.ports.mk, evaluation

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Bakul Shah
Not quite what you asked for but... Given the size and complexity of the port system I have long felt that rather than do everything via more and more complex Mk/*.mk what is is needed is a ports server and a thin CLI frontend to it. This server can store dependency data in an efficient manner,

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Jeremy Chadwick
On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "

Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy operation, but the make has to load in and analyz