Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-30 Thread Stephen Montgomery-Smith
On Wed, 30 May 2007, Bakul Shah wrote: Peter Jeremy <[EMAIL PROTECTED]> wrote: On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: Given the size and complexity of the port system I have long felt that rather than do everything via more and more complex Mk/*.mk what is is ne

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-30 Thread Bakul Shah
Peter Jeremy <[EMAIL PROTECTED]> wrote: > On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: > >Given the size and complexity of the port system I have long > >felt that rather than do everything via more and more complex > >Mk/*.mk what is is needed is a ports server and a thin C

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Jeremy Chadwick
On Tue, May 29, 2007 at 08:34:29PM +1000, Peter Jeremy wrote: > On 2007-May-27 15:30:48 -0700, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: > >This sounds like a good solution. In fact, I'm lead to believe that > >heavy reliance on /bin/sh is part of why the ports collection is slow. > > Someone ne

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 15:30:48 -0700, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: >This sounds like a good solution. In fact, I'm lead to believe that >heavy reliance on /bin/sh is part of why the ports collection is slow. Someone needs to enable accounting on a recent -current (with the high-resolution

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 15:52:16 -0500, Stephen Montgomery-Smith <[EMAIL PROTECTED]> wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. No

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-29 Thread Peter Jeremy
On 2007-May-27 16:12:54 -0700, Bakul Shah <[EMAIL PROTECTED]> wrote: >Given the size and complexity of the port system I have long >felt that rather than do everything via more and more complex >Mk/*.mk what is is needed is a ports server and a thin CLI >frontend to it. I don't believe this is pra

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsin

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Stephen Montgomery-Smith wrote: Roman Divacky wrote: On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "mak

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Roman Divacky wrote: On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and thin

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Alexander Nedotsukov
Correct me if I wrong. Don't you missed the fact that chdir(2) changes process wide attribute? Though it's easy to fix with -C option. Stephen Montgomery-Smith wrote: Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot abo

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Mike Meyer
In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > Mike Meyer wrote: > > In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > >> 1. make and its sub-makes for a) reading the file; b) parsing the file > >> (note that .if and .for processing is done while parsing); c)

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Roman Divacky
On Mon, May 28, 2007 at 11:34:24AM -0500, Stephen Montgomery-Smith wrote: > Jeremy Chadwick wrote: > >On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: > >> I have been thinking a lot about looking for speed increases for "make > >> index" and pkg_version and things like th

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone over already. See http:/

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Garrett Cooper wrote: Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone o

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Mike Meyer
In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: > 1. make and its sub-makes for a) reading the file; b) parsing the file > (note that .if and .for processing is done while parsing); c) processing > targets. Make and submakes have been gone over already. See http://miller.emu.id.a

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Garrett Cooper
Mike Meyer wrote: In <[EMAIL PROTECTED]>, Hartmut Brandt <[EMAIL PROTECTED]> typed: 1. make and its sub-makes for a) reading the file; b) parsing the file (note that .if and .for processing is done while parsing); c) processing targets. Make and submakes have been gone over already. See http:/

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Stephen Montgomery-Smith wrote: Hartmut Brandt wrote: Having done a great deal of rewriting of make some two years ago I can tell you that even a small change to make is a tough job testing-wise: run all the combinations of !-j and -j on all architectures and run the change through the port-bu

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Stephen Montgomery-Smith
Hartmut Brandt wrote: Having done a great deal of rewriting of make some two years ago I can tell you that even a small change to make is a tough job testing-wise: run all the combinations of !-j and -j on all architectures and run the change through the port-building cluster. That's a warning

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy operation, but th

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Hartmut Brandt
Matthew Seaman wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every install

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-28 Thread Matthew Seaman
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
I'm looking for something that will work with the existing framework. But yes, I get the feeling that maybe using "make" to process the ports might be the source of the problem. Make is a program primarily designed for figuring out which was made first, the target or the source, but in the por

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
Jeremy Chadwick wrote: On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Bakul Shah
Not quite what you asked for but... Given the size and complexity of the port system I have long felt that rather than do everything via more and more complex Mk/*.mk what is is needed is a ports server and a thin CLI frontend to it. This server can store dependency data in an efficient manner,

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Jeremy Lea
Hi, On Sun, May 27, 2007 at 03:30:48PM -0700, Jeremy Chadwick wrote: > Does it need to be done this way? Can we just iterate through all of > the ports, call make -V _DEPEND_DIRS, then sort | uniq the results? This is exactly what ALL-DEPENDS-LIST does. Except it's faster. It keeps two lists a

Re: Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Jeremy Chadwick
On Sun, May 27, 2007 at 03:52:16PM -0500, Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "

Looking for speed increases in "make index" and pkg_version for ports

2007-05-27 Thread Stephen Montgomery-Smith
I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy operation, but the make has to load in and analyz