Re: More speed increases for make-ing ports
Stephen Montgomery-Smith wrote:
Ulrich Spoerlein wrote:
The question is: Has it been fixed in all Branches that are currently
supported by the Ports tree? What about 5.x?
Anyway, if you are running 5.x, why not try it out? Try makefile like:
a=abc
all:
echo ${a:Mab}, ${a:Mabc}, ${a:Mabcd}
You should get an output like
echo ,abc,
,abc,
I get this output from an old 4.10 system. So I think we should be safe
...
Ulrich Spoerlein
--
The trouble with the dictionary is you have to know how the word is
spelled before you can look it up to see how it is spelled.
-- Will Cuppy
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith wrote: That makes perfect sense. It does look like you had to work around a bug in make. I have actually looked at the code in make where it does the :M (it is the function Str_Match is str.c) and this bug has clearly been fixed now. The question is: Has it been fixed in all Branches that are currently supported by the Ports tree? What about 5.x? Ulrich Spoerlein -- The trouble with the dictionary is you have to know how the word is spelled before you can look it up to see how it is spelled. -- Will Cuppy ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Ulrich Spoerlein wrote:
Stephen Montgomery-Smith wrote:
That makes perfect sense. It does look like you had to work around a bug in make. I have
actually looked at the code in make where it does the :M (it is the function Str_Match is
str.c) and this bug has clearly been fixed now.
The question is: Has it been fixed in all Branches that are currently
supported by the Ports tree? What about 5.x?
Ulrich Spoerlein
I looked through the cvs logs for make, and I couldn't see when make had
this bug, or when it had been fixed. I do recall Joe sounding uncertain
as to whether this was really the problem, as it is something he did a
few years ago. For example, maybe this was a bug in Solaris make that
he was remembering, and at the time he committed the change he didn't
realize that FreeBSD make was correct in this regard.
Anyway, if you are running 5.x, why not try it out? Try makefile like:
a=abc
all:
echo ${a:Mab}, ${a:Mabc}, ${a:Mabcd}
You should get an output like
echo ,abc,
,abc,
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
Prostý textový dokument příloha (eee)
--- bsd.gnome.mk-orig Mon Mar 19 23:13:36 2007
+++ bsd.gnome.mk Wed May 23 19:25:19 2007
@@ -689,9 +689,9 @@
# Then traverse through all components, check which of them
# exist in ${_USE_GNOME} and set variables accordingly
+.ifdef _USE_GNOME
. for component in ${_USE_GNOME_ALL}
-_COMP_TEST= ${_USE_GNOME:M${component}}
-. if ${_COMP_TEST:S/${component}//}!=${_COMP_TEST:S/ / /g}
+. if ${_USE_GNOME:M${component}}!=
PATCH_DEPENDS+= ${${component}_PATCH_DEPENDS}
FETCH_DEPENDS+= ${${component}_FETCH_DEPENDS}
EXTRACT_DEPENDS+=${${component}_EXTRACT_DEPENDS}
@@ -725,6 +725,7 @@
. endif
. endfor
+.endif
.endif
.if defined(GNOME_PRE_PATCH)
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
The Linimon's Rule: The More You Close, The More Will Come
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
On 5/24/07, Mark Linimon [EMAIL PROTECTED] wrote: On Tue, May 22, 2007 at 05:34:32PM +0400, Andrew Pantyukhin wrote: A seemingly better way may be to make these system vars available in make by default. Doesn't help anyone who runs -RELEASE, so a non-starter. You can go ahead and say that about almost every commit to src. They don't help release-runners immediately, but we're not in a hurry, are we. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely, if
${component} appears more than once in ${_USE_GNOME} - something that
will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that if
someone was working with what Joe was working with, that one could be
lead to think up his revision. I know that I myself have missed the
obvious in many circumstance.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v čt 24. 05. 2007 v 07:35 -0500:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely, if
${component} appears more than once in ${_USE_GNOME} - something that
will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that if
someone was working with what Joe was working with, that one could be
lead to think up his revision. I know that I myself have missed the
obvious in many circumstance.
Please send me the patch again, as an attachment, I'll give it some
workout :)
Here it is.
--- bsd.gnome.mk-orig Mon Mar 19 23:13:36 2007
+++ bsd.gnome.mkWed May 23 19:25:19 2007
@@ -689,9 +689,9 @@
# Then traverse through all components, check which of them
# exist in ${_USE_GNOME} and set variables accordingly
+.ifdef _USE_GNOME
. for component in ${_USE_GNOME_ALL}
-_COMP_TEST=${_USE_GNOME:M${component}}
-. if ${_COMP_TEST:S/${component}//}!=${_COMP_TEST:S/ / /g}
+. if ${_USE_GNOME:M${component}}!=
PATCH_DEPENDS+=${${component}_PATCH_DEPENDS}
FETCH_DEPENDS+=${${component}_FETCH_DEPENDS}
EXTRACT_DEPENDS+=${${component}_EXTRACT_DEPENDS}
@@ -725,6 +725,7 @@
. endif
. endfor
+.endif
.endif
.if defined(GNOME_PRE_PATCH)
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v čt 24. 05. 2007 v 07:35 -0500:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely, if
${component} appears more than once in ${_USE_GNOME} - something that
will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that if
someone was working with what Joe was working with, that one could be
lead to think up his revision. I know that I myself have missed the
obvious in many circumstance.
Please send me the patch again, as an attachment, I'll give it some
workout :)
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
EMACS: Eight Megabytes And Continually Swapping
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith wrote:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v čt 24. 05. 2007 v 07:35 -0500:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in bsd.gnome.mk and it seems to me that the suggested patch is
really equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely,
if ${component} appears more than once in ${_USE_GNOME} - something
that will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that
if someone was working with what Joe was working with, that one could
be lead to think up his revision. I know that I myself have missed
the obvious in many circumstance.
Please send me the patch again, as an attachment, I'll give it some
workout :)
Here it is.
And one more thing - I found the same construction in bsd.python.mk.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v čt 24. 05. 2007 v 07:55 -0500: Please send me the patch again, as an attachment, I'll give it some workout :) Here it is. Without patch: real12m28.513s user12m30.657s sys 4m6.943s With patch: real12m3.077s user11m48.727s sys 4m8.407s That's from 996s to 956s - about same speed up as the unique patch. I like it. -- Pav Lucistnik [EMAIL PROTECTED] [EMAIL PROTECTED] A computer programmer is a device for turning requirements into undocumented features. It runs on cola, pizza and Dilbert cartoons. -- Bram Moolenaar signature.asc Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
On Thu, 2007-05-24 at 07:35 -0500, Stephen Montgomery-Smith wrote:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely, if
${component} appears more than once in ${_USE_GNOME} - something that
will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that if
someone was working with what Joe was working with, that one could be
lead to think up his revision. I know that I myself have missed the
obvious in many circumstance.
I believe this will work, but it's been so long since I did this work.
I think the problem was that :M used to match eel when USE_GNOME
contained eel2. But that doesn't appear to be the case anymore.
Joe
--
PGP Key : http://www.marcuscom.com/pgp.asc
signature.asc
Description: This is a digitally signed message part
Re: More speed increases for make-ing ports
Joe Marcus Clarke wrote:
On Thu, 2007-05-24 at 07:35 -0500, Stephen Montgomery-Smith wrote:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v st 23. 05. 2007 v 20:04 -0500:
I'm getting kind of uncomfortable with the patch. I looked some more
in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
I don't know, we'd have to ask Joe Marcus. Here is the revision in
question:
http://www.freebsd.org/cgi/cvsweb.cgi/ports/Mk/bsd.gnome.mk.diff?r1=1.22;r2=1.23
You're basically reverting that revision. Perhaps our make(1) wasn't
smart enough those four years ago, I don't know.
I'm not really reverting that revision. As it was prior to that
revision, I can see it not working under many circumstances. Namely, if
${component} appears more than once in ${_USE_GNOME} - something that
will happen frequently - the test will not work.
My revision seems the obvious way to do it to me, but I can see that if
someone was working with what Joe was working with, that one could be
lead to think up his revision. I know that I myself have missed the
obvious in many circumstance.
I believe this will work, but it's been so long since I did this work.
I think the problem was that :M used to match eel when USE_GNOME
contained eel2. But that doesn't appear to be the case anymore.
That makes perfect sense. It does look like you had to work around a
bug in make. I have actually looked at the code in make where it does
the :M (it is the function Str_Match is str.c) and this bug has clearly
been fixed now.
Stephen
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 20:29 -0500:
Stephen Montgomery-Smith wrote:
On Wed, 23 May 2007, Pav Lucistnik wrote:
Stephen Montgomery-Smith p?e v ?t 22. 05. 2007 v 16:56 -0500:
I have generated two INDEXes, one with the patch and one without.
They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
Sorry it didn't work out. Thanks for trying it.
Don't get me wrong - the minute saved here is good enough reason to
apply that patch.
Could you try the shell one also? I got the impression it was a bit
faster.
Can you send me the patch?
Yes. In the same place as the previous one you put:
.ifdef _USE_GNOME
_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
.endif
I'm going to try it out myself also to see if it is faster.
I tried it out. On my rather fast computer, I am getting similar times.
The non-shell patch seems slightly better, but there is not much
between that and the shell patch. Both patches give something like 7%
speed improvements over the original. It only potentially speeds up
gnome ports, and then only a few of them like alacarte, so even if it
dramatically speeds those few up, the average speed up over all ports
will not be so great. Like I said earlier, overall improvement is
noticeable albeit not dramatic.
Thanks for testing. I'll stick with pure-make patch and will see to
commit it after xorg freeze and some more formal testing.
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
Like 'Do Notte Buye Betamacks.' That was a prediction for 1972.
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
On Tue, May 22, 2007 at 05:34:32PM +0400, Andrew Pantyukhin wrote: A seemingly better way may be to make these system vars available in make by default. Doesn't help anyone who runs -RELEASE, so a non-starter. mcl ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 20:29 -0500:
Stephen Montgomery-Smith wrote:
On Wed, 23 May 2007, Pav Lucistnik wrote:
Stephen Montgomery-Smith p?e v ?t 22. 05. 2007 v 16:56 -0500:
I have generated two INDEXes, one with the patch and one without.
They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
Sorry it didn't work out. Thanks for trying it.
Don't get me wrong - the minute saved here is good enough reason to
apply that patch.
Could you try the shell one also? I got the impression it was a bit
faster.
Can you send me the patch?
Yes. In the same place as the previous one you put:
.ifdef _USE_GNOME
_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
.endif
I'm going to try it out myself also to see if it is faster.
I tried it out. On my rather fast computer, I am getting similar times.
The non-shell patch seems slightly better, but there is not much
between that and the shell patch. Both patches give something like 7%
speed improvements over the original. It only potentially speeds up
gnome ports, and then only a few of them like alacarte, so even if it
dramatically speeds those few up, the average speed up over all ports
will not be so great. Like I said earlier, overall improvement is
noticeable albeit not dramatic.
Thanks for testing. I'll stick with pure-make patch and will see to
commit it after xorg freeze and some more formal testing.
I'm getting kind of uncomfortable with the patch. I looked some more in
bsd.gnome.mk and it seems to me that the suggested patch is really
equivalent to the patch enclosed here.
Why did the writer of bsd.gnome.mk have the rather complicated
construction which I am proposing to replace?
--- bsd.gnome.mk-orig Mon Mar 19 23:13:36 2007
+++ bsd.gnome.mkWed May 23 19:25:19 2007
@@ -689,9 +689,9 @@
# Then traverse through all components, check which of them
# exist in ${_USE_GNOME} and set variables accordingly
+.ifdef _USE_GNOME
. for component in ${_USE_GNOME_ALL}
-_COMP_TEST=${_USE_GNOME:M${component}}
-. if ${_COMP_TEST:S/${component}//}!=${_COMP_TEST:S/ / /g}
+. if ${_USE_GNOME:M${component}}!=
PATCH_DEPENDS+=${${component}_PATCH_DEPENDS}
FETCH_DEPENDS+=${${component}_FETCH_DEPENDS}
EXTRACT_DEPENDS+=${${component}_EXTRACT_DEPENDS}
@@ -725,6 +725,7 @@
. endif
. endfor
+.endif
.endif
.if defined(GNOME_PRE_PATCH)
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
More speed increases for make-ing ports
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mkTue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+=GTK2_VERSION=${GTK2_VERSION}
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith wrote:
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Kris
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On 5/22/07, Kris Kennaway [EMAIL PROTECTED] wrote:
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith wrote:
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Right, and uniqueness logic can be implemented in make.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Quoting Andrew Pantyukhin [EMAIL PROTECTED] (from Tue, 22 May
2007 11:55:39 +0400):
On 5/22/07, Kris Kennaway [EMAIL PROTECTED] wrote:
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith wrote:
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Right, and uniqueness logic can be implemented in make.
Be proactive and tell/point out how... :)
Bye,
Alexander.
--
Despite the sign that says wet paint,
please don't.
http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On 5/22/07, Alexander Leidinger [EMAIL PROTECTED] wrote:
Quoting Andrew Pantyukhin [EMAIL PROTECTED] (from Tue, 22 May
2007 11:55:39 +0400):
On 5/22/07, Kris Kennaway [EMAIL PROTECTED] wrote:
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith wrote:
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Right, and uniqueness logic can be implemented in make.
Be proactive and tell/point out how... :)
TMTOWTDI. There are several examples in bsd.*.mk. The
obvious one is flags (you set or unset flag vars
first, then traverse them and add what you need to
the list). In recent versions of our make you can
also use ${VAR:O:u}
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On 5/22/07, Stephen Montgomery-Smith [EMAIL PROTECTED] wrote:
Andrew Pantyukhin wrote:
On 5/22/07, Alexander Leidinger [EMAIL PROTECTED] wrote:
Quoting Andrew Pantyukhin [EMAIL PROTECTED] (from Tue, 22 May
2007 11:55:39 +0400):
On 5/22/07, Kris Kennaway [EMAIL PROTECTED] wrote:
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith
wrote:
This small modification cuts off about 25% off pkg_version on my
system.
Basically bsd.gnome.mk recursively finds all the dependencies, but
many
of them are listed many times. This makes make work extra hard
when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) |
sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method
modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Right, and uniqueness logic can be implemented in make.
Be proactive and tell/point out how... :)
TMTOWTDI. There are several examples in bsd.*.mk. The
obvious one is flags (you set or unset flag vars
first, then traverse them and add what you need to
the list). In recent versions of our make you can
also use ${VAR:O:u}
I must admit I was looking for the :u. Definitely a good feature -
maybe it could be invoked in the make file conditional on an appropriate
value of OSVERSION.
Incidently if you want to save a few more != assignments, I notice that
setting the variables
ARCH=i386
OPSYS=FreeBSD
OSREL=6.2
OSVERSION=602110
in /etc/make.conf will do this for you.
A seemingly better way may be to make these system vars
available in make by default. They may even be compiled
in - to achieve virtually no performance impact (except
for a bit larger default var table).
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Quoting Andrew Pantyukhin [EMAIL PROTECTED] (from Tue, 22 May 2007 17:34:32 +0400): On 5/22/07, Stephen Montgomery-Smith [EMAIL PROTECTED] wrote: Incidently if you want to save a few more != assignments, I notice that setting the variables ARCH=i386 OPSYS=FreeBSD OSREL=6.2 OSVERSION=602110 in /etc/make.conf will do this for you. A seemingly better way may be to make these system vars available in make by default. They may even be compiled in - to achieve virtually no performance impact (except for a bit larger default var table). OPSYS can be made static (cvs commit to bsd.port.mk). Bye, Alexander. -- It used to be the fun was in The capture and kill. In another place and time I did it all for thrills. -- Lust to Love http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Kris Kennaway wrote:
On Tue, May 22, 2007 at 01:47:23AM -0500, Stephen Montgomery-Smith wrote:
This small modification cuts off about 25% off pkg_version on my system.
Basically bsd.gnome.mk recursively finds all the dependencies, but many
of them are listed many times. This makes make work extra hard when it
doesn't have to. I simply weed out the repeated entries.
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mkTue May 22 01:29:22 2007
@@ -655,6 +655,8 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+=GTK2_VERSION=${GTK2_VERSION}
Be careful, != assignments may add thousands of process invocations to
large targets like 'make index' and can slow it down dramatically.
Kris
Thanks for the warnings. It turns out that this can be done without
!=, but even if this were not the case, I think that in this situation
the savings would definitely outweigh the costs. For example, this
dramatically improves the time for invocations of make -V PKGNAME for
deskutils/alacarte (on my system from about 1.5 seconds to .3 seconds).
It only affects a few ports, but enough, I think, to increase the speed
of make index noticeably albeit not dramatically.
Stephen
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 09:35 -0500: For example, this dramatically improves the time for invocations of make -V PKGNAME for deskutils/alacarte (on my system from about 1.5 seconds to .3 seconds). It only affects a few ports, but enough, I think, to increase the speed of make index noticeably albeit not dramatically. Sounds good. Can you get us a fresh patch? -- Pav Lucistnik [EMAIL PROTECTED] [EMAIL PROTECTED] XML is a giant step in no direction at all. -- Erik Naggum signature.asc Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 09:35 -0500:
For example, this
dramatically improves the time for invocations of make -V PKGNAME for
deskutils/alacarte (on my system from about 1.5 seconds to .3 seconds).
It only affects a few ports, but enough, I think, to increase the speed
of make index noticeably albeit not dramatically.
Sounds good. Can you get us a fresh patch?
Try the patch I sent last night. The modifier :u seems only to be
available in CURRENT, and writing it without invoking a shell without :u
is really beyond my skills. (And I think the overhead of a single extra
shell invocation is really quite insignificant - for starters it will
only affect gnome ports.)
But if you are using a recent CURRENT, I guess
_USE_GNOME=${_USE_GNOME:O:u}
in the same place that I modified it last night will do the trick.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith wrote:
Pav Lucistnik wrote:
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 09:35 -0500:
For example, this dramatically improves the time for invocations of
make -V PKGNAME for deskutils/alacarte (on my system from about 1.5
seconds to .3 seconds). It only affects a few ports, but enough, I
think, to increase the speed of make index noticeably albeit not
dramatically.
Sounds good. Can you get us a fresh patch?
Try the patch I sent last night. The modifier :u seems only to be
available in CURRENT, and writing it without invoking a shell without :u
is really beyond my skills.
Or maybe it is not beyond my skills. This is what I came up with:
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mkTue May 22 13:56:16 2007
@@ -655,6 +655,13 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+.for component in ${_USE_GNOME_ALL}
+. if ${_USE_GNOME:M${component}} !=
+_USE_GNOME_UNIQ+=${component}
+. endif
+.endfor
+_USE_GNOME=${_USE_GNOME_UNIQ}
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+=GTK2_VERSION=${GTK2_VERSION}
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 13:58 -0500:
Or maybe it is not beyond my skills. This is what I came up with:
Prostý textový dokument příloha (ddd)
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mk Tue May 22 13:56:16 2007
@@ -655,6 +655,13 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+.for component in ${_USE_GNOME_ALL}
+. if ${_USE_GNOME:M${component}} !=
+_USE_GNOME_UNIQ+=${component}
+. endif
+.endfor
+_USE_GNOME=${_USE_GNOME_UNIQ}
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+= GTK2_VERSION=${GTK2_VERSION}
Too bad it does not work. Breaks on, for example, graphics/opencv.
You need to handle the case of USE_GNOME being defined as empty.
Wrapping the code block inside
.ifdef _USE_GNOME
.endif
seems to do the trick here. Still testing ...
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
What do you mean? An African or a European swallow?
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
Pav Lucistnik píše v út 22. 05. 2007 v 23:16 +0200:
Stephen Montgomery-Smith píše v út 22. 05. 2007 v 13:58 -0500:
Or maybe it is not beyond my skills. This is what I came up with:
Prostý textový dokument příloha (ddd)
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mkTue May 22 13:56:16 2007
@@ -655,6 +655,13 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+.for component in ${_USE_GNOME_ALL}
+. if ${_USE_GNOME:M${component}} !=
+_USE_GNOME_UNIQ+=${component}
+. endif
+.endfor
+_USE_GNOME=${_USE_GNOME_UNIQ}
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+=GTK2_VERSION=${GTK2_VERSION}
Too bad it does not work. Breaks on, for example, graphics/opencv.
You need to handle the case of USE_GNOME being defined as empty.
Wrapping the code block inside
.ifdef _USE_GNOME
.endif
seems to do the trick here. Still testing ...
I have generated two INDEXes, one with the patch and one without. They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
/usr/src scares me. I'm just not man enough to commit there.
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: More speed increases for make-ing ports
On Tue, 22 May 2007, Pav Lucistnik wrote:
Pav Lucistnik p??e v ?t 22. 05. 2007 v 23:16 +0200:
Stephen Montgomery-Smith p??e v ?t 22. 05. 2007 v 13:58 -0500:
Or maybe it is not beyond my skills. This is what I came up with:
Prost? textov? dokument p??loha (ddd)
--- bsd.gnome.mk-orig Tue May 22 01:29:08 2007
+++ bsd.gnome.mkTue May 22 13:56:16 2007
@@ -655,6 +655,13 @@
_USE_GNOME+= ${${component}_USE_GNOME_IMPL} ${component}
. endfor
+.for component in ${_USE_GNOME_ALL}
+. if ${_USE_GNOME:M${component}} !=
+_USE_GNOME_UNIQ+=${component}
+. endif
+.endfor
+_USE_GNOME=${_USE_GNOME_UNIQ}
+
# Setup the GTK+ API version for pixbuf loaders, input method modules,
# and theme engines.
PLIST_SUB+=GTK2_VERSION=${GTK2_VERSION}
Too bad it does not work. Breaks on, for example, graphics/opencv.
You need to handle the case of USE_GNOME being defined as empty.
Wrapping the code block inside
.ifdef _USE_GNOME
.endif
seems to do the trick here. Still testing ...
I have generated two INDEXes, one with the patch and one without. They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
--
Pav Lucistnik [EMAIL PROTECTED]
[EMAIL PROTECTED]
/usr/src scares me. I'm just not man enough to commit there.
Sorry it didn't work out. Thanks for trying it.
Could you try the shell one also? I got the impression it was a bit
faster.
Stephen
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On Wed, 23 May 2007, Pav Lucistnik wrote:
Stephen Montgomery-Smith p??e v ?t 22. 05. 2007 v 16:56 -0500:
I have generated two INDEXes, one with the patch and one without.
They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
Sorry it didn't work out. Thanks for trying it.
Don't get me wrong - the minute saved here is good enough reason to
apply that patch.
Could you try the shell one also? I got the impression it was a bit
faster.
Can you send me the patch?
Yes. In the same place as the previous one you put:
.ifdef _USE_GNOME
_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
.endif
I'm going to try it out myself also to see if it is faster.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Would it be possible to check in some of these speedups? Hoping that helps the xorg-7.2 upgrade some Thanks! ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
On Tue, May 22, 2007 at 04:54:11PM -0700, Bakul Shah wrote: Would it be possible to check in some of these speedups? Hoping that helps the xorg-7.2 upgrade some Thanks! They will be committed later after careful testing. The last thing we want to do is complicate user x.org upgrades by accidentally committing broken changes. A slow but working build process is very much preferable to a fast but broken one :-) Kris ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Stephen Montgomery-Smith wrote:
On Wed, 23 May 2007, Pav Lucistnik wrote:
Stephen Montgomery-Smith p?e v ?t 22. 05. 2007 v 16:56 -0500:
I have generated two INDEXes, one with the patch and one without.
They
are identical, the timings:
INDEX-orig
real16m32.761s
user18m36.802s
sys 8m38.610s
INDEX-ddd
real16m34.620s
user17m25.976s
sys 8m46.333s
Sorry it didn't work out. Thanks for trying it.
Don't get me wrong - the minute saved here is good enough reason to
apply that patch.
Could you try the shell one also? I got the impression it was a bit
faster.
Can you send me the patch?
Yes. In the same place as the previous one you put:
.ifdef _USE_GNOME
_USE_GNOME!=(for i in ${_USE_GNOME}; do ${ECHO_CMD} $$i; done) | sort -u
.endif
I'm going to try it out myself also to see if it is faster.
I tried it out. On my rather fast computer, I am getting similar times.
The non-shell patch seems slightly better, but there is not much
between that and the shell patch. Both patches give something like 7%
speed improvements over the original. It only potentially speeds up
gnome ports, and then only a few of them like alacarte, so even if it
dramatically speeds those few up, the average speed up over all ports
will not be so great. Like I said earlier, overall improvement is
noticeable albeit not dramatic.
Stephen
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: More speed increases for make-ing ports
Kris Kennaway wrote: On Tue, May 22, 2007 at 04:54:11PM -0700, Bakul Shah wrote: Would it be possible to check in some of these speedups? Hoping that helps the xorg-7.2 upgrade some Thanks! They will be committed later after careful testing. The last thing we want to do is complicate user x.org upgrades by accidentally committing broken changes. A slow but working build process is very much preferable to a fast but broken one :-) While I am quite proud of some of the speed improvements I found (especially the sortdeps in pkg_create), I would hate to bear responsibility for severe port breakage, and so I am glad that these proposed changes will be carefully tested. Best regards, Stephen ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to [EMAIL PROTECTED]
