Re: MAKE_JOBS_UNSAFE (some more ports)
On Saturday 06 June 2009 22:56:47 Ion-Mihai Tetcu wrote: On Sat, 6 Jun 2009 18:05:14 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.S. Is anyone interested in a list of ports that do not compile under tmpfs? Me. The following are on my blacklist for tmpfs build, where: # df -h | grep tmp tmpfs 8.3G 12M8.3G 0%/tmp # grep WRKDIRPREFIX /etc/make.conf WRKDIRPREFIX=/tmp editors/openoffice.org-3 (just to big for my computer to handle) security/gpgme* lang/ocaml** java/openjdk6*** * Confirmed build failure on 7.1p2 and -Current from December * Confirmed build success on -Current from Saturday ** I had a strange problems with math/facile that it wouldn't build if ocaml was built on tmpfs (didn't confirm this one) *** Cannot reproduce (although do remember it) From what I read it appeared that tmpfs had an internal locking problem however it appears to be fixed in current. signature.asc Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
David Naylor wrote: On Saturday 06 June 2009 22:56:47 Ion-Mihai Tetcu wrote: On Sat, 6 Jun 2009 18:05:14 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.S. Is anyone interested in a list of ports that do not compile under tmpfs? Me. The following are on my blacklist for tmpfs build, where: # df -h | grep tmp tmpfs 8.3G 12M8.3G 0%/tmp # grep WRKDIRPREFIX /etc/make.conf WRKDIRPREFIX=/tmp editors/openoffice.org-3 (just to big for my computer to handle) security/gpgme* lang/ocaml** java/openjdk6*** * Confirmed build failure on 7.1p2 and -Current from December * Confirmed build success on -Current from Saturday ** I had a strange problems with math/facile that it wouldn't build if ocaml was built on tmpfs (didn't confirm this one) *** Cannot reproduce (although do remember it) From what I read it appeared that tmpfs had an internal locking problem however it appears to be fixed in current. Last I tried it, procmail did not build on tmpfs either, I didn't have time to report the full details. After getting a compile error, I looked into what was in the build directory and found two files with the same name!! I think when I deleted one, I think the size of the other showed a change. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi David and * thanks for your patch, I verified and committed. Best, From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Sat, 06 Jun 2009 02:39:42 +0300 On Sat, 06 Jun 2009 08:26:01 +0900 (JST) Maho NAKATA cha...@mac.com wrote: From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Sat, 06 Jun 2009 01:38:18 +0300 On Sat, 06 Jun 2009 07:25:35 +0900 (JST) Maho NAKATA cha...@mac.com wrote: thanks for raising as PR :) http://www.freebsd.org/cgi/query-pr.cgi?pr=135262 Some support has been committed by Pav, can you please check his commit and adjust OOo ports to make use of it? I just checked Pav's commit and I checked David's newest patch, and his patch seems to make use of Pav's support. Cool :) This way I could have all OOo ports tested on-commit on QAT ;-) Yes, really appreciated. Up until now two OOo commits would busy QAT for half a day; with this changes committed it's two hours or less :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B pgpAPb9DlmlgE.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Thursday 21 May 2009 13:56:46 Pav Lucistnik wrote: On Thu, 21 May 2009 12:05:22 +0200, David Naylor wrote The following ports failed to build on my system (with a quad core) and FORCE_MAKE_JOBS set. They did success to build once I added MAKE_JOBS_UNSAFE=yes to their Makefile's. Marked in CVS, thank you! I believe java/jdk* should be marked as unsafe. They define their own do-build targets (and don't use _MAKE_JOBS) so no functional change. I've checked jdk16 with `make MAKE_ARGS=-j4` and build fails. I've found the following ports that are UNSAFE: audio/cdparanoia (under heavy load) devel/dbus-qt4 (under heavy load) java/openjdk6 Is there any effort to mark ports as MAKE_JOBS_SAFE: is it desired for ports that are successful with FORCE_MAKE_JOBS to be reported? Yes, I believe they should be reported. Here are all the ports that compile with -DFORCE_MAKE_JOBS, do not have MAKE_JOBS_* set and do not define a do-build target: (NOTE: FORCE_MAKE_JOBS=yes is in make.conf for all builds) # for i in `pkg_info -oqa`; do cd /usr/ports/$i; if [ -z `make -V MAKE_JOBS_SAFE -V MAKE_JOBS_UNSAFE` -a -z `grep do-build Makefile`]; then echo $i; fi; done | sort [ See attached for output ] Regards, David P.S. Is anyone interested in a list of ports that do not compile under tmpfs? accessibility/atk accessibility/linux-f8-atk accessibility/qt4-accessible archivers/cabextract archivers/libzip archivers/p5-Archive-Zip archivers/p5-Compress-Bzip2 archivers/p5-Compress-Raw-Zlib archivers/p5-Compress-Zlib archivers/p5-IO-Compress-Base archivers/p5-IO-Compress-Zlib archivers/p5-PerlIO-gzip archivers/p5-PerlIO-via-Bzip2 archivers/rpm archivers/unrar archivers/unzip archivers/zip astro/cfitsio astro/libnova audio/aacgain audio/amarok-kde4 audio/faac audio/faad audio/flac audio/gsm audio/lame audio/liba52 audio/libamrnb audio/libamrwb audio/libao audio/libcddb audio/libgpod audio/libid3tag audio/libmad audio/libmikmod audio/libmodplug audio/libmtp audio/libmusicbrainz audio/libofa audio/libogg audio/libtunepimp audio/libvorbis audio/madplay audio/mp3gain audio/normalize audio/sdl_mixer audio/speex audio/taglib audio/vorbis-tools audio/vorbisgain audio/wavpack comms/gnokii converters/fribidi converters/p5-MIME-Base64 databases/db46 databases/gdbm databases/mysql51-client databases/mysql51-server databases/py-bsddb databases/py-qt4-sql databases/qt4-mysql-plugin databases/qt4-sql databases/qt4-sqlite3-plugin databases/rrdtool databases/sqlite3 devel/ORBit2 devel/apache-ant devel/autoconf-wrapper devel/autoconf213 devel/autoconf262 devel/automake-wrapper devel/automake110 devel/automake14 devel/automake15 devel/automake19 devel/bison devel/boost-python devel/clanlib devel/cmake devel/dbus devel/dbus-glib devel/dbus-qt4 devel/desktop-file-utils devel/eric4 devel/gamin devel/gccmakedep devel/gconf2 devel/gettext devel/gio-fam-backend devel/glib12 devel/glib20 devel/gmake devel/gnome-vfs devel/imake devel/kdesvn-kde4 devel/libIDL devel/libcheck devel/libdaemon devel/libexecinfo devel/libglade2 devel/libical devel/libltdl15 devel/liboil devel/libpci devel/libpciaccess devel/libpthread-stubs devel/libstatgrab devel/libtool15 devel/libvolume_id devel/m4 devel/makedepend devel/newfile devel/nspr devel/p5-Algorithm-Annotate devel/p5-Algorithm-Diff devel/p5-App-CLI devel/p5-BFD devel/p5-Class-Accessor devel/p5-Class-Autouse devel/p5-Class-Data-Inheritable devel/p5-Data-Hierarchy devel/p5-Data-UUID devel/p5-ExtUtils-CBuilder devel/p5-ExtUtils-ParseXS devel/p5-File-Temp devel/p5-File-chdir devel/p5-FreezeThaw devel/p5-Getopt-Long devel/p5-IO-Digest devel/p5-IO-Pager devel/p5-IPC-Run3 devel/p5-Locale-Maketext devel/p5-Locale-Maketext-Lexicon devel/p5-Locale-Maketext-Simple devel/p5-Locale-gettext devel/p5-Log-Log4perl devel/p5-Module-Build devel/p5-Path-Class devel/p5-PathTools devel/p5-PerlIO-eol devel/p5-PerlIO-via-dynamic devel/p5-PerlIO-via-symlink devel/p5-Regexp-Shellish devel/p5-SVN-Dump devel/p5-SVN-Mirror devel/p5-SVN-Simple devel/p5-Storable devel/p5-Term-ReadKey devel/p5-Time-Progress devel/p5-TimeDate devel/p5-UNIVERSAL-require devel/p5-VCP-autrijus devel/p5-prefork devel/p5-version devel/patch devel/pcre devel/pkg-config devel/popt devel/py-astng devel/py-dbus devel/py-logilab-common devel/py-qt4-assistant devel/py-qt4-core devel/py-qt4-dbus devel/py-qt4-designer devel/py-qt4-designerplugin devel/py-qt4-help devel/py-qt4-qscintilla2 devel/py-qt4-script devel/py-qt4-test devel/py-sip devel/pylint devel/qca devel/qmake4 devel/qscintilla2 devel/qt4 devel/qt4-assistant devel/qt4-assistant-adp devel/qt4-corelib devel/qt4-designer devel/qt4-help devel/qt4-libqtassistantclient devel/qt4-linguist devel/qt4-makeqpf devel/qt4-moc devel/qt4-porting devel/qt4-qdbusviewer devel/qt4-qt3support devel/qt4-qtestlib devel/qt4-qvfb devel/qt4-rcc devel/qt4-script devel/qt4-uic devel/qt4-uic3 devel/sdl12 devel/subversion devel/t1lib devel/xorg-macros devel/yasm dns/libidn emulators/wine ftp/curl ftp/wget
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sat, 6 Jun 2009 18:05:14 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.S. Is anyone interested in a list of ports that do not compile under tmpfs? Me. -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
thanks for raising as PR :) http://www.freebsd.org/cgi/query-pr.cgi?pr=135262 -- Nakata Maho http://accc.riken.jp/maho/ , http://ja.openoffice.org/ Nakata Maho's PGP public keys: http://accc.riken.jp/maho/maho.pgp.txt pgpOxw8vi6mZA.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sat, 06 Jun 2009 07:25:35 +0900 (JST) Maho NAKATA cha...@mac.com wrote: thanks for raising as PR :) http://www.freebsd.org/cgi/query-pr.cgi?pr=135262 Some support has been committed by Pav, can you please check his commit and adjust OOo ports to make use of it? This way I could have all OOo ports tested on-commit on QAT ;-) Thanks, -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Sat, 06 Jun 2009 01:38:18 +0300 On Sat, 06 Jun 2009 07:25:35 +0900 (JST) Maho NAKATA cha...@mac.com wrote: thanks for raising as PR :) http://www.freebsd.org/cgi/query-pr.cgi?pr=135262 Some support has been committed by Pav, can you please check his commit and adjust OOo ports to make use of it? I just checked Pav's commit and I checked David's newest patch, and his patch seems to make use of Pav's support. This way I could have all OOo ports tested on-commit on QAT ;-) Yes, really appreciated. Thanks -- Nakata Maho http://accc.riken.jp/maho/ , http://ja.openoffice.org/ Nakata Maho's PGP public keys: http://accc.riken.jp/maho/maho.pgp.txt pgpcmZa4kqYvl.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sat, 06 Jun 2009 08:26:01 +0900 (JST) Maho NAKATA cha...@mac.com wrote: From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Sat, 06 Jun 2009 01:38:18 +0300 On Sat, 06 Jun 2009 07:25:35 +0900 (JST) Maho NAKATA cha...@mac.com wrote: thanks for raising as PR :) http://www.freebsd.org/cgi/query-pr.cgi?pr=135262 Some support has been committed by Pav, can you please check his commit and adjust OOo ports to make use of it? I just checked Pav's commit and I checked David's newest patch, and his patch seems to make use of Pav's support. Cool :) This way I could have all OOo ports tested on-commit on QAT ;-) Yes, really appreciated. Up until now two OOo commits would busy QAT for half a day; with this changes committed it's two hours or less :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
2009/5/21 David Naylor naylor.b.da...@gmail.com: Hi, The following ports failed to build on my system (with a quad core) and FORCE_MAKE_JOBS set. They did success to build once I added MAKE_JOBS_UNSAFE=yes to their Makefile's. devel/nasm graphics/libart_lgpl lang/ocaml multimedia/mplayer multimedia/smplayer security/nss Please also mark misc/e2fsprogs-libuuid as MAKE_JOBS_UNSAFE as it always fails to build on my quad-core system until I use MAKE_JOBS_UNSAFE. Thanks ! Is there any effort to mark ports as MAKE_JOBS_SAFE: is it desired for ports that are successful with FORCE_MAKE_JOBS to be reported? Regards David P.S. I'm not on the list P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: oliv...@gid0.org- against HTML email vCards X www: http://www.gid0.org- against proprietary attachments / \ Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: MAKE_JOBS_UNSAFE (some more ports)
On Tuesday 26 May 2009 23:23:16 Pav Lucistnik wrote:
David Naylor píše v út 26. 05. 2009 v 18:17 +0200:
What about the change that exposes MAKE_JOBS_NUMBER when MAKE_JOBS_SAFE
or FORCE_MAKE_JOBS are defined (to avoid using ${_MAKE_JOBS:C/-j//}, not
sure what the policy is of ports using *.mk internals). I think that is
a reasonable change???
I think it's reasonable. It will need to be tested widely. Can you file
a PR with just that change, to help me track it while in testing?
Done, please see PR ports/134977. This should not have any functional change
and the only ports (at this stage) that will use MAKE_JOBS_NUMBER is OOo*
(although games/teeworld is the next closest candidate).
I've also made some changes to how OOo2 handles concurrency, as pav@ pointed
out `make -j1` is different to `make` and OOo2 now differentiates between the
two, could someone please check if the following work:
(cd /usr/ports/editors/openoffice.org-2; make MAKE_JOBS_NUMBER=1)
OOo3 should be functionally the same to the previous patch however it does not
make the distinction between `make -j1` and `make` and I don't know enough
about the build process to know how to add a normal `make`.
Thanks for all your patience.
David
P.S. This should have been sent ~9 hours ago, but internet went down.
P.P.S. This should be the final patch (pending OOo2 verification).
diff -ur /usr/ports/Mk/bsd.port.mk ports/Mk/bsd.port.mk
--- /usr/ports/Mk/bsd.port.mk 2009-05-23 13:20:58.0 +0200
+++ ports/Mk/bsd.port.mk 2009-05-27 08:38:44.0 +0200
@@ -2185,11 +2185,8 @@
_MAKE_JOBS= #
.else
.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
.if defined(FORCE_MAKE_JOBS)
BUILD_FAIL_MESSAGE+= You have chosen to use multiple make jobs (parallelization) for all ports. This port was not tested for this setting. Please remove FORCE_MAKE_JOBS and retry the build before reporting the failure to the maintainer.
.endif
diff -ur /usr/ports/editors/openoffice.org-2/Makefile ports/editors/openoffice.org-2/Makefile
--- /usr/ports/editors/openoffice.org-2/Makefile 2009-01-25 10:45:45.0 +0200
+++ ports/editors/openoffice.org-2/Makefile 2009-05-27 08:38:27.0 +0200
@@ -51,6 +51,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include bsd.port.pre.mk
@@ -132,7 +133,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-NUMOFPROCESSES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -192,8 +192,8 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
# PR:84786 #i53289#
-.if (${NUMOFPROCESSES}1)
- @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all
+.if !defined(DISABLE_MAKE_JOBS)
+ @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all
.else
@cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake
.endif
diff -ur /usr/ports/editors/openoffice.org-2-RC/Makefile ports/editors/openoffice.org-2-RC/Makefile
--- /usr/ports/editors/openoffice.org-2-RC/Makefile 2009-01-25 10:45:45.0 +0200
+++ ports/editors/openoffice.org-2-RC/Makefile 2009-05-27 08:41:53.0 +0200
@@ -52,6 +52,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include bsd.port.pre.mk
@@ -134,7 +135,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-NUMOFPROCESSES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -194,8 +194,8 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
# PR:84786 #i53289#
-.if (${NUMOFPROCESSES}1)
- @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all
+.if !defined(DISABLE_MAKE_JOBS)
+ @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all
.else
@cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake
.endif
diff -ur /usr/ports/editors/openoffice.org-2-devel/Makefile ports/editors/openoffice.org-2-devel/Makefile
--- /usr/ports/editors/openoffice.org-2-devel/Makefile 2009-01-25 10:45:45.0 +0200
+++ ports/editors/openoffice.org-2-devel/Makefile 2009-05-27 08:46:03.0 +0200
@@ -52,6 +52,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include bsd.port.pre.mk
@@
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi David, I'll go to Canada for attending conference tomorrow.be back on 6/4. see you -- Nakata Maho http://accc.riken.jp/maho/ , http://ja.openoffice.org/ Nakata Maho's PGP public keys: http://accc.riken.jp/maho/maho.pgp.txt pgpVkDXepD1Ih.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Tuesday 26 May 2009 10:48:25 Pav Lucistnik wrote:
David Naylor píše v út 26. 05. 2009 v 08:19 +0200:
pav: ${_MAKE_JOBS:C/-j//} won't work with DISABLE_MAKE_JOBS (or
MAKE_JOBS_UNSAFE) since it needs to always be a positive number, secondly
it still cannot be used for conditional code (since it is defined in the
post section, but the whole code could always be moved to the pre
section).
I'm hesitant to modify bsd.port.mk for benefit of just four ports.
Also, I think having MAKE_JOBS_NUMBER set to 1 when the feature is in
fact disable, is counter-intuitive (because -j1 is very different to no
-j at all).
I understand, I see the light. By the way it is two ports requiring the
below.
What about the change that exposes MAKE_JOBS_NUMBER when MAKE_JOBS_SAFE or
FORCE_MAKE_JOBS are defined (to avoid using ${_MAKE_JOBS:C/-j//}, not sure
what the policy is of ports using *.mk internals). I think that is a
reasonable change???
So how about just having
.if defined(DISABLE_MAKE_JOBS)
MAKE_JOBS_NUMBER= 1
.else
+.if !defined(MAKE_JOBS_NUMBER)
MAKE_JOBS_NUMBER!=echo `${SYSCTL} -n kern.smp.cpus`
+.endif
.endif
in ooo makefile?
This will work in OOo2*, the OOo3 will also need a check for DISABLE_MAKE_JOBS
since they rely on MKAE_JOBS_NUMBER always being set (just the way they do
things).
Will fix and send another patch.
signature.asc
Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
David Naylor píše v po 25. 05. 2009 v 10:11 +0200:
This part looks OK, I wonder if there's any reason t ain't like
this now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
Wouldn't that mean an evaluation of the backtick command in every
make(1) invocation? That would be highly undesirable.
I don't believe that is the case.
Here is what I get with the patch applied (MAKE_JOBS_NUMBER not defined):
/usr/ports/editors/openoffice.org-3# make -V MAKE_JOBS_NUMBER -V
_MAKE_JOBS `/sbin/sysctl -n kern.smp.cpus`
-j`/sbin/sysctl -n kern.smp.cpus`
Wouldn't this indicate that the backtick command is not being evaluated?
Seems correct. But explain again, why you need this change?
Not all ports use make but are concurrent capable and require different
arguments to be passed which is why I needed to expose MAKE_JOBS_NUMBER
(since it is just a number) and why _MAKE_JOBS was not an option.
Ok, how about doing a dirty hack and using ${_MAKE_JOBS:C/-j//} ?
Then we wouldn't have to modify bsd.port.mk ..
--
Pav Lucistnik p...@oook.cz
p...@freebsd.org
Why do we need a film of Lord of the Rings when we have the book?
Because watching a cg enhanced Legolas fire a flaming arrow
into the heart of a warg is cool?
- a...@asdf.com in rec.games.roguelike.angband
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: MAKE_JOBS_UNSAFE (some more ports)
On Mon, 25 May 2009 10:03:12 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 21:37:45 Ion-Mihai Tetcu wrote:
On Sun, 24 May 2009 10:26:23 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you
explicitly set MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl
kern.smp.cpus` as the command is not resolved.
w/o patch
editors/openoffice.org-3openoffice.org-3.1.04:53:27
with patch:
+ MAKE_JOBS_SAFE= yes
+ MAKE_JOBS_NUMBER= 4
+ MAXPROCESSES?=${MAKE_JOBS_NUMBER}
+ MAXMODULES?= ${MAKE_JOBS_NUMBER}
editors/openoffice.org-3openoffice.org-3.1.048:51
The build is done in
/dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs,
asynchronous, local, noatime)
Wow, that is quite a speedup. Is it even possible (4 * 60 + 53)/4 =
73, and you get 48 (that is 152% scaling efficiency). This would
mean a serious performance problem with the ooo3 build script and
MAX* =1.
I'll make a patch tonight (+10 hours) that will fix ooo2 in the
default case. You can test ooo2 with patch and MAKE_JOBS_NUMBER
preset (not using default value) and MAKE_JOBS_SAFE=yes.
BTW, what about using the same vars for parallel building in all OOo
port?
--
IOnut - Un^d^dregistered ;) FreeBSD user
Intellectual Property is nowhere near as valuable as Intellect
FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B
signature.asc
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Monday 25 May 2009 20:01:25 Ion-Mihai Tetcu wrote:
On Mon, 25 May 2009 10:03:12 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 21:37:45 Ion-Mihai Tetcu wrote:
On Sun, 24 May 2009 10:26:23 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you
explicitly set MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl
kern.smp.cpus` as the command is not resolved.
w/o patch
editors/openoffice.org-3 openoffice.org-3.1.04:53:27
with patch:
+ MAKE_JOBS_SAFE= yes
+ MAKE_JOBS_NUMBER= 4
+ MAXPROCESSES?= ${MAKE_JOBS_NUMBER}
+ MAXMODULES?=${MAKE_JOBS_NUMBER}
editors/openoffice.org-3 openoffice.org-3.1.048:51
The build is done in
/dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs,
asynchronous, local, noatime)
Wow, that is quite a speedup. Is it even possible (4 * 60 + 53)/4 =
73, and you get 48 (that is 152% scaling efficiency). This would
mean a serious performance problem with the ooo3 build script and
MAX* =1.
I'll make a patch tonight (+10 hours) that will fix ooo2 in the
default case. You can test ooo2 with patch and MAKE_JOBS_NUMBER
preset (not using default value) and MAKE_JOBS_SAFE=yes.
BTW, what about using the same vars for parallel building in all OOo
port?
Done, the following patch uses MAKE_JOBS_NUMBER for all the variables in OOo.
It also tries to be efficient when resolving the MAKE_JOBS_NUMBER to a value
(only done when a port sets USE_MAKE_JOBS, as in the OOo2-RC and OOo2 case).
This should fix OOo2* builds and support such use cases for other ports...
diff -ru /usr/ports/Mk/bsd.port.mk ports/Mk/bsd.port.mk
--- /usr/ports/Mk/bsd.port.mk 2009-05-23 13:20:58.0 +0200
+++ ports/Mk/bsd.port.mk 2009-05-25 22:08:58.0 +0200
@@ -1361,6 +1361,19 @@
.include ${PORTSDIR}/Mk/bsd.linux-apps.mk
.endif
+.if defined(USE_MAKE_JOBS)
+.if defined(MAKE_JOBS_UNSAFE)
+.error USE_MAKE_JOBS requested yet port is marked as MAKE_JOBS_UNSAFE
+.endif
+.if defined(DISABLE_MAKE_JOBS)
+MAKE_JOBS_NUMBER= 1
+.elif defined(FORCE_MAKE_JOBS) || defined(MAKE_JOBS_SAFE)
+.if !defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER!= ${SYSCTL} -n kern.smp.cpus
+.endif
+.endif
+.endif
+
.if defined(X_WINDOW_SYSTEM) ${X_WINDOW_SYSTEM:L} != xorg
IGNORE= cannot be installed: bad X_WINDOW_SYSTEM setting; valid value is 'xorg'
.endif
@@ -2182,15 +2195,13 @@
# Multiple make jobs support
.if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE)
+MAKE_JOBS_NUMBER= 1
_MAKE_JOBS= #
.else
.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
-.if defined(FORCE_MAKE_JOBS)
+.if defined(FORCE_MAKE_JOBS) !defined(MAKE_JOBS_SAFE)
BUILD_FAIL_MESSAGE+= You have chosen to use multiple make jobs (parallelization) for all ports. This port was not tested for this setting. Please remove FORCE_MAKE_JOBS and retry the build before reporting the failure to the maintainer.
.endif
.endif
diff -ru /usr/ports/editors/openoffice.org-2/Makefile ports/editors/openoffice.org-2/Makefile
--- /usr/ports/editors/openoffice.org-2/Makefile 2009-01-25 10:45:45.0 +0200
+++ ports/editors/openoffice.org-2/Makefile 2009-05-25 22:10:21.0 +0200
@@ -48,9 +48,11 @@
USE_XORG= x11 ice xaw xau xext xrender xrandr \
xi xt xcursor xdamage xcomposite xfixes
USE_GMAKE= yes
+USE_MAKE_JOBS= yes
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include bsd.port.pre.mk
@@ -132,7 +134,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-NUMOFPROCESSES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -192,8 +193,8 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
# PR:84786 #i53289#
-.if (${NUMOFPROCESSES}1)
- @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all
+.if (${MAKE_JOBS_NUMBER}1)
+ @cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all
.else
@cd ${WRKSRC} ; ${SETENV} LANG=C LC_ALL=C ${TCSH} -c source
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi David,
Thanks kudos for tough works and discussions!
David, is this the final patch which I should test?
Best,
From: David Naylor naylor.b.da...@gmail.com
Subject: Re: MAKE_JOBS_UNSAFE (some more ports)
Date: Mon, 25 May 2009 22:47:29 +0200
On Monday 25 May 2009 20:01:25 Ion-Mihai Tetcu wrote:
On Mon, 25 May 2009 10:03:12 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 21:37:45 Ion-Mihai Tetcu wrote:
On Sun, 24 May 2009 10:26:23 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you
explicitly set MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl
kern.smp.cpus` as the command is not resolved.
w/o patch
editors/openoffice.org-3 openoffice.org-3.1.04:53:27
with patch:
+ MAKE_JOBS_SAFE=yes
+ MAKE_JOBS_NUMBER= 4
+ MAXPROCESSES?= ${MAKE_JOBS_NUMBER}
+ MAXMODULES?= ${MAKE_JOBS_NUMBER}
editors/openoffice.org-3 openoffice.org-3.1.048:51
The build is done in
/dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs,
asynchronous, local, noatime)
Wow, that is quite a speedup. Is it even possible (4 * 60 + 53)/4 =
73, and you get 48 (that is 152% scaling efficiency). This would
mean a serious performance problem with the ooo3 build script and
MAX* =1.
I'll make a patch tonight (+10 hours) that will fix ooo2 in the
default case. You can test ooo2 with patch and MAKE_JOBS_NUMBER
preset (not using default value) and MAKE_JOBS_SAFE=yes.
BTW, what about using the same vars for parallel building in all OOo
port?
Done, the following patch uses MAKE_JOBS_NUMBER for all the variables in OOo.
It also tries to be efficient when resolving the MAKE_JOBS_NUMBER to a value
(only done when a port sets USE_MAKE_JOBS, as in the OOo2-RC and OOo2 case).
This should fix OOo2* builds and support such use cases for other ports...
pgpgPOCyCj044.pgp
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you explicitly set
MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl kern.smp.cpus`
as the command is not resolved.
3. ooo3, 3-rc, 3-devel are okay with patch 1.
Good to hear.
signature.asc
Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sun, 24 May 2009 10:26:23 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you explicitly
set MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl
kern.smp.cpus` as the command is not resolved.
Adding MAKE_JOBS_NUMBER!=`sysctl kern.smp.cpus` in the port Makefile
should help.
--
IOnut - Un^d^dregistered ;) FreeBSD user
Intellectual Property is nowhere near as valuable as Intellect
FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B
signature.asc
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
Ion-Mihai Tetcu píše v so 23. 05. 2009 v 13:51 +0300:
- MAKE_JOBS_NUMBER defaults (but user defined) to number of cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
Wouldn't that mean an evaluation of the backtick command in every
make(1) invocation? That would be highly undesirable.
--
Pav Lucistnik p...@oook.cz
p...@freebsd.org
See file. Click file. Get file.
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sun, 24 May 2009 16:10:23 +0200
Pav Lucistnik p...@freebsd.org wrote:
Ion-Mihai Tetcu píše v so 23. 05. 2009 v 13:51 +0300:
- MAKE_JOBS_NUMBER defaults (but user defined) to number of
cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS=-j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS=-j`${SYSCTL} -n kern.smp.cpus`
-.endif
Wouldn't that mean an evaluation of the backtick command in every
make(1) invocation? That would be highly undesirable.
Umm, why? it shouldn't be evaluated if MAKE_JOBS_NUMBER is defined, no?
Am I missing some make magic here?
--
IOnut - Un^d^dregistered ;) FreeBSD user
Intellectual Property is nowhere near as valuable as Intellect
FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B
signature.asc
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
Ion-Mihai Tetcu píše v ne 24. 05. 2009 v 19:01 +0300:
On Sun, 24 May 2009 16:10:23 +0200
Pav Lucistnik p...@freebsd.org wrote:
Ion-Mihai Tetcu píše v so 23. 05. 2009 v 13:51 +0300:
- MAKE_JOBS_NUMBER defaults (but user defined) to number of
cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
Wouldn't that mean an evaluation of the backtick command in every
make(1) invocation? That would be highly undesirable.
Umm, why? it shouldn't be evaluated if MAKE_JOBS_NUMBER is defined, no?
Am I missing some make magic here?
But for 99.99% of the users, MAKE_JOBS_NUMBER is not defined.
--
Pav Lucistnik p...@oook.cz
p...@freebsd.org
... the obese drugged penguin used by Linux. -- Scott Long
signature.asc
Description: Toto je digitálně podepsaná část zprávy
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sunday 24 May 2009 18:27:57 Pav Lucistnik wrote:
Ion-Mihai Tetcu píše v ne 24. 05. 2009 v 19:01 +0300:
On Sun, 24 May 2009 16:10:23 +0200
Pav Lucistnik p...@freebsd.org wrote:
Ion-Mihai Tetcu píše v so 23. 05. 2009 v 13:51 +0300:
- MAKE_JOBS_NUMBER defaults (but user defined) to number of
cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS=-j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS=-j`${SYSCTL} -n kern.smp.cpus`
-.endif
Wouldn't that mean an evaluation of the backtick command in every
make(1) invocation? That would be highly undesirable.
I don't believe that is the case.
Here is what I get with the patch applied (MAKE_JOBS_NUMBER not defined):
/usr/ports/editors/openoffice.org-3# make -V MAKE_JOBS_NUMBER -V _MAKE_JOBS
`/sbin/sysctl -n kern.smp.cpus`
-j`/sbin/sysctl -n kern.smp.cpus`
Wouldn't this indicate that the backtick command is not being evaluated?
The following does, however, make MAKE_JOBS_NUMBER resolve, and fixes ooo2
with parallel build.
# Multiple make jobs support
.if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE)
MAKE_JOBS_NUMBER= 1
_MAKE_JOBS= #
.else
.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
.if !defined(MAKE_JOBS_NUMBER)
MAKE_JOBS_NUMBER!= ${SYSCTL} -n kern.smp.cpus
.endif
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
[etc]
and then I get:
/usr/ports/editors/openoffice.org-3# make -V MAKE_JOBS_NUMBER -V _MAKE_JOBS
4
-j4
I agree that having sysctl being called every time is not good. I don't think
it is unavoidable in the ooo2 case (but that is only a few ports).
signature.asc
Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sun, 24 May 2009 10:26:23 +0200
David Naylor naylor.b.da...@gmail.com wrote:
On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
Hi I tested it yesterday,
1.
I need
MAKE_JOBS_SAFE=yes
in the Makefile.
Yes, you would need that. I believe that will be default.
2. with above patch, ooo2 doesn't launch parallele jobs.
I spotted that problem after submitting the patch, if you explicitly
set MAKE_JOBS_NUMBER to something it will work.
The problem is that ooo2 does (in effect):
.if (${MAKE_JOBS_NUMBER} 1)
# Stuff
.else
# Other stuff
.endif
and that doesn't work as expected with MAKE_JOBS_NUMBER=`sysctl
kern.smp.cpus` as the command is not resolved.
w/o patch
editors/openoffice.org-3openoffice.org-3.1.04:53:27
with patch:
+ MAKE_JOBS_SAFE= yes
+ MAKE_JOBS_NUMBER= 4
+ MAXPROCESSES?=${MAKE_JOBS_NUMBER}
+ MAXMODULES?= ${MAKE_JOBS_NUMBER}
editors/openoffice.org-3openoffice.org-3.1.048:51
The build is done in
/dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs, asynchronous, local,
noatime)
--
IOnut - Un^d^dregistered ;) FreeBSD user
Intellectual Property is nowhere near as valuable as Intellect
FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B
signature.asc
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi David Many many thanks for your patch. I'll test it very soon. Just one comment I believe openoffice-2* can me marked as SAFE while openoffice-3* should not be marked at all (since it sometimes works..., very well for me :-). you can mark as SAFE for all of our ports. If it's broken, its OOo issue. We should identify if dependencies are missing. -devel ports can be unsafe but 3, 3-RC 2, 2-RC must be safe. Please wait a few days to say ok. Best, From: David Naylor naylor.b.da...@gmail.com Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Sat, 23 May 2009 11:01:56 +0200 On Friday 22 May 2009 15:41:38 Ion-Mihai Tetcu wrote: On Fri, 22 May 2009 19:53:50 +0900 (JST) Maho NAKATA cha...@mac.com wrote: In massive parallel build, OOo can be broken. We explicitly fix them otherwise broken. I just test with MAXJOB = 4 or something like that. I had it complain about perl (or something) needing to be recompiled but that was because I interrupted the build process. It has always completed for me when using MAX* from the start. me too. but - note it just works for you. I can make the patch, only thing is bsd.port.mk will need to be patched (simple enough though). I can run a few test builds on QAT and maybe Phillip can do the same on his tindy. Just drop us the patch ;-) Even if it only works with MAXJOB = 2, we mark it as such and the build will be faster. Please see attached for the patch. The changes to bsd.port.mk: - MAKE_JOBS_NUMBER always defined - MAKE_JOBS_NUMBER forced to 1 if UNSAFE of DISABLE - MAKE_JOBS_NUMBER defaults (but user defined) to number of cores I've then used MAKE_JOBS_NUMBER to set MAXPROCESSES, MAXMODULES and NUMOFPROCESSES for openoffice-* (not including 1.*). I believe openoffice-2* can me marked as SAFE while openoffice-3* should not be marked at all (since it sometimes works..., very well for me :-). This patch just makes openoffice-* behave like other ports in regards to parallel builds and the usual MAKE_JOBS variables now works as expected. Happy testing, David pgpq2xMEndG96.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Sat, 23 May 2009 18:24:26 +0900 (JST)
Maho NAKATA cha...@mac.com wrote:
Hi David
Many many thanks for your patch. I'll test it very soon.
Just one comment
I believe openoffice-2* can me marked as SAFE while openoffice-3*
should not be marked at all (since it sometimes works..., very well
for me :-).
you can mark as SAFE for all of our ports.
For testing, right?
If it's broken, its OOo issue.
Obviously.
We should identify if dependencies are missing. -devel ports can be
unsafe but 3, 3-RC 2, 2-RC must be safe.
Please wait a few days to say ok.
I'll give them a try during this weekend.
From: David Naylor naylor.b.da...@gmail.com
Subject: Re: MAKE_JOBS_UNSAFE (some more ports)
Date: Sat, 23 May 2009 11:01:56 +0200
[ .. ]
Please see attached for the patch. The changes to bsd.port.mk:
- MAKE_JOBS_NUMBER always defined
- MAKE_JOBS_NUMBER forced to 1 if UNSAFE of DISABLE
AFAIR there are ports that compile OK w/o MAKE_JOBS_SAFE but fail with
MAKE_JOBS_NUMBER=1
- MAKE_JOBS_NUMBER defaults (but user defined) to number of cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS=-j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS=-j`${SYSCTL} -n kern.smp.cpus`
-.endif
I believe pav@ didn't put the ' !defined(MAKE_JOBS_SAFE)' part
intentionally until we get to test all our ports.
-.if defined(FORCE_MAKE_JOBS)
+.if defined(FORCE_MAKE_JOBS) !defined(MAKE_JOBS_SAFE)
BUILD_FAIL_MESSAGE+= You have chosen to use multiple make jobs
(parallelization) for all ports. This port was not tested for this setting.
Please remove FORCE_MAKE_JOBS and retry the build before reporting the failure
to the maintainer.
I've then used MAKE_JOBS_NUMBER to set MAXPROCESSES, MAXMODULES and
NUMOFPROCESSES for openoffice-* (not including 1.*).
I believe openoffice-2* can me marked as SAFE while openoffice-3*
should not be marked at all (since it sometimes works..., very well
for me :-).
This patch just makes openoffice-* behave like other ports in
regards to parallel builds and the usual MAKE_JOBS variables now
works as expected.
Nice, thanks.
--
IOnut - Un^d^dregistered ;) FreeBSD user
Intellectual Property is nowhere near as valuable as Intellect
FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B
signature.asc
Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Saturday 23 May 2009 12:51:33 Ion-Mihai Tetcu wrote:
On Sat, 23 May 2009 18:24:26 +0900 (JST)
Maho NAKATA cha...@mac.com wrote:
Please see attached for the patch. The changes to bsd.port.mk:
- MAKE_JOBS_NUMBER always defined
- MAKE_JOBS_NUMBER forced to 1 if UNSAFE of DISABLE
AFAIR there are ports that compile OK w/o MAKE_JOBS_SAFE but fail with
MAKE_JOBS_NUMBER=1
That is quite a problem. And this reveals a problem with openoffice-2*, it
doesn't work since it does (in-effect):
.if (${MAKE_JOBS_NUMBER} 1)
which will not work for MAKE_JOBS_NUMBER=`${SYSCTL} -n kern.smp.cpus`. Is
there anyway for MAKE_JOBS_NUMBER to get a resolved value (I think expanding
make to expose the number of cores on the system [rather radical, I know]).
If MAKE_JOBS_NUMBER can be 'fixed' then the solution is straight forward:
.if (${MAKE_JOBS_NUMBER} 1)
# Use concurrent build
.else
# Use standard build
.endif
- MAKE_JOBS_NUMBER defaults (but user defined) to number of cores
This part looks OK, I wonder if there's any reason t ain't like this
now; Pav?
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
I believe pav@ didn't put the ' !defined(MAKE_JOBS_SAFE)' part
intentionally until we get to test all our ports.
-.if defined(FORCE_MAKE_JOBS)
+.if defined(FORCE_MAKE_JOBS) !defined(MAKE_JOBS_SAFE)
BUILD_FAIL_MESSAGE+= You have chosen to use multiple make jobs
(parallelization) for all ports. This port was not tested for this
setting. Please remove FORCE_MAKE_JOBS and retry the build before
reporting the failure to the maintainer.
Sorry but I don't see how this would change anything. The message will only
get displayed if the port fails AND -DFORCE_MAKE_JOBS, which is the less
likely scenario.
I only changed it because when testing the command output with `make build -n`
the offset changed with -DFORCE_MAKE_JOBS on a safe port. I just found it
annoying...
I've then used MAKE_JOBS_NUMBER to set MAXPROCESSES, MAXMODULES and
NUMOFPROCESSES for openoffice-* (not including 1.*).
I believe openoffice-2* can me marked as SAFE while openoffice-3*
should not be marked at all (since it sometimes works..., very well
for me :-).
This patch just makes openoffice-* behave like other ports in
regards to parallel builds and the usual MAKE_JOBS variables now
works as expected.
Nice, thanks.
signature.asc
Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi I tested it yesterday, 1. I need MAKE_JOBS_SAFE=yes in the Makefile. 2. with above patch, ooo2 doesn't launch parallele jobs. 3. ooo3, 3-rc, 3-devel are okay with patch 1. thanks -- Nakata Maho http://accc.riken.jp/maho/ , http://ja.openoffice.org/ Nakata Maho's PGP public keys: http://accc.riken.jp/maho/maho.pgp.txt pgpLp4gHHSs1u.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Friday 22 May 2009 07:11:19 Maho NAKATA wrote: Dear, I appriciate David or Ion-Mihai make a patch for that. just seetting MAXMODULE=4 and/or MAXPROCESSES=4 or something like that. But note that sometimes it's broken :-( by missing dependencey. What do you mean by missing dependency? I had it complain about perl (or something) needing to be recompiled but that was because I interrupted the build process. It has always completed for me when using MAX* from the start. I can make the patch, only thing is bsd.port.mk will need to be patched (simple enough though). From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Fri, 22 May 2009 08:03:42 +0300 On Thu, 21 May 2009 12:05:22 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). Anything reducing OOo build time would be great :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
Hi David, From: David Naylor naylor.b.da...@gmail.com Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Fri, 22 May 2009 12:24:30 +0200 On Friday 22 May 2009 07:11:19 Maho NAKATA wrote: Dear, I appriciate David or Ion-Mihai make a patch for that. just seetting MAXMODULE=4 and/or MAXPROCESSES=4 or something like that. But note that sometimes it's broken :-( by missing dependencey. What do you mean by missing dependency? see http://www.openoffice.org/issues/show_bug.cgi?id=51745 http://www.openoffice.org/issues/show_bug.cgi?id=51755 ... ... ... ... In massive parallel build, OOo can be broken. We explicitly fix them otherwise broken. I just test with MAXJOB = 4 or something like that. I had it complain about perl (or something) needing to be recompiled but that was because I interrupted the build process. It has always completed for me when using MAX* from the start. me too. but - note it just works for you. I can make the patch, only thing is bsd.port.mk will need to be patched (simple enough though). Thank you. From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Fri, 22 May 2009 08:03:42 +0300 On Thu, 21 May 2009 12:05:22 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). Anything reducing OOo build time would be great :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B pgpHUkE8LIwdi.pgp Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
On Fri, 22 May 2009 19:53:50 +0900 (JST) Maho NAKATA cha...@mac.com wrote: In massive parallel build, OOo can be broken. We explicitly fix them otherwise broken. I just test with MAXJOB = 4 or something like that. I had it complain about perl (or something) needing to be recompiled but that was because I interrupted the build process. It has always completed for me when using MAX* from the start. me too. but - note it just works for you. I can make the patch, only thing is bsd.port.mk will need to be patched (simple enough though). I can run a few test builds on QAT and maybe Phillip can do the same on his tindy. Just drop us the patch ;-) Even if it only works with MAXJOB = 2, we mark it as such and the build will be faster. -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
Ion-Mihai Tetcu wrote: I can run a few test builds on QAT and maybe Phillip can do the same on his tindy. Just drop us the patch ;-) Even if it only works with MAXJOB = 2, we mark it as such and the build will be faster. LOL. Do it once and I get volunteered. Too funny. Sure I'm up for some patches, I make no gaurantees my TBs won't be doing other things at the same time though. -- Philip M. Gollucci (phi...@ridecharge.com) did: 703.579.6947, o: 703.549.2050x206 Senior System Admin - RideCharge Inc. http://ridecharge.com 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
MAKE_JOBS_UNSAFE (some more ports)
Hi, The following ports failed to build on my system (with a quad core) and FORCE_MAKE_JOBS set. They did success to build once I added MAKE_JOBS_UNSAFE=yes to their Makefile's. devel/nasm graphics/libart_lgpl lang/ocaml multimedia/mplayer multimedia/smplayer security/nss Is there any effort to mark ports as MAKE_JOBS_SAFE: is it desired for ports that are successful with FORCE_MAKE_JOBS to be reported? Regards David P.S. I'm not on the list P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). signature.asc Description: This is a digitally signed message part.
Re: MAKE_JOBS_UNSAFE (some more ports)
On Thu, 21 May 2009 12:05:22 +0200, David Naylor wrote The following ports failed to build on my system (with a quad core) and FORCE_MAKE_JOBS set. They did success to build once I added MAKE_JOBS_UNSAFE=yes to their Makefile's. Marked in CVS, thank you! Is there any effort to mark ports as MAKE_JOBS_SAFE: is it desired for ports that are successful with FORCE_MAKE_JOBS to be reported? Yes, I believe they should be reported. -- Pav Lucistnik p...@oook.cz p...@freebsd.org ___ freebsd-ports@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org
Re: MAKE_JOBS_UNSAFE (some more ports)
On Thu, 21 May 2009 12:05:22 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). Anything reducing OOo build time would be great :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B signature.asc Description: PGP signature
Re: MAKE_JOBS_UNSAFE (some more ports)
Dear, I appriciate David or Ion-Mihai make a patch for that. just seetting MAXMODULE=4 and/or MAXPROCESSES=4 or something like that. But note that sometimes it's broken :-( by missing dependencey. From: Ion-Mihai Tetcu ite...@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Date: Fri, 22 May 2009 08:03:42 +0300 On Thu, 21 May 2009 12:05:22 +0200 David Naylor naylor.b.da...@gmail.com wrote: P.P.S. editors/openoffice-3 does not obey MAKE_JOBS, it requires MAXMODULES and MAXPROCESSES set (should I file a PR?). Anything reducing OOo build time would be great :-) -- IOnut - Un^d^dregistered ;) FreeBSD user Intellectual Property is nowhere near as valuable as Intellect FreeBSD committer - ite...@freebsd.org, PGP Key ID 057E9F8B493A297B pgp1hsNptFU86.pgp Description: PGP signature
