Re: poudriere: howto build ports with 'make -j x'?
On 7/08/2016 1:16 AM, Michael Grimm wrote: > Hi, > > I am used to build my ports by poudriere in parallel (-J x). But I do want to > build specific ports in parallel per se (make -j x) just to speed up build > times of huge ports like mariadb101-server. > > I did try to add an MAKEOPTS="-j8" into poudriere's specific > jailname-make.conf file, and I did try to use a jailname-setfile-make.conf as > well. But that didn't work for me. > > What would be the correct way to achieve what I want? > > Thanks and regards, > Michael > LOCALBASE/etc/poudriere.conf has some settings for this: # By default MAKE_JOBS is disabled to allow only one process per cpu # Use the following to allow it anyway #ALLOW_MAKE_JOBS=yes # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. ALLOW_MAKE_JOBS_PACKAGES="pkg gcc cmake py*" MAKE_JOBS is a ports framework variable (see Mk/bsd.port.mk) ./koobs ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
Hi Michael, you can set ALLOW_MAKE_JOBS_PACKAGES in /usr/local/etc/poudriere.conf to do that. # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. #ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*" and MAKE_JOBS_NUMBER in jailname-make.conf to set the number of jobs you want. On Sat, Aug 6, 2016, at 17:16, Michael Grimm wrote: > Hi, > > I am used to build my ports by poudriere in parallel (-J x). But I do > want to build specific ports in parallel per se (make -j x) just to speed > up build times of huge ports like mariadb101-server. > > I did try to add an MAKEOPTS="-j8" into poudriere's specific > jailname-make.conf file, and I did try to use a > jailname-setfile-make.conf as well. But that didn't work for me. > > What would be the correct way to achieve what I want? > > Thanks and regards, > Michael > > ___ > freebsd-ports@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
>From my poudriere.conf: # By default MAKE_JOBS is disabled to allow only one process per cpu # Use the following to allow it anyway ALLOW_MAKE_JOBS=no # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py* perl5.20" Hope this helps! -- Christian Schwarz ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
Christian Schwarz wrote: > From my poudriere.conf: > > # By default MAKE_JOBS is disabled to allow only one process per cpu > # Use the following to allow it anyway > ALLOW_MAKE_JOBS=no > > # List of packages that will always be allowed to use MAKE_JOBS > # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports > # which holdup the rest of the queue to build more quickly. > ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py* perl5.20" > > Hope this helps! Yes, it helped. Especially me being blind to find that in the first place :-( Thanks to all who helped me finding these knobs. Regards, Michael ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
If you plan on running build in parallel it maybe necessary to also set MAKE_JOBS_NUMBER_LIMIT= into the make.conf file as well. On Sat, Aug 6, 2016 at 11:49 AM, Michael Grimm wrote: > Christian Schwarz wrote: > > > From my poudriere.conf: > > > > # By default MAKE_JOBS is disabled to allow only one process per cpu > > # Use the following to allow it anyway > > ALLOW_MAKE_JOBS=no > > > > # List of packages that will always be allowed to use MAKE_JOBS > > # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports > > # which holdup the rest of the queue to build more quickly. > > ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py* perl5.20" > > > > Hope this helps! > > Yes, it helped. Especially me being blind to find that in the first place > :-( > > Thanks to all who helped me finding these knobs. > > Regards, > Michael > > ___ > freebsd-ports@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" > ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
El día Saturday, August 06, 2016 a las 02:13:39PM -0400, Ultima escribió: > If you plan on running build in parallel it maybe necessary to also > set MAKE_JOBS_NUMBER_LIMIT= into the make.conf file as well. I have a list of some 270 ports which poudriere should work through with 3 builders, resulting in some 1700 ports to build: poudriere bulk -f /usr/local/etc/poudriere-list -J 3 -j freebsd-head -p ports-head This list does not include some ports which are verry time consuming on my hardware (2 CPU 3.7 GHz) like www/firefox www/chromium editors/openoffice-devel editors/libreoffice ... These I do run as single steps each from a shell script poudriere bulk -j freebsd-head -p ports-head www/firefox poudriere bulk -j freebsd-head -p ports-head www/chromium ... matthias -- Matthias Apitz, ✉ g...@unixarea.de, ⌂ http://www.unixarea.de/ ☎ +49-176-38902045 "Ohne die Mauer hätte es Krieg gegeben" Fritz Streletz u.a. "Sin el Muro hubiese habido guerra." ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: poudriere: howto build ports with 'make -j x'?
Chromium and Libreoffice are two programs that are massive and have the longest compile times in the ports tree along with eclipse. If the system only has 2 cpus I would expect those to take at least 4-8 hours minimum. If they are also building in parallel you can expect that to also increase. My system takes 40 minutes for Chromium and 50 for Libreoffice, and thats with jobs=40 on 2.0 cores. Sorry if my comment was confusing, this variable's use-case is better to limit the amount of jobs per jail on a given system. Enabling ALLOW_MAKE_JOBS=yes will set the makejobs on all jails to maximum amount of cpu's, in you're case 2. The reason I made my comment, if makejobs > 64 some ports will begin to fail to build due to too many makejobs. Also, for multiple threaded cpu's it may increase performance setting it lower when running many jails in parallel. > These I do run as single steps each from a shell script > poudriere bulk -j freebsd-head -p ports-head www/firefox > poudriere bulk -j freebsd-head -p ports-head www/chromium It may run faster, but only because they are not building in parallel. On Thu, Aug 11, 2016 at 3:49 AM, Matthias Apitz wrote: > El día Saturday, August 06, 2016 a las 02:13:39PM -0400, Ultima escribió: > > > If you plan on running build in parallel it maybe necessary to also > > set MAKE_JOBS_NUMBER_LIMIT= into the make.conf file as well. > > I have a list of some 270 ports which poudriere should work through with > 3 builders, resulting in some 1700 ports to build: > > poudriere bulk -f /usr/local/etc/poudriere-list -J 3 -j freebsd-head -p > ports-head > > This list does not include some ports which are verry time consuming on > my hardware (2 CPU 3.7 GHz) like > > www/firefox > www/chromium > editors/openoffice-devel > editors/libreoffice > ... > > These I do run as single steps each from a shell script > > poudriere bulk -j freebsd-head -p ports-head www/firefox > poudriere bulk -j freebsd-head -p ports-head www/chromium > ... > > matthias > -- > Matthias Apitz, ✉ g...@unixarea.de, ⌂ http://www.unixarea.de/ ☎ > +49-176-38902045 > "Ohne die Mauer hätte es Krieg gegeben" Fritz Streletz u.a. > "Sin el Muro hubiese habido guerra." > ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"