Re: We need to do something about build times

2024-02-02 Thread Bryan Drewery
On 1/31/2024 12:16 PM, Bryan Drewery wrote: On 10/24/2023 12:12 PM, Robert Clausecker wrote:   - rework Poudriere's rebuild detection to not rebuild every port for     every random bullshit thing.  For example, I don't see why ports need     to be rebuilt for transitive changes in build

Re: We need to do something about build times

2024-01-31 Thread Bryan Drewery
On 10/24/2023 12:12 PM, Robert Clausecker wrote: - rework Poudriere's rebuild detection to not rebuild every port for every random bullshit thing. For example, I don't see why ports need to be rebuilt for transitive changes in build dependencies. E.g. if port A has build depends

Re: We need to do something about build times

2023-12-17 Thread Daniel Engberg
Hi, Given the discussion about build times I decided test a handful of ports to see if we could reduce time spent just by converting to either CMake or Meson depending on what upstream offers. Hardware: Oracle VM, 4x Neoverse N1 cores, 24Gb of RAM Software: FreeBSD 14.0, Poudriere (devel)

Re: We need to do something about build times

2023-11-01 Thread Tatsuki Makino
Sorry for the repetition. The previous fix for the critical flaw does not work properly. There should be no problem this time. .if 1 STAGEDIRPREFIX= ${LOCALBASE}/.stage STAGEDIR= ${STAGEDIRPREFIX}${.CURDIR}/${_WRKDIR}/stage _PORTS_DIRECTORIES+=${STAGEDIRPREFIX} _USES_fetch+=

Re: We need to do something about build times

2023-11-01 Thread Tatsuki Makino
Sorry for the repetition. Please stop using the following section, as a terrible flaw exists in it. Never use it. Tatsuki Makino wrote on 2023/10/31 18:20: > ${STAGEDIRPREFIX}/: .NOTMAIN .PHONY ${STAGEDIRPREFIX} > @${CHMOD} -R 1777 ${STAGEDIRPREFIX} The following will be the correct. .if

Re: We need to do something about build times

2023-10-31 Thread Tatsuki Makino
Sorry, I think this is getting too far off topic by now. But there is something that bothered me when using this. Tatsuki Makino wrote on 2023/10/31 18:20: > .if 1 > STAGEDIRPREFIX= ${LOCALBASE}/.stage > STAGEDIR= ${STAGEDIRPREFIX}${.CURDIR}/${_WRKDIR}/stage > _PORTS_DIRECTORIES+=

Re: We need to do something about build times

2023-10-31 Thread Tatsuki Makino
Tatsuki Makino wrote on 2023/10/31 17:57: > Changes for /usr/local/etc/poudriere.d/make.conf > The following block will be added so that the stage can also be placed in > tmpfs. > > .if 1 > STAGEDIRPREFIX= ${LOCALBASE}/.stage > STAGEDIR= ${STAGEDIRPREFIX}${.CURDIR}/${_WRKDIR}/stage >

Re: We need to do something about build times

2023-10-31 Thread Tatsuki Makino
Hello. Robert Clausecker wrote on 2023/10/25 04:12: > And it seems I'm slowly killing my build SSD like that. After just about > 9 months, it is already at 100 TB of writes just from port builds. > Building with workdirs in memory is no longer an option as that frequently > kills my build server

Re: We need to do something about build times

2023-10-30 Thread Edward Sanford Sutton, III
On 10/29/23 21:39, Tatsuki Makino wrote: Daniel Engberg wrote on 2023/10/28 17:48: If upstream uses GNU Autotools, use upstream release archives as they usually contains a configure script ready to run which means that you can avoid USES= autoreconf which is slow and adds unncessary

Re: We need to do something about build times

2023-10-29 Thread Tatsuki Makino
Daniel Engberg wrote on 2023/10/28 17:48: > If upstream uses GNU Autotools, use upstream release archives as they > usually contains a configure script ready to run which means that you > can avoid USES= autoreconf which is slow and adds unncessary > dependencies. This has the following problem

Re: We need to do something about build times

2023-10-29 Thread Tatsuki Makino
Tomoaki AOKI wrote on 2023/10/29 20:51: > On Sat, 28 Oct 2023 16:10:11 +0200 > Moin Rahman wrote: >> >> Maybe there are still some place of improvements in poudriere's change >> detection mechanism specially BUILD_DEPENDS. :P >> > > And maybe indirect dependencies, too. ;-( > > At worst,

Re: We need to do something about build times

2023-10-29 Thread Tomoaki AOKI
On Sat, 28 Oct 2023 16:10:11 +0200 Moin Rahman wrote: > > On Oct 28, 2023, at 12:32 AM, Moin Rahman wrote: > > > > > > > >> On Oct 27, 2023, at 11:37 PM, Tatsuki Makino > >> wrote: > >> > >> Moin Rahman wrote on 2023/10/28 01:50: > >>> But by no means it reduces the build > >>> time of

Re: We need to do something about build times

2023-10-28 Thread Tatsuki Makino
Moin Rahman wrote on 2023/10/28 23:10: >> I believe you do not have the latest tree. I have removed the build time >> dependency to texlive-base. >> >> And there is not call to texlive-base itself. > > In contrary to my previous comment I think that somehow poudriere detected > the change that

Re: We need to do something about build times

2023-10-28 Thread Moin Rahman
> On Oct 28, 2023, at 12:32 AM, Moin Rahman wrote: > > > >> On Oct 27, 2023, at 11:37 PM, Tatsuki Makino >> wrote: >> >> Moin Rahman wrote on 2023/10/28 01:50: >>> But by no means it reduces the build >>> time of texlive-texmf. Hope everyone enjoys that. :) >>> >> >> There are updates

Re: We need to do something about build times

2023-10-28 Thread Daniel Engberg
Hi, Many interesting thoughts about current situation, here's my take on it and I'm also trying to catch up. Some ports do require quite a lot of CPU time and memory to build, I fully understand that not everyone is using the latest hardware available but there's little we can do about upstream

Re: We need to do something about build times

2023-10-27 Thread Moin Rahman
> On Oct 27, 2023, at 11:37 PM, Tatsuki Makino > wrote: > > Moin Rahman wrote on 2023/10/28 01:50: >> But by no means it reduces the build >> time of texlive-texmf. Hope everyone enjoys that. :) >> > > There are updates in the current port tree that cause packages rebuilt by > glib updates

Re: We need to do something about build times

2023-10-27 Thread Tatsuki Makino
Moin Rahman wrote on 2023/10/28 01:50: > But by no means it reduces the build > time of texlive-texmf. Hope everyone enjoys that. :) > There are updates in the current port tree that cause packages rebuilt by glib updates to be rebuilt again :) > [00:01:19] [Dry Run] Deleting

Re: We need to do something about build times

2023-10-27 Thread Moin Rahman
> On Oct 27, 2023, at 6:43 AM, Tatsuki Makino > wrote: > > Hello. > > Moin Rahman wrote on 2023/10/27 02:07: >> texlive-texmf is one major requirement for doxygen which is a >> requirement of many other ports. This is one reason texlive-texmf is a >> common requirement. If you know of a

Re: We need to do something about build times

2023-10-26 Thread Tatsuki Makino
Hello. Moin Rahman wrote on 2023/10/27 02:07: > texlive-texmf is one major requirement for doxygen which is a > requirement of many other ports. This is one reason texlive-texmf is a > common requirement. If you know of a specific reason this one is rebuilt > more frequently let me know I will

Re: We need to do something about build times

2023-10-26 Thread Moin Rahman
> On Oct 24, 2023, at 9:12 PM, Robert Clausecker wrote: > > And it seems I'm slowly killing my build SSD like that. After just about > 9 months, it is already at 100 TB of writes just from port builds. > Building with workdirs in memory is no longer an option as that frequently > kills my

Re: We need to do something about build times

2023-10-25 Thread Tatsuki Makino
Hello. All the packages I have had poudriere make and hold were ready this morning (TZ=Asia/Tokyo :), 5 hours ago). However, this equilibrium was broken by glib update 4 hours ago :) 232 ports are now queued again. If only graphics/cairo is built manually first here, only 2 packages, glib and

Re: We need to do something about build times

2023-10-25 Thread Kevin Bowling
On Tue, Oct 24, 2023 at 1:55 PM Brooks Davis wrote: > > I'll reply to LLVM specific suggestions A big win would be to reduce the number of LLVMs required for common desktop ports. Aside from the base system LLVM, my desktop system seems to build llvm 12, 13, 15. > > On Tue, Oct 24, 2023 at

Re: We need to do something about build times

2023-10-25 Thread Fernando Apesteguía
On Tue, Oct 24, 2023 at 9:12 PM Robert Clausecker wrote: > The build times have gone up to the point where they are unsustainable. > Frequent updates to key ports (like llvm*, rust, gcc*) make it so that > basically every time I prepare a new batch of commits, I have to rebuild > a variety of

Re: We need to do something about build times

2023-10-24 Thread Tatsuki Makino
Hello. In the machine where those packages are installed, there's no need to update ports that don't show up in the result of pkg version. However, since the version number is also recorded in the dependencies in the package, everything is rebuilt just to rewrite it. For example, information

Re: We need to do something about build times

2023-10-24 Thread Brooks Davis
I'll reply to LLVM specific suggestions On Tue, Oct 24, 2023 at 09:12:13PM +0200, Robert Clausecker wrote: > - untangle some of the dependencies so that less ports may trigger >rebuilds of critical ports. For example, llvm docs could be moved to >separate ports so that updates in the

Re: We need to do something about build times

2023-10-24 Thread fuz
Hi Lorenzo, Am Tue, Oct 24, 2023 at 08:11:29PM + schrieb Lorenzo Salvadore: > Disabling LTO_BOOTSTRAP option by default has > already been done for the devel ports on the i386, amd64 > and aarch64 architectures (so for all tier 1 platforms): >

Re: We need to do something about build times

2023-10-24 Thread Lorenzo Salvadore
Hello, I am the maintainer of the GCC ports and thus I reply about this particular topic. Disabling LTO_BOOTSTRAP option by default has already been done for the devel ports on the i386, amd64 and aarch64 architectures (so for all tier 1 platforms):

Re: We need to do something about build times

2023-10-24 Thread Sysadmin Lists
> > From: Robert Clausecker > Date: Oct 24, 2023, 12:12:13 PM > To: > Subject: We need to do something about build times > > > The build times have gone up to the point where they are unsustainable. > Frequent updates to key p

Re: We need to do something about build times

2023-10-24 Thread Kurt Jaeger
Hi! > The build times have gone up to the point where they are unsustainable. Yes. Thank you for the great problem statement! -- p...@freebsd.org +49 171 3101372 Now what ?

We need to do something about build times

2023-10-24 Thread Robert Clausecker
The build times have gone up to the point where they are unsustainable. Frequent updates to key ports (like llvm*, rust, gcc*) make it so that basically every time I prepare a new batch of commits, I have to rebuild a variety of toolchain ports across 8 jails (amd64/i386/arm64/armv7 each for