Yeah thanks, it could have explained this indeed. But I guess bots would be green again if it was linked to this mlir issue 62687, as the fix for it was merged several hours ago and part of these jobs:
https://lab.llvm.org/buildbot/#/builders/198/builds/1879 https://lab.llvm.org/buildbot/#/builders/176/builds/2357 Actually Maxim probably found the issue: The swap was not enabled on these g3 machines, which is a problem as the build of flang (-j34) now needs a bit more than the 58GB of RAM allocated for the buildbot container. I also understand that a custom ninja binary is used on these containers, behaving differently when processes are waiting for something (memory/cpu). Enabling the swap on these linaro-g3 machines will probably fix the bot failures. And yes indeed, building flang looks to require a lot of memory: ~60Go RAM for a full build (j34) vs <20Go RAM for a build without flang ! Antoine On Tue, 16 May 2023 at 10:56, Andrzej Warzynski <andrzej.warzyn...@arm.com> wrote: > + @Kiran Chandramohan <kiran.chandramo...@arm.com> > > Kiran suggested that this might be this recent change in MLIR that’s > increased build times: https://github.com/llvm/llvm-project/issues/62687 > > -Andrzej > > Thanks for looking into this! > > Based on the logs and your experiment, looks like it's Flang's fault. 64GB > used to be sufficient, but perhaps that's no longer the case (sadly, Flang > has insane requirements). I will ask internally. > > If this is indeed Flang's fault, I'd be tempted to simply disable it on > this bot. We know that that sub-project is tested quite extensively through > dedicated Flang bots. What do you think? > > -Andrzej > > On 16/05/2023 07:23, Antoine Moynault wrote: > > Yeah sure, I am looking at this. > > Here the timeout happens after 1200 seconds without any new output on > stdout/stderr. > I reproduce the issue on tcwg-g3-01 with -j34 (the default on this > machine): the build is stuck near the end when building flang objects. > The same build succeeds using -j16 on this machine. > So I wonder if this could simply be due to insufficient memory. > > I will relaunch a few builds to check if something significant changed > with last commits. > > KO * 0245dc94ebee [test][sanitizers] Disable new test on Android > ?? * 795f93007e10 [AArch64] Add test for #62620. > ?? * 4fe91e083a25 [llvm-profdata] ProfileReader cleanup - preparation for > MD5 refactoring - 3 > ?? * be5747e51693 [flang] Fixed global name creation for literal constants. > OK * 26a7f4230613 [mlir][tosa] Add accumulator type attribute to TOSA > dialect > > > > Antoine > > > > On Mon, 15 May 2023 at 22:37, Maxim Kuvyrkov <maxim.kuvyr...@linaro.org> > wrote: > > Thanks, Andrzej! > > Hi Antoine, would you please take a look at this? > > Kind regards, > > -- > Maxim Kuvyrkov > https://www.linaro.org > > > > > > On May 15, 2023, at 22:55, Andrzej Warzynski <andrzej.warzyn...@arm.com> > wrote: > > > > Hi, > > > > Has anyone noticed that https://lab.llvm.org/buildbot/#/builders/198 > (clang-aarch64-sve-vla-2stage<https://lab.llvm.org/buildbot/#/builders/198>) > has been timing out for the past few days? "Duration" is often less than > 1hr, so that's odd. And all Flang buildbots are green, so it's unlikely > caused by changes to that sub-project ( > https://lab.llvm.org/buildbot/#/builders/198/builds/1804). Would anyone > be able to take a look? > > > > Best regards, > > > > Andrzej > > > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > > _______________________________________________ > > linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org > > To unsubscribe send an email to linaro-toolchain-le...@lists.linaro.org > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > _______________________________________________ linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org To unsubscribe send an email to linaro-toolchain-le...@lists.linaro.org
