Hi David, This is a false positive, sorry for the noise. Our CI bisects performance regressions down to a single commit, and notifies patch authors about significant regressions.
Since this is benchmarking CI, some noise is expected. Apparently, 433.milc started to show bi-modal performance, and bisection [mistakenly] converged on this patch. We work to reduce benchmarking noise and otherwise improve benchmarking CI. Regards! -- Maxim Kuvyrkov https://www.linaro.org > On 7 Dec 2021, at 06:12, David Blaikie <dblai...@gmail.com> wrote: > > Seems... unlikely this change had a performance impact. > > Also is this email meant to be sent to public contributors like myself, or > only intended for some Linaro folks? > > On Sun, Dec 5, 2021 at 6:18 AM <ci_not...@linaro.org> wrote: > >> After llvm commit bd4c6a476fd037fb07a1c484f75d93ee40713d3d >> Author: David Blaikie <dblai...@gmail.com> >> >> Add missing header >> >> the following benchmarks slowed down by more than 2%: >> - 433.milc slowed down by 4% from 12427 to 12916 perf samples >> >> Below reproducer instructions can be used to re-build both "first_bad" and >> "last_good" cross-toolchains used in this bisection. Naturally, the >> scripts will fail when triggerring benchmarking jobs if you don't have >> access to Linaro TCWG CI. >> >> For your convenience, we have uploaded tarballs with pre-processed source >> and assembly files at: >> - First_bad save-temps: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-bd4c6a476fd037fb07a1c484f75d93ee40713d3d/save-temps/ >> - Last_good save-temps: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7/save-temps/ >> - Baseline save-temps: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-baseline/save-temps/ >> >> Configuration: >> - Benchmark: SPEC CPU2006 >> - Toolchain: Clang + Glibc + LLVM Linker >> - Version: all components were built from their tip of trunk >> - Target: aarch64-linux-gnu >> - Compiler flags: -O2 -flto >> - Hardware: NVidia TX1 4x Cortex-A57 >> >> This benchmarking CI is work-in-progress, and we welcome feedback and >> suggestions at linaro-toolchain@lists.linaro.org . In our improvement >> plans is to add support for SPEC CPU2017 benchmarks and provide "perf >> report/annotate" data behind these reports. >> >> THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, >> REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT. >> >> This commit has regressed these CI configurations: >> - tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O2_LTO >> >> First_bad build: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-bd4c6a476fd037fb07a1c484f75d93ee40713d3d/ >> Last_good build: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7/ >> Baseline build: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-baseline/ >> Even more details: >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/ >> >> Reproduce builds: >> <cut> >> mkdir investigate-llvm-bd4c6a476fd037fb07a1c484f75d93ee40713d3d >> cd investigate-llvm-bd4c6a476fd037fb07a1c484f75d93ee40713d3d >> >> # Fetch scripts >> git clone https://git.linaro.org/toolchain/jenkins-scripts >> >> # Fetch manifests and test.sh script >> mkdir -p artifacts/manifests >> curl -o artifacts/manifests/build-baseline.sh >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/manifests/build-baseline.sh >> --fail >> curl -o artifacts/manifests/build-parameters.sh >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/manifests/build-parameters.sh >> --fail >> curl -o artifacts/test.sh >> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/test.sh >> --fail >> chmod +x artifacts/test.sh >> >> # Reproduce the baseline build (build all pre-requisites) >> ./jenkins-scripts/tcwg_bmk-build.sh @@ >> artifacts/manifests/build-baseline.sh >> >> # Save baseline build state (which is then restored in artifacts/test.sh) >> mkdir -p ./bisect >> rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ >> --exclude /llvm/ ./ ./bisect/baseline/ >> >> cd llvm >> >> # Reproduce first_bad build >> git checkout --detach bd4c6a476fd037fb07a1c484f75d93ee40713d3d >> ../artifacts/test.sh >> >> # Reproduce last_good build >> git checkout --detach 7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7 >> ../artifacts/test.sh >> >> cd .. >> </cut> >> >> Full commit (up to 1000 lines): >> <cut> >> commit bd4c6a476fd037fb07a1c484f75d93ee40713d3d >> Author: David Blaikie <dblai...@gmail.com> >> Date: Mon Nov 29 16:29:25 2021 -0800 >> >> Add missing header >> --- >> llvm/lib/Demangle/DLangDemangle.cpp | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/llvm/lib/Demangle/DLangDemangle.cpp >> b/llvm/lib/Demangle/DLangDemangle.cpp >> index faf91b239490..f380aa90035e 100644 >> --- a/llvm/lib/Demangle/DLangDemangle.cpp >> +++ b/llvm/lib/Demangle/DLangDemangle.cpp >> @@ -17,6 +17,7 @@ >> #include "llvm/Demangle/StringView.h" >> #include "llvm/Demangle/Utility.h" >> >> +#include <cctype> >> #include <cstring> >> #include <limits> >> >> </cut> > _______________________________________________ > linaro-toolchain mailing list > linaro-toolchain@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/linaro-toolchain _______________________________________________ linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain
