OK , thanks - appreciate all the hard work of making performance regression analysis & continued improvement to reduce false positives!
On Tue, Dec 7, 2021 at 4:03 AM Maxim Kuvyrkov <maxim.kuvyr...@linaro.org> wrote: > Hi David, > > This is a false positive, sorry for the noise. Our CI bisects performance > regressions down to a single commit, and notifies patch authors about > significant regressions. > > Since this is benchmarking CI, some noise is expected. Apparently, > 433.milc started to show bi-modal performance, and bisection [mistakenly] > converged on this patch. We work to reduce benchmarking noise and > otherwise improve benchmarking CI. > > Regards! > > -- > Maxim Kuvyrkov > https://www.linaro.org > > > On 7 Dec 2021, at 06:12, David Blaikie <dblai...@gmail.com> wrote: > > > > Seems... unlikely this change had a performance impact. > > > > Also is this email meant to be sent to public contributors like myself, > or > > only intended for some Linaro folks? > > > > On Sun, Dec 5, 2021 at 6:18 AM <ci_not...@linaro.org> wrote: > > > >> After llvm commit bd4c6a476fd037fb07a1c484f75d93ee40713d3d > >> Author: David Blaikie <dblai...@gmail.com> > >> > >> Add missing header > >> > >> the following benchmarks slowed down by more than 2%: > >> - 433.milc slowed down by 4% from 12427 to 12916 perf samples > >> > >> Below reproducer instructions can be used to re-build both "first_bad" > and > >> "last_good" cross-toolchains used in this bisection. Naturally, the > >> scripts will fail when triggerring benchmarking jobs if you don't have > >> access to Linaro TCWG CI. > >> > >> For your convenience, we have uploaded tarballs with pre-processed > source > >> and assembly files at: > >> - First_bad save-temps: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-bd4c6a476fd037fb07a1c484f75d93ee40713d3d/save-temps/ > >> - Last_good save-temps: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7/save-temps/ > >> - Baseline save-temps: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-baseline/save-temps/ > >> > >> Configuration: > >> - Benchmark: SPEC CPU2006 > >> - Toolchain: Clang + Glibc + LLVM Linker > >> - Version: all components were built from their tip of trunk > >> - Target: aarch64-linux-gnu > >> - Compiler flags: -O2 -flto > >> - Hardware: NVidia TX1 4x Cortex-A57 > >> > >> This benchmarking CI is work-in-progress, and we welcome feedback and > >> suggestions at linaro-toolchain@lists.linaro.org . In our improvement > >> plans is to add support for SPEC CPU2017 benchmarks and provide "perf > >> report/annotate" data behind these reports. > >> > >> THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, > >> REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT. > >> > >> This commit has regressed these CI configurations: > >> - tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O2_LTO > >> > >> First_bad build: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-bd4c6a476fd037fb07a1c484f75d93ee40713d3d/ > >> Last_good build: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7/ > >> Baseline build: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/build-baseline/ > >> Even more details: > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/ > >> > >> Reproduce builds: > >> <cut> > >> mkdir investigate-llvm-bd4c6a476fd037fb07a1c484f75d93ee40713d3d > >> cd investigate-llvm-bd4c6a476fd037fb07a1c484f75d93ee40713d3d > >> > >> # Fetch scripts > >> git clone https://git.linaro.org/toolchain/jenkins-scripts > >> > >> # Fetch manifests and test.sh script > >> mkdir -p artifacts/manifests > >> curl -o artifacts/manifests/build-baseline.sh > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/manifests/build-baseline.sh > >> --fail > >> curl -o artifacts/manifests/build-parameters.sh > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/manifests/build-parameters.sh > >> --fail > >> curl -o artifacts/test.sh > >> > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O2_LTO/34/artifact/artifacts/test.sh > >> --fail > >> chmod +x artifacts/test.sh > >> > >> # Reproduce the baseline build (build all pre-requisites) > >> ./jenkins-scripts/tcwg_bmk-build.sh @@ > >> artifacts/manifests/build-baseline.sh > >> > >> # Save baseline build state (which is then restored in > artifacts/test.sh) > >> mkdir -p ./bisect > >> rsync -a --del --delete-excluded --exclude /bisect/ --exclude > /artifacts/ > >> --exclude /llvm/ ./ ./bisect/baseline/ > >> > >> cd llvm > >> > >> # Reproduce first_bad build > >> git checkout --detach bd4c6a476fd037fb07a1c484f75d93ee40713d3d > >> ../artifacts/test.sh > >> > >> # Reproduce last_good build > >> git checkout --detach 7d4da4e1ab7f79e51db0d5c2a0f5ef1711122dd7 > >> ../artifacts/test.sh > >> > >> cd .. > >> </cut> > >> > >> Full commit (up to 1000 lines): > >> <cut> > >> commit bd4c6a476fd037fb07a1c484f75d93ee40713d3d > >> Author: David Blaikie <dblai...@gmail.com> > >> Date: Mon Nov 29 16:29:25 2021 -0800 > >> > >> Add missing header > >> --- > >> llvm/lib/Demangle/DLangDemangle.cpp | 1 + > >> 1 file changed, 1 insertion(+) > >> > >> diff --git a/llvm/lib/Demangle/DLangDemangle.cpp > >> b/llvm/lib/Demangle/DLangDemangle.cpp > >> index faf91b239490..f380aa90035e 100644 > >> --- a/llvm/lib/Demangle/DLangDemangle.cpp > >> +++ b/llvm/lib/Demangle/DLangDemangle.cpp > >> @@ -17,6 +17,7 @@ > >> #include "llvm/Demangle/StringView.h" > >> #include "llvm/Demangle/Utility.h" > >> > >> +#include <cctype> > >> #include <cstring> > >> #include <limits> > >> > >> </cut> > > _______________________________________________ > > linaro-toolchain mailing list > > linaro-toolchain@lists.linaro.org > > https://lists.linaro.org/mailman/listinfo/linaro-toolchain > > _______________________________________________ linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain
