Hi Quingsi,
This report is a false positive. We will investigate why noise levels have
increased in our benchmarking setup.
Regards,
--
Maxim Kuvyrkov
https://www.linaro.org
> On 13 Oct 2021, at 04:43, ci_not...@linaro.org wrote:
>
> After llvm commit 75127bce6de78b83b70b898a04473f213451f13e
> Author: Qiongsi Wu
>
>[AIX][ZOS] Excluding merge-objc-interface.m from Tests
>
> the following hot functions slowed down by more than 10% (but their
> benchmarks slowed down by less than 2%):
> - 433.milc:[.] mult_su3_mat_vec slowed down by 16% from 1615 to 1871 perf
> samples
>
> Below reproducer instructions can be used to re-build both "first_bad" and
> "last_good" cross-toolchains used in this bisection. Naturally, the scripts
> will fail when triggerring benchmarking jobs if you don't have access to
> Linaro TCWG CI.
>
> For your convenience, we have uploaded tarballs with pre-processed source and
> assembly files at:
> - First_bad save-temps:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-75127bce6de78b83b70b898a04473f213451f13e/save-temps/
> - Last_good save-temps:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-d01ae990e1fd6561ed86dc8004a7147dd09fb13c/save-temps/
> - Baseline save-temps:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-baseline/save-temps/
>
> Configuration:
> - Benchmark: SPEC CPU2006
> - Toolchain: Clang + Glibc + LLVM Linker
> - Version: all components were built from their tip of trunk
> - Target: aarch64-linux-gnu
> - Compiler flags: -O3
> - Hardware: NVidia TX1 4x Cortex-A57
>
> This benchmarking CI is work-in-progress, and we welcome feedback and
> suggestions at linaro-toolchain@lists.linaro.org . In our improvement plans
> is to add support for SPEC CPU2017 benchmarks and provide "perf
> report/annotate" data behind these reports.
>
> THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS,
> REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT.
>
> This commit has regressed these CI configurations:
> - tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3
>
> First_bad build:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-75127bce6de78b83b70b898a04473f213451f13e/
> Last_good build:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-d01ae990e1fd6561ed86dc8004a7147dd09fb13c/
> Baseline build:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/build-baseline/
> Even more details:
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/
>
> Reproduce builds:
>
> mkdir investigate-llvm-75127bce6de78b83b70b898a04473f213451f13e
> cd investigate-llvm-75127bce6de78b83b70b898a04473f213451f13e
>
> # Fetch scripts
> git clone https://git.linaro.org/toolchain/jenkins-scripts
>
> # Fetch manifests and test.sh script
> mkdir -p artifacts/manifests
> curl -o artifacts/manifests/build-baseline.sh
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/manifests/build-baseline.sh
> --fail
> curl -o artifacts/manifests/build-parameters.sh
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/manifests/build-parameters.sh
> --fail
> curl -o artifacts/test.sh
> https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/27/artifact/artifacts/test.sh
> --fail
> chmod +x artifacts/test.sh
>
> # Reproduce the baseline build (build all pre-requisites)
> ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh
>
> # Save baseline build state (which is then restored in artifacts/test.sh)
> mkdir -p ./bisect
> rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/
> --exclude /llvm/ ./ ./bisect/baseline/
>
> cd llvm
>
> # Reproduce first_bad build
> git checkout --detach 75127bce6de78b83b70b898a04473f213451f13e
> ../artifacts/test.sh
>
> # Reproduce last_good build
> git checkout --detach d01ae990e1fd6561ed86dc8004a7147dd09fb13c
> ../artifacts/test.sh
>
> cd ..
>
>
> Full commit (up to 1000 lines):
>
> commit 75127bce6de78b83b70b898a04473f213451f13e
> Author: Qiongsi Wu
> Date: Fri Oct 8 13:58:32 2021 +
>
>[AIX][ZOS] Excluding merge-objc-interface.m from Tests
>
>Objective C is not supported on AIX or ZOS. This patch excludes the newly
> added `clang/test/Modules/merge-objc-interface.m` (added by
> https://reviews.llvm.org/D110280) from AIX and ZOS testing.
>
>Many existing tests are already disabled by
>
