From: Deepesh Varatharajan <[email protected]>
This patch series introduces a test framework to run selftests for
target builds of Clang, LLVM, and LLD.
v3 changes:
- llvm_sysroot_preprocess_testsuite function needs be added to
llvm:class-target not llvm:class-nativesdk.
v2 changes:
- Enable required CMake flags only when running the testsuite.
- Ensure default build time and package sizes remain unaffected.
Summary of changes:
-Patch 1: Disable clang-tools-extra tests, as Clang's CMake cannot
detect the llvm-bcanalyzer target.
-Patch 2: Enabled the necessary CMake flags for clang, llvm & lld tests
when CLANG_ENABLE_TESTSUITE is set to "1".
-Patch 3: Selftest script for Clang, LLVM, and LLD including exclude list
-Patch 4: Fix rust build failure by replacing target llvm-config with native
Testing:
1. The test suite has been successfully validated on x86-64, arm64, riscv64
and arm32 for all Clang, LLD & LLVM components.
2. Clang and LLD tests passed successfully on x86. LLVM tests are skipped
on x86 since it's broken. A follow-up bug will be filed and addressed.
3. Clang, LLVM & LLD all together ~80K tests are there and we currently
~250 tests to exclude list which are failing (0.32%).
Impact on build time:
These builds were executed in a controlled environment without poky-sstate
enabled.
Default(without changes):
+-----------+------------+------------+------------+----------------+
| Target | Run 1 | Run 2 | Run 3 | Average |
| --------- | ---------- | ---------- | ---------- | -------------- |
| llvm | 22m18.441s | 22m31.905s | 22m23.718s | 22m24.688s |
| clang | 30m09.774s | 30m22.680s | 30m15.312s | 30m15.922s |
| lld | 1m26.994s | 1m30.415s | 1m29.006s | 1m28.805s |
+-----------+------------+------------+------------+----------------+
With the testsuite changes applied:
+-----------+------------+------------+------------+----------------+
| Target | Run 1 | Run 2 | Run 3 | Average |
| --------- | ---------- | ---------- | ---------- | -------------- |
| llvm | 23m32.104s | 23m45.887s | 23m38.157s | 23m38.716s |
| clang | 30m31.402s | 30m46.198s | 30m39.445s | 30m39.015s |
| lld | 1m29.845s | 1m34.212s | 1m31.925s | 1m31.994s |
+-----------+------------+------------+------------+----------------+
Comparison:
The additional overhead is minimal and primarily due to enabling
test-related build configurations.
+-----------+-------------+------------------+------------------+
| Target | Default Avg | With Changes Avg | **Diff** |
| --------- | ----------- | ---------------- | ---------------- |
| llvm | 22m24.688s | 23m38.716s | +1m14.028s |
| clang | 30m15.922s | 30m39.015s | +0m23.093s |
| lld | 1m28.805s | 1m31.994s | +0m3.189s |
+-----------+-------------+------------------+------------------+
Impact on package size and Fix:
The testsuite changes has a negative impact on certain package sizes,
as below:
+---------------------+----------+----------+-----------+------------+
| Package | Old (MB) | New (MB) | Diff (MB) | Change (%) |
+=====================+==========+==========+===========+============+
| clang/clang-dbg | 44.99 | 59.79 | +14.81 | +32.92% |
| clang/clang-dev | 30.22 | 30.22 | -0.00 | -0.00% |
| clang/clang-tools | 5.05 | 29.58 | +24.53 | +485.87% |
| llvm/llvm-bin | 16.40 | 71.77 | +55.36 | +337.50% |
| llvm/llvm-dbg | 386.79 | 646.12 | +259.33 | +67.05% |
| llvm/llvm-dev | 32.67 | 33.67 | +1.00 | +3.05% |
| llvm/llvm-src | 387.54 | 389.44 | +1.90 | +0.49% |
| llvm/llvm-staticdev | 609.68 | 616.80 | +7.12 | +1.17% |
+---------------------+----------+----------+-----------+------------+
FIX:
All testsuite-related changes are now gated by the CLANG_ENABLE_TESTSUITE
variable (default "0"), ensuring:
- Test-specific CMake flags are applied only during test execution.
- Standard builds remain fully unaffected in both build time and package size.
Deepesh Varatharajan (4):
clang-tools-extra: disable tests
clang: Use CLANG_ENABLE_TESTSUITE to enable LLVM, Clang, and LLD tests
oeqa/selftest/clang: Add oe-seltests for Clang/LLVM/LLD
rust: Fix rust build failure
meta/lib/oeqa/selftest/cases/clang.py | 299 ++++++++++++++++++
.../0042-guard-clang-tools-extra-test.patch | 74 +++++
meta/recipes-devtools/clang/clang_git.bb | 8 +
meta/recipes-devtools/clang/common.inc | 1 +
meta/recipes-devtools/clang/lld_git.bb | 5 +
meta/recipes-devtools/clang/llvm_git.bb | 18 +-
meta/recipes-devtools/rust/rust_1.93.0.bb | 5 +-
7 files changed, 407 insertions(+), 3 deletions(-)
create mode 100644 meta/lib/oeqa/selftest/cases/clang.py
create mode 100644
meta/recipes-devtools/clang/clang/0042-guard-clang-tools-extra-test.patch
--
2.49.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#232189):
https://lists.openembedded.org/g/openembedded-core/message/232189
Mute This Topic: https://lists.openembedded.org/mt/118091332/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
