** Description changed:
- In src:rocm we have a `run-tests` script that runs the tests for all (ideally)
- packages in rocm stack. See the logic we have as of today in src:rocm
- ```
- my @pkgs = qw(
- libhipblas3-tests
- libhipcub-tests
- libhipfft0-tests
- libhiprand1-tests
- libhipsolver1-tests
- libhipsparse4-tests
- libmiopen1-tests
- librccl1-tests
- librocblas5-tests
- librocfft0-tests
- librocprim-tests
- librocrand1-tests
- librocsolver0-tests
- librocsparse1-tests
- librocthrust-tests
- );
+ [ Impact ]
- my $result = 0;
- foreach my $pkg (@pkgs) {
- my $test = "/usr/libexec/rocm/$pkg/run-tests";
- if (-x $test) {
- my $options = '';
- print("--- Starting $pkg ---\n");
- my $status = system("$test$options");
- my $msg = ($status == 0) ? 'PASS' : 'FAIL';
- print("--- Finished $pkg: $msg ---\n");
- $result += $status;
- } else {
- warn("WARNING: Skipping $pkg - test binary not found at $test\n");
- }
- }
- exit $result;
- ```
+ * It is standard in the ROCm stack for each test package to provide a
`/usr/libexec/rocm/<package>/run-tests` entry point. The `librccl1-tests`
package does not follow this convention.
+ As a result, when the ROCm stack's main test runner
(`/usr/libexec/rocm/rocm-tests/run-tests` from `src:rocm`) iterates through all
ROCm packages to execute their tests, it skips RCCL and reports:
- The lack of standard in test files causing us to create exceptional `if`
blocks which
- makes things look a bit ugly in my opinion. So lets follow the standard set
by other
- packages in the stack to make our life easier.
+ > WARNING: Skipping librccl1-tests - test binary not found at
+ /usr/libexec/rocm/librccl1-tests/run-tests
- See the example run of `run-tests` in bin:rocm-tests
- ```
- root@fit-mole:~# /usr/libexec/rocm/rocm-tests/run-tests
- --- Starting libhipblas3-tests ---
- --- Finished libhipblas3-tests: PASS ---
- WARNING: Skipping libhipcub-tests - test binary not found at
/usr/libexec/rocm/libhipcub-tests/run-tests
- --- Starting libhipfft0-tests ---
- --- Finished libhipfft0-tests: PASS ---
- --- Starting libhiprand1-tests ---
- --- Finished libhiprand1-tests: PASS ---
- --- Starting libhipsolver1-tests ---
- --- Finished libhipsolver1-tests: PASS ---
- --- Starting libhipsparse4-tests ---
- --- Finished libhipsparse4-tests: PASS ---
- WARNING: Skipping libmiopen1-tests - test binary not found at
/usr/libexec/rocm/libmiopen1-tests/run-tests
- WARNING: Skipping librccl1-tests - test binary not found at
/usr/libexec/rocm/librccl1-tests/run-tests
- --- Starting librocblas5-tests ---
- --- Finished librocblas5-tests: PASS ---
- --- Starting librocfft0-tests ---
- --- Finished librocfft0-tests: PASS ---
- --- Starting librocprim-tests ---
- --- Finished librocprim-tests: PASS ---
- --- Starting librocrand1-tests ---
- --- Finished librocrand1-tests: PASS ---
- --- Starting librocsolver0-tests ---
- --- Finished librocsolver0-tests: PASS ---
- --- Starting librocsparse1-tests ---
- --- Finished librocsparse1-tests: PASS ---
- WARNING: Skipping librocthrust-tests - test binary not found at
/usr/libexec/rocm/librocthrust-tests/run-tests
- --- Starting libhsa-runtime64-tests ---
- --- Finished libhsa-runtime64-tests: PASS ---
- ```
+ * The update adds the standard `run-tests` entry point to
+ `librccl1-tests`, accounting for RCCL-specific test requirements. It
+ also updates autopkgtest to use this entry point and the `rocm-test-
+ launcher` provided by `pkg-rocm-tools`, replacing the package-specific
+ `upstream-binaries` script.
+
+ * This change ensures RCCL tests are included in stack-wide test
+ execution, both through autopkgtest and through the ROCm stack test
+ runner, bringing `librccl1-tests` into compliance with ROCm stack
+ conventions.
+
+ [ Test Plan ]
+
+ * Reproduce the issue with the current release package.
+ - Install `librccl1-tests` and `rocm-tests` (from `src:rocm`).
+ - Confirm that `/usr/libexec/rocm/librccl1-tests/run-tests` is absent.
+ - Run `/usr/libexec/rocm/rocm-tests/run-tests`.
+ - Confirm that the output includes the warning that `librccl1-tests` is
skipped because its `run-tests` entry point is missing (as shown in the example
output provided).
+
+ * Verify the updated package directly.
+ - Install the candidate `librccl1-tests` package.
+ - Confirm that `/usr/libexec/rocm/librccl1-tests/run-tests` exists and is
executable.
+ - Run `/usr/libexec/rocm/librccl1-tests/run-tests`.
+ - Confirm that the RCCL tests are actually executed, rather than merely
that the wrapper exits successfully (for this a system with at lest 2 AMD
supported GPUs is needed)
+
+ * Verify integration with the ROCm stack test runner.
+ - Run `/usr/libexec/rocm/rocm-tests/run-tests` (from `src:rocm`).
+ - Confirm that it now includes `librccl1-tests` in the test iteration,
starts it and executes its tests.
+ - Confirm that the missing-entry-point warning for `librccl1-tests` no
longer appears.
+ - Check that the other installed ROCm test packages continue to execute as
before.
+
+ * Run autopkgtest for the updated source package.
+ - Run the package's autopkgtest in a GPU-enabled environment with the
candidate packages available.
+ - Confirm that the updated test definition installs `pkg-rocm-tools` and
invokes the packaged `run-tests` entry point.
+ - Confirm that the RCCL tests execute successfully through
`rocm-test-launcher`.
+
+ * The successful test runs have been confirmed with the SRU: New
+ upstream version 7.2.4 -
+ https://bugs.launchpad.net/ubuntu/+source/rccl/+bug/2154290 bug
+
+ [ Where problems could occur ]
+
+ * The new entry point could be installed at the wrong location or
+ without executable permissions, causing the ROCm stack runner to
+ continue skipping RCCL.
+
+ * Incorrect handling of RCCL test requirements could prevent tests from
+ starting, omit tests, or change their execution environment.
+
+ * Replacing `upstream-binaries` with `rocm-test-launcher` could change
+ test discovery or failure reporting. In particular, a wrapper that
+ succeeds without executing tests, or fails to propagate test failures,
+ could produce misleading results.
+
+ Testing should therefore verify actual test execution and results, not
+ only the presence of the new entry point or a successful wrapper exit
+ status.
+
+ [ Other Info ]
+
+ * This update addresses LP: #2148387 and brings `librccl1-tests` into
+ compliance with the standard ROCm stack test packaging convention.
+
+ * The packaging changes are:
+ - d/bin/run-tests: introduce the new binary with the special rccl
+ circumstances taken in account; Drop fixed ManagedMem gtest filter
+ - d/librccl1-tests.install: add run-tests
+ - d/tests/control: update autopkgtest to run the run-tests binary
+ instead of upstream-binaries and add pkg-rocm-tools as depends
+ - d/tests/upstream-binaries: remove in favor of rocm-test-launcher
+ provided by pkg-rocm-tools
+
+ * The change is limited to test infrastructure; it does not change RCCL
+ library functionality. Hardware-backed validation is required to
+ demonstrate that the previously skipped tests now execute successfully.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2148387
Title:
Missing run-tests binary to be used in rocm-tests
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rccl/+bug/2148387/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
