Public bug reported:
[ Impact ]
On arm64, Ubuntu 26.04's OpenBLAS (0.3.32+ds-5) skips three CBLAS calls
entirely on non-SME kernel sets.
These calls leave the output matrix unchanged and report no error:
* cblas_strsm and cblas_strmm with CblasRowMajor, CblasLeft and CblasNonUnit,
when lda == m and ldb == n;
* cblas_ssymm with CblasRowMajor and CblasLeft, when lda == m, ldb == n and
ldc == n.
These are ordinary, tightly packed row-major arrays of C and C++ code. The same
calls with padded leading
dimensions compute correctly. A program solving a triangular system with
cblas_strsm silently keeps the
right-hand side as its "solution".
The faulty entry-point code is shared by the pthread, openmp and serial
flavours. The local before/after
package build and runtime tests used pthread. The census exercised all 13
non-SME kernel sets: nine natively
on an Apple M5 VM and four SVE sets under qemu-aarch64 -cpu max, using
OPENBLAS_CORETYPE. This tests the
different kernel selections; it is not a set of measurements on 13 physical CPU
models. The additional
ARMV9SME set has separate faults which this patch does not fix, as described
below.
The fault is in the CBLAS entry points. OpenBLAS 0.3.31 added an SME
fast path for these three routines, behind
if (strcmp(gotoblas_corename(), "armv9sme") == 0)
if (args.m == 0 || args.n == 0) return;
if (order == CblasRowMajor && ... && m == lda && n == ldb) { <SME kernel>;
return; }
The core test governs only the quick return, so the fast path runs on every
core. On every core except ARMV9SME,
the "SME kernel" is an empty function. Upstream issue #5763 was fixed in 0.3.33
by PR #5767, which moved the
quick return in front of the core test. This proposal backports those two
one-line moves
(d/p/arm64-cblas-sme-direct-gate.patch).
Not affected by this dispatch defect:
* the Fortran interface (strsm_, strmm_, ssymm_);
* the double and complex CBLAS variants;
* column-major or padded arrays;
* other architectures (the SME fast path is arm64-only).
[ Test Plan ]
On an arm64 machine or VM running resolute, with the attached cblas-sme-
direct.c:
sudo apt install libopenblas-pthread-dev gcc
sudo update-alternatives --set libopenblas.so.0-aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu/openblas-pthread/libopenblas.so.0
apt-cache policy libopenblas0-pthread
gcc -O2 cblas-sme-direct.c -o cblas-sme-direct -lopenblas -lm
./cblas-sme-direct; echo "exit $?"
OPENBLAS_CORETYPE=ARMV8 ./cblas-sme-direct; echo "exit $?"
The program checks 10 calls (STRSM and STRMM: Upper/Lower x NoTrans/Trans;
SSYMM: Upper/Lower) against a
double-precision reference, first tightly packed and then with every leading
dimension one larger.
With 0.3.32+ds-5, or with the sdot-only 0.3.32+ds-5ubuntu0.1, the forced ARMV8
run fails all 10 tight calls and
passes all 10 padded calls. Natural dispatch gives the same result on non-SME
kernel sets:
cblas_strsm Upper NoTrans tight WRONG (30 of 30 elements; [0][0] = 1,
want 0.990145)
...
cblas_strsm Upper NoTrans padded ok
...
FAIL: 10 of 20 calls wrong
exit 1
After this SRU is available in resolute-proposed, enable that pocket following
https://wiki.ubuntu.com/Testing/EnableProposed, then install and verify the
package version:
sudo apt install -t resolute-proposed libopenblas0-pthread
libopenblas-pthread-dev
apt-cache policy libopenblas0-pthread # Expected SRU version:
0.3.32+ds-5ubuntu0.2
sudo update-alternatives --set libopenblas.so.0-aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu/openblas-pthread/libopenblas.so.0
./cblas-sme-direct; echo "exit $?"
OPENBLAS_CORETYPE=ARMV8 ./cblas-sme-direct; echo "exit $?"
The forced ARMV8 run, and natural dispatch on non-SME kernel sets, must
now print:
PASS: 0 of 20 calls wrong
exit 0
On ARMV9SME, the natural-dispatch invocation intentionally prints SKIP and
exits 77 before and after this
update. That is not a successful numerical test. The forced ARMV8 invocation
still performs the required
before/after check. This patch leaves the ARMV9SME direct path unchanged.
Repeat the before/after runs for openmp and serial. For each flavour, install
its corresponding runtime and
development packages from the pocket being tested, check the installed runtime
version with apt-cache policy,
and explicitly select it before executing the program. For example, for the
proposed openmp build:
sudo apt install -t resolute-proposed libopenblas0-openmp
libopenblas-openmp-dev
apt-cache policy libopenblas0-openmp # Expected SRU version:
0.3.32+ds-5ubuntu0.2
sudo update-alternatives --set libopenblas.so.0-aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu/openblas-openmp/libopenblas.so.0
./cblas-sme-direct; echo "exit $?"
OPENBLAS_CORETYPE=ARMV8 ./cblas-sme-direct; echo "exit $?"
Use libopenblas0-serial, libopenblas-serial-dev and
/usr/lib/aarch64-linux-gnu/openblas-serial/libopenblas.so.0
for the serial flavour. Record the selected flavour and version with each
result.
The proposed package also adds an arm64 autopkgtest, cblas-sme-direct, in three
stanzas, one per flavour.
It runs the program on the core that dispatch selects, then forces ARMV8 and,
where the CPU allows,
NEOVERSEN1. ARMV9SME's natural-dispatch exit 77 is handled inside the test
wrapper; the forced-core checks
still run. Local pthread runs fail against 0.3.32+ds-5 and pass against the
proposed source build. Under
qemu -cpu max, natural dispatch skips on ARMV9SME, while forced ARMV8 fails
before the fix and passes after it.
[ Where problems could occur ]
* Behaviour changes in the tight row-major Left calls listed above on non-SME
kernel sets. They now use the
regular Level-3 driver, which the padded and column-major calls already use.
A regression would show up as
a wrong STRSM/STRMM/SSYMM result for those shapes. The autopkgtest compares
them against a reference.
* Programs that were unknowingly relying on the no-op will see their output
matrix change. Run time rises
from the erroneous immediate return to the normal cost of the call.
* The quick return for m == 0 or n == 0 now comes before the core test in the
CBLAS functions of every
precision in trsm.c/symm.c. The same condition already returned a few lines
later. Invalid-argument
handling still occurs first.
* trsm.c and symm.c are compiled on every architecture, so every binary is
rebuilt. Off arm64 the SME block
is compiled out and the only source change is the quick-return move. The
object comparison was on arm64.
* Two upstream deletion hunks are deliberately omitted: the shared Fortran
quick returns in symm.c and
trsm.c are retained. Removing the trsm.c return caused upstream #5810 and
was reversed in PR #5813.
The Fortran objects strsm.o, strmm.o and ssymm.o have identical .text with
and without this patch in the
local comparison.
* ARMV9SME calls still take the SME path and remain wrong in this GCC build
for separate reasons. The new
autopkgtest skips its natural-dispatch run there and tests the forced
non-SME paths. Passing this test
does not establish correctness of the ARMV9SME kernels.
* The proposed 0.3.32+ds-5ubuntu0.2 debdiff is based on the sdot-only
0.3.32+ds-5ubuntu0.1 (LP: #2169719).
Sponsorship and upload of this second SRU will wait until that exact base is
published to resolute-updates.
The patches touch different files, and the sdot-v0 autopkgtest still passes
on the local combined build.
If the base changes before then, the second SRU must be regenerated and
reviewed against the new base.
[ Other Info ]
* Upstream: https://github.com/OpenMathLib/OpenBLAS/issues/5763, fixed by
https://github.com/OpenMathLib/OpenBLAS/pull/5767 (commits 74486799bb0c and
59cfea08592d), released in 0.3.33.
* Development release: stonking's 0.3.33+ds-3 already contains the fix. The
same program passes there on
non-SME kernel selections, so no development-series upload is needed for
this defect.
* Affected supported series: resolute. noble (0.3.26) and jammy (0.3.20)
predate the fast path, which arrived
in 0.3.31.
* Deviation from upstream: the third hunk of 59cfea08 (symm.c) and commit
fd862d43 (trsm.c) are omitted because
both remove a quick return shared with the Fortran interface. Upstream
restored the trsm.c return in
0.3.34 (PR #5813) after #5810. The retained changes are upstream's two
quick-return moves.
* Source comparison: the shipped pthread cblas_strsm, cblas_strmm and
cblas_ssymm functions match a rebuild
of the 0.3.32+ds-5 source with the package flags after masking link-time
address fields. The package's
faulty dispatch therefore agrees with the source being patched.
* Separate from this SRU: on ARMV9SME, tightly packed row-major SGEMM with
alpha != 1 or beta != 0, SSYMM,
SSYRK, SSYR2K and STRMM/STRSM are wrong in the tested GCC build. The census
identified two causes:
C SME kernels are compiled as empty stubs while the interface still calls
them; STRSM reaches the STRMM
kernel (fixed upstream in 0.3.34 by PR #5898). These are outside this
backport.
** Affects: openblas (Ubuntu)
Importance: Undecided
Status: New
** Tags: arm64 patch resolute
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2169788
Title:
cblas_strsm, cblas_strmm and cblas_ssymm return without computing on
arm64 (row-major, tightly packed)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openblas/+bug/2169788/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
