Hapy holidays … but I have an issue still that hopefully can be
addressed with the meson blas detection you are upstreaming(?).
Am Wed, 6 Dec 2023 18:06:01 +0100
schrieb Ralf Gommers <ralf.gomm...@gmail.com>:
> > Well, now is another day. Pkgsrc uses python -m build and I added
> >
> > -Csetup-args=-Dblas=${CBLAS_PC} -Csetup-args=-Dlapack=${LAPACK_PC}
> >
> > which seems to work out fine using cblas.pc and lapack.pc in the case
> > of the netlib install. In fact, most linking is done only to libblas.so
> > instead of libcblas.so, as the linker is smart enough to throw away the
> > unused lib.
> >
>
> Great, thanks for confirming!
This works for numpy and also installs scipy nicely, but this produces
a broken scipy install when using netlib reference libraries from
pkgsrc. These come as
libblas.so
liblapack.so (NEEDing libblas.so)
libcblas.so (NEEDing libblas.so)
libpapacke.so (NEEDing liblapack.so, hence libblas.so)
and their respective .pc files. This is the natural order that occus to
me when building from netlib upstream. This also means that one could
just replace BLAS and put stock LAPACK on top, what optimized BLAS libs
usually start out with. Only that they tend to pack all symbols into
one common library, which then project builds like numpy rely on.
Telling the meson build that BLAS is libcblas works as long as actually
CBLAS symbols are used. If not — I presume now, as I didn't yet see the
actual build lines that are triggered via the python -m build and meson
indirections — the linker might discard the -lcblas and leave symbols
unresolved (--as-needed but no --no-undefined).
This happens with scipy:
$ LANG=C readelf -d
/data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so
|grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
It would link against libopenblas_openmp.so if that had been the CBLAS
(and LAPACK) choice and all would be fine, but here, it should link
with libcblas.so, or directly to libblas.so, just like our regular
install of superlu:
$ LANG=C readelf -d /data/pkg/lib/libsuperlu.so|grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libblas.so.3]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Of course, just not vendoring superlu would be one solution for scipy,
but I think the deeper issue with the meson BLAS support should be
solved: The 4 parts of the BLAS canon (not talking about SCALAPACK etc.
yet) need to be handled explicitly.
It is confusing, though, as meson prints this:
Run-time dependency blas found: YES 3.11.0
Run-time dependency cblas found: YES 3.11.0
Run-time dependency lapack found: YES 3.11.0
It suggests that it looked for and found 3 libraries, but actually, it
only cared for -llapack and -lcblas. It needs to find -lblas directly,
too (or the cblas package separately, for that matter, not as component
of blas).
Is that easily fixable from your side? (I'm assuming numpy, scipy and
the future stock BLAS support of meson are handled together.) Is this
just an oversight on the scipy side and they could link the vendored
superly with -lblas without changing the meson machinery?
Alrighty then,
Thomas
--
Dr. Thomas Orgis
HPC @ Universität Hamburg
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
