So thanks again for staying with this. Though it is obvious that the
two viewpoints of upstream developer and package maintainer are not
easily aligned.
My main point about smartness detection certain implementations of
BLAS: I would like projects like NumPy not to waste a single bit on the
question which BLAS family they're using. In the case of Debian with
alternatives switching, they will be always built with Netlib anyway,
and then that is switched around at runtime. So the NumPy build is even
wrong in assuming what it will get at runtime.
Naturally, you want smartness in the build that avoids users asking why
their BLAS stuff is not picked up automatically.
I'd be happy with no detection at all and simple use of provided
BLAS_LIBS and LAPACK_LIBS from the environment (and fallback to -lblas,
-lcblas, -llapack, -llapack3), like it used to be for many projects.
You say you depend on a _project_ that offers certain libraries, not
the individual libraries. In the BLAS/LAPACK case, I'd be happier if
you didn't care which implementation it is. If you do want to use other
parts of MKL, too, it gets messier. You might want to actively avoid
mixing with a differing BLAS in that case, even.
I'll continue to investigate the situation with libblas+libcblas for
Netlib, see
https://github.com/Reference-LAPACK/lapack/issues/968
(hopefully picking up some activity as we pass over into the new year).
I'm not able to convince you to remove all implementation-specific
detection code and instead error out with such if no cblas was found:
Setup error: Did not find working BLAS+CBLAS package. Please point
-Dblas=your-library-here to a pkg-config module that provides
linkage to libraries offering BLAS and CBLAS symbols, as well
as the location of the cblas.h header file.
You know … somewhat like NPY_BLAS_LIBS, NPY_CBLAS_LIBS before;-) But
you did provide your own cblas.h back then. That was suboptimal. Better
would have been to force downstreams to provide a (symlink to) cblas.h
somewhere, IMHO.
Anyhow, somehow we will get along … some (final) remarks inline. Please
note the one about SuperLU: Is the vendoring necessary?
Am Sun, 31 Dec 2023 19:26:33 +0100
schrieb Ralf Gommers <ralf.gomm...@gmail.com>:
> So we want "MKL" or "Netlib BLAS", where MKL is not
> only a shared library (or set of them), but for example also the
> corresponding header file (mkl_cblas.h rather than cblas.h). The situation
> you are worrying about is basically that of an unknown package with a set
> of shared libraries and headers that have non-standard names. I'd say that
> that's then simply a non-supported package, until someone comes to report
> the situation and we can add support for it (or file a bug against that
> package and convince the authors not to make such a mess).
I hope to be able to make any such 'nonstandard names' work to be
presented to NumPy and meson as Netlib (compatible), worst case by
providing blas.pc and cblas.pc via build scripts.
> I'll note that the ATL_ symbols prefixes are an internal implementation
> detail; we don't prefix our own symbols when calling BLAS APIs. And there
> is no way Debian lets you install both of Netlib BLAS and ATLAS (it's one
> or the other, through `update-alternatives`), so this can't really go wrong
> in the way you imagine I think.
Nope.
$ ls -l /usr/lib/x86_64-linux-gnu/lib*blas.so.3*
lrwxrwxrwx 1 root root 47 Feb 7 2019
/usr/lib/x86_64-linux-gnu/libblas.so.3 ->
/etc/alternatives/libblas.so.3-x86_64-linux-gnu
lrwxrwxrwx 1 root root 18 Dez 20 2021
/usr/lib/x86_64-linux-gnu/libcblas.so.3 -> libcblas.so.3.10.3
-rw-r--r-- 1 root root 151144 Dez 20 2021
/usr/lib/x86_64-linux-gnu/libcblas.so.3.10.3
lrwxrwxrwx 1 root root 20 Dez 20 2021
/usr/lib/x86_64-linux-gnu/libf77blas.so.3 -> libf77blas.so.3.10.3
-rw-r--r-- 1 root root 146840 Dez 20 2021
/usr/lib/x86_64-linux-gnu/libf77blas.so.3.10.3
only libblas.so is linked to alternatives, as is liblapack.so. Both
libcblas.so and liblapacke.so are not (one is ATLAS, the other is
Netlib!). But we already agreed that this is a mess in Debian;-)
> Note that Debian in addition decided to use custom pkg-config names, so it
> works with `-Dblas=blas-atlas -Dlapack=lapack-atlas`.
Yes, there is only one blas.pc symlink to the chosen alternative and
specific names otherwise. In pkgsrc, it's blas.pc from netlib and
specific names for all other variants. We might switch to a similar
system and rename blas.pc to blas-netlib.pc, so that you are forced to
choose and not have it found automatically.
> If you're renaming build outputs from an
> upstream package from their defaults (e.g. libcblas.so ->
> libcblas-foobar.so), you are breaking things and get to keep the pieces.
> The correct name for Netlib BLAS is `libcblas.so` and `cblas.pc`. So
> provide `cblas.pc`, and you unbreak yourself.
In the case of OpenBLAS, the default library names are (or were at some
point) just unusable in a packaging system, referencing the build CPU
name and what not. Or rather, the upstream build system explictly
contains provisions to configure library names. And at least in the
past, you really wanted to provide differing builds of them (e.g. the
OpenMP variant being incompatible with some client programs, while
pthread worked, or only serial) and I am not aware of a workable
standard naming scheme for them, just distro choices that happen to
match by accident or obviousness.
On a HPC cluster, I can easily imagine providing a common software tree
for most things, but a differently built BLAS for a specific CPU in a
partition of the cluster. A user building their own stack for that
partition then would be told to use
-Dblas=openblas-for-partition-special. Well, and that wouldn't be a
problem with the current NumPy setup as long as this blas contains
CBLAS. So lots of words about nothing from me, I guess;-)
> Sure, I was talking about NumPy/SciPy. In general, the LAPACKE situation is
> pretty much identical to the CBLAS one.
Except that some packagers didn't decide yet that LAPACKE needs the
same treatment, see
https://packages.debian.org/experimental/amd64/liblapacke/filelist
and another little detail that emerges with this is that Debian doesn't
enforce a consistent choice of _package_ that provides the libs.
$ sudo update-alternatives --get-selections|grep blas
libblas64.so.3-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/libblas64.so.3
libblas64.so-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/libblas64.so
libblas.so.3-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
libblas.so-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/blas/libblas.so
liblapack64.so.3-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/liblapack64.so.3
liblapack64.so-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/liblapack64.so
liblapack.so.3-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
libopenblas64.so.0-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/libopenblas64.so.0
libopenblas64.so-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas64-openmp/libopenblas64.so
libopenblas.so.0-x86_64-linux-gnu auto
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so.0
You can choose the implementation for each individual library. This
leads to these dependencies for liblapacke.so in my case:
linux-vdso.so.1 (0x00007ffebdbce000)
libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007f737d503000)
liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3
(0x00007f737ce3e000)
libtmglib.so.3 => /lib/x86_64-linux-gnu/libtmglib.so.3
(0x00007f737cdd3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f737cbab000)
libopenblas.so.0 => /lib/x86_64-linux-gnu/libopenblas.so.0
(0x00007f737a758000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f737a671000)
libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5
(0x00007f737a396000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f737a376000)
/lib64/ld-linux-x86-64.so.2 (0x00007f737d7fd000)
libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0
(0x00007f737a32e000)
They build openblas libs without LAPACKE, so there is only the generic
liblapacke, but it links to both Netlib BLAS and OpenBLAS here. Feels
like this case should be avoided.
> https://github.com/scipy/scipy/blob/main/scipy/sparse/linalg/_dsolve/SuperLU/SRC/slu_Cnames.h
That again raises a point: Do you do anything to SuperLU that warrants
vendoring the code? We have a package that provides it as shared
library in pkgsrc and would rather see NumPy use it instead of
installing a redundant copy. Our current approach is to support one
common BLAS choice for all packages, so it should fit NumPy, too.
> Yep, that should be okay - the two test failures should be pretty harmless.
Thanks.
> Running `python -c "import numpy.linalg; numpy.test()" should do
Yes, this runs and returns those two log prefix failures alongside
thousands of successful tests.
> So what is standard == whatever configure/make produces for Netlib
> BLAS or OpenBLAS, without renaming or deleting anything.
As I elaborated above … OpenBLAS with renaming _is_ the standard;-) I
must admit that I also still carry a patch for integrating our variant
renaming business into the upstream Makefile. Not sure right now if I
tried upstreaming it before. The default build of OpenBLAS produces a
libopenblas.so.0 that links to a very custom library name that may
reference the build (target) CPU and doesn't change if I
enable/disable threading (or in funny ways). You need to change that,
or hide all names behind some alternatives system.
> Yes indeed. At least, I cannot think of any package/project that
> doesn't provide CBLAS with BLAS. While BLAS and LAPACK clearly are
> separate from each other, and there are projects that provide only
> one (e.g., BLIS is BLAS-only).
Thanks for pointing that out about BLIS, that pkgsrc doesn't carry yet.
So in our scheme, we'd need to do an extra build of netlib LAPACK that
links to libblis. So that would be
blis.pc (BLIS with BLAS+CBLAS)
lapack-blis.pc (wrapping the above)
lapacke-blis.pc (wrapping the above)
and for NumPy build -Dblas=blis (or -Dblas=blis-for-cluster-partition) and
-Dlapack=lapack-blis (or …).
In pkgsrc, we introduced your view of the overall package, actually.
User sets PKGSRC_BLAS_TYPES="openblas-pthread" that chooses one
provider for all 4 APIs. But we're not in a world where that simply is
exposed as a single entity — except for all our openblas builds that
include all parts …
> For OpenBLAS we
> know it's the default to include LAPACK but it can be built without,
> so we need to check symbols, and if they're missing we look for
> another library.
Another library entirely or only one providing LAPACK on top (a
possible disaster, I presume)?
> Yes, the search order and which packages are searched for is fully
> customizable. But you should be good with the default `-Dblas=auto`.
> If not, I'd like to understand why.
As I said, we for sure do _not_ want to use -Dblas=auto in package
builds (unless we installed AutoBLAS?;-). We want to be specific and
fail early if the configured choice is not available in the build
environment. Pkgsrc is especially trying hard to avoid locating
libraries from the host system (possibly a full GNU/Linux distro) if
not explicitly told to.
Also, my HPC users also should pick the correct BLAS from a list
available in a common prefix for their builds (we're avoiding too many
environment modules for libraries that can be installed into a common
prefix with pkgsrc). Auto-detection ist not always desired.
Alrighty then,
Thomas
--
Dr. Thomas Orgis
HPC @ Universität Hamburg
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com
