Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
Dear tiziano,
On Sat, Feb 23, 2008 at 11:28:22AM -0800, [EMAIL PROTECTED] wrote:
and the following code snippet was added to the file:
if ('NO_ATLAS_INFO',1) in blas_info.get('define_macros',[]):
return None # dotblas needs ATLAS, Fortran compiled blas will
not be sufficient.
which clearly indicates that dotblas is now, for some reason which I really
don't know, dependent on ATLAS. To get dotblas built in, it is enough to
remove the debian patch: 02_dontuse_lapack.diff . I tested it and the
resulting python-numpy packages contain _dotblas.so correctly linked to
the optimized atlas library.
This of course would mean Depends: libatlas3-headers, libatlas3gf-base |
libatlas3gf-sse2 | etc...
I would think that the bug actually is not a bug but a wishlist item and
that it has nothing to do with the gfortran transition.
Would you add dependency to atlas and give numpy a manyfold speed
improvement?
I agree that not using Atlas is going to result in a huge performance
penalty. However, the trouble is that Atlas has not really completed
the gfortran transition, and has not yet built on all architectures
yet:
http://buildd.debian.org/~jeroen/status/package.php?p=atlas
In case we start Build-depending on Atlas now, this would mean that we
will not be able to get numpy built on these architectures.
Also, numpy seems to be special, in that it seems to be very sensitive
to being built against Atlas. Other packages seem to want only
Blas/Lapack and happily use the Atlas system when it is installed,
without a rebuild.
Thanks for reporting this, but for now, I can just say you might have
to rebuild with Atlas. A better suggestion to overcome this problem is
welcome though.
Thanks!
Kumar
--
Kumar Appaiah,
458, Jamuna Hostel,
Indian Institute of Technology Madras,
Chennai - 600 036
signature.asc
Description: Digital signature
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
On Fri, Feb 22, 2008 at 8:40 PM, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi! Unfortunately I don't think I'll manage to investigate this further today, I'll try to have a look at it in the week end. Regarding the _dotblas.so problem: - on an etch system $ ldd /usr/lib/python2.4/site-packages/numpy/core/_dotblas.so linux-gate.so.1 = (0xe000) liblapack.so.3 = /usr/lib/atlas/sse2/liblapack.so.3 (0xb794e000) libblas.so.3 = /usr/lib/atlas/sse2/libblas.so.3 (0xb737) libpthread.so.0 = /lib/tls/i686/cmov/libpthread.so.0 (0xb735e000) libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb722d000) libg2c.so.0 = /usr/lib/libg2c.so.0 (0xb7205000) /lib/ld-linux.so.2 (0x8000) libm.so.6 = /lib/tls/i686/cmov/libm.so.6 (0xb71df000) and $ ldd /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so linux-gate.so.1 = (0xe000) liblapack.so.3 = /usr/lib/atlas/sse2/liblapack.so.3 (0xb7919000) libblas.so.3 = /usr/lib/atlas/sse2/libblas.so.3 (0xb733b000) libg2c.so.0 = /usr/lib/libg2c.so.0 (0xb7313000) libm.so.6 = /lib/tls/i686/cmov/libm.so.6 (0xb72ee000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb72e3000) libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb71b2000) /lib/ld-linux.so.2 (0x8000) - on a sid system the _dotblas.so file is missing, and : $ ldd /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so linux-gate.so.1 = (0xe000) liblapack.so.3gf = /usr/lib/sse2/atlas/liblapack.so.3gf (0xb7844000) libblas.so.3gf = /usr/lib/sse2/atlas/libblas.so.3gf (0xb7274000) libgfortran.so.3 = /usr/lib/libgfortran.so.3 (0xb71cc000) libm.so.6 = /lib/i686/cmov/libm.so.6 (0xb71a7000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb719a000) libc.so.6 = /lib/i686/cmov/libc.so.6 (0xb704d000) /lib/ld-linux.so.2 (0x8000) which is right, I think. If I recall correctly, _dotblas.so is built only if lapack libraries are found when numpy is built. But the linalg module is linked against lapack, so I really don't see why dotblas was not built. Concerning the ATLAS problem: installing the liblapack3-dev package made it possible to compile external modules linking to ATLAS using f2py, so sorry. that was my mistake. But since python-numpy now provides python-numpy-dev, I would suggest to add a Depends: liblapack3-dev. An example of such an extension module is my symeig package: http://mdp-toolkit.sourceforge.net/symeig.html building without liblapack3-dev results in a failure to import the module. Instructions for building can be found on the web site. let me know if you need further details, Thanks for the details. Unfortunately, I am not an expert in this blas, dotblas, ... thing. So what is the problem in concrete terms - missing liblapack3-dev depends? There doesn't seem to by andy liblapack3-dev package in unstable. If you tell me exactly what to fix, I'll do that. Kumar (- he is the expert:), any thoughts on this? Ondrej -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
I think I found out where the problem is.
In upstream numpy svn repository, in numpy/core/setup.py I found the
following log entry:
Fix: _dotblas will not ork with fortran compiled blas.
r3834 | cookedm | 2007-05-28 19:47:47 +0200 (Mon, 28 May 2007) | 2 lines
and the following code snippet was added to the file:
if ('NO_ATLAS_INFO',1) in blas_info.get('define_macros',[]):
return None # dotblas needs ATLAS, Fortran compiled blas will
not be sufficient.
which clearly indicates that dotblas is now, for some reason which I really
don't know, dependent on ATLAS. To get dotblas built in, it is enough to
remove the debian patch: 02_dontuse_lapack.diff . I tested it and the
resulting python-numpy packages contain _dotblas.so correctly linked to
the optimized atlas library.
This of course would mean Depends: libatlas3-headers, libatlas3gf-base |
libatlas3gf-sse2 | etc...
I would think that the bug actually is not a bug but a wishlist item and
that it has nothing to do with the gfortran transition.
Would you add dependency to atlas and give numpy a manyfold speed
improvement?
thank you!
tiziano
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
Package: python-numpy Version: 1:1.0.4-6 Followup-For: Bug #464784 After transition to gfortran it appears that the matrix multiplication function (numpy.dot) is now linked to the slow version in numpy/core/multiarray.so instead of being linked to the faster ATLAS version (in previous package version available under numpy/core/_dotblas.so). Another side effect of the transition is that ATLAS libraries are not found by /usr/lib/python2.4/site-packages/numpy/distutils/system_info.py which makes it impossible to compile extension modules linking to ATLAS and LAPACK with f2py. ld seems to be configured correctly, here is my /etc/ld.so.conf: include /etc/ld.so.conf.d/*.conf and my /etc/ld.so.conf.d/atlas.conf: /usr/lib/sse2/atlas /usr/lib/sse2 /usr/lib/atlas let me know if you need further information to help in debugging! keep up with the excellent work, tiziano -- System Information: Debian Release: lenny/sid APT prefers unstable APT policy: (500, 'unstable') Architecture: i386 (i686) Kernel: Linux 2.6.24-1-686 (SMP w/2 CPU cores) Locale: LANG=en_US.ISO-8859-15, LC_CTYPE=en_US.ISO-8859-15 (charmap=ISO-8859-15) Shell: /bin/sh linked to /bin/bash Versions of packages python-numpy depends on: ii libatlas3gf-base [libla 3.6.0-21.3 Automatically Tuned Linear Algebra ii libatlas3gf-sse2 [libla 3.6.0-21.3 Automatically Tuned Linear Algebra ii libblas3gf [libblas.so. 1.2-1.5 Basic Linear Algebra Subroutines 3 ii libc6 2.7-8GNU C Library: Shared libraries ii libgcc1 1:4.3-20080219-1 GCC support library ii libgfortran34.3-20080219-1 Runtime library for GNU Fortran ap ii liblapack3gf [liblapack 3.1.1-0.4library of linear algebra routines ii python 2.4.4-6 An interactive high-level object-o ii python-central 0.5.15-0.1 register and build utility for Pyt python-numpy recommends no packages. -- no debconf information -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
On Fri, Feb 22, 2008 at 4:09 PM, [EMAIL PROTECTED] wrote: Package: python-numpy Version: 1:1.0.4-6 Followup-For: Bug #464784 After transition to gfortran it appears that the matrix multiplication function (numpy.dot) is now linked to the slow version in numpy/core/multiarray.so instead of being linked to the faster ATLAS version (in previous package version available under numpy/core/_dotblas.so). Another side effect of the transition is that ATLAS libraries are not found by /usr/lib/python2.4/site-packages/numpy/distutils/system_info.py which makes it impossible to compile extension modules linking to ATLAS and LAPACK with f2py. ld seems to be configured correctly, here is my /etc/ld.so.conf: include /etc/ld.so.conf.d/*.conf and my /etc/ld.so.conf.d/atlas.conf: /usr/lib/sse2/atlas /usr/lib/sse2 /usr/lib/atlas let me know if you need further information to help in debugging! keep up with the excellent work, Hi, thanks very much for the bug report. Could you please give me a simple (python?) example that worked before and doesn't work now? I'll look at this over the weekend. Of course, if you know, how to fix it, feel free to fix it in our svn in Debian Python Modules Team. Thanks, Ondrej -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
Hi! Unfortunately I don't think I'll manage to investigate this further today, I'll try to have a look at it in the week end. Regarding the _dotblas.so problem: - on an etch system $ ldd /usr/lib/python2.4/site-packages/numpy/core/_dotblas.so linux-gate.so.1 = (0xe000) liblapack.so.3 = /usr/lib/atlas/sse2/liblapack.so.3 (0xb794e000) libblas.so.3 = /usr/lib/atlas/sse2/libblas.so.3 (0xb737) libpthread.so.0 = /lib/tls/i686/cmov/libpthread.so.0 (0xb735e000) libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb722d000) libg2c.so.0 = /usr/lib/libg2c.so.0 (0xb7205000) /lib/ld-linux.so.2 (0x8000) libm.so.6 = /lib/tls/i686/cmov/libm.so.6 (0xb71df000) and $ ldd /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so linux-gate.so.1 = (0xe000) liblapack.so.3 = /usr/lib/atlas/sse2/liblapack.so.3 (0xb7919000) libblas.so.3 = /usr/lib/atlas/sse2/libblas.so.3 (0xb733b000) libg2c.so.0 = /usr/lib/libg2c.so.0 (0xb7313000) libm.so.6 = /lib/tls/i686/cmov/libm.so.6 (0xb72ee000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb72e3000) libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb71b2000) /lib/ld-linux.so.2 (0x8000) - on a sid system the _dotblas.so file is missing, and : $ ldd /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so linux-gate.so.1 = (0xe000) liblapack.so.3gf = /usr/lib/sse2/atlas/liblapack.so.3gf (0xb7844000) libblas.so.3gf = /usr/lib/sse2/atlas/libblas.so.3gf (0xb7274000) libgfortran.so.3 = /usr/lib/libgfortran.so.3 (0xb71cc000) libm.so.6 = /lib/i686/cmov/libm.so.6 (0xb71a7000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb719a000) libc.so.6 = /lib/i686/cmov/libc.so.6 (0xb704d000) /lib/ld-linux.so.2 (0x8000) which is right, I think. If I recall correctly, _dotblas.so is built only if lapack libraries are found when numpy is built. But the linalg module is linked against lapack, so I really don't see why dotblas was not built. Concerning the ATLAS problem: installing the liblapack3-dev package made it possible to compile external modules linking to ATLAS using f2py, so sorry. that was my mistake. But since python-numpy now provides python-numpy-dev, I would suggest to add a Depends: liblapack3-dev. An example of such an extension module is my symeig package: http://mdp-toolkit.sourceforge.net/symeig.html building without liblapack3-dev results in a failure to import the module. Instructions for building can be found on the web site. let me know if you need further details, tiziano Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#464784: python-numpy: After transition to gfortran _dotblas.so is lost
I would suggest to add a Depends: liblapack3-dev This of course does not make any sense. It's my module which needs lapack3-dev, not numpy, which is perfectly functional without it! Sorry! Writing emails five minutes before living work is never a good idea :-) ciao, tiziano Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
