Re: [Numpy-discussion] Building numpy with ATLAS
As it turns out, I just had to run "python setup.py build --force" after changing my site.cfg file in order to recompile numpy/core/multiarray.so. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Building numpy with ATLAS
Hey Everyone,
I am trying to build numpy-1.11.1 with ATLAS libraries. The build completes
successfully, and
I am able to use numpy, but there seems to be some problem with my config
because
numpy only ever uses one core, even when multiplying 5000x5000 matrices with
numpy.dot.
My site.cfg is:
[default]
library_dirs =
/u3/s4kashya/.local/atlas3.11/lib:/u3/s4kashya/.local/lib:/usr/local/lib:/usr/lib
include_dirs =
/u3/s4kashya/.local/atlas3.11/include:/u3/s4kashya/.local/include:/usr/local/include:/usr/include
[atlas]
libraries = ptlapack, ptf77blas, ptcblas, atlas
library_dirs = /u3/s4kashya/.local/atlas3.11/lib
include_dirs = /u3/s4kashya/.local/atlas3.11/include
Output from numpy.__config__.show():
atlas_3_10_blas_threads_info:
libraries = ['tatlas', 'ptlapack', 'ptf77blas', 'ptcblas', 'atlas']
library_dirs = ['/u3/s4kashya/.local/atlas3.11/lib']
define_macros = [('HAVE_CBLAS', None), ('NO_ATLAS_INFO', -1)]
language = c
include_dirs = ['/u3/s4kashya/.local/atlas3.11/include']
lapack_opt_info:
libraries = ['tatlas', 'tatlas', 'tatlas', 'ptlapack', 'ptf77blas',
'ptcblas', 'atlas']
library_dirs = ['/u3/s4kashya/.local/atlas3.11/lib']
define_macros = [('NO_ATLAS_INFO', -1)]
language = f77
include_dirs = ['/u3/s4kashya/.local/atlas3.11/include']
blas_opt_info:
libraries = ['tatlas', 'ptlapack', 'ptf77blas', 'ptcblas', 'atlas']
library_dirs = ['/u3/s4kashya/.local/atlas3.11/lib']
define_macros = [('HAVE_CBLAS', None), ('NO_ATLAS_INFO', -1)]
language = c
include_dirs = ['/u3/s4kashya/.local/atlas3.11/include']
openblas_info:
NOT AVAILABLE
openblas_lapack_info:
NOT AVAILABLE
atlas_3_10_threads_info:
libraries = ['tatlas', 'tatlas', 'tatlas', 'ptlapack', 'ptf77blas',
'ptcblas', 'atlas']
library_dirs = ['/u3/s4kashya/.local/atlas3.11/lib']
define_macros = [('NO_ATLAS_INFO', -1)]
language = f77
include_dirs = ['/u3/s4kashya/.local/atlas3.11/include']
lapack_mkl_info:
NOT AVAILABLE
blas_mkl_info:
NOT AVAILABLE
mkl_info:
NOT AVAILABLE
Any ideas what could be going wrong?
Thanks,
Shitikanth
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/17/07, David Cournapeau [EMAIL PROTECTED] wrote: Now that you mention it, I am also puzzled by this one: I can see why you would use atlas3-sse2-dev without atlas3-base-dev (for the static library), but not having atlas3-base-dev makes it imposible to dynamically link to atlas libraries without customizing makefiles and/or configure options. I'd like to compile atlas so that I can take full advantage of my core 2 duo. Numpy dynamically links to the debian binary of atlas-sse that I installed. But the atlas website says that they recommend static linking. Which do you recommend, static or dynamic? Are there good directions for either? ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/18/07, Keith Goodman [EMAIL PROTECTED] wrote: On 4/17/07, David Cournapeau [EMAIL PROTECTED] wrote: Now that you mention it, I am also puzzled by this one: I can see why you would use atlas3-sse2-dev without atlas3-base-dev (for the static library), but not having atlas3-base-dev makes it imposible to dynamically link to atlas libraries without customizing makefiles and/or configure options. I'd like to compile atlas so that I can take full advantage of my core 2 duo. Numpy dynamically links to the debian binary of atlas-sse that I installed. But the atlas website says that they recommend static linking. Which do you recommend, static or dynamic? Are there good directions for either? I don't know which is best, although I suspect the statically linked version will be larger. It might seem that just pulling in the gemm routines wouldn't add much, but they pull in lots of supporting routines. To get numpy to link statically you will also probably need to have a directory that contains only the *.a versions because the linker will default to the *.so if they are present; i don't think there is a way to specify the -static flag to the gcc compiler. Maybe someone else knows how to do that. For ATLAS, I believe the latest versions are also recommended because the stable version is so old. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/18/07, Charles R Harris [EMAIL PROTECTED] wrote: On 4/18/07, Keith Goodman [EMAIL PROTECTED] wrote: I'd like to compile atlas so that I can take full advantage of my core 2 duo. Numpy dynamically links to the debian binary of atlas-sse that I installed. But the atlas website says that they recommend static linking. Which do you recommend, static or dynamic? Are there good directions for either? I don't know which is best, although I suspect the statically linked version will be larger. It might seem that just pulling in the gemm routines wouldn't add much, but they pull in lots of supporting routines. To get numpy to link statically you will also probably need to have a directory that contains only the *.a versions because the linker will default to the *.so if they are present; i don't think there is a way to specify the -static flag to the gcc compiler. Maybe someone else knows how to do that. For ATLAS, I believe the latest versions are also recommended because the stable version is so old. At the moment best is equal to easiest since I have never compiled atlas. Does anyone know of a howto on compiling atlas (dynamically linked)? Besides speed I'm also interested in seeing if I can get rid of the repeatability problems I have with the debian atlas-sse2 binary. (Repeated calulations, as discuss on this list, give give difference results in numpy but not octave.) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Keith Goodman [EMAIL PROTECTED] [2007-04-18 10:49]: I'd like to compile atlas so that I can take full advantage of my core 2 duo. If your use is entirely non-commercial you can use Intel's MKL with built-in optimized BLAS and LAPACK and avoid the need for ATLAS. http://www.intel.com/cd/software/products/asmo-na/eng/266858.htm The BLAS and LAPACK libraries are time-honored standards for solving a large variety of linear algebra problems. The Intel� Math Kernel Library (Intel� MKL) contains an implementation of BLAS and LAPACK that is highly optimized for Intel� processors. Intel MKL can enable you to achieve significant performance improvements over alternative implementations of BLAS and LAPACK. [...] The charts immediately below show that, for Itanium� 2-based systems, Intel MKL performs approximately 20 percent faster than ATLAS for large matrices, and even faster for small matrices. On the new Dual-Core Intel� Xeon� processors, Intel MKL provides similar performance advantages. I've compiled both Python (icc) and Numpy using icc 9.1 and MKL 9.1_beta. It's significantly faster than using gcc on my Core 2 Duo system. I'm still looking for a broad performance test (something like Scimark, say). The best compiler flags I've found are: -fast -parallel In some cases -funroll-loops and -fno-alias helps. -rex -- Time flies like wind. Fruit flies like pears. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/18/07, rex [EMAIL PROTECTED] wrote: Keith Goodman [EMAIL PROTECTED] [2007-04-18 10:49]: I'd like to compile atlas so that I can take full advantage of my core 2 duo. If your use is entirely non-commercial you can use Intel's MKL with built-in optimized BLAS and LAPACK and avoid the need for ATLAS. Thanks for that. For a variety of reasons I'm sticking with atlas. Does the parallel flag give you a big speed increase? I imagine it speeds things up more for larger matrices. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
rex wrote: Keith Goodman [EMAIL PROTECTED] [2007-04-18 10:49]: I'd like to compile atlas so that I can take full advantage of my core 2 duo. If your use is entirely non-commercial you can use Intel's MKL with built-in optimized BLAS and LAPACK and avoid the need for ATLAS. Just to clarify, my understanding is that if you buy a developer's license, you can also use it for commercial use, including distributing binaries. (Otherwise it would seem kind of silly for Intel to invest so much in their performance libraries and compilers.) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Sun has recently released their compilers under an opensource license for Linux as well (Sun Studio Express or something), including their perflib - which includes Blas and Lapack. Has somebody tried how that combination performs, compared to Intel MKL or Atlas? I think they are free even for commercial use. Just a thought... Christian. On Wed, April 18, 2007 22:27, rex wrote: Andrew Straw [EMAIL PROTECTED] [2007-04-18 13:22]: rex wrote: If your use is entirely non-commercial you can use Intel's MKL with built-in optimized BLAS and LAPACK and avoid the need for ATLAS. Just to clarify, my understanding is that if you buy a developer's license, you can also use it for commercial use, including distributing binaries. (Otherwise it would seem kind of silly for Intel to invest so much in their performance libraries and compilers.) Yes, I should have included that. icc and MKL licenses for commercial use are $399 each. -rex -- It's a Singer, Captain Picard. Make it sew. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
I'm having great difficulty building numpy on solaris with the sun compilers and libsunperf. I may try this on ubuntu x86_64 to see if the setup is less painful. Thanks for the idea, -Peter On Wed, Apr 18, 2007 at 11:04:26PM +0200, Christian Marquardt wrote: Sun has recently released their compilers under an opensource license for Linux as well (Sun Studio Express or something), including their perflib - which includes Blas and Lapack. Has somebody tried how that combination performs, compared to Intel MKL or Atlas? I think they are free even for commercial use. Just a thought... Christian. On Wed, April 18, 2007 22:27, rex wrote: Andrew Straw [EMAIL PROTECTED] [2007-04-18 13:22]: rex wrote: If your use is entirely non-commercial you can use Intel's MKL with built-in optimized BLAS and LAPACK and avoid the need for ATLAS. Just to clarify, my understanding is that if you buy a developer's license, you can also use it for commercial use, including distributing binaries. (Otherwise it would seem kind of silly for Intel to invest so much in their performance libraries and compilers.) Yes, I should have included that. icc and MKL licenses for commercial use are $399 each. -rex -- It's a Singer, Captain Picard. Make it sew. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- The 5 year plan: In five years we'll make up another plan. Or just re-use this one. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Robert Kern wrote: Charles R Harris wrote: I don't know which is best, although I suspect the statically linked version will be larger. It might seem that just pulling in the gemm routines wouldn't add much, but they pull in lots of supporting routines. To get numpy to link statically you will also probably need to have a directory that contains only the *.a versions because the linker will default to the *.so if they are present; i don't think there is a way to specify the -static flag to the gcc compiler. Maybe someone else knows how to do that. Not really. Since you are *making* a shared library, the flags are already set to *look up* shared libraries. I do not believe they are overridable, much to my frequent vexation. I think there is a way, but not trivial. I had to do all kind of tricks to use fftw in mex (matlab extension written in C, practically dynamically loaded .so inside matlab), and linking statically some libraries was one of them. If you still need it (linking some libraries statically, other dynamically, to build a .so), I can try to dig in my matlab scripts (unused for some time thanks to scipy :) ) cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Charles R Harris wrote: I'm wondering if the static libraries could simply be compiled with the -fPIC flag and linked with the program to produce the dynamic library. The static libraries are just collections of *.o files, so I don't see why that shouldn't work. I don't think there is a reason why it *wouldn't* work; it's just that one needs to do gcc -shared to make a .so and that flag also tells the linker to choose .so's over .a's for the libraries that get linked into the target .so. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Charles R Harris wrote: On 4/18/07, *Keith Goodman* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 4/18/07, Charles R Harris [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: On 4/18/07, Keith Goodman [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: I'd like to compile atlas so that I can take full advantage of my core 2 duo. Numpy dynamically links to the debian binary of atlas-sse that I installed. But the atlas website says that they recommend static linking. Which do you recommend, static or dynamic? Are there good directions for either? I don't know which is best, although I suspect the statically linked version will be larger. It might seem that just pulling in the gemm routines wouldn't add much, but they pull in lots of supporting routines. To get numpy to link statically you will also probably need to have a directory that contains only the *.a versions because the linker will default to the *.so if they are present; i don't think there is a way to specify the -static flag to the gcc compiler. Maybe someone else knows how to do that. For ATLAS, I believe the latest versions are also recommended because the stable version is so old. At the moment best is equal to easiest since I have never compiled atlas. Does anyone know of a howto on compiling atlas (dynamically linked)? The instructions that come with ATLAS are useable, and with the 3.7.30 version the process is pretty easy. Just make sure you read the part concerning relocatable libraries first, as you will need to add some flags to the ./configure command line, -fPIC and a couple of others. After compiling and install, IIRC, the libraries will be in /usr/local/lib/ATLAS, which should be OK for debian. Then you need to add a file in /etc/ld.conf.d with the name of the directory and run ldconfig in order to let the system know where the libraries are and what they contain. Or you could save the old /usr/lib/atlas directory and put the new libraries there and then run ldconfig. I also found it necessary to delete the old numpy site-package before reinstalling numpy. Here is what I do to compile atlas with full lapack, dynamically, on debian systems (ubuntu is the same for that matter): - sudo apt-get install g77 lapack3-pic lapack3-test refblas3-test - untar atlas, go into ATLAS directory - mkdir MyObj, cd MyObj (this is where the lib will be built) - configure the build: ../configure -Si archdef 1 -Fa alg -fPIC -C if g77 -Ss flapack /usr/lib/liblapack_pic.a - build the lib: make - time and test the library: make time; make test - build the shared library: cd lib; make shared. Explanations: - lapack3-test will be handy to test the compiled atlas, and lapack3-pic gives you a lapack library which can be used to build a full shared LAPACK library using atlas. - archdef 1 means uses arch default, much faster - up to 10 times - to compile the library - -Fa alg -fPIC says to add -fPIC to build all objects files (needed for shared lib) - -C if g77 says to use g77 and not gfortran as the fortran compiler. On debian, g77 is still the default, and gfortran has a different ABI, meaning all kind of problems if you use gfortran for now on debian if you do not know what you are doing. I strongly advise you to use the officiel lapack and blas testers afterwards. On debian, as they are dynamically linked to blas/lapack, you can choose which library to test thanks to the LD_LIBRARY_PATH trick. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Robert Kern wrote: Charles R Harris wrote: I'm wondering if the static libraries could simply be compiled with the -fPIC flag and linked with the program to produce the dynamic library. The static libraries are just collections of *.o files, so I don't see why that shouldn't work. I don't think there is a reason why it *wouldn't* work; it's just that one needs to do gcc -shared to make a .so and that flag also tells the linker to choose .so's over .a's for the libraries that get linked into the target .so. Ok, I found it much faster than I expected: you should use the option -Bdynamic of the *linker*, that is, for example: gcc -shared -o c_gaussd.mexglx c_gaussd.o mex_alloc.o -L/usr/local/matlab/bin/glnx86 -L/usr/media/boulot/local/lib -L/usr/lib/atlas -Wl,-Bstatic,-lem_full -llapack_atlas -latlas -Wl,-Bdynamic,-lmx -lmex will build a shared library c_gaussd.mexglx from c_gaussd.o and mex_alloc.o, statically link libem_full, lapack_atlas and atlas, and dynamically everything else. This was working, as otherwise, matlab would have picked up atlas symbols from its existing, already linked own atlas, and not from the atlas installed on my system (which was crashing matlab and was the reason why I went through this pain in the first place). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
David Cournapeau wrote: Robert Kern wrote: Charles R Harris wrote: I'm wondering if the static libraries could simply be compiled with the -fPIC flag and linked with the program to produce the dynamic library. The static libraries are just collections of *.o files, so I don't see why that shouldn't work. I don't think there is a reason why it *wouldn't* work; it's just that one needs to do gcc -shared to make a .so and that flag also tells the linker to choose .so's over .a's for the libraries that get linked into the target .so. Ok, I found it much faster than I expected: you should use the option -Bdynamic of the *linker*, that is, for example: gcc -shared -o c_gaussd.mexglx c_gaussd.o mex_alloc.o -L/usr/local/matlab/bin/glnx86 -L/usr/media/boulot/local/lib -L/usr/lib/atlas -Wl,-Bstatic,-lem_full -llapack_atlas -latlas -Wl,-Bdynamic,-lmx -lmex will build a shared library c_gaussd.mexglx from c_gaussd.o and mex_alloc.o, statically link libem_full, lapack_atlas and atlas, and dynamically everything else. This was working, as otherwise, matlab would have picked up atlas symbols from its existing, already linked own atlas, and not from the atlas installed on my system (which was crashing matlab and was the reason why I went through this pain in the first place). Thank you! That's good information to know. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/18/07, Robert Kern [EMAIL PROTECTED] wrote: Charles R Harris wrote: I'm wondering if the static libraries could simply be compiled with the -fPIC flag and linked with the program to produce the dynamic library. The static libraries are just collections of *.o files, so I don't see why that shouldn't work. I don't think there is a reason why it *wouldn't* work; it's just that one needs to do gcc -shared to make a .so and that flag also tells the linker to choose .so's over .a's for the libraries that get linked into the target .so. The linker picks dynamic over static even without the -shared flag. I think that putting the static libraries in their own directory *without* the dynamic libraries might do the trick. Hmm... something to try. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Charles R Harris wrote: On 4/18/07, *Robert Kern* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Charles R Harris wrote: I'm wondering if the static libraries could simply be compiled with the -fPIC flag and linked with the program to produce the dynamic library. The static libraries are just collections of *.o files, so I don't see why that shouldn't work. I don't think there is a reason why it *wouldn't* work; it's just that one needs to do gcc -shared to make a .so and that flag also tells the linker to choose .so's over .a's for the libraries that get linked into the target .so. The linker picks dynamic over static even without the -shared flag. Oh sorry, using -static would normally make the linker choose static libraries over shared libraries, but that only works when linking an executable. Of course, David found the right way to bypass this and talk directly to the linker. I think that putting the static libraries in their own directory *without* the dynamic libraries might do the trick. Hmm... something to try. That's usually the way I worked around it. Things may be difficult if your shared libraries are installed to /usr/lib and you don't want to muck about as root in there. IIRC (but I possibly don't), you'd have to remove the .so's from the entire library search path in order to find the .a's. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Christian K wrote: David Cournapeau wrote: On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with atlas support. Just install the package atlas3-base-dev, and you are done. The reason is that when *compiling* a software which needs atlas, the linker will try to find libblas.so in /usr/lib, not in /usr/lib/sse2. If you install atlas3-base-dev, the package will install those at the correct locations. I have updated the instructions for Ubuntu (also works for debian) on the wiki a few days ago: Indeed, installing atlas3-base-dev helps. I only had atlas3-base, atlas3-sse and atlas3-sse2-dev installed. Sorry for the noise. Hmm, atlas3-sse2-dev doesn't depend on atlas3-base-dev? That sounds like a bug... Off to investigate and possibly file a bug report with Debian or Ubuntu, Andrew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Andrew Straw wrote: Christian K wrote: David Cournapeau wrote: On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with atlas support. Just install the package atlas3-base-dev, and you are done. The reason is that when *compiling* a software which needs atlas, the linker will try to find libblas.so in /usr/lib, not in /usr/lib/sse2. If you install atlas3-base-dev, the package will install those at the correct locations. I have updated the instructions for Ubuntu (also works for debian) on the wiki a few days ago: Indeed, installing atlas3-base-dev helps. I only had atlas3-base, atlas3-sse and atlas3-sse2-dev installed. Sorry for the noise. Hmm, atlas3-sse2-dev doesn't depend on atlas3-base-dev? That sounds like a bug... Now that you mention it, I am also puzzled by this one: I can see why you would use atlas3-sse2-dev without atlas3-base-dev (for the static library), but not having atlas3-base-dev makes it imposible to dynamically link to atlas libraries without customizing makefiles and/or configure options. I will report this as a bug, see what the maintainers say. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Christian K wrote:
Thanks, but that didn't help:
atlas_info:
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS installation.
*
warnings.warn(message)
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
lapack_info:
libraries lapack not found in /usr/lib/sse2
libraries lapack not found in /usr/lib
NOT AVAILABLE
Christian
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with
atlas support. Just install the package atlas3-base-dev, and you are
done. The reason is that when *compiling* a software which needs atlas,
the linker will try to find libblas.so in /usr/lib, not in
/usr/lib/sse2. If you install atlas3-base-dev, the package will install
those at the correct locations. I have updated the instructions for
Ubuntu (also works for debian) on the wiki a few days ago:
http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe
Note that if you have also optimized version installed (sse, sse2), it
will use those automatically when you launch numpy (you can check by
doing ldd on the file numpy/core/_dotblas.so inside your numpy installation.
cheers,
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
On 4/15/07, David Cournapeau [EMAIL PROTECTED] wrote:
Christian K wrote:
Thanks, but that didn't help:
atlas_info:
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS installation.
*
warnings.warn(message)
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
lapack_info:
libraries lapack not found in /usr/lib/sse2
libraries lapack not found in /usr/lib
NOT AVAILABLE
Christian
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with
atlas support. Just install the package atlas3-base-dev, and you are
done. The reason is that when *compiling* a software which needs atlas,
the linker will try to find libblas.so in /usr/lib, not in
/usr/lib/sse2. If you install atlas3-base-dev, the package will install
those at the correct locations. I have updated the instructions for
Ubuntu (also works for debian) on the wiki a few days ago:
http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe
Note that if you have also optimized version installed (sse, sse2), it
will use those automatically when you launch numpy (you can check by
doing ldd on the file numpy/core/_dotblas.so inside your numpy
installation.
Be aware that on recent 64 bit Intel processors running a 64 bit OS the base
Atlas package appears to be buggy. You might be better off building your own
version, on a fast machine it won't take long.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Charles R Harris wrote:
On 4/15/07, *David Cournapeau* [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] wrote:
Christian K wrote:
Thanks, but that didn't help:
atlas_info:
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS installation.
*
warnings.warn(message)
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
lapack_info:
libraries lapack not found in /usr/lib/sse2
libraries lapack not found in /usr/lib
NOT AVAILABLE
Christian
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org mailto:Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Ubuntu and debian, you do NOT need any site.cfg to compile
numpy with
atlas support. Just install the package atlas3-base-dev, and you are
done. The reason is that when *compiling* a software which needs
atlas,
the linker will try to find libblas.so in /usr/lib, not in
/usr/lib/sse2. If you install atlas3-base-dev, the package will
install
those at the correct locations. I have updated the instructions for
Ubuntu (also works for debian) on the wiki a few days ago:
http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe
Note that if you have also optimized version installed (sse, sse2), it
will use those automatically when you launch numpy (you can check by
doing ldd on the file numpy/core/_dotblas.so inside your numpy
installation.
Be aware that on recent 64 bit Intel processors running a 64 bit OS
the base Atlas package appears to be buggy. You might be better off
building your own version, on a fast machine it won't take long.
I do not use any 64 bits machines myself, so I didn't know about this.
Compiling atlas may take a long time on fast machine if you don't have
arch defaults, though (it takes several hours for my workstation which
has two xeon @ 3.2 ghz).
My impression is that using 64 bits OS is still a bit rough, anyway.
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Robert Kern wrote:
Christian K wrote:
Hi,
I'm trying to build numpy from svn on ubuntu edgy with atlas provided by
ubuntu
package atlas3-sse2-dev which contains:
/usr
/usr/lib
/usr/lib/sse2
/usr/lib/sse2/libatlas.a
/usr/lib/sse2/libcblas.a
/usr/lib/sse2/libf77blas.a
/usr/lib/sse2/liblapack_atlas.a
/usr/lib/atlas
/usr/lib/atlas/sse2
/usr/lib/atlas/sse2/libblas.a
/usr/lib/atlas/sse2/liblapack.a
/usr/share
/usr/share/doc
/usr/share/doc/atlas3-sse2-dev
/usr/share/doc/atlas3-sse2-dev/copyright
/usr/share/doc/atlas3-sse2-dev/changelog.Debian.gz
/usr/lib/sse2/libatlas.so
/usr/lib/sse2/libcblas.so
/usr/lib/sse2/libf77blas.so
/usr/lib/sse2/liblapack_atlas.so
/usr/lib/atlas/sse2/libblas.so
/usr/lib/atlas/sse2/liblapack.so
I tried both with and without a site.cfg:
[DEFAULT]
library_dirs = /usr/lib/sse2
include_dirs = /usr/include
[blas_opt]
libraries = f77blas, cblas, atlas
[lapack_opt]
libraries = lapack, f77blas, cblas, atlas
Confusingly lapack_atlas resides in /usr/lib but even though setup.py looks
for
it in that place it reports 'not found'.
What should I try next?
Change this:
[lapack_opt]
libraries = lapack, f77blas, cblas, atlas
to this:
[lapack_opt]
libraries = lapack_atlas, f77blas, cblas, atlas
Thanks, but that didn't help:
atlas_info:
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS installation.
*
warnings.warn(message)
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
lapack_info:
libraries lapack not found in /usr/lib/sse2
libraries lapack not found in /usr/lib
NOT AVAILABLE
Christian
___
Numpy-discussion mailing list
[EMAIL PROTECTED]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] building numpy with atlas on ubuntu edgy
Hi,
I'm trying to build numpy from svn on ubuntu edgy with atlas provided by ubuntu
package atlas3-sse2-dev which contains:
/usr
/usr/lib
/usr/lib/sse2
/usr/lib/sse2/libatlas.a
/usr/lib/sse2/libcblas.a
/usr/lib/sse2/libf77blas.a
/usr/lib/sse2/liblapack_atlas.a
/usr/lib/atlas
/usr/lib/atlas/sse2
/usr/lib/atlas/sse2/libblas.a
/usr/lib/atlas/sse2/liblapack.a
/usr/share
/usr/share/doc
/usr/share/doc/atlas3-sse2-dev
/usr/share/doc/atlas3-sse2-dev/copyright
/usr/share/doc/atlas3-sse2-dev/changelog.Debian.gz
/usr/lib/sse2/libatlas.so
/usr/lib/sse2/libcblas.so
/usr/lib/sse2/libf77blas.so
/usr/lib/sse2/liblapack_atlas.so
/usr/lib/atlas/sse2/libblas.so
/usr/lib/atlas/sse2/liblapack.so
I tried both with and without a site.cfg:
[DEFAULT]
library_dirs = /usr/lib/sse2
include_dirs = /usr/include
[blas_opt]
libraries = f77blas, cblas, atlas
[lapack_opt]
libraries = lapack, f77blas, cblas, atlas
and tested wether numpy is actually using the optimized libs as demonstrated in
a posting by Simon Burton
(http://article.gmane.org/gmane.comp.python.numeric.general/5849).
It apparently is linked to
/usr/lib/sse2/libatlas.so.3.0
/usr/lib/sse2/libcblas.so.3.0
/usr/lib/sse2/libf77blas.so.3.0
/usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so
The optimized lapack lib is not used. This is consistent with the output of the
build script:
[EMAIL PROTECTED]:~/prog/scipy/numpy$ python setup.py build
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2_3714
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/atlas
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
customize GnuFCompiler
customize GnuFCompiler
customize GnuFCompiler using config
compiling '_configtest.c':
/* This file is generated from numpy_distutils/system_info.py */
void ATL_buildinfo(void);
int main(void) {
ATL_buildinfo();
return 0;
}
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall
-Wstrict-prototypes -fPIC
compile options: '-c'
gcc: _configtest.c
gcc -pthread _configtest.o -L/usr/lib/sse2 -lf77blas -lcblas -latlas -o
_configtest
ATLAS version 3.6.0 built by root on Fri Jan 9 15:57:20 UTC 2004:
UNAME: Linux intech67 2.4.20 #1 SMP Fri Jan 10 18:29:51 EST 2003 i686
GNU/Linux
INSTFLG :
MMDEF: /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/gemm
ARCHDEF : /fix/g/camm/atlas3-3.6.0/CONFIG/ARCHS/P4SSE2/gcc/misc
F2CDEFS : -DAdd__ -DStringSunStyle
CACHEEDGE: 1048576
F77 : /usr/bin/g77, version GNU Fortran (GCC) 3.3.3 20031229
(prerelease) (Debian)
F77FLAGS : -fomit-frame-pointer -O
CC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease)
(Debian)
CC FLAGS : -fomit-frame-pointer -O3 -funroll-all-loops
MCC : /usr/bin/gcc, version gcc (GCC) 3.3.3 20031229 (prerelease)
(Debian)
MCCFLAGS : -fomit-frame-pointer -O
success!
removing: _configtest.c _configtest.o _configtest
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_INFO', '\\3.6.0\\')]
lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info:
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
libraries lapack_atlas not found in /usr/lib/atlas
libraries lapack not found in /usr/lib/sse2
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS
Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy
Christian K wrote: Hi, I'm trying to build numpy from svn on ubuntu edgy with atlas provided by ubuntu package atlas3-sse2-dev which contains: [...] I tried both with and without a site.cfg: [DEFAULT] library_dirs = /usr/lib/sse2 include_dirs = /usr/include [blas_opt] libraries = f77blas, cblas, atlas [lapack_opt] libraries = lapack, f77blas, cblas, atlas and tested wether numpy is actually using the optimized libs as demonstrated in a posting by Simon Burton (http://article.gmane.org/gmane.comp.python.numeric.general/5849). It apparently is linked to /usr/lib/sse2/libatlas.so.3.0 /usr/lib/sse2/libcblas.so.3.0 /usr/lib/sse2/libf77blas.so.3.0 /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so I tried with a modified site.cfg [DEFAULT] library_dirs = /usr/lib/atlas/sse2:/usr/lib include_dirs = /usr/include [blas_opt] libraries = f77blas, cblas, atlas [lapack_opt] libraries = lapack, f77blas, cblas, atlas and now numpy seems to use both its own lapack_lite and the optimized one: /usr/lib/atlas/sse2/liblapack.so.3.0 /usr/lib/atlas/sse2/liblapack.so.3.0 /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so Is that ok? Christian ___ Numpy-discussion mailing list [EMAIL PROTECTED] http://projects.scipy.org/mailman/listinfo/numpy-discussion
