Re: [Numpy-discussion] Problems when using ACML with numpy
12.05.2012 00:54, Thomas Unterthiner kirjoitti: [clip] The process will have 100% CPU usage and will not show any activity under strace. A gdb backtrace looks as follows: (gdb) bt #0 0x7fdcc000e524 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so [clip] The backtrace looks like it does not use ACML. Does from numpy.core._dotblas import dot work? -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problems when using ACML with numpy
On 05/12/2012 03:27 PM, numpy-discussion-requ...@scipy.org wrote: 12.05.2012 00:54, Thomas Unterthiner kirjoitti: [clip] The process will have 100% CPU usage and will not show any activity under strace. A gdb backtrace looks as follows: (gdb) bt #0 0x7fdcc000e524 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so [clip] The backtrace looks like it does not use ACML. Does from numpy.core._dotblas import dot work? Thanks for having a look at this. The following was tried with the numpy that comes from the Ubuntu repo and symlinked ACML: $ python Python 2.7.3 (default, Apr 20 2012, 22:39:59) [GCC 4.6.3] on linux2 Type help, copyright, credits or license for more information. from numpy.core._dotblas import dot Traceback (most recent call last): File stdin, line 1, in module ImportError: /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so: undefined symbol: cblas_cdotc_sub Following up: $ ldd /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so linux-vdso.so.1 = (0x7fff3de0) libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f10965f8000) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f1096238000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f109603) libgfortran.so.3 = /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x7f1095d18000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f1095a18000) /lib64/ld-linux-x86-64.so.2 (0x7f1098a88000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f10957f8000) libquadmath.so.0 = /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x7f10955c) $ ls -lh /usr/lib/libblas.so.3gf lrwxrwxrwx 1 root root 32 May 11 22:27 /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf $ ls -lh /etc/alternatives/libblas.so.3gf lrwxrwxrwx 1 root root 45 May 11 22:36 /etc/alternatives/libblas.so.3gf - /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so Cheers ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problems when using ACML with numpy
Does ACML now provide a CBLAS interface? Matthieu 2012/5/12 Thomas Unterthiner thomas_unterthi...@web.de On 05/12/2012 03:27 PM, numpy-discussion-requ...@scipy.org wrote: 12.05.2012 00:54, Thomas Unterthiner kirjoitti: [clip] The process will have 100% CPU usage and will not show any activity under strace. A gdb backtrace looks as follows: (gdb) bt #0 0x7fdcc000e524 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so [clip] The backtrace looks like it does not use ACML. Does from numpy.core._dotblas import dot work? Thanks for having a look at this. The following was tried with the numpy that comes from the Ubuntu repo and symlinked ACML: $ python Python 2.7.3 (default, Apr 20 2012, 22:39:59) [GCC 4.6.3] on linux2 Type help, copyright, credits or license for more information. from numpy.core._dotblas import dot Traceback (most recent call last): File stdin, line 1, in module ImportError: /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so: undefined symbol: cblas_cdotc_sub Following up: $ ldd /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so linux-vdso.so.1 = (0x7fff3de0) libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f10965f8000) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f1096238000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f109603) libgfortran.so.3 = /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x7f1095d18000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f1095a18000) /lib64/ld-linux-x86-64.so.2 (0x7f1098a88000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f10957f8000) libquadmath.so.0 = /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x7f10955c) $ ls -lh /usr/lib/libblas.so.3gf lrwxrwxrwx 1 root root 32 May 11 22:27 /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf $ ls -lh /etc/alternatives/libblas.so.3gf lrwxrwxrwx 1 root root 45 May 11 22:36 /etc/alternatives/libblas.so.3gf - /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so Cheers ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problems when using ACML with numpy
On 05/12/2012 04:00 PM, Matthieu Brucher wrote: Does ACML now provide a CBLAS interface? Matthieu D'oh! Very good point, I wasn't aware that numpy needed a CLBAS interface. I now followed the steps outlined at the end of http://mail.scipy.org/pipermail/numpy-discussion/2006-February/018379.html (Basically: I built a libcblas.a for ACML, then tried to build numpy with that). However it didn't seem to work. The same 5000x5000 matrix-multiply is still spinning at 100% CPU usage. I attached to the process after I let it run for over 3 minutes, and the stacktrace looked like this: #0 DOUBLE_dot (ip1=optimized out, is1=8, ip2= [...gibberish...], is2=4, op=0x7f8633086000 , n=5000, __NPY_UNUSED_TAGGEDignore=0x23f40f0) at numpy/core/src/multiarray/arraytypes.c.src:3077 #1 0x7f864dea1466 in PyArray_MatrixProduct2 (op1=optimized out, op2=optimized out, out=optimized out) at numpy/core/src/multiarray/multiarraymodule.c:847 #2 0x7f864dea18ed in array_matrixproduct (__NPY_UNUSED_TAGGEDdummy=optimized out, args=optimized out, kwds=optimized out) at numpy/core/src/multiarray/multiarraymodule.c:2025 #3 0x0042a485 in PyEval_EvalFrameEx () #4 0x004317f2 in PyEval_EvalCodeEx () #5 0x0054bd50 in PyRun_InteractiveOneFlags () #6 0x0054c045 in PyRun_InteractiveLoopFlags () #7 0x0054ce9f in Py_Main () #8 0x7f864ec7976d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 #9 0x0041b931 in _start () So either the process needed changed since that message was written in 2006, or I did something wrong (I am absolutely unfamiliar with the build-system used by numpy) or missed something :( Thomas 2012/5/12 Thomas Unterthiner thomas_unterthi...@web.de mailto:thomas_unterthi...@web.de On 05/12/2012 03:27 PM, numpy-discussion-requ...@scipy.org mailto:numpy-discussion-requ...@scipy.org wrote: 12.05.2012 00:54, Thomas Unterthiner kirjoitti: [clip] The process will have 100% CPU usage and will not show any activity under strace. A gdb backtrace looks as follows: (gdb) bt #0 0x7fdcc000e524 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so [clip] The backtrace looks like it does not use ACML. Does from numpy.core._dotblas import dot work? Thanks for having a look at this. The following was tried with the numpy that comes from the Ubuntu repo and symlinked ACML: $ python Python 2.7.3 (default, Apr 20 2012, 22:39:59) [GCC 4.6.3] on linux2 Type help, copyright, credits or license for more information. from numpy.core._dotblas import dot Traceback (most recent call last): File stdin, line 1, in module ImportError: /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so: undefined symbol: cblas_cdotc_sub Following up: $ ldd /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so linux-vdso.so.1 = (0x7fff3de0) libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f10965f8000) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f1096238000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f109603) libgfortran.so.3 = /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x7f1095d18000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f1095a18000) /lib64/ld-linux-x86-64.so.2 (0x7f1098a88000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f10957f8000) libquadmath.so.0 = /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x7f10955c) $ ls -lh /usr/lib/libblas.so.3gf lrwxrwxrwx 1 root root 32 May 11 22:27 /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf $ ls -lh /etc/alternatives/libblas.so.3gf lrwxrwxrwx 1 root root 45 May 11 22:36 /etc/alternatives/libblas.so.3gf - /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so Cheers ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problems when using ACML with numpy
12.05.2012 17:30, Thomas Unterthiner kirjoitti: [clip] However it didn't seem to work. The same 5000x5000 matrix-multiply is still spinning at 100% CPU usage. I attached to the process after I let it run for over 3 minutes, and the stacktrace looked like this: #0 DOUBLE_dot (ip1=optimized out, is1=8, ip2= [...gibberish...], is2=4, op=0x7f8633086000 , n=5000, __NPY_UNUSED_TAGGEDignore=0x23f40f0) at numpy/core/src/multiarray/arraytypes.c.src:3077 #1 0x7f864dea1466 in PyArray_MatrixProduct2 (op1=optimized out, op2=optimized out, out=optimized out) at numpy/core/src/multiarray/multiarraymodule.c:847 This is also not using ACML. You'll need to adjust things until the line from numpy.core._dotblas import dot works. CBLAS is indeed needed. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Problems when using ACML with numpy
On 05/12/2012 05:34 PM, Pauli Virtanen wrote:
12.05.2012 17:30, Thomas Unterthiner kirjoitti:
[clip]
However it didn't seem to work. The same 5000x5000 matrix-multiply is
still spinning at 100% CPU usage. I attached to the process after I let
it run for over 3 minutes, and the stacktrace looked like this:
#0 DOUBLE_dot (ip1=optimized out, is1=8, ip2=
[...gibberish...], is2=4, op=0x7f8633086000 ,
n=5000, __NPY_UNUSED_TAGGEDignore=0x23f40f0) at
numpy/core/src/multiarray/arraytypes.c.src:3077
#1 0x7f864dea1466 in PyArray_MatrixProduct2 (op1=optimized out,
op2=optimized out, out=optimized out)
at numpy/core/src/multiarray/multiarraymodule.c:847
This is also not using ACML. You'll need to adjust things until the line
from numpy.core._dotblas import dot
works. CBLAS is indeed needed.
It seems like I just can't get it to work. Here is what I did:
I downloaded cblas from netlib, and changed the Makefile.LINUX to read:
BLLIB = /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so
CBLIB = ../lib/cblas.a
CFLAGS = -O3 -DADD_ -march=native -flto -fPIC
FFLAGS = -O3 -march=native -flto
RANLIB = ranlib
Then renamed Makefile.LINUX to Makefile.in, and built cblas. I copied
the resulting libcblas.a to /opt/acml5.1.0/gfortran64_fma4/lib/ . I
also copied the files in $CBLAS_DIR/include to
/opt/acml5.1.0/gfortran64_fma4/include (not sure if that was needed).
I then modified my .bashrc by adding the line
export LD_LIBRARY_PATH=/opt/acml5.1.0/gfortran64_fma4/lib
I started a new shell afterwards to make sure the variable was set.
Next, I downloaded the numpy-1.6.1 source distribution. I then created
the following 'site.cfg':
[blas]
blas_libs = cblas,acml
library_dirs = /opt/acml5.1.0/gfortran64_fma4/lib
include_dirs = /opt/acml5.1.0/gfortran64_fma4/include
[lapack]
language = f77
lapack_libs = acml
library_dirs = /opt/acml5.1.0/gfortran64_fma4/lib
include_dirs = /opt/acml5.1.0/gfortran64_fma4/include
I'm not sure where 'site.cfg' is supposed to go, so I placed one copy in
the root-directory of the package, one in ./numpy/ and one in
./numpy/distutils/
I then did 'sudo python setup.py install --prefix=/usr/local' which
built and installed without a hitch. (Sidenote: in other tries, I
noticed that if I export CFLAGS and LDFLAGS before calling this, there
will be an error, is that normal? How am I supposed to set CFLAGS when
building?)
Anyways, when I don't export any environment variables and just do
'sudo python setup.py install --prefix=/usr/local', evertying works
smoothly. Amongst other things, I get the following output while compiling:
$ sudo python setup.py install --prefix=/usr/local 21
[...]
/media/scratch/software/numpy/numpy-1.6.1/numpy/distutils/system_info.py:1414:
UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
FOUND:
libraries = ['cblas', 'acml']
library_dirs = ['/opt/acml5.1.0/gfortran64_fma4/lib']
language = f77
FOUND:
libraries = ['cblas', 'acml']
library_dirs = ['/opt/acml5.1.0/gfortran64_fma4/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
[...]
lapack_info:
FOUND:
libraries = ['acml']
library_dirs = ['/opt/acml5.1.0/gfortran64_fma4/lib']
language = f77
FOUND:
libraries = ['acml', 'cblas', 'acml']
library_dirs = ['/opt/acml5.1.0/gfortran64_fma4/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
[...]
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h',
'build/src.linux-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h']
building extension numpy.core._dotblas sources
building extension numpy.core.umath_tests sources
conv_template:
build/src.linux-x86_64-2.7/numpy/core/src/umath/umath_tests.c
[...]
/usr/bin/gfortran -Wall -Wall -shared
build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o
build/temp.linux-x86_64-2.7/numpy/linalg/python_xerbla.o
-L/opt/acml5.1.0/gfortran64_fma4/lib -Lbuild/temp.linux-x86_64-2.7
-lacml -lcblas -lacml -lgfortran -o
build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so
The output contains no other mention of anything BLAS-related. Notice
the absense of any calls to gcc/gfortran after the 'building extension
numpy.core._dotblas sources' message.
After the build is done, the directory
./build/lib.linux-x86_64-2.7/numpy/core/ does not contain any dotblas
files, and neither does
/usr/local/lib/python2.7/dist-packages/numpy/core/ afterwards. Thus it
is no surprise that the following still fails:
tom@blucomp:~$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59)
[GCC 4.6.3] on linux2
Type help, copyright, credits or license for more information.
from numpy.core._dotblas import dot
Traceback
[Numpy-discussion] Problems when using ACML with numpy
Hi there! I'm having troubles getting numpy to work with ACML. I'm running Ubuntu 12.04 on an x86-64 system and use acml 5.1.0. On my first try, I installed numpy/scipy from the official ubuntu repository, then just changed the symlink to the blas/lapack libraries of my system to use acml. i.e. I did: ln -s /usr/lib/liblapack.so.3gf /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so ln -s /usr/lib/libblas.so.3gf /opt/acml5.1.0/gfortran64_fma4/lib/libacml.so This method worked perfectly fine for Octave. However with numpy the following code will always run into what seems to be an endless loop: import numpy as np import time a = np.random.randn(5000, 5000) t0 = time.clock() b = np.dot(a, a) t1 = time.clock() print t1 - t0 The process will have 100% CPU usage and will not show any activity under strace. A gdb backtrace looks as follows: (gdb) bt #0 0x7fdcc000e524 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so #1 0x7fdcc008bcb9 in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so #2 0x7fdcc00a304d in ?? () from /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so #3 0x0042a485 in PyEval_EvalFrameEx () #4 0x004317f2 in PyEval_EvalCodeEx () #5 0x0054bd50 in PyRun_InteractiveOneFlags () #6 0x0054c045 in PyRun_InteractiveLoopFlags () #7 0x0054ce9f in Py_Main () #8 0x7fdcc0e7976d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 Curiously enough, when changing the matrix-dimensions from 5000x5000 to 500x500, all is good and the code executes in 0.32 seconds. (As a reference: the 5000x5000 matrix multiply takes 65 seconds in octave with ATLAS and 10 seconds with ACML, so the problem is not that the matrix-multiply just takes too long). I then tried compiling numpy-1.6.1 myself, doing: tar xzf numpy-1.6.1.tar.gz cd numpy-1.6.1/ export CFLAGS=-O3 -march=native export CXXFLAGS=-O3 -march=native export FFLAGS=-O3 -march=native export FCFLAGS=-O3 -march=native export LDFLAGS=-O3 export BLAS=/opt/acml5.1.0/gfortran64_fma4/lib/libacml.so export LAPACK=/opt/acml5.1.0/gfortran64_fma4/lib/libacml.so export ATLAS=None python setup.py build This worked (apart from a missing '-shared' flag when linking lapack_lite.so, which I then had to link by hand by adding the flag), however the error persisted. The same with numpy-1.6.2rc1. Fromt here I don't know how to proceed. Any help would be greatly appreciated :) Cheers Thomas ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
