Re: [Numpy-discussion] another numpy/ATLAS problem
(we hashed this out on IRC, but replying here for the sake of recording it) On 5-Dec-09, at 9:04 PM, Pauli Virtanen wrote: > Can you try to change linalg/setup.py so that it *only* includes > lapack_litemodule.c in the build? Yup; it turns out it wasn't NumPy's lapack_lite calling dlamc3_ but rather other routines in LAPACK. It was accepted as a bug and it should be fixed in future 3.9.x's of ATLAS. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
la, 2009-12-05 kello 20:27 -0500, David Warde-Farley kirjoitti: [clip] > I've briefly been trying to figure out how to work around this, but > I'm a bit confused. I was under the (mistaken?) impression that the > f2c'd code in dlapack_lite.c, zlapack_lite.c and dlamch.c was only > used in the absence of an external BLAS/LAPACK, so that NumPy can fall > back on its own; yet these are the only places dlamc3_ is referenced. I think your impression is not mistaken, the f2c'd stuff really is intended to be included only when a real lapack is not available. And lamc3_ is not called in lapack_litemodule.c... Even more curiously, dlamc3_ is actually included in dlamch.c, so if that file is included in the build, one would expect the routine be available... Perhaps there is some distutils mystery going on, maybe the depends list becomes reordered or something, messing up the list of files included in the build. In fact, I don't understand why linalg/setup.py uses a callback to construct the source file list -- it would be simpler to construct the source file list before passing it on to config.add_extension. Can you try to change linalg/setup.py so that it *only* includes lapack_litemodule.c in the build? -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On 29-Nov-09, at 4:46 PM, David Warde-Farley wrote: > On 27-Nov-09, at 6:29 PM, Charles R Harris wrote: > >> 3.9.12 segfaulted on me while running, so I haven't bothered with >> versions after that. Why not try the stable version 3.8.3? > > Just to follow up, I went back to 3.9.11 and numpy works without > incident, using the exact same ATLAS build procedure; I guess it > wasn't something I was doing wrong after all. > > I've filed it on the ATLAS tracker, so hopefully it'll be addressed > sooner or later. Below is the response I received from Clint Whaley re: dlamc3_ and NumPy. It seems that NumPy's lapack_lite is making some assumptions about routines which aren't actually there in recent (3.9.16 and on) versions of ATLAS, as it provides a speedier version of DLAMCH. I've briefly been trying to figure out how to work around this, but I'm a bit confused. I was under the (mistaken?) impression that the f2c'd code in dlapack_lite.c, zlapack_lite.c and dlamch.c was only used in the absence of an external BLAS/LAPACK, so that NumPy can fall back on its own; yet these are the only places dlamc3_ is referenced. The ATLAS ticket is at http://tinyurl.com/y8dv5w8 if you wish to respond directly as well. David, Well, dlamc3 is an internal routine called by dlamch in LAPACK in order to keep register assignment from screwing up their internal tests. It is not part of the official LAPACK API AFAIK. It looks like some part of NumPy is calling this routine, expecting that DLAMCH will provide it. However, from 3.9.16 on, ATLAS provides a native LAMCH implementation, that reads in a generated file for improved speed. This LAMCH does no computation at all, and so obviously does not provide the internal routine DLAM3. Can you ask the NumPy people to comment on why they are calling this routine? Thanks, Clint ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On 27-Nov-09, at 6:29 PM, Charles R Harris wrote: > 3.9.12 segfaulted on me while running, so I haven't bothered with > versions after that. Why not try the stable version 3.8.3? Just to follow up, I went back to 3.9.11 and numpy works without incident, using the exact same ATLAS build procedure; I guess it wasn't something I was doing wrong after all. I've filed it on the ATLAS tracker, so hopefully it'll be addressed sooner or later. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On 27-Nov-09, at 7:49 PM, David Cournapeau wrote: > I guess because of the updated threading support. Right you are; the 3.9 series is rather faster I find, at least in parallel. > I think the solution is to simply bypass the atlas mechanism to build > lapack, and do it manually. Tried that... see below: > "and so I unarchived the liblapack.a that ATLAS spit out and added > those files to the one built by Netlib's makefile. The result is still > the same." I'm gonna try what Chuck suggested and go back a few versions at least. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On Sat, Nov 28, 2009 at 8:29 AM, Charles R Harris wrote: > > > On Fri, Nov 27, 2009 at 4:18 PM, David Warde-Farley > wrote: >> >> On 27-Nov-09, at 5:02 PM, Charles R Harris wrote: >> >> > What version of ATLAS? >> >> 3.9.17, the latest development branch version. At some point in the > > 3.9.12 segfaulted on me while running, so I haven't bothered with versions > after that. Why not try the stable version 3.8.3? I guess because of the updated threading support. I think the solution is to simply bypass the atlas mechanism to build lapack, and do it manually. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On Fri, Nov 27, 2009 at 4:18 PM, David Warde-Farley wrote: > On 27-Nov-09, at 5:02 PM, Charles R Harris wrote: > > > What version of ATLAS? > > 3.9.17, the latest development branch version. At some point in the 3.9.12 segfaulted on me while running, so I haven't bothered with versions after that. Why not try the stable version 3.8.3? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On 27-Nov-09, at 5:02 PM, Charles R Harris wrote: > What version of ATLAS? 3.9.17, the latest development branch version. At some point in the changelog the author mentions he removed other methods building with LAPACK, leaving only the tarfile one. I'm giving the entire build another try with gfortran-4.2 rather than gfortran-4.4, since I already had a problem on another Ubuntu 9.10 machine with gfortran 4.4 and that exact same function, except it wasn't an unresolved symbol issue, rather it was an infinite loop: http://mail.scipy.org/pipermail/numpy-discussion/2009-November/046373.html David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On Fri, Nov 27, 2009 at 2:04 PM, David Warde-Farley wrote: > On 26-Nov-09, at 8:56 PM, Charles R Harris wrote: > > > I never had luck with the netlib-lapack-tarfile= > > option, it didn't build lapack correctly. Try doing them separately. > > I just tried that. > > -Ss flapack path/to/lapack_LINUX.a didn't seem to build a proper > > --with-netlib-lapack= What version of ATLAS? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On 26-Nov-09, at 8:56 PM, Charles R Harris wrote: > I never had luck with the netlib-lapack-tarfile= > option, it didn't build lapack correctly. Try doing them separately. I just tried that. -Ss flapack path/to/lapack_LINUX.a didn't seem to build a proper LAPACK (it was missing all sorts of stuff and was only about 800kb) and so In unarchived the liblapack.a that ATLAS spit out and added those files to the one build by Netlib's makefile. The result is still the same. I'm wondering if this is maybe a gfortran version issue, somehow. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] another numpy/ATLAS problem
On Thu, Nov 26, 2009 at 6:53 PM, David Warde-Farley wrote: > It seems like I've done this enough times on enough machines but I'm > still running into problems... > > After installing what I'm pretty sure is a complete ATLAS+LAPACK and > building numpy against it (appropriately editing site.cfg, etc.) I'm > faced with this when building numpy-1.3.0: > > d...@mirage:~/tmp$ python > Python 2.6.4 (r264:75706, Nov 2 2009, 14:44:17) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > Traceback (most recent call last): > File "", line 1, in > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > __init__.py", line 130, in > import add_newdocs > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > add_newdocs.py", line 9, in > from lib import add_newdoc > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > lib/__init__.py", line 13, in > from polynomial import * > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > lib/polynomial.py", line 18, in > from numpy.linalg import eigvals, lstsq > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > linalg/__init__.py", line 47, in > from linalg import * > File "/home/dwf/virtualenv/stable/lib/python2.6/site-packages/numpy/ > linalg/linalg.py", line 22, in > from numpy.linalg import lapack_lite > ImportError: /home/dwf/virtualenv/stable/lib/python2.6/site-packages/ > numpy/linalg/lapack_lite.so: undefined symbol: dlamc3_ > > FWIW these are the files that ATLAS has installed: > > d...@mirage:~/Downloads$ ls /opt/sw/ATLAS/lib/ -al > total 23472 > drwxr-xr-x 2 root root 4096 2009-11-26 20:36 . > drwxr-xr-x 4 root root 4096 2009-11-26 20:36 .. > -rw-r--r-- 1 root root 12963138 2009-11-26 20:36 libatlas.a > -rw-r--r-- 1 root root 466244 2009-11-26 20:36 libcblas.a > -rw-r--r-- 1 root root 572812 2009-11-26 20:36 libf77blas.a > -rw-r--r-- 1 root root 8979016 2009-11-26 20:36 liblapack.a > -rw-r--r-- 1 root root 466678 2009-11-26 20:36 libptcblas.a > -rw-r--r-- 1 root root 573196 2009-11-26 20:36 libptf77blas.a > > liblapack.a looks big enough for it to be a complete LAPACK. > > ATLAS (3.9.17) was configured with the following: > > ../configure -Ss kern /usr/bin/gcc-4.2 --prefix=/opt/sw/ATLAS --with- > netlib-lapack-tarfile=/home/dwf/tmp/lapack-lite-3.1.1.tgz -Fa alg - > fPIC -D c -DPentiumCPS=2668 -t 4 -b 64 > > I never had luck with the netlib-lapack-tarfile= option, it didn't build lapack correctly. Try doing them separately. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
