Re: [Numpy-discussion] Building numpy with ATLAS

[Shitikanth Kashyap]

As it turns out, I just had to run "python setup.py build --force" after 
changing my site.cfg file in order to recompile numpy/core/multiarray.so.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Robert Kern]

Charles R Harris wrote:
> 
> On 4/18/07, *Robert Kern* <[EMAIL PROTECTED]
> > wrote:
> 
> Charles R Harris wrote:
> > I'm wondering if the static libraries could simply be compiled
> with the
> > -fPIC flag and linked with the program to produce the dynamic library.
> > The static libraries are just collections of *.o files, so I don't
> see
> > why that shouldn't work.
> 
> I don't think there is a reason why it *wouldn't* work; it's just
> that one needs
> to do "gcc -shared" to make a .so and that flag also tells the
> linker to choose
> .so's over .a's for the libraries that get linked into the target .so.
> 
> 
> The linker picks dynamic over static even without the -shared flag.

Oh sorry, using -static would normally make the linker choose static libraries
over shared libraries, but that only works when linking an executable. Of
course, David found the right way to bypass this and talk directly to the 
linker.

> I
> think that putting the static libraries in their own directory *without*
> the dynamic libraries might do the trick. Hmm... something to try.

That's usually the way I worked around it. Things may be difficult if your
shared libraries are installed to /usr/lib and you don't want to muck about as
root in there. IIRC (but I possibly don't), you'd have to remove the .so's from
the entire library search path in order to find the .a's.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Charles R Harris]

On 4/18/07, Robert Kern <[EMAIL PROTECTED]> wrote:


Charles R Harris wrote:
> I'm wondering if the static libraries could simply be compiled with the
> -fPIC flag and linked with the program to produce the dynamic library.
> The static libraries are just collections of *.o files, so I don't see
> why that shouldn't work.

I don't think there is a reason why it *wouldn't* work; it's just that one
needs
to do "gcc -shared" to make a .so and that flag also tells the linker to
choose
.so's over .a's for the libraries that get linked into the target .so.



The linker picks dynamic over static even without the -shared flag. I think
that putting the static libraries in their own directory *without* the
dynamic libraries might do the trick. Hmm... something to try.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Robert Kern]

David Cournapeau wrote:
> Robert Kern wrote:
>> Charles R Harris wrote:
>>> I'm wondering if the static libraries could simply be compiled with the
>>> -fPIC flag and linked with the program to produce the dynamic library.
>>> The static libraries are just collections of *.o files, so I don't see
>>> why that shouldn't work.
>> I don't think there is a reason why it *wouldn't* work; it's just that one 
>> needs
>> to do "gcc -shared" to make a .so and that flag also tells the linker to 
>> choose
>> .so's over .a's for the libraries that get linked into the target .so.
>>
> Ok, I found it much faster than I expected: you should use the option 
> -Bdynamic of the *linker*, that is, for example:
> 
> gcc -shared -o c_gaussd.mexglx c_gaussd.o mex_alloc.o 
> -L/usr/local/matlab/bin/glnx86  -L/usr/media/boulot/local/lib 
> -L/usr/lib/atlas -Wl,-Bstatic,-lem_full -llapack_atlas -latlas 
> -Wl,-Bdynamic,-lmx -lmex
> 
> will build a shared "library" c_gaussd.mexglx from c_gaussd.o and 
> mex_alloc.o, statically link libem_full, lapack_atlas and atlas, and 
> dynamically everything else. This was working, as otherwise, matlab 
> would have picked up atlas symbols from its existing, already linked own 
> atlas, and not from the atlas installed on my system (which was crashing 
> matlab and was the reason why I went through this pain in the first place).

Thank you! That's good information to know.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Robert Kern wrote:
> Charles R Harris wrote:
>> I'm wondering if the static libraries could simply be compiled with the
>> -fPIC flag and linked with the program to produce the dynamic library.
>> The static libraries are just collections of *.o files, so I don't see
>> why that shouldn't work.
>
> I don't think there is a reason why it *wouldn't* work; it's just that one 
> needs
> to do "gcc -shared" to make a .so and that flag also tells the linker to 
> choose
> .so's over .a's for the libraries that get linked into the target .so.
>
Ok, I found it much faster than I expected: you should use the option 
-Bdynamic of the *linker*, that is, for example:

gcc -shared -o c_gaussd.mexglx c_gaussd.o mex_alloc.o 
-L/usr/local/matlab/bin/glnx86  -L/usr/media/boulot/local/lib 
-L/usr/lib/atlas -Wl,-Bstatic,-lem_full -llapack_atlas -latlas 
-Wl,-Bdynamic,-lmx -lmex

will build a shared "library" c_gaussd.mexglx from c_gaussd.o and 
mex_alloc.o, statically link libem_full, lapack_atlas and atlas, and 
dynamically everything else. This was working, as otherwise, matlab 
would have picked up atlas symbols from its existing, already linked own 
atlas, and not from the atlas installed on my system (which was crashing 
matlab and was the reason why I went through this pain in the first place).

cheers,

David



   

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Charles R Harris wrote:
>
>
> On 4/18/07, *Keith Goodman* <[EMAIL PROTECTED] 
> > wrote:
>
> On 4/18/07, Charles R Harris <[EMAIL PROTECTED]
> > wrote:
> > On 4/18/07, Keith Goodman <[EMAIL PROTECTED]
> > wrote:
> > > I'd like to compile atlas so that I can take full advantage of
> my core
> > > 2 duo. Numpy dynamically links to the debian binary of
> atlas-sse that
> > > I installed. But the atlas website says that they recommend
> static
> > > linking.
> > >
> > > Which do you recommend, static or dynamic? Are there good
> directions for
> > either?
> >
> > I don't know which is best, although I suspect the statically
> linked version
> > will be larger. It might seem that just pulling in the gemm routines
> > wouldn't add much, but they pull in lots of supporting routines.
> To get
> > numpy to link statically you will also probably need to have a
> directory
> > that contains only the *.a versions because the linker will
> default to the
> > *.so if they are present; i don't think there is a way to
> specify the
> > -static flag to the gcc compiler. Maybe someone else knows how
> to do that.
> > For ATLAS, I believe the latest versions are also recommended
> because the
> > stable version is so old.
>
> At the moment best is equal to easiest since I have never compiled
> atlas. Does anyone know of a howto on compiling atlas (dynamically
> linked)?
>
>
> The instructions that come with ATLAS are useable, and with the 3.7.30 
> version the process is pretty easy. Just make sure you read the part 
> concerning relocatable libraries first, as you will need to add some 
> flags to the ./configure command line, -fPIC  and a couple of others. 
> After compiling and install, IIRC, the libraries will be in 
> /usr/local/lib/ATLAS, which should be OK for debian. Then you need to 
> add a file in /etc/ld.conf.d with the name of the directory and run 
> ldconfig in order to let the system know where the libraries are and 
> what they contain. Or you could save the old /usr/lib/atlas directory 
> and put the new libraries there and then run ldconfig. I also found it 
> necessary to delete the old numpy site-package before reinstalling numpy.
Here is what I do to compile atlas with full lapack, dynamically, on 
debian systems (ubuntu is the same for that matter):

- sudo apt-get install g77 lapack3-pic lapack3-test refblas3-test
- untar atlas, go into ATLAS directory
- mkdir MyObj, cd MyObj (this is where the lib will be built)
- configure the build: ../configure -Si archdef 1 -Fa alg -fPIC -C 
if g77 -Ss flapack /usr/lib/liblapack_pic.a
- build the lib: make
- time and test the library: make time; make test
- build the shared library: cd lib; make shared.

Explanations:
- lapack3-test will be handy to test the compiled atlas, and 
lapack3-pic gives you a lapack library which can be used to build a full 
shared LAPACK library using atlas.
- archdef 1 means uses arch default, much faster - up to 10 times - 
to compile the library
- -Fa alg -fPIC says to add -fPIC to build all objects files (needed 
for shared lib)
- -C if g77 says to use g77 and not gfortran as the fortran 
compiler. On debian, g77 is still the default, and gfortran has a 
different ABI, meaning all kind of problems if you use gfortran for now 
on debian if you do not know what you are doing.

I strongly advise you to use the officiel lapack and blas testers 
afterwards. On debian, as they are dynamically linked to blas/lapack, 
you can choose which library to test thanks to the LD_LIBRARY_PATH trick.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Robert Kern]

Charles R Harris wrote:
> I'm wondering if the static libraries could simply be compiled with the
> -fPIC flag and linked with the program to produce the dynamic library.
> The static libraries are just collections of *.o files, so I don't see
> why that shouldn't work.

I don't think there is a reason why it *wouldn't* work; it's just that one needs
to do "gcc -shared" to make a .so and that flag also tells the linker to choose
.so's over .a's for the libraries that get linked into the target .so.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Charles R Harris]

On 4/18/07, David Cournapeau <[EMAIL PROTECTED]> wrote:


Robert Kern wrote:
> Charles R Harris wrote:
>
>
>> I don't know which is best, although I suspect the statically linked
>> version will be larger. It might seem that just pulling in the gemm
>> routines wouldn't add much, but they pull in lots of supporting
>> routines. To get numpy to link statically you will also probably need
to
>> have a directory that contains only the *.a versions because the linker
>> will default to the *.so if they are present; i don't think there is a
>> way to specify the -static flag to the gcc compiler. Maybe someone else
>> knows how to do that.
>>
>
> Not really. Since you are *making* a shared library, the flags are
already set
> to *look up* shared libraries. I do not believe they are overridable,
much to my
> frequent vexation.
>
I think there is a way, but not trivial. I had to do all kind of tricks
to use fftw in mex (matlab extension written in C, practically
dynamically loaded .so inside matlab), and linking statically some
libraries was one of them. If you still need it (linking some libraries
statically, other dynamically, to build a .so), I can try to dig in my
matlab scripts (unused for some time thanks to scipy :) )



I'm wondering if the static libraries could simply be compiled with the
-fPIC flag and linked with the program to produce the dynamic library. The
static libraries are just collections of *.o files, so I don't see why that
shouldn't work.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Robert Kern wrote:
> Charles R Harris wrote:
>
>   
>> I don't know which is best, although I suspect the statically linked
>> version will be larger. It might seem that just pulling in the gemm
>> routines wouldn't add much, but they pull in lots of supporting
>> routines. To get numpy to link statically you will also probably need to
>> have a directory that contains only the *.a versions because the linker
>> will default to the *.so if they are present; i don't think there is a
>> way to specify the -static flag to the gcc compiler. Maybe someone else
>> knows how to do that.
>> 
>
> Not really. Since you are *making* a shared library, the flags are already set
> to *look up* shared libraries. I do not believe they are overridable, much to 
> my
> frequent vexation.
>   
I think there is a way, but not trivial. I had to do all kind of tricks 
to use fftw in mex (matlab extension written in C, practically 
dynamically loaded .so inside matlab), and linking statically some 
libraries was one of them. If you still need it (linking some libraries 
statically, other dynamically, to build a .so), I can try to dig in my 
matlab scripts (unused for some time thanks to scipy :) )

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Peter C. Norton]

I'm having great difficulty building numpy on solaris with the sun
compilers and libsunperf. I may try this on ubuntu x86_64 to see if
the setup is less painful.

Thanks for the idea,

-Peter

On Wed, Apr 18, 2007 at 11:04:26PM +0200, Christian Marquardt wrote:
> Sun has recently released their compilers under an opensource license for
> Linux as well (Sun Studio Express or something), including their perflib -
> which includes Blas and Lapack. Has somebody tried how that combination
> performs, compared to Intel MKL or Atlas? I think they are free even for
> commercial use.
> 
> Just a thought...
> 
>   Christian.
> 
> 
> On Wed, April 18, 2007 22:27, rex wrote:
> > Andrew Straw <[EMAIL PROTECTED]> [2007-04-18 13:22]:
> >> rex wrote:
> >> > If your use is entirely non-commercial you can use Intel's MKL with
> >> > built-in optimized BLAS and LAPACK and avoid the need for ATLAS.
> >> >
> >> Just to clarify, my understanding is that if you buy a developer's
> >> license, you can also use it for commercial use, including distributing
> >> binaries. (Otherwise it would seem kind of silly for Intel to invest so
> >> much in their performance libraries and compilers.)
> >
> > Yes, I should have included that. icc and MKL licenses for commercial
> > use are $399 each.
> >
> > -rex
> > --
> > "It's a Singer, Captain Picard."   "Make it sew."
> >
> > ___
> > Numpy-discussion mailing list
> > Numpy-discussion@scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
> 
> 
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

-- 
The 5 year plan:
In five years we'll make up another plan.
Or just re-use this one.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Christian Marquardt]

Sun has recently released their compilers under an opensource license for
Linux as well (Sun Studio Express or something), including their perflib -
which includes Blas and Lapack. Has somebody tried how that combination
performs, compared to Intel MKL or Atlas? I think they are free even for
commercial use.

Just a thought...

  Christian.


On Wed, April 18, 2007 22:27, rex wrote:
> Andrew Straw <[EMAIL PROTECTED]> [2007-04-18 13:22]:
>> rex wrote:
>> > If your use is entirely non-commercial you can use Intel's MKL with
>> > built-in optimized BLAS and LAPACK and avoid the need for ATLAS.
>> >
>> Just to clarify, my understanding is that if you buy a developer's
>> license, you can also use it for commercial use, including distributing
>> binaries. (Otherwise it would seem kind of silly for Intel to invest so
>> much in their performance libraries and compilers.)
>
> Yes, I should have included that. icc and MKL licenses for commercial
> use are $399 each.
>
> -rex
> --
> "It's a Singer, Captain Picard."   "Make it sew."
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[rex]

Andrew Straw <[EMAIL PROTECTED]> [2007-04-18 13:22]:
> rex wrote:
> > If your use is entirely non-commercial you can use Intel's MKL with
> > built-in optimized BLAS and LAPACK and avoid the need for ATLAS.
> >   
> Just to clarify, my understanding is that if you buy a developer's
> license, you can also use it for commercial use, including distributing
> binaries. (Otherwise it would seem kind of silly for Intel to invest so
> much in their performance libraries and compilers.)

Yes, I should have included that. icc and MKL licenses for commercial
use are $399 each.

-rex
-- 
"It's a Singer, Captain Picard."   "Make it sew."

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Andrew Straw]

rex wrote:
> Keith Goodman <[EMAIL PROTECTED]> [2007-04-18 10:49]:
>   
>> I'd like to compile atlas so that I can take full advantage of my core
>> 2 duo. 
>> 
>
> If your use is entirely non-commercial you can use Intel's MKL with
> built-in optimized BLAS and LAPACK and avoid the need for ATLAS.
>   
Just to clarify, my understanding is that if you buy a developer's
license, you can also use it for commercial use, including distributing
binaries. (Otherwise it would seem kind of silly for Intel to invest so
much in their performance libraries and compilers.)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Keith Goodman]

On 4/18/07, rex <[EMAIL PROTECTED]> wrote:
> Keith Goodman <[EMAIL PROTECTED]> [2007-04-18 10:49]:
> > I'd like to compile atlas so that I can take full advantage of my core
> > 2 duo.
>
> If your use is entirely non-commercial you can use Intel's MKL with
> built-in optimized BLAS and LAPACK and avoid the need for ATLAS.

Thanks for that. For a variety of reasons I'm sticking with atlas.
Does the parallel flag give you a big speed increase? I imagine it
speeds things up more for larger matrices.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[rex]

Keith Goodman <[EMAIL PROTECTED]> [2007-04-18 10:49]:
> I'd like to compile atlas so that I can take full advantage of my core
> 2 duo. 

If your use is entirely non-commercial you can use Intel's MKL with
built-in optimized BLAS and LAPACK and avoid the need for ATLAS.

http://www.intel.com/cd/software/products/asmo-na/eng/266858.htm

The BLAS and LAPACK libraries are time-honored standards for solving a
large variety of linear algebra problems. The Intel� Math Kernel Library
(Intel� MKL) contains an implementation of BLAS and LAPACK that is
highly optimized for Intel� processors. Intel MKL can enable you to
achieve significant performance improvements over alternative
implementations of BLAS and LAPACK.

[...]

The charts immediately below show that, for Itanium� 2-based systems,
Intel MKL performs approximately 20 percent faster than ATLAS for large
matrices, and even faster for small matrices. On the new Dual-Core
Intel� Xeon� processors, Intel MKL provides similar performance
advantages.


I've compiled both Python (icc) and Numpy using icc 9.1 and MKL
9.1_beta. It's significantly faster than using gcc on my Core 2 Duo
system. I'm still looking for a broad performance test (something like
Scimark, say).

The best compiler flags I've found are: -fast -parallel 
In some cases -funroll-loops and -fno-alias helps.

-rex
-- 
Time flies like wind. Fruit flies like pears.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Robert Kern]

Charles R Harris wrote:

> I don't know which is best, although I suspect the statically linked
> version will be larger. It might seem that just pulling in the gemm
> routines wouldn't add much, but they pull in lots of supporting
> routines. To get numpy to link statically you will also probably need to
> have a directory that contains only the *.a versions because the linker
> will default to the *.so if they are present; i don't think there is a
> way to specify the -static flag to the gcc compiler. Maybe someone else
> knows how to do that.

Not really. Since you are *making* a shared library, the flags are already set
to *look up* shared libraries. I do not believe they are overridable, much to my
frequent vexation.

> For ATLAS, I believe the latest versions are also
> recommended because the stable version is so old.

And definitely if you have a Core 2 Duo. The optimizations for that processor
have only been incorporated in recent versions.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Charles R Harris]

On 4/18/07, Keith Goodman <[EMAIL PROTECTED]> wrote:


On 4/18/07, Charles R Harris <[EMAIL PROTECTED]> wrote:
> On 4/18/07, Keith Goodman <[EMAIL PROTECTED]> wrote:
> > I'd like to compile atlas so that I can take full advantage of my core
> > 2 duo. Numpy dynamically links to the debian binary of atlas-sse that
> > I installed. But the atlas website says that they recommend static
> > linking.
> >
> > Which do you recommend, static or dynamic? Are there good directions
for
> either?
>
> I don't know which is best, although I suspect the statically linked
version
> will be larger. It might seem that just pulling in the gemm routines
> wouldn't add much, but they pull in lots of supporting routines. To get
> numpy to link statically you will also probably need to have a directory
> that contains only the *.a versions because the linker will default to
the
> *.so if they are present; i don't think there is a way to specify the
> -static flag to the gcc compiler. Maybe someone else knows how to do
that.
> For ATLAS, I believe the latest versions are also recommended because
the
> stable version is so old.

At the moment best is equal to easiest since I have never compiled
atlas. Does anyone know of a howto on compiling atlas (dynamically
linked)?



The instructions that come with ATLAS are useable, and with the
3.7.30version the process is pretty easy. Just make sure you read the
part
concerning relocatable libraries first, as you will need to add some flags
to the ./configure command line, -fPIC  and a couple of others. After
compiling and install, IIRC, the libraries will be in /usr/local/lib/ATLAS,
which should be OK for debian. Then you need to add a file in /etc/ld.conf.d
with the name of the directory and run ldconfig in order to let the system
know where the libraries are and what they contain. Or you could save the
old /usr/lib/atlas directory and put the new libraries there and then run
ldconfig. I also found it necessary to delete the old numpy site-package
before reinstalling numpy.

I am not the expert here, so perhaps someone else will weigh in.

Besides speed I'm also interested in seeing if I can get rid of the

repeatability problems I have with the debian atlas-sse2 binary.
(Repeated calulations, as discuss on this list, give give difference
results in numpy but not octave.)



I suspect this has something to do with resetting floating point flags, so
this might not get fixed.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Keith Goodman]

On 4/18/07, Charles R Harris <[EMAIL PROTECTED]> wrote:
> On 4/18/07, Keith Goodman <[EMAIL PROTECTED]> wrote:
> > I'd like to compile atlas so that I can take full advantage of my core
> > 2 duo. Numpy dynamically links to the debian binary of atlas-sse that
> > I installed. But the atlas website says that they recommend static
> > linking.
> >
> > Which do you recommend, static or dynamic? Are there good directions for
> either?
>
> I don't know which is best, although I suspect the statically linked version
> will be larger. It might seem that just pulling in the gemm routines
> wouldn't add much, but they pull in lots of supporting routines. To get
> numpy to link statically you will also probably need to have a directory
> that contains only the *.a versions because the linker will default to the
> *.so if they are present; i don't think there is a way to specify the
> -static flag to the gcc compiler. Maybe someone else knows how to do that.
> For ATLAS, I believe the latest versions are also recommended because the
> stable version is so old.

At the moment best is equal to easiest since I have never compiled
atlas. Does anyone know of a howto on compiling atlas (dynamically
linked)?

Besides speed I'm also interested in seeing if I can get rid of the
repeatability problems I have with the debian atlas-sse2 binary.
(Repeated calulations, as discuss on this list, give give difference
results in numpy but not octave.)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Charles R Harris]

On 4/18/07, Keith Goodman <[EMAIL PROTECTED]> wrote:


On 4/17/07, David Cournapeau <[EMAIL PROTECTED]> wrote:
> Now that you mention it, I am also puzzled by this one: I can see why
> you would use atlas3-sse2-dev without atlas3-base-dev (for the static
> library), but not having atlas3-base-dev makes it imposible to
> dynamically link to atlas libraries without customizing makefiles and/or
> configure options.

I'd like to compile atlas so that I can take full advantage of my core
2 duo. Numpy dynamically links to the debian binary of atlas-sse that
I installed. But the atlas website says that they recommend static
linking.

Which do you recommend, static or dynamic? Are there good directions for
either?



I don't know which is best, although I suspect the statically linked version
will be larger. It might seem that just pulling in the gemm routines
wouldn't add much, but they pull in lots of supporting routines. To get
numpy to link statically you will also probably need to have a directory
that contains only the *.a versions because the linker will default to the
*.so if they are present; i don't think there is a way to specify the
-static flag to the gcc compiler. Maybe someone else knows how to do that.
For ATLAS, I believe the latest versions are also recommended because the
stable version is so old.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Keith Goodman]

On 4/17/07, David Cournapeau <[EMAIL PROTECTED]> wrote:
> Now that you mention it, I am also puzzled by this one: I can see why
> you would use atlas3-sse2-dev without atlas3-base-dev (for the static
> library), but not having atlas3-base-dev makes it imposible to
> dynamically link to atlas libraries without customizing makefiles and/or
> configure options.

I'd like to compile atlas so that I can take full advantage of my core
2 duo. Numpy dynamically links to the debian binary of atlas-sse that
I installed. But the atlas website says that they recommend static
linking.

Which do you recommend, static or dynamic? Are there good directions for either?
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Andrew Straw wrote:
> Christian K wrote:
>   
>> David Cournapeau wrote:
>>   
>> 
>>> On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with 
>>> atlas support. Just install the package atlas3-base-dev, and you are 
>>> done. The reason is that when *compiling* a software which needs atlas, 
>>> the linker will try to find libblas.so in /usr/lib, not in 
>>> /usr/lib/sse2. If you install atlas3-base-dev, the package will install 
>>> those at the correct locations. I have updated the instructions for 
>>> Ubuntu (also works for debian) on the wiki a few days ago:
>>> 
>>>   
>> Indeed, installing atlas3-base-dev helps. I only had atlas3-base, atlas3-sse 
>> and
>> atlas3-sse2-dev installed. Sorry for the noise.
>>   
>> 
> Hmm, atlas3-sse2-dev doesn't depend on atlas3-base-dev? That sounds like 
> a bug...
>   
Now that you mention it, I am also puzzled by this one: I can see why 
you would use atlas3-sse2-dev without atlas3-base-dev (for the static 
library), but not having atlas3-base-dev makes it imposible to 
dynamically link to atlas libraries without customizing makefiles and/or 
configure options. I will report this as a bug, see what the maintainers 
say.

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Andrew Straw]

Christian K wrote:
> David Cournapeau wrote:
>   
>> On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with 
>> atlas support. Just install the package atlas3-base-dev, and you are 
>> done. The reason is that when *compiling* a software which needs atlas, 
>> the linker will try to find libblas.so in /usr/lib, not in 
>> /usr/lib/sse2. If you install atlas3-base-dev, the package will install 
>> those at the correct locations. I have updated the instructions for 
>> Ubuntu (also works for debian) on the wiki a few days ago:
>> 
>
> Indeed, installing atlas3-base-dev helps. I only had atlas3-base, atlas3-sse 
> and
> atlas3-sse2-dev installed. Sorry for the noise.
>   
Hmm, atlas3-sse2-dev doesn't depend on atlas3-base-dev? That sounds like 
a bug...

Off to investigate and possibly file a bug report with Debian or Ubuntu,
Andrew
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Christian K]

David Cournapeau wrote:
> On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with 
> atlas support. Just install the package atlas3-base-dev, and you are 
> done. The reason is that when *compiling* a software which needs atlas, 
> the linker will try to find libblas.so in /usr/lib, not in 
> /usr/lib/sse2. If you install atlas3-base-dev, the package will install 
> those at the correct locations. I have updated the instructions for 
> Ubuntu (also works for debian) on the wiki a few days ago:

Indeed, installing atlas3-base-dev helps. I only had atlas3-base, atlas3-sse and
atlas3-sse2-dev installed. Sorry for the noise.

Christian

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Charles R Harris wrote:
>
>
> On 4/15/07, *David Cournapeau* <[EMAIL PROTECTED] 
> > wrote:
>
> Christian K wrote:
> >
> > Thanks, but that didn't help:
> >
> > atlas_info:
> >   libraries lapack not found in /usr/lib/sse2
> >   libraries f77blas,cblas,atlas not found in /usr/lib/atlas
> >   libraries lapack_atlas not found in /usr/lib/atlas
> >   libraries lapack not found in /usr/lib/sse2
> >   libraries f77blas,cblas,atlas not found in /usr/lib
> >   libraries lapack_atlas not found in /usr/lib
> > numpy.distutils.system_info.atlas_info
> >
> /media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
> > UserWarning:
> >
> *
> > Could not find lapack library within the ATLAS installation.
> >
> *
> >
> >   warnings.warn(message)
> >   FOUND:
> > libraries = ['f77blas', 'cblas', 'atlas']
> > library_dirs = ['/usr/lib/sse2']
> > language = c
> > define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
> >
> > lapack_info:
> >   libraries lapack not found in /usr/lib/sse2
> >   libraries lapack not found in /usr/lib
> >   NOT AVAILABLE
> >
> >
> > Christian
> >
> > ___
> > Numpy-discussion mailing list
> > Numpy-discussion@scipy.org 
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> On Ubuntu and debian, you do NOT need any site.cfg to compile
> numpy with
> atlas support. Just install the package atlas3-base-dev, and you are
> done. The reason is that when *compiling* a software which needs
> atlas,
> the linker will try to find libblas.so in /usr/lib, not in
> /usr/lib/sse2. If you install atlas3-base-dev, the package will
> install
> those at the correct locations. I have updated the instructions for
> Ubuntu (also works for debian) on the wiki a few days ago:
>
> 
> http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe
>
> Note that if you have also optimized version installed (sse, sse2), it
> will use those automatically when you launch numpy (you can check by
> doing ldd on the file numpy/core/_dotblas.so inside your numpy
> installation.
>
>
> Be aware that on recent 64 bit Intel processors running a 64 bit OS 
> the base Atlas package appears to be buggy. You might be better off 
> building your own version, on a fast machine it won't take long.
I do not use any 64 bits machines myself, so I didn't know about this. 
Compiling atlas may take a long time on fast machine if you don't have 
arch defaults, though (it takes several hours for my workstation which 
has two xeon @ 3.2 ghz).

My impression is that using 64 bits OS is still a bit rough, anyway.

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Charles R Harris]

On 4/15/07, David Cournapeau <[EMAIL PROTECTED]> wrote:


Christian K wrote:
>
> Thanks, but that didn't help:
>
> atlas_info:
>   libraries lapack not found in /usr/lib/sse2
>   libraries f77blas,cblas,atlas not found in /usr/lib/atlas
>   libraries lapack_atlas not found in /usr/lib/atlas
>   libraries lapack not found in /usr/lib/sse2
>   libraries f77blas,cblas,atlas not found in /usr/lib
>   libraries lapack_atlas not found in /usr/lib
> numpy.distutils.system_info.atlas_info
> /media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
> UserWarning:
> *
> Could not find lapack library within the ATLAS installation.
> *
>
>   warnings.warn(message)
>   FOUND:
> libraries = ['f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/lib/sse2']
> language = c
> define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
>
> lapack_info:
>   libraries lapack not found in /usr/lib/sse2
>   libraries lapack not found in /usr/lib
>   NOT AVAILABLE
>
>
> Christian
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with
atlas support. Just install the package atlas3-base-dev, and you are
done. The reason is that when *compiling* a software which needs atlas,
the linker will try to find libblas.so in /usr/lib, not in
/usr/lib/sse2. If you install atlas3-base-dev, the package will install
those at the correct locations. I have updated the instructions for
Ubuntu (also works for debian) on the wiki a few days ago:


http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe

Note that if you have also optimized version installed (sse, sse2), it
will use those automatically when you launch numpy (you can check by
doing ldd on the file numpy/core/_dotblas.so inside your numpy
installation.



Be aware that on recent 64 bit Intel processors running a 64 bit OS the base
Atlas package appears to be buggy. You might be better off building your own
version, on a fast machine it won't take long.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[David Cournapeau]

Christian K wrote:
>
> Thanks, but that didn't help:
>
> atlas_info:
>   libraries lapack not found in /usr/lib/sse2
>   libraries f77blas,cblas,atlas not found in /usr/lib/atlas
>   libraries lapack_atlas not found in /usr/lib/atlas
>   libraries lapack not found in /usr/lib/sse2
>   libraries f77blas,cblas,atlas not found in /usr/lib
>   libraries lapack_atlas not found in /usr/lib
> numpy.distutils.system_info.atlas_info
> /media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
> UserWarning:
> *
> Could not find lapack library within the ATLAS installation.
> *
>
>   warnings.warn(message)
>   FOUND:
> libraries = ['f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/lib/sse2']
> language = c
> define_macros = [('ATLAS_WITHOUT_LAPACK', None)]
>
> lapack_info:
>   libraries lapack not found in /usr/lib/sse2
>   libraries lapack not found in /usr/lib
>   NOT AVAILABLE
>
>
> Christian
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
On Ubuntu and debian, you do NOT need any site.cfg to compile numpy with 
atlas support. Just install the package atlas3-base-dev, and you are 
done. The reason is that when *compiling* a software which needs atlas, 
the linker will try to find libblas.so in /usr/lib, not in 
/usr/lib/sse2. If you install atlas3-base-dev, the package will install 
those at the correct locations. I have updated the instructions for 
Ubuntu (also works for debian) on the wiki a few days ago:

http://www.scipy.org/Installing_SciPy/Linux#head-c5a062b2ecf76f746d78cfcde1dae00ae26109fe

Note that if you have also optimized version installed (sse, sse2), it 
will use those automatically when you launch numpy (you can check by 
doing ldd on the file numpy/core/_dotblas.so inside your numpy installation.

cheers,

David


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Christian K]

Robert Kern wrote:
> Christian K wrote:
>> Hi,
>> I'm trying to build numpy from svn on ubuntu edgy with atlas provided by 
>> ubuntu
>> package atlas3-sse2-dev which contains:
>>
>> /usr
>> /usr/lib
>> /usr/lib/sse2
>> /usr/lib/sse2/libatlas.a
>> /usr/lib/sse2/libcblas.a
>> /usr/lib/sse2/libf77blas.a
>> /usr/lib/sse2/liblapack_atlas.a
>> /usr/lib/atlas
>> /usr/lib/atlas/sse2
>> /usr/lib/atlas/sse2/libblas.a
>> /usr/lib/atlas/sse2/liblapack.a
>> /usr/share
>> /usr/share/doc
>> /usr/share/doc/atlas3-sse2-dev
>> /usr/share/doc/atlas3-sse2-dev/copyright
>> /usr/share/doc/atlas3-sse2-dev/changelog.Debian.gz
>> /usr/lib/sse2/libatlas.so
>> /usr/lib/sse2/libcblas.so
>> /usr/lib/sse2/libf77blas.so
>> /usr/lib/sse2/liblapack_atlas.so
>> /usr/lib/atlas/sse2/libblas.so
>> /usr/lib/atlas/sse2/liblapack.so
>>
>> I tried both with and without a site.cfg:
>>
>>
>> [DEFAULT]
>> library_dirs = /usr/lib/sse2
>> include_dirs = /usr/include
>>
>> [blas_opt]
>> libraries = f77blas, cblas, atlas
>>
>> [lapack_opt]
>> libraries = lapack, f77blas, cblas, atlas
> 
>> Confusingly lapack_atlas resides in /usr/lib but even though setup.py looks 
>> for
>> it in that place it reports 'not found'.
>>
>> What should I try next?
> 
> Change this:
> 
> [lapack_opt]
> libraries = lapack, f77blas, cblas, atlas
> 
> to this:
> 
> [lapack_opt]
> libraries = lapack_atlas, f77blas, cblas, atlas
> 

Thanks, but that didn't help:

atlas_info:
  libraries lapack not found in /usr/lib/sse2
  libraries f77blas,cblas,atlas not found in /usr/lib/atlas
  libraries lapack_atlas not found in /usr/lib/atlas
  libraries lapack not found in /usr/lib/sse2
  libraries f77blas,cblas,atlas not found in /usr/lib
  libraries lapack_atlas not found in /usr/lib
numpy.distutils.system_info.atlas_info
/media/hda6/home/ck/prog/scipy/numpy/numpy/distutils/system_info.py:903:
UserWarning:
*
Could not find lapack library within the ATLAS installation.
*

  warnings.warn(message)
  FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
define_macros = [('ATLAS_WITHOUT_LAPACK', None)]

lapack_info:
  libraries lapack not found in /usr/lib/sse2
  libraries lapack not found in /usr/lib
  NOT AVAILABLE


Christian

___
Numpy-discussion mailing list
[EMAIL PROTECTED]
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Robert Kern]

Christian K wrote:
> Hi,
> I'm trying to build numpy from svn on ubuntu edgy with atlas provided by 
> ubuntu
> package atlas3-sse2-dev which contains:
> 
> /usr
> /usr/lib
> /usr/lib/sse2
> /usr/lib/sse2/libatlas.a
> /usr/lib/sse2/libcblas.a
> /usr/lib/sse2/libf77blas.a
> /usr/lib/sse2/liblapack_atlas.a
> /usr/lib/atlas
> /usr/lib/atlas/sse2
> /usr/lib/atlas/sse2/libblas.a
> /usr/lib/atlas/sse2/liblapack.a
> /usr/share
> /usr/share/doc
> /usr/share/doc/atlas3-sse2-dev
> /usr/share/doc/atlas3-sse2-dev/copyright
> /usr/share/doc/atlas3-sse2-dev/changelog.Debian.gz
> /usr/lib/sse2/libatlas.so
> /usr/lib/sse2/libcblas.so
> /usr/lib/sse2/libf77blas.so
> /usr/lib/sse2/liblapack_atlas.so
> /usr/lib/atlas/sse2/libblas.so
> /usr/lib/atlas/sse2/liblapack.so
> 
> I tried both with and without a site.cfg:
> 
> 
> [DEFAULT]
> library_dirs = /usr/lib/sse2
> include_dirs = /usr/include
> 
> [blas_opt]
> libraries = f77blas, cblas, atlas
> 
> [lapack_opt]
> libraries = lapack, f77blas, cblas, atlas

> Confusingly lapack_atlas resides in /usr/lib but even though setup.py looks 
> for
> it in that place it reports 'not found'.
> 
> What should I try next?

Change this:

[lapack_opt]
libraries = lapack, f77blas, cblas, atlas

to this:

[lapack_opt]
libraries = lapack_atlas, f77blas, cblas, atlas

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
[EMAIL PROTECTED]
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] building numpy with atlas on ubuntu edgy

[Christian K]

Christian K wrote:
> Hi,
> I'm trying to build numpy from svn on ubuntu edgy with atlas provided by 
> ubuntu
> package atlas3-sse2-dev which contains:

[...]

> 
> I tried both with and without a site.cfg:
> 
> 
> [DEFAULT]
> library_dirs = /usr/lib/sse2
> include_dirs = /usr/include
> 
> [blas_opt]
> libraries = f77blas, cblas, atlas
> 
> [lapack_opt]
> libraries = lapack, f77blas, cblas, atlas
> 
> and tested wether numpy is actually using the optimized libs as demonstrated 
> in
> a posting by Simon Burton
> (http://article.gmane.org/gmane.comp.python.numeric.general/5849).
> 
> It apparently is linked to
> /usr/lib/sse2/libatlas.so.3.0
> /usr/lib/sse2/libcblas.so.3.0
> /usr/lib/sse2/libf77blas.so.3.0
> /usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so

I tried with a modified site.cfg

[DEFAULT]
library_dirs = /usr/lib/atlas/sse2:/usr/lib
include_dirs = /usr/include
[blas_opt]
libraries = f77blas, cblas, atlas
[lapack_opt]
libraries = lapack, f77blas, cblas, atlas

and now numpy seems to use both its own lapack_lite and the optimized one:

/usr/lib/atlas/sse2/liblapack.so.3.0
/usr/lib/atlas/sse2/liblapack.so.3.0
/usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so
/usr/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so

Is that ok?

Christian

___
Numpy-discussion mailing list
[EMAIL PROTECTED]
http://projects.scipy.org/mailman/listinfo/numpy-discussion