Re: [Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Skipper Seabold]

On Sat, Mar 23, 2013 at 8:44 PM, Skipper Seabold  wrote:
> On Sat, Mar 23, 2013 at 7:26 PM, Ake Sandgren  
> wrote:
>> On Sat, 2013-03-23 at 14:19 -0400, Skipper Seabold wrote:
>>> Some help on this would be greatly appreciated. It's been recommended
>>> to use OpenBlas over ATLAS, so I've been trying to build numpy with
>>> openblas and have run into a few problems.
>>
>>>
>>> To truly support OpenBlas, is it maybe necessary to make some
>>> additions to numpy/distutils/system_info.py?
>>
>> Here is how.
>>
>> https://github.com/akesandgren/numpy/commit/363339dd3a9826f3e3e7dc4248c258d3c4dfcd7c
>>
>
>
> Thanks that works well for numpy. Test pass. I hope that makes it into
> a pull request. My site.cfg looks like this. I don't know about the
> lapack_opt section. It doesn't seem to work.
>
> [DEFAULT]
> library_dirs = /home/skipper/.local/lib
> include_dirs = /home/skipper/.local/include
>
> [openblas]
> libraries = openblas
>
> [lapack_opt]
> libraries = openblas
>
> Do you have any idea how to get scipy working too. I have a similar
> site.cfg, but it does not find lapack, which is rolled into
> libopenblas from what I understand. I can do
>
> export LAPACK=~/.local/lib/libopenblas.a
> python setup.py build &> build.log
> sudo -E python setup.py install
>
> There are no obvious failures in the build.log, but scipy is still
> broken because it needs lapack from numpy I guess.

The answer is to

export BLAS=~/.local/lib/libopenblas.a
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/

before building and installing. Now everything works. Whew. Thanks a
lot for the help.

>
 import numpy as np
 np.show_config()
> lapack_info:
>   NOT AVAILABLE
> atlas_threads_info:
>   NOT AVAILABLE
> blas_opt_info:
> libraries = ['openblas', 'openblas']
> library_dirs = ['/home/skipper/.local/lib']
> language = f77
> lapack_src_info:
>   NOT AVAILABLE
> openblas_info:
> libraries = ['openblas', 'openblas']
> library_dirs = ['/home/skipper/.local/lib']
> language = f77
> lapack_opt_info:
>   NOT AVAILABLE
> atlas_info:
>   NOT AVAILABLE
> lapack_mkl_info:
>   NOT AVAILABLE
> blas_mkl_info:
>   NOT AVAILABLE
> mkl_info:
>   NOT AVAILABLE
 from scipy import stats
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "/usr/local/lib/python2.7/dist-packages/scipy/stats/__init__.py",
> line 320, in 
> from .stats import *
>   File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py",
> line 242, in 
> import scipy.linalg as linalg
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/__init__.py",
> line 147, in 
> from .misc import *
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/misc.py",
> line 5, in 
> from . import blas
>   File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/blas.py",
> line 113, in 
> from scipy.linalg import _fblas
> ImportError: libopenblas.so.0: cannot open shared object file: No such
> file or directory
>
> Skipper
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Skipper Seabold]

On Sat, Mar 23, 2013 at 7:26 PM, Ake Sandgren  wrote:
> On Sat, 2013-03-23 at 14:19 -0400, Skipper Seabold wrote:
>> Some help on this would be greatly appreciated. It's been recommended
>> to use OpenBlas over ATLAS, so I've been trying to build numpy with
>> openblas and have run into a few problems.
>
>>
>> To truly support OpenBlas, is it maybe necessary to make some
>> additions to numpy/distutils/system_info.py?
>
> Here is how.
>
> https://github.com/akesandgren/numpy/commit/363339dd3a9826f3e3e7dc4248c258d3c4dfcd7c
>


Thanks that works well for numpy. Test pass. I hope that makes it into
a pull request. My site.cfg looks like this. I don't know about the
lapack_opt section. It doesn't seem to work.

[DEFAULT]
library_dirs = /home/skipper/.local/lib
include_dirs = /home/skipper/.local/include

[openblas]
libraries = openblas

[lapack_opt]
libraries = openblas

Do you have any idea how to get scipy working too. I have a similar
site.cfg, but it does not find lapack, which is rolled into
libopenblas from what I understand. I can do

export LAPACK=~/.local/lib/libopenblas.a
python setup.py build &> build.log
sudo -E python setup.py install

There are no obvious failures in the build.log, but scipy is still
broken because it needs lapack from numpy I guess.

>>> import numpy as np
>>> np.show_config()
lapack_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
blas_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/home/skipper/.local/lib']
language = f77
lapack_src_info:
  NOT AVAILABLE
openblas_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/home/skipper/.local/lib']
language = f77
lapack_opt_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE
>>> from scipy import stats
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python2.7/dist-packages/scipy/stats/__init__.py",
line 320, in 
from .stats import *
  File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py",
line 242, in 
import scipy.linalg as linalg
  File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/__init__.py",
line 147, in 
from .misc import *
  File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/misc.py",
line 5, in 
from . import blas
  File "/usr/local/lib/python2.7/dist-packages/scipy/linalg/blas.py",
line 113, in 
from scipy.linalg import _fblas
ImportError: libopenblas.so.0: cannot open shared object file: No such
file or directory

Skipper
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Ake Sandgren]

On Sat, 2013-03-23 at 14:19 -0400, Skipper Seabold wrote:
> Some help on this would be greatly appreciated. It's been recommended
> to use OpenBlas over ATLAS, so I've been trying to build numpy with
> openblas and have run into a few problems.

> 
> To truly support OpenBlas, is it maybe necessary to make some
> additions to numpy/distutils/system_info.py?

Here is how.

https://github.com/akesandgren/numpy/commit/363339dd3a9826f3e3e7dc4248c258d3c4dfcd7c

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Colin J. Williams]

  
  
On 23/03/2013 12:05 AM, Chris Barker -
  NOAA Federal wrote:


  On Fri, Mar 22, 2013 at 2:39 PM, Colin J. Williams
 wrote:

  
I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and
3.3.0.

  
  ...

  
The tests, which are available
here(http://web.ncf.ca/cjw/FP%20Summary%20over%20273-323-330.txt), show that
3.2 is slower, but not to the same degree reported before.

  
  
Have posted your test code anywhere? Anyway, depending on how you did
your timings, that looks to me like 3.* is a bit faster with small
data, and pretty much within measurement error for the large datasets.

And if the large ones are doing things with really big arrays (I'm
assuming pretty big, as you're getting close to 32 bit memory
limits...), then it's really hard to imagine how python version could
make a noticeable difference -- the real work would be in the numpy
code, and that's exactly the same on all python versions.

If you are using BLAS or LAPACK stuff, then there might be some
differences with the different builds, though I wouldn't expect so if
you ar getting them from the same source.

-Chris


I used the versions from PyPi, this choice has been
questioned.  I'll compare with the SourceForge versions.  Also,
I shall be incorporating the random SEED.

I expect to report the results in the next week or so.

The test code used is available here: http://web.ncf.ca/cjw/testFPSpeed.py

Colin W.
  

  




  

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Colin J. Williams]

  
  
On 23/03/2013 11:17 AM, Daπid wrote:


  I am a bit worried about the differences in results. Just to be sure
you are comparing apples with apples, it may be a good idea to set the
seed at the beginning:

np.random.seed( SEED )

where SEED is an int. This way, you will be inverting always the same
matrix, regardless of the Python version. I think, even if the timing
is different, the results should be the same.

http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.seed.html#numpy.random.seed



David.


David,

Thanks for this suggestion.  I will incorporate it, but first I
am following up on the suggestion that the product of
easy_install is inferior to that from Source Forge.

The results for Python 3.2 using a win32 system have been
consistently inferior to 2.7, but using Raspberry Pi, an ARM
processor, the results for the two Python Versions have been
much the same.

I'll post the results in the next week or so.

Colin W.

[snip]
  

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Skipper Seabold]

On Sat, Mar 23, 2013 at 2:32 PM, Hanno Klemm  wrote:

> Skipper,
> this looks like a problem that I had in the bad old days with ATLAS, as
> well. Try compiling openblas with the -fPIC flag that used to help.
>
>
Thanks for having a look. I checked after seeing that odd bento failure
(see here [1]), and it looks to me like OpenBlas uses the -fPIC flag in all
of the gcc and gfortran calls. Possible related? [2]

Skipper

[1] https://github.com/cournape/Bento/issues/116
[2] https://github.com/cournape/Bento/issues/128



> Best of luck,
> Hanno
>
> hanno.kl...@me.com
>
> Sent from my mobile device, please excuse my brevity.
>
> On 23.03.2013, at 19:19, Skipper Seabold  wrote:
>
> Some help on this would be greatly appreciated. It's been recommended to
> use OpenBlas over ATLAS, so I've been trying to build numpy with openblas
> and have run into a few problems.
>
> 1) Build fails using bento master and waf 1.7.9, see below.
> 2) Distutils doesn't seem to be able to find lapack as part of atlas. I
> tried to skip a site.cfg and define environmental variables. No idea what I
> missed.
>
> I followed instructions found scattered over the internet and only
> understand vaguely the issues. Maybe someone can help. I'd be happy to
> update the wiki with any answers.
>
> To truly support OpenBlas, is it maybe necessary to make some additions to
> numpy/distutils/system_info.py?
>
> Thanks for having a look,
>
> Skipper
>
> Install OpenBlas
> -
> git clone git://github.com/xianyi/OpenBLAS
> cd OpenBlas
>
> Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)
>
> |4 $ git diff c_check
> ```
> diff --git a/c_check b/c_check
> index 4d82237..de0fd33 100644
> --- a/c_check
> +++ b/c_check
> @@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if
> $need_fu
>
>  if ($os eq "LINUX") {
>
> -@pthread = split(/\s+/, `nm /lib/libpthread.so* | grep
> _pthread_create`);
> +@pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* |
> grep _pthread_create`);
>
>  if ($pthread[2] ne "") {
> print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";
> ```
>
> make fc=gfortran
> make PREFIX=~/.local install
>
> Everything looks ok, so far.
>
> Install NumPy
> ---
> Using numpy master
>
> I tried to use bento master and waf 1.7.9, following instructions from
> David's blog
>
> bentomaker configure --prefix=/home/skipper/.local
> --with-blas-lapack-libdir=/home/skipper/.local/lib
> --blas-lapack-type=openblas ..
> bentomaker build -j4
>
> ```
> 
> [101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o ->
> build/numpy/core/umath_tests.so
>
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
> when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> ```
>
> No idea, so, let's try distutils
>
> export LAPACK=~/.local/lib/libopenblas.a
> export BLAS=~/.local/lib/libopenblas.a
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
> echo $LD_LIBRARY_PATH
> ```
> :/usr/local/lib64/R/bin:/home/skipper/.local/lib/
> ```
>
> This step seems to be necessary?
>
> python setup.py config
> ```
> Running from numpy source directory.
> non-existing path in 'numpy/distutils': 'site.cfg'
> F2PY Version 2
> numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected,
> the C API version numbers have to be updated. Current C api version is 8,
> with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C
> API version 8 in codegen_dir/cversions.txt is
> 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you
> have to update C_API_VERSION  in numpy/core/setup_common.py.
>   MismatchCAPIWarning)
> blas_opt_info:
> blas_mkl_info:
>   libraries mkl,vml,guide not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
>
> atlas_blas_threads_info:
> Setting PTATLAS=ATLAS
>   libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
>
> atlas_blas_info:
>   libraries f77blas,cblas,atlas not found in ['/usr/local/lib64',
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
>
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501:
> UserWarning:
> Atlas (http://math-atlas.sourceforge.net/) libraries not found.
> Directories to search for the libraries can be specified in the
> numpy/distutils/

Re: [Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Hanno Klemm]

Skipper,
this looks like a problem that I had in the bad old days with ATLAS, as well. 
Try compiling openblas with the -fPIC flag that used to help. 

Best of luck,
Hanno

hanno.kl...@me.com

Sent from my mobile device, please excuse my brevity. 

On 23.03.2013, at 19:19, Skipper Seabold  wrote:

> Some help on this would be greatly appreciated. It's been recommended to use 
> OpenBlas over ATLAS, so I've been trying to build numpy with openblas and 
> have run into a few problems.
> 
> 1) Build fails using bento master and waf 1.7.9, see below.
> 2) Distutils doesn't seem to be able to find lapack as part of atlas. I tried 
> to skip a site.cfg and define environmental variables. No idea what I missed.
> 
> I followed instructions found scattered over the internet and only understand 
> vaguely the issues. Maybe someone can help. I'd be happy to update the wiki 
> with any answers. 
> 
> To truly support OpenBlas, is it maybe necessary to make some additions to 
> numpy/distutils/system_info.py?
> 
> Thanks for having a look,
> 
> Skipper
> 
> Install OpenBlas
> -
> git clone git://github.com/xianyi/OpenBLAS
> cd OpenBlas
> 
> Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)
> 
> |4 $ git diff c_check
> ```
> diff --git a/c_check b/c_check
> index 4d82237..de0fd33 100644
> --- a/c_check
> +++ b/c_check
> @@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if 
> $need_fu
>  
>  if ($os eq "LINUX") {
>  
> -@pthread = split(/\s+/, `nm /lib/libpthread.so* | grep _pthread_create`);
> +@pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* | grep 
> _pthread_create`);
>  
>  if ($pthread[2] ne "") {
> print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";
> ```
> 
> make fc=gfortran
> make PREFIX=~/.local install
> 
> Everything looks ok, so far. 
> 
> Install NumPy
> ---
> Using numpy master
> 
> I tried to use bento master and waf 1.7.9, following instructions from 
> David's blog
> 
> bentomaker configure --prefix=/home/skipper/.local 
> --with-blas-lapack-libdir=/home/skipper/.local/lib 
> --blas-lapack-type=openblas ..
> bentomaker build -j4
> 
> ```
> 
> [101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o -> 
> build/numpy/core/umath_tests.so   
>
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation 
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when 
> making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation 
> R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when 
> making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> ```
> 
> No idea, so, let's try distutils
> 
> export LAPACK=~/.local/lib/libopenblas.a
> export BLAS=~/.local/lib/libopenblas.a
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
> echo $LD_LIBRARY_PATH
> ```
> :/usr/local/lib64/R/bin:/home/skipper/.local/lib/
> ```
> 
> This step seems to be necessary?
> 
> python setup.py config
> ```
> Running from numpy source directory.
> non-existing path in 'numpy/distutils': 'site.cfg'
> F2PY Version 2
> numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected, 
> the C API version numbers have to be updated. Current C api version is 8, 
> with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C 
> API version 8 in codegen_dir/cversions.txt is 
> 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you 
> have to update C_API_VERSION  in numpy/core/setup_common.py.
>   MismatchCAPIWarning)
> blas_opt_info:
> blas_mkl_info:
>   libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', 
> '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> atlas_blas_threads_info:
> Setting PTATLAS=ATLAS
>   libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64', 
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> atlas_blas_info:
>   libraries f77blas,cblas,atlas not found in ['/usr/local/lib64', 
> '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501: 
> UserWarning: 
> Atlas (http://math-atlas.sourceforge.net/) libraries not found.
> Directories to search for the libraries can be specified in the
> numpy/distutils/site.cfg file (section [atlas]) or by setting
> the ATLAS environment variable.
>   warnings.warn(AtlasNotFoundError.__doc__)
> blas_info:
> Replacing _lib_names[0]=='blas' with 'openblas'
> Replacing _lib_names[0]=='openblas' with 'openblas'
>   FOUND:
> 

[Numpy-discussion] Unable to building numpy with openblas using bento or distutils

[Skipper Seabold]

Some help on this would be greatly appreciated. It's been recommended to
use OpenBlas over ATLAS, so I've been trying to build numpy with openblas
and have run into a few problems.

1) Build fails using bento master and waf 1.7.9, see below.
2) Distutils doesn't seem to be able to find lapack as part of atlas. I
tried to skip a site.cfg and define environmental variables. No idea what I
missed.

I followed instructions found scattered over the internet and only
understand vaguely the issues. Maybe someone can help. I'd be happy to
update the wiki with any answers.

To truly support OpenBlas, is it maybe necessary to make some additions to
numpy/distutils/system_info.py?

Thanks for having a look,

Skipper

Install OpenBlas
-
git clone git://github.com/xianyi/OpenBLAS
cd OpenBlas

Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)

|4 $ git diff c_check
```
diff --git a/c_check b/c_check
index 4d82237..de0fd33 100644
--- a/c_check
+++ b/c_check
@@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if
$need_fu

 if ($os eq "LINUX") {

-@pthread = split(/\s+/, `nm /lib/libpthread.so* | grep
_pthread_create`);
+@pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* |
grep _pthread_create`);

 if ($pthread[2] ne "") {
print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";
```

make fc=gfortran
make PREFIX=~/.local install

Everything looks ok, so far.

Install NumPy
---
Using numpy master

I tried to use bento master and waf 1.7.9, following instructions from
David's blog

bentomaker configure --prefix=/home/skipper/.local
--with-blas-lapack-libdir=/home/skipper/.local/lib
--blas-lapack-type=openblas ..
bentomaker build -j4

```

[101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o ->
build/numpy/core/umath_tests.so

/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
```

No idea, so, let's try distutils

export LAPACK=~/.local/lib/libopenblas.a
export BLAS=~/.local/lib/libopenblas.a
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
echo $LD_LIBRARY_PATH
```
:/usr/local/lib64/R/bin:/home/skipper/.local/lib/
```

This step seems to be necessary?

python setup.py config
```
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2
numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected,
the C API version numbers have to be updated. Current C api version is 8,
with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C
API version 8 in codegen_dir/cversions.txt is
17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you
have to update C_API_VERSION  in numpy/core/setup_common.py.
  MismatchCAPIWarning)
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

atlas_blas_info:
  libraries f77blas,cblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501:
UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
Replacing _lib_names[0]=='blas' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
  FOUND:
libraries = ['openblas']
library_dirs = ['/home/skipper/.local/lib']
language = f77

  FOUND:
libraries = ['openblas']
library_dirs = ['/home/skipper/.local/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77

non-existing path in 'numpy/lib': 'benchmarks'
lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
  libraries lapack_atlas not found in /usr/local/lib64
 

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Daπid]

I am a bit worried about the differences in results. Just to be sure
you are comparing apples with apples, it may be a good idea to set the
seed at the beginning:

np.random.seed( SEED )

where SEED is an int. This way, you will be inverting always the same
matrix, regardless of the Python version. I think, even if the timing
is different, the results should be the same.

http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.seed.html#numpy.random.seed



David.


On 23 March 2013 15:39, Colin J. Williams  wrote:
> On 23/03/2013 7:21 AM, Ralf Gommers wrote:
>
>
>
>
> On Fri, Mar 22, 2013 at 10:39 PM, Colin J. Williams 
> wrote:
>>
>> On 20/03/2013 11:12 AM, Frédéric Bastien wrote:
>>
>> On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams
>>  wrote:
>>
>> On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
>>
>> Hi,
>>
>> win32 do not mean it is a 32 bits windows. sys.platform always return
>> win32 on 32bits and 64 bits windows even for python 64 bits.
>>
>> But that is a good question, is your python 32 or 64 bits?
>>
>> 32 bits.
>>
>> That explain why you have memory problem but not other people with 64
>> bits version. So if you want to work with bigger input, change to a
>> python 64 bits.
>>
>> Fred
>>
>> Thanks to the people who responded to my report that numpy, with Python
>> 3.2 was significantly slower than with Python 2.7.
>>
>> I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and
>> 3.3.0.
>>
>> The Pythons came from python.org and the Numpys from PyPi.  The SciPy site
>> still points to Source Forge, I gathered from the responses that Source
>> Forge is no longer recommended for downloads.
>
>
> That's not the case. The official binaries for NumPy and SciPy are on
> SourceForge. The Windows installers on PyPI are there to make easy_install
> work, but they're likely slower than the SF installers (no SSE2/SSE3
> instructions).
>
> Ralf
>
> Thanks, I'll read over Robert Kern's comments.  PyPi is the simpler process,
> but, if the result is unoptimized code, then easy_install is not the way to
> go.
>
> The code is available here(http://web.ncf.ca/cjw/testFPSpeed.py)
> and the most recent test results are
> here(http://web.ncf.ca/cjw/FP%2023-Mar-13%20Test%20Summary.txt).  These are
> using PyPi, I'll look into SourceForge.
>
> Colin W.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Colin J. Williams]

  
  
On 23/03/2013 7:21 AM, Ralf Gommers
  wrote:


  

  
  On Fri, Mar 22, 2013 at 10:39 PM,
Colin J. Williams 
wrote:

  
On 20/03/2013 11:12 AM, Frédéric Bastien wrote:


  On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams
 wrote:

  
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:


  Hi,

win32 do not mean it is a 32 bits windows. sys.platform always return
win32 on 32bits and 64 bits windows even for python 64 bits.

But that is a good question, is your python 32 or 64 bits?


32 bits.

  
  That explain why you have memory problem but not other people with 64
bits version. So if you want to work with bigger input, change to a
python 64 bits.

Fred



Thanks to the people who responded to my
report that numpy, with Python 3.2 was significantly
slower than with Python 2.7.

I have updated to numpy 1.7.0 for each of the
Pythons 2.7.3, 3.2.3 and 3.3.0.

The Pythons came from python.org
and the Numpys from PyPi.  The SciPy site still
points to Source Forge, I gathered from the
responses that Source Forge is no longer recommended
for downloads.
  

  
  

That's not the case. The official
  binaries for NumPy and SciPy are on SourceForge. The Windows
  installers on PyPI are there to make easy_install work, but
  they're likely slower than the SF installers (no SSE2/SSE3
  instructions).
  
  Ralf

  

Thanks, I'll read over Robert Kern's comments.  PyPi is
the simpler process, but, if the result is unoptimized code,
then easy_install is not the way to go.

The code is available here(http://web.ncf.ca/cjw/testFPSpeed.py)
and the most recent test results are here(http://web.ncf.ca/cjw/FP%2023-Mar-13%20Test%20Summary.txt). 
These are using PyPi, I'll look into SourceForge.

Colin W.
  
  

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Ralf Gommers]

On Sat, Mar 23, 2013 at 12:23 PM, Todd  wrote:

> On Sat, Mar 23, 2013 at 12:21 PM, Ralf Gommers wrote:
>
>>
>> That's not the case. The official binaries for NumPy and SciPy are on
>> SourceForge. The Windows installers on PyPI are there to make easy_install
>> work, but they're likely slower than the SF installers (no SSE2/SSE3
>> instructions).
>>
>> Ralf
>>
>>
> Is there a reason why the same binaries can't be used for both?
>

The SF .exe superpack installers contains three installers: plain, SSE2 and
SSE3 support. easy_install doesn't know what to do with such an installer.
See
http://thread.gmane.org/gmane.comp.python.numeric.general/29395/focus=29582for
the discussion on why things are as they are now.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Todd]

On Sat, Mar 23, 2013 at 12:21 PM, Ralf Gommers wrote:

>
> That's not the case. The official binaries for NumPy and SciPy are on
> SourceForge. The Windows installers on PyPI are there to make easy_install
> work, but they're likely slower than the SF installers (no SSE2/SSE3
> instructions).
>
> Ralf
>
>
Is there a reason why the same binaries can't be used for both?
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Execution time difference between 2.7 and 3.2 using numpy

[Ralf Gommers]

On Fri, Mar 22, 2013 at 10:39 PM, Colin J. Williams
wrote:

>  On 20/03/2013 11:12 AM, Frédéric Bastien wrote:
>
> On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams 
>  wrote:
>
>  On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
>
>  Hi,
>
> win32 do not mean it is a 32 bits windows. sys.platform always return
> win32 on 32bits and 64 bits windows even for python 64 bits.
>
> But that is a good question, is your python 32 or 64 bits?
>
>  32 bits.
>
>  That explain why you have memory problem but not other people with 64
> bits version. So if you want to work with bigger input, change to a
> python 64 bits.
>
> Fred
>
>
>  Thanks to the people who responded to my report that numpy, with Python
> 3.2 was significantly slower than with Python 2.7.
>
> I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and
> 3.3.0.
>
> The Pythons came from python.org and the Numpys from PyPi.  The SciPy
> site still points to Source Forge, I gathered from the responses that
> Source Forge is no longer recommended for downloads.
>

That's not the case. The official binaries for NumPy and SciPy are on
SourceForge. The Windows installers on PyPI are there to make easy_install
work, but they're likely slower than the SF installers (no SSE2/SSE3
instructions).

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion