Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
On Fri, Sep 11, 2009 at 5:25 AM, David Cournapeau wrote: > V. Armando Solé wrote: >> David Cournapeau wrote: >> >>> V. Armando Solé wrote: >>> >>> Hello, It seems to point towards a packaging problem. In python 2.5, I can do: import numpy.core._dotblas as dotblas dotblas.__file__ and I get: C:\\Python25\\lib\\site-packages\\numpy\\core\\_dotblas.pyd >>> That's where the error lies: if you install with nosse, you should not >>> get _dotblas.pyd at all. >>> >> Why? The nosse for python 2.5 has _dotblas.pyd >> > > Yes, and it should not - because the _dotblas.pyd uses SSE2 > instructions. The python 2.6 installer is the correct one, python 2.5 > is not. >> Is it impossible to get it compiled under python 2.6 without using sse2 >> or sse3? >> > > It is possible to compile anything you want if you are willing to go > through the hassle of compiling ATLAS on windows. The binary installer > only uses ATLAS (and hence build _dotblas) for SSE2 and SSE3. The low > availability of machines with only SSE does not worth the hassle to do > it anymore (but again, that's only an issue of the official binaries, > you can still compile your own atlas). > It's also possible to use the older ATLAS binaries at http://www.scipy.org/Installing_SciPy/Windows#head-cd37d819e333227e327079e4c2a2298daf625624 with MingW, which does not require building ATLAS yourself. I'm using the sse2 from there and don't have any problems. David, If you have updated ATLAS binaries (e.g. also for sse3), is it possible to add them to the webpage? Thanks, Josef > cheers, > > David > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
V. Armando Solé wrote: > David Cournapeau wrote: > >> V. Armando Solé wrote: >> >> >>> Hello, >>> >>> It seems to point towards a packaging problem. >>> >>> In python 2.5, I can do: >>> >>> import numpy.core._dotblas as dotblas >>> dotblas.__file__ >>> >>> and I get: >>> >>> C:\\Python25\\lib\\site-packages\\numpy\\core\\_dotblas.pyd >>> >>> >>> >> That's where the error lies: if you install with nosse, you should not >> get _dotblas.pyd at all. >> > Why? The nosse for python 2.5 has _dotblas.pyd > Yes, and it should not - because the _dotblas.pyd uses SSE2 instructions. The python 2.6 installer is the correct one, python 2.5 is not. > Is it impossible to get it compiled under python 2.6 without using sse2 > or sse3? > It is possible to compile anything you want if you are willing to go through the hassle of compiling ATLAS on windows. The binary installer only uses ATLAS (and hence build _dotblas) for SSE2 and SSE3. The low availability of machines with only SSE does not worth the hassle to do it anymore (but again, that's only an issue of the official binaries, you can still compile your own atlas). cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
David Cournapeau wrote: > V. Armando Solé wrote: > >> Hello, >> >> It seems to point towards a packaging problem. >> >> In python 2.5, I can do: >> >> import numpy.core._dotblas as dotblas >> dotblas.__file__ >> >> and I get: >> >> C:\\Python25\\lib\\site-packages\\numpy\\core\\_dotblas.pyd >> >> > > That's where the error lies: if you install with nosse, you should not > get _dotblas.pyd at all. Why? The nosse for python 2.5 has _dotblas.pyd Is it impossible to get it compiled under python 2.6 without using sse2 or sse3? If so it should be somewhere written in the release notes. > I will look into the packaging problem - could you open an issue on > numpy trac, so that I don't forget about it ? > OK. I'll try to do it. Best regards, Armando ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
Sturla Molden wrote: > V. Armando Solé skrev: > >> In python 2.6: >> >> >>>import numpy.core._dotblas as dotblas >> ... >> ImportError: No module named _dotblas >> >> > > >>> import numpy.core._dotblas as dotblas > >>> dotblas.__file__ > 'C:\\Python26\\lib\\site-packages\\numpy\\core\\_dotblas.pyd' > That's because you have installed either the sse2 or the sse3 versions. As I said in my post, the problem affects the nosse version. _dotblas.pyd is missing in the nosse version and that is a problem unless one forgets about supporting Pentium III and Socket A processors when developing code. Armando ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
V. Armando Solé skrev: > In python 2.6: > > >>>import numpy.core._dotblas as dotblas > ... > ImportError: No module named _dotblas > >>> import numpy.core._dotblas as dotblas >>> dotblas.__file__ 'C:\\Python26\\lib\\site-packages\\numpy\\core\\_dotblas.pyd' ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
V. Armando Solé wrote: > Hello, > > It seems to point towards a packaging problem. > > In python 2.5, I can do: > > import numpy.core._dotblas as dotblas > dotblas.__file__ > > and I get: > > C:\\Python25\\lib\\site-packages\\numpy\\core\\_dotblas.pyd > That's where the error lies: if you install with nosse, you should not get _dotblas.pyd at all. The 15 second is the 'normal' speed if you don't use ATLAS. I will look into the packaging problem - could you open an issue on numpy trac, so that I don't forget about it ? cheers, David > In python 2.6: > > >>>import numpy.core._dotblas as dotblas > ... > ImportError: No module named _dotblas > > and, of course, I cannot find the _dotblas.pyd file in the relevant > directories. > > Best regards, > > Armando > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
V. Armando Solé skrev: > import numpy > import time > a=numpy.arange(100.) > a.shape=1000,1000 > t0=time.time() > b=numpy.dot(a.T,a) > print "Elapsed time = ",time.time()-t0 > > reports an "Elapsed time" of 1.4 seconds under python 2.5 and 15 seconds > under python 2.6 > My computer reports 0.34 seconds (Pytin 2.6.2, Win32). S.M. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
Hello, It seems to point towards a packaging problem. In python 2.5, I can do: import numpy.core._dotblas as dotblas dotblas.__file__ and I get: C:\\Python25\\lib\\site-packages\\numpy\\core\\_dotblas.pyd In python 2.6: >>>import numpy.core._dotblas as dotblas ... ImportError: No module named _dotblas and, of course, I cannot find the _dotblas.pyd file in the relevant directories. Best regards, Armando ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
David Cournapeau wrote: > V. Armando Solé wrote: > >> Hello, >> >> I have found performance problems under windows when using python 2.6 >> In my case, they seem to be related to the dot product. >> >> The following simple script: >> >> import numpy >> import time >> a=numpy.arange(100.) >> a.shape=1000,1000 >> t0=time.time() >> b=numpy.dot(a.T,a) >> print "Elapsed time = ",time.time()-t0 >> >> reports an "Elapsed time" of 1.4 seconds under python 2.5 and 15 seconds >> under python 2.6 >> >> Same version of numpy, same machine, official numpy installers for >> windows (both with nosse flag) >> >> > > Could you confirm this by pasting the output of numpy.show_config() in > both versions The output of: python -c "import numpy; import sys; print sys.executable;numpy.show_config()" > python26.txt and python -c "import numpy; import sys; print sys.executable; numpy.show_config()" > python25.txt are identical except for the first line: diff python25.txt python26.txt 1c1 < C:\Python25\python.exe --- > C:\Python26\python.exe I paste the python26.txt because the other one is the same except for the first line: C:\Python26\python.exe blas_info: libraries = ['blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 lapack_info: libraries = ['lapack'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 atlas_threads_info: NOT AVAILABLE blas_opt_info: libraries = ['blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_blas_threads_info: NOT AVAILABLE lapack_opt_info: libraries = ['lapack', 'blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 define_macros = [('NO_ATLAS_INFO', 1)] atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE Any hint? Armando ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dot product performance on python 2.6 (windows)
V. Armando Solé wrote: > Hello, > > I have found performance problems under windows when using python 2.6 > In my case, they seem to be related to the dot product. > > The following simple script: > > import numpy > import time > a=numpy.arange(100.) > a.shape=1000,1000 > t0=time.time() > b=numpy.dot(a.T,a) > print "Elapsed time = ",time.time()-t0 > > reports an "Elapsed time" of 1.4 seconds under python 2.5 and 15 seconds > under python 2.6 > > Same version of numpy, same machine, official numpy installers for > windows (both with nosse flag) > Could you confirm this by pasting the output of numpy.show_config() in both versions ? cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Dot product performance on python 2.6 (windows)
Hello, I have found performance problems under windows when using python 2.6 In my case, they seem to be related to the dot product. The following simple script: import numpy import time a=numpy.arange(100.) a.shape=1000,1000 t0=time.time() b=numpy.dot(a.T,a) print "Elapsed time = ",time.time()-t0 reports an "Elapsed time" of 1.4 seconds under python 2.5 and 15 seconds under python 2.6 Same version of numpy, same machine, official numpy installers for windows (both with nosse flag) Are some libraries missing in the windows superpack for python 2.6? Perhaps the reported problem is already known, but I did not find any information about it. Best regards, Armando ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion