[Numpy-discussion] about SIMD (SSE2 SSE3)

2010-11-06 Thread qihua wu
I used the following command to install the numpy to enable the SSE3 numpy-1.5.1rc1-win32-superpack-python3.1.exe /arch sse3 Then how can I know whether numpy is running with SSE or not? I have a program to process the data from sql server using java to process 600M rows, it takes 7 hours to

[Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Damien Moore
Hi List, I'm trying to import csv data as a numpy array using genfromtxt. The csv file contains mixed data, some floating point, others string codes and dates that I want to convert to floating point. The strange thing is that when I use the 'converters' argument to convert a subset of the

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Damien Moore
In reply to my own question, the trivial, but massively inefficient solution is: s=StringIO.StringIO('q1,2\nq3,4') a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) a1 = numpy.array(a.tolist()) But what I really want to do is have genfromtxt do the conversion for me.

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Lluís
Damien Moore writes: Hi List, I'm trying to import csv data as a numpy array using genfromtxt. The csv file contains mixed data, some floating point, others string codes and dates that I want to convert to floating point. The strange thing is that when I use the ' converters' argument to

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread Lluís
Sorry, I got it wrong and ignored the StringIO part. Lluis Lluís writes: Damien Moore writes: [...] import numpy, StringIO s=StringIO.StringIO('q1,2\nq3,4') a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) s=StringIO.StringIO('q1,2\nq3,4')

Re: [Numpy-discussion] numpy.genfromtxt converters issue

2010-11-06 Thread josef . pktd
On Sat, Nov 6, 2010 at 11:52 AM, Damien Moore damienlmo...@gmail.com wrote: In reply to my own question, the trivial, but massively inefficient solution is: s=StringIO.StringIO('q1,2\nq3,4') a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s[1:])}) a1 = numpy.array(a.tolist())

[Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread K. Sun
Hello, I wrote the following code with numpy to implement the Floyd-Wallshall algorithm to compute the pair-wise shortest path in a undirected weighted graph. It is really slow when N ~ 10k, while the same implementation in matlab is much faster. I am sorry I don't want to run it again to

Re: [Numpy-discussion] Optimize Floyd-Wallshall algorithm with Numpy

2010-11-06 Thread John Salvatier
The difference is that dis[k,:] eliminates the first dimension since you are using a single number as an index, but dis[k:k+1,:] does not eliminate that dimension. On Sat, Nov 6, 2010 at 1:24 PM, josef.p...@gmail.com wrote: On Sat, Nov 6, 2010 at 4:14 PM, K. Sun sunk...@gmail.com wrote: Thanks

Re: [Numpy-discussion] about SIMD (SSE2 SSE3)

2010-11-06 Thread qihua wu
Thank David, the java program takes 3 hours to read data, after read the data into memory, it takes 4 hours to process/calculate somthing on all these data. The data is the sale data which contains both promoted sale and non-promoted sale, the program needs to predict the non-promoted sale: so

Re: [Numpy-discussion] about SIMD (SSE2 SSE3)

2010-11-06 Thread David Warde-Farley
On 2010-11-06, at 7:46 PM, qihua wu wrote: day 1,2,3 have the non-promoted sales, day 4 have the promoted sales, day 5,6,7 have the non-promted sales, the output for day 1~7 are all non-promoted sales. During the process, we might need to sum all the data for day 1~7, is this what you

Re: [Numpy-discussion] about SIMD (SSE2 SSE3)

2010-11-06 Thread David Cournapeau
On Sat, Nov 6, 2010 at 9:51 PM, qihua wu staywith...@gmail.com wrote: I used the following command to install the numpy to enable the SSE3 numpy-1.5.1rc1-win32-superpack-python3.1.exe /arch sse3 The whole point of the super pack installer is to install the most optimized one possible on your