[Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-13 Thread Bruno Santos
Dear Sirs, I'm trying to use Numpy to solve a speed problem with Python, I need to perform agglomerative clustering as a first step to k-means clustering. My problem is that I'm using a very large list in Pyhton and the script is taking more than 9minutes to process all the information, so I'm try

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-13 Thread Charles R Harris
On 4/13/07, Bruno Santos <[EMAIL PROTECTED]> wrote: Dear Sirs, I'm trying to use Numpy to solve a speed problem with Python, I need to perform agglomerative clustering as a first step to k-means clustering. My problem is that I'm using a very large list in Pyhton and the script is taking more th

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-17 Thread Bruno Santos
I try to use the expression as you said, but I'm not getting the desired result, My text file look like this: # num rows=115 num columns=2634 AbassiM.txt 0.033023 0.033023 0.033023 0.165115 0.4623210.00 AgricoleW.txt 0.038691 0.038691 0.038691 0.232147 0.5416760.215300 AliR.txt 0.0418

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-17 Thread Christian K.
Bruno Santos wrote: > I try to use the expression as you said, but I'm not getting the desired > result, > My text file look like this: > > # num rows=115 num columns=2634 > AbassiM.txt 0.033023 0.033023 0.033023 0.165115 0.4623210.00 > AgricoleW.txt 0.038691 0.038691 0.038691 0.232147 0.

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-18 Thread Bruno Santos
Finally I was able to read the data, by using the command you sair with some small changes: matrix = numpy.array([[float(x) for x in line.split()[1:]] for line in vecfile]) But that doesn't solve my speed problem, now instead of taking 40seconds in the slow step, takes 1min ant 10seconds :( The s

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-18 Thread Timothy Hochberg
On 4/18/07, Bruno Santos <[EMAIL PROTECTED]> wrote: Finally I was able to read the data, by using the command you sair with some small changes: matrix = numpy.array([[float(x) for x in line.split()[1:]] for line in vecfile]) But that doesn't solve my speed problem, now instead of taking 40secon

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-18 Thread Christopher Barker
Bruno Santos wrote: > Finally I was able to read the data, by using the command you sair with > some small changes: > matrix = numpy.array([[float(x) for x in line.split()[1:]] for line in > vecfile]) it doesn't sound like you're concerned about the speed of reading the files, but you can still

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-18 Thread Nick Fotopoulos
On 4/18/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > -- > > Message: 5 > Date: Wed, 18 Apr 2007 09:11:32 -0700 > From: Christopher Barker <[EMAIL PROTECTED]> > Subject: Re: [Numpy-discussion] Help using numPy to create a very >

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-18 Thread Gael Varoquaux
On Wed, Apr 18, 2007 at 06:09:38PM -0500, Nick Fotopoulos wrote: > Devs, is there any possibility of moving/copying pylab.load to numpy? +1 Gaƫl ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/num

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-19 Thread Travis Oliphant
Nick Fotopoulos wrote: > Devs, is there any possibility of moving/copying pylab.load to numpy? > I don't see anything in the source that requires the rest of > matplotlib. Among convenience functions, I think that this function > ranks pretty highly in convenience. > I'm supportive of this. B

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-19 Thread Lisandro Dalcin
On 4/19/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: > Nick Fotopoulos wrote: > > Devs, is there any possibility of moving/copying pylab.load to numpy? > > I don't see anything in the source that requires the rest of > > matplotlib. Among convenience functions, I think that this function > > ran

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-19 Thread Christopher Barker
Lisandro Dalcin wrote: > I am also +1 on this, but this functionality should be implemented in > C, I think. well, maybe. > I've just tested numpy.fromfile('name.txt', sep=' ') > against pylab.load('name.txt') for a 35MB text file, the number are: > > numpy.fromfile: 2.66 sec. > pylab.load: 16

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-19 Thread Vincent Nijs
I think it would be a great idea to have pylab.load in numpy. It also seems to be a lot faster than scipy.io. One thing that is very nice about pylab.load is that it can read-in dates. However, it can't, as far a I know, handle other non-float data. I played around with python's csv module and py

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-22 Thread Markus Rosenstihl
Whats wrong with scipy.io.read_array? Am 19.04.2007 um 15:50 schrieb Lisandro Dalcin: On 4/19/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: Nick Fotopoulos wrote: Devs, is there any possibility of moving/copying pylab.load to numpy? I don't see anything in the source that requires the rest o

Re: [Numpy-discussion] Help using numPy to create a very large multi dimensional array

2007-04-22 Thread Vincent Nijs
It seems to be a lot slower than pylab.load for large arrays. Also, it doesn't handle dates. Vincent On 4/22/07 10:33 AM, "Markus Rosenstihl" <[EMAIL PROTECTED]> wrote: > Whats wrong with scipy.io.read_array? > > Am 19.04.2007 um 15:50 schrieb Lisandro Dalcin: > >> On 4/19/07, Travis Oliphant