Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-04 Thread Gökhan Sever
On Thu, Sep 3, 2009 at 12:22 AM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 23:59, Gökhan Sever wrote: > > > Robert, > > > > You must have thrown a couple RTFM's while replying my emails :) > > Not really. There's no manual for this. Greg Wilson's _Data Crunching_ > may be a good general introdu

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 23:59, Gökhan Sever wrote: > Robert, > > You must have thrown a couple RTFM's while replying my emails :) Not really. There's no manual for this. Greg Wilson's _Data Crunching_ may be a good general introduction to how to think about these problems. http://www.pragprog.com

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 1:58 PM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 13:28, Gökhan Sever wrote: > > Put the reference manual in: > > > > http://drop.io/1plh5rt > > > > First few pages describe the data format they use. > > Ah. The fields are *not* delimited by a fixed value. Regexes are no

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 13:28, Gökhan Sever wrote: > Put the reference manual in: > > http://drop.io/1plh5rt > > First few pages describe the data format they use. Ah. The fields are *not* delimited by a fixed value. Regexes are no help to you for pulling out the information you need, except perhap

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 12:46 PM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 12:33, Gökhan Sever wrote: > > How your find suggestion work? It just returns the location of the first > > occurrence. > > http://docs.python.org/library/stdtypes.html#str.find > > str.find(sub[, start[, end]]) >Ret

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 12:29 PM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 12:27, Gökhan Sever wrote: > > > > On Wed, Sep 2, 2009 at 12:01 PM, Citi, Luca wrote: > >> > >> If I understand the problem... > >> if you are 100% sure that "', '" only occurs between fields > >> and never within, you

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 12:29 PM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 12:27, Gökhan Sever wrote: > > > > On Wed, Sep 2, 2009 at 12:01 PM, Citi, Luca wrote: > >> > >> If I understand the problem... > >> if you are 100% sure that "', '" only occurs between fields > >> and never within, you

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 12:33, Gökhan Sever wrote: > How your find suggestion work? It just returns the location of the first > occurrence. http://docs.python.org/library/stdtypes.html#str.find str.find(sub[, start[, end]]) Return the lowest index in the string where substring sub is found, su

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 12:27, Gökhan Sever wrote: > > On Wed, Sep 2, 2009 at 12:01 PM, Citi, Luca wrote: >> >> If I understand the problem... >> if you are 100% sure that "', '" only occurs between fields >> and never within, you can use the 'split' method of the string >> which could be faster th

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 12:04 PM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 11:53, Gökhan Sever wrote: > > > How to use recarrays with variable-length data fields as well as > metadata? > > You don't. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 12:01 PM, Citi, Luca wrote: > If I understand the problem... > if you are 100% sure that "', '" only occurs between fields > and never within, you can use the 'split' method of the string > which could be faster than regexp in this simple case. > ___

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Citi, Luca
If I understand the problem... if you are 100% sure that "', '" only occurs between fields and never within, you can use the 'split' method of the string which could be faster than regexp in this simple case. ___ NumPy-Discussion mailing list NumPy-Discus

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 11:53, Gökhan Sever wrote: > How to use recarrays with variable-length data fields as well as metadata? You don't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 10:34 AM, Sturla Molden wrote: > Gökhan Sever skrev: > > What would be wisest and fastest way to tackle this issue? > Get the format, read the binary data directly, skip the ascii/regex part. > > I sometimes use recarrays with formatted binary data; just constructing > a dt

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
On Wed, Sep 2, 2009 at 10:11 AM, Robert Kern wrote: > On Wed, Sep 2, 2009 at 09:38, Gökhan Sever wrote: > > Hello, > > > > I want to be able to parse a binary file which hold information regarding > to > > experiment configuration and data obviously. Both configuration and data > > sections are v

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Sturla Molden
Gökhan Sever skrev: > What would be wisest and fastest way to tackle this issue? Get the format, read the binary data directly, skip the ascii/regex part. I sometimes use recarrays with formatted binary data; just constructing a dtype and use numpy.fromfile to read. That works when the binary fi

Re: [Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Robert Kern
On Wed, Sep 2, 2009 at 09:38, Gökhan Sever wrote: > Hello, > > I want to be able to parse a binary file which hold information regarding to > experiment configuration and data obviously. Both configuration and data > sections are variable-length. A chuck this data is shown as below (after a > binar

[Numpy-discussion] Fastest way to parsing a specific binay file

2009-09-02 Thread Gökhan Sever
Hello, I want to be able to parse a binary file which hold information regarding to experiment configuration and data obviously. Both configuration and data sections are variable-length. A chuck this data is shown as below (after a binary read operation) '\x00\...@\x00$\x00\x02\x00\x12\x00\xff\x0