OK, thanks. I think in a previous version this was possible. Or am I
getting something wrong? I have many files with tables of 1002 rows,
with one column that holds a 32 x 3488 float32 matrix, for example:
/bird1/IJ_5000/erp/lfp (Table(1002L,)) ''
description := {
"signal": Float32Col(shape=(32, 3488), dflt=0.0, pos=0)}
byteorder := 'little'
chunkshape := (1L,)
I admit that it is a weird thing to do, but I didn't know a better
solution.
Here is my application:
I store long 32-channel recordings (say 1.5 hours at 14000 Hz) of brain
activity in a PyTables file. In this recording there are relatively
short episodes (events, n=1002) where I want to study brain activity,
across the 32 channels. Ideally I would 'cut out' these 1002 events from
the recording (say 32 x 3488 samples per event) and store the result in
a big 1002 x 32 x 3488 array for further analyses. This is no problem of
course, but for my analyses I very often I need to make selections of
events that are not sequential. Say, event numbers [3, 8, 23, 57, 576,
578]. But usually the sets are much bigger. I cannot access those with
normal slicing in Array types. So as an alternative I stored the 32 x
3488 events in a 1002-row Table. Tables have a readCoordinates method
that does exactly what I want. If there is a better solution I would be
glad to learn about it.
Many thanks, cheers,
Gabriel
On Fri, 2009-02-20 at 17:47 +0100, Francesc Alted wrote:
> This seems a limitation in HDF5 itself, I'm afraid. Using HDF5 1.6.5
> gives a little better explanation about this:
>
> #005: ../../../src/H5O.c line 2204 in H5O_new_mesg(): object header
> message is too large (16k max)
>
> In fact, I was able to create a compound type with up to (16k - 3)
> components. However, you are trying with one with 122k components
(!).
> I think that defining such a large data types is really
> counter-productive, mainly because of performance reasons, so you
> should avoid them.
>
> What are you trying to do? Perhaps using an EArray is your best bet
for
> this case.
>
> Cheers,
>
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Pytables-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pytables-users
