[Pytables-users] Numpy Arrays to Structure Array or Table

2013-08-07 Thread David Reed
Hi there, I have some generic functions that take time series data with 2 numpy array arguments, time and value, and return 2 numpy arrays of time and value. I would like to place these arrays into a Numpy structured array or directly into a new pytables table with fields, time and value. Now Iv

Re: [Pytables-users] suitable for storing data like k-v style?

2013-08-07 Thread Anthony Scopatz
Hi Jason, A key-value store pattern is definitely supported. However, be forewarned that groups are implemented using B-trees, not hash tables. However, with data of your size most of the access time will be in the leaf nodes and not getting the group. I'd say try it out and see. Be Well Anthon

Re: [Pytables-users] should I use pytables?

2013-08-07 Thread Chao YUE
Thanks Anthony, I think I will give a try, apprently at some stage I would like to flush the data into disk :p cheers, Chao On Wed, Aug 7, 2013 at 6:44 PM, Anthony Scopatz wrote: > On Wed, Aug 7, 2013 at 5:44 AM, Chao YUE wrote: > >> Dear all, >> >> I have a hierachical nested python dictiona

Re: [Pytables-users] suitable for storing data like k-v style?

2013-08-07 Thread Xianli Xu
oops sorry, seem auto-correction of my email client created some typo for me : P here's the corrections, On 8 Aug, 2013, at 2:33 AM, Xianli Xu wrote: > Hi all, > > I'm developing data processing service and evaluating if Pytable. Since hdf5 > supports hierarchical data like a tree of folder

[Pytables-users] suitable for storing data like k-v style?

2013-08-07 Thread Xianli Xu
Hi all, I'm developing data processing service and evaluating if Pytable. Since hdf5 supports hierarchical data like a tree of folder, can I use such a tree-like structure as a K-V store like possibly store million of tables or arrays under one group and randomly access any one of them in O(1)

Re: [Pytables-users] searching for group names

2013-08-07 Thread Anthony Scopatz
On Wed, Aug 7, 2013 at 4:39 AM, Gabriel J.L. Beckers < pytables-u...@gbeckers.nl> wrote: > Hi, > > I don't know if this is related in any way to Gergo's problem, but I > have slow responses when querying which children a group contains, if > that group contains big leafs. I am using pytables 2.5 a

Re: [Pytables-users] should I use pytables?

2013-08-07 Thread Anthony Scopatz
On Wed, Aug 7, 2013 at 5:44 AM, Chao YUE wrote: > Dear all, > > I have a hierachical nested python dictionaries with the end of the branch > as either pandas dataframe, or np.ndarray or list or plain scalars. > > let's say the different levels of keys are: > > 1st level: ['top1', 'top2', 'top3']

[Pytables-users] should I use pytables?

2013-08-07 Thread Chao YUE
Dear all, I have a hierachical nested python dictionaries with the end of the branch as either pandas dataframe, or np.ndarray or list or plain scalars. let's say the different levels of keys are: 1st level: ['top1', 'top2', 'top3'] 2nd level: ['mid1','mid2','mid3'] 3rd level: ['bot1','bot2','bo

Re: [Pytables-users] searching for group names

2013-08-07 Thread Gabriel J.L. Beckers
Hi, I don't know if this is related in any way to Gergo's problem, but I have slow responses when querying which children a group contains, if that group contains big leafs. I am using pytables 2.5 and hdf5 1.8.9 on linux 64 bit. Specifically, I found that using the _g_get_objinfo method (w