Re: [Pytables-users] Multithreaded decompress unexpectedly does not help

2012-12-06 Thread Alvaro Tejero Cantero
I'll answer myself on the size-checking: the right attributes are Leaf.size_in_memory and Leaf.size_on_disk (per http://pytables.github.com/usersguide/libref/hierarchy_classes.html) -รก. On 6 December 2012 12:42, Alvaro Tejero Cantero wrote: > Thank you for the comprehensive round-up. I have

Re: [Pytables-users] Problems with flush(): RuntimeError: dictionary changed size during iteration

2012-12-06 Thread Alan Marchiori
Josh, Thanks for the detailed response. I would like to avoid going through a separate process if at all possible due to the performance penalty. I have also tried your last suggestion to create a dedicated pytables thread and send everything through that but still see the same problem (Runtime

Re: [Pytables-users] Multithreaded decompress unexpectedly does not help

2012-12-06 Thread Alvaro Tejero Cantero
Thank you for the comprehensive round-up. I have some ideas and reports below. What about ctables? The documentation says that it is specificly column-access optimized, which is what I need in this scenario (sometimes sequential, sometimes random). Unfortunately I could not get the rootdir parame

Re: [Pytables-users] Multithreaded decompress unexpectedly does not help

2012-12-06 Thread Francesc Alted
On 12/5/12 7:55 PM, Alvaro Tejero Cantero wrote: > My system was benched for reads and writes with Blosc[1]: > > with pt.openFile(paths.braw(block), 'r') as handle: > pt.setBloscMaxThreads(1) > %timeit a = handle.root.raw.c042[:] > pt.setBloscMaxThreads(6) > %timeit a = handle.root.raw.c0