Re: [Pytables-users] Large (to very large) datasets...

2012-10-31 Thread Andrea Gavana
Hi Francesc and All, On 31 October 2012 21:02, Francesc Alted wrote: > On 10/31/12 10:12 AM, Andrea Gavana wrote: >> Hi Francesc & All, >> >> On 31 October 2012 14:13, Francesc Alted wrote: >>> On 10/31/12 4:30 AM, Andrea Gavana wrote: >>>> Th

Re: [Pytables-users] Large (to very large) datasets...

2012-10-31 Thread Andrea Gavana
Hi Francesc & All, On 31 October 2012 14:13, Francesc Alted wrote: > On 10/31/12 4:30 AM, Andrea Gavana wrote: >> Thank you for all your suggestions. I managed to slightly modify the >> script you attached and I am also experimenting with compression. >> However, in the

Re: [Pytables-users] Large (to very large) datasets...

2012-10-31 Thread Andrea Gavana
Hi Anthony & All, On 30 October 2012 23:31, Anthony Scopatz wrote: > On Tue, Oct 30, 2012 at 6:20 PM, Andrea Gavana > wrote: >> >> Hi Anthony, >> >> On 30 October 2012 22:52, Anthony Scopatz wrote: >> > Hi Andrea, >> > >> > Your probl

Re: [Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Andrea Gavana
Hi Anthony, On 30 October 2012 22:52, Anthony Scopatz wrote: > Hi Andrea, > > Your problem is two fold. > > 1. Your timing wasn't reporting the time per data set, but rather the total > time since writing all data sets. You need to put the start time in the > loop to get the time per data set. >

[Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Andrea Gavana
Hi All, I am pretty new to pytables and I am facing a problem of actually storing and retrieving data to/from a large dataset. My situation is the following: 1. I am running stochastic simulations of a number of objects (typically between 100-1,000 simulations); 2. For every simulation, I hav