Re: [Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Anthony Scopatz
On Tue, Oct 30, 2012 at 6:20 PM, Andrea Gavana wrote: > Hi Anthony, > > On 30 October 2012 22:52, Anthony Scopatz wrote: > > Hi Andrea, > > > > Your problem is two fold. > > > > 1. Your timing wasn't reporting the time per data set, but rather the > total > > time since writing all data sets. You

Re: [Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Andrea Gavana
Hi Anthony, On 30 October 2012 22:52, Anthony Scopatz wrote: > Hi Andrea, > > Your problem is two fold. > > 1. Your timing wasn't reporting the time per data set, but rather the total > time since writing all data sets. You need to put the start time in the > loop to get the time per data set. >

Re: [Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Anthony Scopatz
Hi Andrea, Your problem is two fold. 1. Your timing wasn't reporting the time per data set, but rather the total time since writing all data sets. You need to put the start time in the loop to get the time per data set. 2. Your larger problem was that you were writing too many times. Generally

[Pytables-users] Large (to very large) datasets...

2012-10-30 Thread Andrea Gavana
Hi All, I am pretty new to pytables and I am facing a problem of actually storing and retrieving data to/from a large dataset. My situation is the following: 1. I am running stochastic simulations of a number of objects (typically between 100-1,000 simulations); 2. For every simulation, I hav

Re: [Pytables-users] Is it possible to manipulate a VariableNode in a query?

2012-10-30 Thread Francesc Alted
On 10/30/12 10:44 AM, Aquil H. Abdullah wrote: > Hello All, > > I am querying a table that has a field with a string value. I would > like to determine if the string matches a pattern. Is there a simple > way to do that through readWhere and the condition syntax? None of > the following work, b

Re: [Pytables-users] Is it possible to manipulate a VariableNode in a query?

2012-10-30 Thread Aquil H. Abdullah
Bah! Thanks for the quick reply! -- Aquil H. Abdullah "I never think of the future. It comes soon enough" - Albert Einstein On Tuesday, October 30, 2012 at 10:49 AM, Anthony Scopatz wrote: > Hello Aquil, > > Unfortunately, You currently cannot use indexing in queries (ie "symbol[:3] > == x

Re: [Pytables-users] Is it possible to manipulate a VariableNode in a query?

2012-10-30 Thread Anthony Scopatz
Hello Aquil, Unfortunately, You currently cannot use indexing in queries (ie "symbol[:3] == x") and may only use the whole variable ("symbol == x". This is a limitation of numexpr. Please file a ticket with them, if you would like to see this changed. Sorry! Be Well Anthony On Tue, Oct 30, 20

[Pytables-users] Is it possible to manipulate a VariableNode in a query?

2012-10-30 Thread Aquil H. Abdullah
Hello All, I am querying a table that has a field with a string value. I would like to determine if the string matches a pattern. Is there a simple way to do that through readWhere and the condition syntax? None of the following work, but I was wondering if it were possible to do something si