In case you want more feedback on compression filters, this study might
be interesting for you:
http://gael-varoquaux.info/blog/?p=159
Francesc
On 3/28/12 12:33 PM, Alvaro Tejero Cantero wrote:
> Hi,
>
> Trying to evaluate compression filters, I was looking for a call in
> PyTables to get the s
On Wed, Mar 28, 2012 at 1:05 PM, Francesc Alted wrote:
> On 3/28/12 12:33 PM, Alvaro Tejero Cantero wrote:
> > Hi,
> >
> > Trying to evaluate compression filters, I was looking for a call in
> > PyTables to get the size of a dataset (in bytes). As I didn't find it
> > I remembered the many benchm
On 3/28/12 12:33 PM, Alvaro Tejero Cantero wrote:
> Hi,
>
> Trying to evaluate compression filters, I was looking for a call in
> PyTables to get the size of a dataset (in bytes). As I didn't find it
> I remembered the many benchmarks and found instead [1] that the way to
> do it is to create singl
On 3/28/12 10:15 AM, Alvaro Tejero Cantero wrote:
> That is a perfectly fine solution for me, as long as the arrays aren't
> copied in memory for the query.
No, the arrays are not copied in memory. They are just read from disk
block-by-block and then the output is directed to the iterator, or an
Hi,
Trying to evaluate compression filters, I was looking for a call in
PyTables to get the size of a dataset (in bytes). As I didn't find it
I remembered the many benchmarks and found instead [1] that the way to
do it is to create single-dataset files and interrogate the
filesystem. Curiously eno
That is a perfectly fine solution for me, as long as the arrays aren't
copied in memory for the query.
Thank you!
Thinking that your proposed solution uses iterables to avoid it I tried
boolcond = pt.Expr('(exp(a)<0.9)&(a*b>0.7)|(b*sin(a)<0.1)')
indices = [i for i,v in boolcond if v]
(...) TypeE
On 3/27/12 6:34 PM, Francesc Alted wrote:
> Another option that occurred to me recently is to save all your
> columns as unidimensional arrays (Array object, or, if you want
> compression, a CArray or EArray), and then use them as components of a
> boolean expression using the class `tables.Expr
