Thanks Francesc, we're getting there :).
Some more precise questions below.
> Here it is how you can do that in PyTables:
>
> my_condition = '(col1>0.5) && (col2<24) && (col3 == "novel")'
> mycol4_values = [ r['col4'] for r in tbl.where(my_condtion) ]
ok, but having data upon which I want to ope
Hi,
I have created five tables in a Hdf5 file.I have created index during the
creation of the file.I have about 140 million records in my postgresql
database.I am trying to divide it into 20 hdf5 chunks.The problem is that i
have one master table which has relationships with other tables.After I
i
Hola Alvaro,
On Mar 15, 2012, at 4:58 PM, Alvaro Tejero Cantero wrote:
> Hi Anthony and Francesc,
>
> please bear with me for one more.
>
> I was thinking of having this huge array in memory and be able to
> write nice indexing expressions, the kind that one writes all the time
> with numpy; e.
Hi Anthony and Francesc,
please bear with me for one more.
I was thinking of having this huge array in memory and be able to
write nice indexing expressions, the kind that one writes all the time
with numpy; e.g.
arr[ fast && novel && checked, 1:4]
where fast, novel and checked are boolean arra
Hi Angel
On Thu, Mar 15, 2012 at 3:29 PM, Angel Medrano wrote:
> Hi,
> I come from RDMS background and am experimenting with Pytables + HDF5 to
> store a massive amount of records we keep. Im trying to understand how to
> make the most out of pytables but am having some trouble understanding som
Hi,
I come from RDMS background and am experimenting with Pytables + HDF5 to
store a massive amount of records we keep. Im trying to understand how to
make the most out of pytables but am having some trouble understanding some
concepts.
Currently my data structure looks like this
Group_1
nest
On Mar 15, 2012, at 1:43 PM, Anthony Scopatz wrote:
> Hello Alvaro
>
> On Thu, Mar 15, 2012 at 1:20 PM, Alvaro Tejero Cantero
> wrote:
> Hi!
>
> Thanks for the prompt answer. Actually I am not clear about switching
> from NxM array to N columns (64 in my case). How do I make a
> rectangular se
On Thu, Mar 15, 2012 at 1:44 PM, Francesc Alted wrote:
> On Mar 15, 2012, at 1:30 PM, Alvaro Tejero Cantero wrote:
>
> > I had seen this, but I don't see how to point to regions inside a
> > leaf. It seems that h5py supports this. Quoting from their
> > documentation
> >
> > <>
> > Region referen
On Mar 15, 2012, at 1:30 PM, Alvaro Tejero Cantero wrote:
> I had seen this, but I don't see how to point to regions inside a
> leaf. It seems that h5py supports this. Quoting from their
> documentation
>
> <>
> Region references always contain a selection. You create them using
> the dataset pro
Hello Alvaro
On Thu, Mar 15, 2012 at 1:20 PM, Alvaro Tejero Cantero wrote:
> Hi!
>
> Thanks for the prompt answer. Actually I am not clear about switching
> from NxM array to N columns (64 in my case). How do I make a
> rectangular selection with columns? With an NxM array I just have to
> do arr
Ok, thanks for the hint!
Maybe and advantage of keeping these strings in HDF5 is that if the
dataset they refer to is deleted they also go to oblivion. Otherwise I
have to keep things synchronized. Or decorate deletion of datasets. Or
use the undo mechanism. Just trying to wrap my head around the
I had seen this, but I don't see how to point to regions inside a
leaf. It seems that h5py supports this. Quoting from their
documentation
<>
Region references always contain a selection. You create them using
the dataset property “regionref” and standard NumPy slicing syntax:
>>> myds = myfile.c
Hi!
Thanks for the prompt answer. Actually I am not clear about switching
from NxM array to N columns (64 in my case). How do I make a
rectangular selection with columns? With an NxM array I just have to
do arr[1:2,1:4] to select columns 1,2,3 and time samples 1
to 2.. While it is
Cool idea, but why not just have a log file on the side that the decorator
writes to? HDF5 only allocates a certain amount of space for attributes /
attribute names. (You can check the spec but I think it is something like
64 k.) So if you are writing an excessive number of attributes you may run
Links are covered here.
soft: http://pytables.github.com/usersguide/libref.html?#the-softlink-class
hard:
http://pytables.github.com/usersguide/libref.html?#the-externallink-class
On Thu, Mar 15, 2012 at 7:56 AM, Alvaro Tejero Cantero wrote:
> Does PyTables support object region references[1]?
>
Hello Alvaro,
Thanks for your excitement!
On Thu, Mar 15, 2012 at 7:52 AM, Alvaro Tejero Cantero wrote:
> Hi everybody!
>
> I plan to start using PyTables for an application at the University of
> Oxford where data is collected in sessions of 2Gb Int16 data organized
> as 64 parallel time series
Hi,
Here's my last question for today (I sent them separately because they
are quite unrelated).
I am thinking of writing a python decorator that for any processing
function (e.g. band-pass filter of median of data[0:3,:]) logs to the
attributes of the target HDF5 column
* the name of the functi
Does PyTables support object region references[1]?
When using soft links to other files, is a performance penalty
incurred? I like the idea of having the raw data, that never changes,
referenced from another file that is read-only. How do you guys
normally deal with this scenario?
Álvaro.
[1] I
Hi everybody!
I plan to start using PyTables for an application at the University of
Oxford where data is collected in sessions of 2Gb Int16 data organized
as 64 parallel time series (64 detectors), each holding 15 million
points (15M).
I could handle this sessions separately, but ideally I would
19 matches
Mail list logo
