Hi Tamas,
So my question is: how would an experienced HDF5 user structure this simple
data to maximise the performance of the event-by-event readout?
I see two solutions for your purposes.
First - try to switch from Python to C++ - it's much faster.
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=python3&lang2=gpp
Second - I know this is HDF5 forum, but for such a huge but simple set
of data, I would suggest to use some SQL engine as a backend.
MySQL or PostgreSQL would be a good choice if you need a full set of
relational database engine features for your data analysis, but
file-based solutions (SQLite) could be also taken into consideration.
In your case data would be stored into two tables (hits and events) with
a proper key-based join between them.
Regards,
Rafal
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
