On 16 July 2012 16:13, Nicolas <nblouve...@gmail.com> wrote: > Hi Ger, > > Le 16/07/2012 16:09, Ger van Diepen a écrit : > > What is the amount of data you have to write at the rates you mention? Is > it all numeric, or also strings? Can the data be compressed? > > Data are collected from several driving simulators connected to the same > map (so we store numeric data about speed, position, engine dynamic etc.), > eye-trackers (numeric) and message sent by several embedded devices (mostly > string data). > > > However, beforehand you have to define the total data size. Furthermore, > the data need to be regularly shaped. > > > It is impossible to know the total size of data as it depends of human > behaviour. > > You say you use meta-data hdf5 containers that in turn point on > measurement data (also stored in hdf5). How do you achieve this ? Is it > possible to point to datasets across files ? > > Many thanks for everyone's answer. > > Cheers, > Nicolas > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > Hdf-forum@hdfgroup.org > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > You're probably best looking at something like Clouderas flume, Twitters storm project or mongo/couchdb for collecting your data. These also then give you a computational framework within which to process the data.
James
_______________________________________________ Hdf-forum is for HDF software users discussion. Hdf-forum@hdfgroup.org http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
