Hi Petr, For appending data to an existing dataset in HDF5 you need a chunked (or extensible) dataset instead of a simple (or contiguous) one. There is plenty of info about HDF5 chunked datasets out there, but the basics are here:
https://www.hdfgroup.org/HDF5/doc/H5.user/Chunking.html Now, you simply need to create a chunked dataset and append data to it, so the next tutorial: https://www.hdfgroup.org/HDF5/Tutor/extend.html should basically be all you need. Also, and to make things a bit simpler, the H5TB is a 'high-level' API that uses the regular HDF5 API to create primitives so as to make the calls easier (although somewhat less flexible). Even if there is not a .NET API for H5TB, what you can do is to have a look at the sources and port what you need and include your own 'simple enough' H5TB version to your project. For example, PyTables is using this route, and you might just grab the H5TBOmake_table, H5TBOread_records and H5TBOappend_records from: https://github.com/PyTables/PyTables/blob/develop/src/H5TB-opt.c convert them to .NET calls, and you are ready to go and use them in your project. Francesc 2015-08-10 21:09 GMT+02:00 Petr KLAPKA <[email protected]>: > Hello all, > > Now that I'm able to write types of pretty much unlimited complexity into > the HDF5 files, the next step is writing additional elements of data to > existing datasets. > > I'll go diving through the manuals again, but if anyone has a "quick start > guide" link for how to create a dataset and append to (stream to) it > throughout the life of the application, I'd be most grateful. > > My next goal is to add elements to the "myMajorCycles" dataset that is > pictured below with just a single element. I need to accomplish this in a > way where the dataset is kept open and can accept new data as it comes in, > but it must not remain completely in memory because the final dataset on > disk will be ~2GB. > > D:\Programs\HDF5Utils>h5dump c:\temp\myfirstfile.h5 > HDF5 "c:\temp\myfirstfile.h5" { > GROUP "/" { > DATASET "myMajorCycles" { > DATATYPE H5T_COMPOUND { > H5T_STD_U32LE "UniqueID"; > H5T_STD_U32LE "MsgCount"; > H5T_ARRAY { [5] H5T_COMPOUND { > H5T_STD_U32LE "MsgID"; > H5T_STD_U64LE "MsgHWTimestamp"; > H5T_STD_U32LE "MsgLengthBytes"; > H5T_ARRAY { [8] H5T_STD_U8LE } "MsgBody"; > } } "Messages"; > } > DATASPACE SIMPLE { ( 1 ) / ( 1 ) } > DATA { > (0): { > 23, > 5, > [ { > 10, > 18446744073709551615, > 8, > [ 0, 1, 2, 3, 4, 5, 6, 7 ] > }, > { > 11, > 18446744073709551615, > 8, > [ 10, 11, 12, 13, 14, 15, 16, 17 ] > }, > { > 12, > 18446744073709551615, > 8, > [ 20, 21, 22, 23, 24, 25, 26, 27 ] > }, > { > 13, > 18446744073709551615, > 8, > [ 30, 31, 32, 33, 34, 35, 36, 37 ] > }, > { > 14, > 18446744073709551615, > 8, > [ 40, 41, 42, 43, 44, 45, 46, 47 ] > } ] > } > } > } > } > } > > Many thanks! > > > Best regards, > > Petr Klapka > System Tools Engineer > *Valeo* Radar Systems > 46 River Rd > Hudson, NH 03051 > Mobile: (603) 921-4440 > Office: (603) 578-8045 > *"Festina lente."* > > > *This e-mail message is intended only for the use of the intended > recipient(s). > The information contained therein may be confidential or privileged, > and its disclosure or reproduction is strictly prohibited. > If you are not the intended recipient, please return it immediately to its > sender > at the above address and destroy it. * > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > -- Francesc Alted
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
