Hi Walter, hi All,
Thank you for sharing your work.
I've analyzed your codes briefly and it seems you manually create a
dataset with compound type and then you put your values in such a
dataset - is that correct?
But that means for my use case I need to collect all columns (their
types) first and then create a compound dataset.
When the number of such columns is really huge this operation can be
time and resource consuming. But that's OK if there is no other solution...
If I well understand your codes, you are collecting your columns (which
are separate classes in your case) just in vectors and then you
calculating columns offsets basing on std::vector::data - is that correct?
Any other suggestions from HDF Forum Team which could help to solve my
use case?
Thank you.
Regards,
Rafal
W dniu 2017-08-23 o 16:25, Walter Landry pisze:
Rafal Lichwala <[email protected]> wrote:
Let's say my final HDF table will look like this:
['a', 1, 3.14]
['b', 2, 2.11]
['c', 3, 1.89]
So we simply have a HDF table with 3 columns of types: char, int,
float
and 3 rows with some values.
Creation of that table must be divided into some "steps".
After 1st "step" I should have a table:
['a']
After 2nd step:
['a', 1]
After 3rd step:
['a', 1, 3.14]
After 4th step:
['a', 1, 3.14]
['b', x, x]
where x after 4th step is undefined and can be some default values
which will be overwritten in the next steps.
How to achieve that use case?
I have to do something similar for my program tablator
https://github.com/Caltech-IPAC/tablator
I read in tables in other formats and write out as HDF5. So I do not
know the types of the rows at compile time. It is all in C++. The
details of writing HDF5 are in
src/Table/write_hdf5/
Is it possible to create a table by calling H5TBmake_table(), but
having no fields and no records at the beginning and then just call
H5TBinsert_field() in the next steps?
I do not think that is going to work, because you need to know the
sizes of rows when you create the table.
Is it possible to have "data" attribute of H5TBinsert_field() function
a NULL value when we insert a new field to a table dataset with no
records yet?
What about 4th step - can I create just a first column value for a new
record in a table?
I do not know of a way to do that. I would end up creating a whole
new table with the new field. You can then populate the empty fields
with appropriate default values.
I know it's maybe a strange use case, but the problem is that I could
have really huge structure model (a lot of columns and a lot of
records) which should be stored in the HDF table dataset, so I need to
avoid "collecting" required information (number of fields, their
types, values) by initial iterating over whole structure.
The second problem is that I have a vector of objects which need to be
stored as HDF table (where table row is the given object and columns
are its fields), but all examples I've seen just work on C struct.
That sounds similar to the internal data structure I use in tablator.
Hope that helps,
Walter Landry
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
