Thanks Elena. The application I had in mind was generating an alternate
schema. A data acquisition system may produce a more complicated schema
with a number of compound types, while users may need a simpler schema
for their analysis. I was wondering if one could generate a virtual view
conforming to the simpler schema that gets the data from the more
complicated schema. Being able to slice compounds would help, but I
think it is not that important. The main thing is to be able to generate
a view to the 'big data' which is usually large arrays of float or
int's. At least for our DAQ, the data in the compounds tend to be small
and I think it would be cheap to make a copy of what users need for
their schemas.
best,
David
On 08/14/15 14:36, Elena Pourmal wrote:
Hi David,
On Aug 13, 2015, at 12:30 AM, Schneider, David A. <[email protected]>
wrote:
I have a question about the virtual view feature that the hdf5 group is
developing. My understanding is that I can build a view dataset out of elements
from different datasets of the same type. I'm wondering if I can also build a
view dataset that has a different but compatible type. For instance, if I had
datasetA
type = Compound { float x, int a}
could I make a view dataset that is just of float, that pulls out the float x
in the compound from in datasetA?
HDF5 virtual dataset uses only spacial information for setting a view (or
defining mappings), i.e., there is no way to define the self-describing mapping
as above. Also, the source datasets have to have compatible datatypes (one type
can be converted to another by the HDF5 library). There is no restriction that
they have the same datatype.
In HDF5, atomic type and a field of a compound type of the same atomic type are
not compatible. This is a current limitation and it can be probably addressed
in the future.
Said this, it is possible to pull data out as you described ( we still need to
test it!), but it is clunky.
If all source datasets in the mapping use compatible compound datatypes, the
library should be able to read or write virtual dataset data by a field. It
will be similar to reading a field from a compound dataset.
If the source datasets have non-compatible datatypes, then one can do I/O only
on a selection in the virtual dataset that maps to the elements of compatible
datatypes. It will be application’s responsibility to make sure that all
elements in the selection have compatible types, otherwise I/O will fail - not
fun ;-)
Elena
best,
David
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
