Thank you Jason! I actually just found a solution after digging really hard into the documentation. I've posted my answer here http://stackoverflow.com/questions/33085684/hide-certain-fields-of-a-compound-datatype-from-being-written-to-or-read-back-f
The key is to specify a properly packed datatype in file that is different from the datatype in memory. Bests, Jiaxin 2015-10-14 2:42 GMT+01:00 Jason Newton <[email protected]>: > You can't hide it. > > Your options are copy it to a new packed struct layout - maybe performing > the conversions in chunks of a suitable number. This would probably be the > fastest and probably pretty friendly on memory & high performance if you > choose the right constants. You can use the type conversion routines to go > from the originanl memory layout of the structs to the packed ones, or > simply reduce it yourself - which will of course be higher performance / > lower overhead if you preallocate (rather than append) and avoid all the > complicated conversion code. > > Your second option is structure of arrays which decouples where each field > goes, which of course allows you to specify exactly what you want. You > could then either post-process or preprocess them back in as structures > wherever is most appropriate. > > HTH, > -Jason > > On Mon, Oct 12, 2015 at 11:55 AM, Jiaxin Han <[email protected]> wrote: > >> Hello, >> >> I was expecting that by not inserting a field into memtype, I could avoid >> outputing that field to disk. However, the attached test file shows that >> this is not the case. Even if I do not insert the field, everything is >> written and everything is read back as well. >> >> I guess it has to do with the size of memtype. I tried pack() to reduce >> the size but the data interpretation then went wrong. >> >> Defining a new struct containing only these wanted fields is not optimal, >> since it would require copying the data to the new struct or back, while my >> application involves huge amounts of data. What I try to hide is actually a >> vector field, which I write out separately as an array of variable-length >> arrays. Currently although I have omitted the vector field in memtype, it >> is still written and then read back as well, which corrupts the memory (the >> reading automatically fills the size and memory pointer of the vectors with >> their written values, which are not valid pointers anymore). >> >> So is there a way that I can truly hide a particular field from being >> written as well as from being read back, without having to define a new >> temporary class? >> >> Thank you! >> >> Jiaxin >> >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 >> > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
