As a clarification to hopefully avoid some knee jerk reactions: If I use uint32_t instead of uint64_t, everything works. I'll have to try double.
In the aforementioned unix file packet logs, in a seperate h5 file, I commit an H5::CompType type computed in the same runtime so I can safely parse the binary file later while keeping most advantages of HDF5. So when I read the data in with python and convert it to HDF, I am using this named type as the src (from binary) and dst (to HDF) types. I hope this increases confidence that I'm not making a trivial mistake on the CompType construction. I do intend to look at the offsets of the fields at some point today to verify in the context where I have the problem, it is correct. -Jason On Fri, Oct 16, 2015 at 11:47 AM, Jason Newton <[email protected]> wrote: > Hi, > > I have structs like the following: > > struct record{ > uint32_t count; > uint32_t event; > uint64_t timestamp_nsec; > uint64_t time_of_receipt_nsec; > uint32_t some_other_value; > uint8_t is_chunked; > }; > > > I use the c++ api with 1.8.15 to make an H5::CompType containing those > fields, using the H5::PredType::Native's. The H5::CompType is constructed > as H5::CompType(sizeof(record)). Now when I write this to an append table > in online non-trivial programs (2 completely different programs so far), > things will be fairly deterministically corrupted. __attribute__((packed)) > will make a difference how this corruption shows up, I've noted count and > is_chunked getting the same values before without it. With it, there tends > to only be corruption after the uint64_t's. Some fields are corrupted but > it's not easy to know what's up - other fields have the values of fields > somewhere else in the structure's memory which strongly implies there's > some incorrect offset in the struct used. I use HOFFSET to get the field > offsets in the above for H5::CompType::inesrtMember. I believe the > H5::CompType is correctly describing the structure's but the append process > is not being performed correctly - I use the same types of structs in > programs that write unix file packet logs and parse it with these memtypes > (in python) and it's fine I convert them to hdf5 datasets with h5py.. > > I have this problem with HDF 1.8.13 and 1.8.15, with gcc 4.8.2 and clang > 4.5, on 64bit (the 64bit may be very relevant here, I'm suspecting > somewhere the uint64_t's are treated as uint32_t's or INT's or something). > > Any thoughts? I've tried producing it in some boiled down programs and > have yet failed to get a minimum case. > > -Jason >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
