Hi Barbara and Mark, thanks for getting back to me.
As we tried to implement direct MPI I/O we discovered that HDF5 does not directly allocate created datasets so we could not get the offsets to do the direct writing. Doing auto plist = H5Pcreate(H5P_DATASET_CREATE); H5Pset_layout(plist, H5D_CONTIGUOUS); H5Pset_alloc_time(plist, H5D_ALLOC_TIME_EARLY); before calling H5Dcreate(); and passing plist does allocate the data - then the offsets works. Turns out that closing the serially written file, reopening it in parallel and writing independently with H5DOpen() and H5Dwrite() then works fine. So I guess the modification of the dataset does then not count as a modification and it does not have to be collective. The files close cleanly and the output is correct. Hope it helps someone who runs into the same issue! It would also be very helpful to document what exactly counts as a modification the collective/independent call page. Cheers! Lion On 20/03/2017 18:13, Barbara Jones wrote: > > Hi Lion, > > > > I checked with the developers… > > H5DOwrite_chunk() does not work with parallel HDF5. > > > > -Barbara > > [email protected] > > > > *From:*Hdf-forum [mailto:[email protected]] *On > Behalf Of *Lion Krischer > *Sent:* Tuesday, March 14, 2017 3:43 PM > *To:* [email protected] > *Subject:* Re: [Hdf-forum] Collective H5Dopen > > > > Thanks a lot for your input! > > We'll try to directly use MPI I/O to write to the file after getting > the offsets from the initially created file. Thanks for the > suggestion! As we use none of the advanced features a straight mmap or > its equivalent with MPI I/O should work. > > Our code guarantees that each data set is only written to from a > single rank so that should work and metadata synchronization issues do > not really exist. It would be great if something like this could be > supported directly via the hdf5 library. Maybe a H5DOwrite_directly() > that does some very basic checks and then writes straight to the > uncompressed, unchunked, .. dataset without even touching the metadata > so it should work fine with parallel I/O? > > Does H5DOwrite_chunk() work for parallel code? I cannot really tell > from the documentation and it also appears to solve the way more > complicated problem of efficiently writing chunked data. > > We'd still be happy to hear additional suggestions :) > > > Cheers! > > Lion > > > > Hmm. Interesting question. I would think *any* operation that changes > data (either > > metadata or raw data) is a *modification*. But, I think I kinda see > where you are > > going here...If you've created a (non-extendible) dataset with no > checksum or compression > > filters etc., all you wanna do is change the raw data but not perterb > any of the HDF5 file's > > metadata. I don't think the HDF5 library would treat that as a > non-modification though. I think a write > > operation (even on the raw data) can wind up changing how the library > caches dataset and file > > metadata in memory thereby creating a situation where two different > tasks have a different idea > > of the file's cached metadata. When it comes time to close the file, > which tasks' view of the metadata > > is correct? > > > > Its concievable you could manually do it by obtaining the dataset's > offset in the file, calling > > any necessary H5Tconvert method on your buffer just prior to writing > in and then writing the > > buffer yourself to the file via pwrite or something. That would > essentially skirt HDF5 though > > and probably be too complex to be worth it. > > > > It might be worth looking at, H5DOwrite_chunk() though, > https://support.hdfgroup.org/HDF5/doc/HL/RM_HDF5Optimized.html > > to ask how that works and if it achieves or gets close to behavior you > want. > > > > Mark > > > > > > > > > > _______________________________________________ > > Hdf-forum is for HDF software users discussion. > > [email protected] <mailto:[email protected]> > > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > > Twitter: https://twitter.com/hdf5 > > > > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] <mailto:[email protected]> > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
