"Hdf-forum on behalf of Aaron Friesz" wrote: I take it this is not something that hdf5 supports within the API?
On Mon, Dec 12, 2016 at 10:18 AM, Aaron Friesz <[email protected]<mailto:[email protected]>> wrote: Hello, I have a very specific usage question. I would like to output a part of an array. Other libraries may call this a slice. By way of an example, say I have a 3 dimensional array with a size of 5 in each direction spread over several separate processes. I would like to write to file all the data with y coordinate 3. I currently use parallel file I/O by hyperslab, following the example here: https://support.hdfgroup.org/HDF5/Tutor/phypecont.html It seems to me that there might be a very easy way to do this using the memory and file datasets which doesn't require my code to check each process for participation in the output, etc, but I have failed to find examples or documentation that outline the constraints. Any help is appreciated. I am pretty sure you should be able to do what you describe (e.g. write to the file all the data with y coordinate 3 from several separate processors). Once you have *created* the target dataset (and its dataspace) in the file (which *requiires* all processors that opened the file to participate in the creation -- there is no way around that), then each processor that *has*data* to write can define its own memory and file dataspaces to write data and then each can, using independent rather than collective I/O, write the data. The trick is in defining the size/shape of the memory and file dataspaces. But, I am pretty confident it can be done. Alas, I don't have any examples to send. But, you could play with your code writing bogus data (e.g. 1's from proc 1, 2's from proc 2, etc.) and then examine the results with h5dump or h5ls to confirm you have code that is touching the correct parts of the array. Mark -- Mark C. Miller, LLNL Happiness is a shear act of will, not simply a by product of happenstance. - MCM
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
