Rob, Matthew, It turns out as you correctly guess that it wasn't actually hanging, just reading painfully slowly. The test they ran took 2 hours to read, but it did complete.
Changing the read to use the mpio fapl was a 5 line change to the code and adding MPICH_SKIP_MPICXX got rid of the 'incompatibilities'. It is rather sad if that was all that was stopping the use of parallel io in the code. The code did not run any faster unfortunately, so I also asked them to add a transfer property list using collective IO, but this has also not made a significant difference I'm told. I guess I'll have to actually look at the code now and see if I can help with some profiling. Thanks for the feedback JB > -----Original Message----- > From: Hdf-forum [mailto:[email protected]] On Behalf > Of Matthew Turk > Sent: 04 April 2014 23:06 > To: HDF Users Discussion List > Subject: Re: [Hdf-forum] Hang on large file read > > On Fri, Apr 4, 2014 at 4:34 PM, Rob Latham <[email protected]> wrote: > > > > > > On 04/04/2014 10:08 AM, Matthew Turk wrote: > > > >>> What kind of incompatibilities? Are these bugs in the MPI-IO > >>> implementation? > >> > >> > >> No, they're just incompatibilities with it in Enzo. Here's an > >> example thread from the mailing list. > >> > >> https://groups.google.com/d/msg/enzo- > users/2as4N0iOS5Y/LftkV0oXuHEJ > > > > > > well that *particular* build error is because of a mix-mash of namespaces. > > no one needs mpicxx.h anyway, so you can add "-DMPICH_SKIP_MPICXX > > -DOMPI_SKIP_MPICXX" to your MACH_DEFINES or MACH_CPPFLAGS. > > > > Was that seriously the only reason enzo hasn't looked at using > > parallel hdf5? > > To be perfectly candid, the reason Enzo hasn't looked at using parallel HDF5 is > probably because the person who designed the IO system (who has, very > sadly, now passed on) saw no measurable benefit to it, and structured the IO > in such a way that we did not use it. > > There is only one time that multiple processors read from the same file, > which is during the initialization of a simulation that utilizes > ParallelRootGridIO. Once that has completed, the grids are decomposed > across processors, and each processor writes to a single file. As it stands, > traversing the leaf nodes in large simulations is already extremely costly, and > unless we address that I do not think moving to a monolithic file per snapshot > is going to provide us with much benefit, as it would exacerbate that > problem. (I would be keen to be proven wrong on this, however.) > > All of that aside, I apologize for jumping in and re-directing the conversation > to Enzo rather than to the issue that John brought up initially. The enzo-dev > mailing list ( https://groups.google.com/forum/#!forum/enzo-dev ) would > be a good place to continue further discussions. > > -Matt > > > > > ==rob > > > > > >> > >> -Matt > >> > >>> > >>> > >>> ==rob > >>> > >>> -- > >>> Rob Latham > >>> Mathematics and Computer Science Division Argonne National Lab, IL > >>> USA > >>> > >>> _______________________________________________ > >>> Hdf-forum is for HDF software users discussion. > >>> [email protected] > >>> > >>> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfg > >>> roup.org > >> > >> > >> _______________________________________________ > >> Hdf-forum is for HDF software users discussion. > >> [email protected] > >> > >> http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgr > >> oup.org > >> > > > > -- > > Rob Latham > > Mathematics and Computer Science Division Argonne National Lab, IL USA > > > > _______________________________________________ > > Hdf-forum is for HDF software users discussion. > > [email protected] > > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgro > > up.org > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
