Hi, Rob Latham <[email protected]> writes:
> If you see a lot of read in your write-only workloads, it suggests that a > "data > sieving" optimization is kicking in. If there are only partial updates to a > block of data, then something will read the whole block, update the changed > bits, and write the new block out. I'm vauge about where the optimization > happens because raid devices, file systems, and the ROMIO MPI-IO > implementation > could all be doing this. > > With Collective I/O, you can transform your workload into something more > contiguous and less likely to trigger data sieving, but it can still happen. > > You can pass MPI-IO hints to hdf5 to turn off collective I/O *and* turn of > data > sieving -- this is the way Lustre folks got good performance in the 2008-ish > time frame. it could either help you a lot or hurt you a lot. I cannot tell > you more over email. thanks. I'm not sure if these could be tuned a bit better, but with the following hints the problem is all gone in the two problematic clusters (for a given file size, one of the writing modes of the program was taking about ~200x more time. With these hints all is back to normal, and the problematic mode takes just the same time as the other ones). call MPI_Info_create(info, error) call MPI_Info_set(info,"IBM_largeblock_io","true", error) call MPI_Info_set(info,"stripping_unit","4194304", error) CALL MPI_INFO_SET(info,"H5F_ACS_CORE_WRITE_TRACKING_PAGE_SIZE_DEF","524288",error) CALL MPI_INFO_SET(info,"ind_rd_buffer_size","41943040", error) CALL MPI_INFO_SET(info,"ind_wr_buffer_size","5242880", error) CALL MPI_INFO_SET(info,"romio_ds_read","disable", error) CALL MPI_INFO_SET(info,"romio_ds_write","disable", error) CALL MPI_INFO_SET(info,"romio_cb_write","enable", error) CALL MPI_INFO_SET(info,"cb_buffer_size","4194304", error) For the moment, problem solved. Thanks a lot, -- Ángel de Vicente http://www.iac.es/galeria/angelv/ --------------------------------------------------------------------------------------------- ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protecci�n de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
