I pushed PetscCommBuildTwoSided() which is a scalable interface for setting up two-sided communication given one-sided information.
https://bitbucket.org/petsc/petsc-dev/commits/721cf8e8a07d6de2abc9b4f621f09a70e0d48012#Lsrc/sys/utils/mpits.cT257 The default interface uses the O(size) MPI_Reduce, but if you have an MPI-3 implementation (providing MPI_Ibarrier), it uses the nonblocking consensus algorithm from http://www.unixer.de/publications/img/hoefler-dsde-protocols.pdf You can revert to the old implementation using -build_twosided ALLREDUCE (versus IBARRIER). I'd like to eventually update most uses of PetscGatherNumberOfMessages() and PetscGatherMessageLengths() to use this interface, and to use it in new code whenever possible. In the implementation, I wrote a utility similar to PetscFreeSpace, but with what I think is a safer and more generic interface. It's currently called SegArray, but I'd be happy to change the name (maybe StreamBuffer). It's essentially used to buffer a chunked stream (with low-latency incremental allocation) and later return it as a contiguous array. I'd appreciate comments on the API before moving it to include/petsc-private. https://bitbucket.org/petsc/petsc-dev/commits/721cf8e8a07d6de2abc9b4f621f09a70e0d48012#Lsrc/sys/utils/mpits.cT42 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130106/001d6e99/attachment.html>