On http://www.mcs.anl.gov/petsc/documentation/faq.html#computers
the website states The software http://open-mx.org provides faster speed for ethernet systems, we have not tried it but it claims it can dramatically reduce latency and increase bandwidth on Linux system. You must first install this software and then install MPICH or Open MPI to use it. However, it looks like open-mx.org has expired; it's just an ad selling the domain. Patrick