Dear Gromacs users: I have access to a new cluster that has GigE interconnect (selected vs. IB for reasons other than cost). As expected, systems that scale nicely to two nodes with IB end up running faster on 1 node than they do in 2 nodes when using GigE. SysAdmins are wondering if software RoCE (Software RDMA over Converged Ethernet) will help. Anybody have any experience with this?
here is what the sysadmin said: " For large message sizes (>64k), SoftRoCE can provide performance comparable to hardware RoCE. Latency improvements are more modest, ~50% better than straight ethernet but still about 3x higher than hardware RoCE. Some references: http://www.lanl.gov/projects/national-security-education-center/information-science-technology/_assets/docs/2010-si-docs/Team_CYAN_Implementation_and_Comparison_of_RDMA_Over_Ethernet_Presentation.pdf http://www.iosrjournals.org/iosr-jce/papers/Vol15-issue4/N01548187.pdf?id=7557 " I found this: http://quick.hcs.ufl.edu/pubs/UF_HPIDC.pdf but that is suggesting that there is a speedup when going to multiple nodes even for GigE and that is not what I see. Thank you, Chris. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to [email protected].
