> I agree that there is some improvement in going from one QP per > UDP socket to one per node, but it still will likely not > scale to 10,000 node clusters, which is something that Oracle > probably does not care about, but others in HPC do. >
To put the improvement in perspective: For Mpi running on a 10,000 node cluster with 2 or 4 way nodes, here are the QP/ CM connection requirements: (assuming intra node communication doesn't use IB) Procs per node uDapl/Sdp Rds 2 19996 9999 4 39984 9999 Clearly, there is tradeoff in performance as we go from uDapl/Sdp to Rds. The choice will have to depend on the requirements of performance Vs Scalability. Btw, for this large a cluster, there is a huge overhead in just setting up the connections. Rds connections are setup only once. > If we are going to invent a Reliable Datagram Service, shouldn't > it be made to scale so that MPIs that currently use datagrams > could also benefit ? > > woody > > > > > _______________________________________________ > openib-general mailing list > [email protected] > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
