On Sep 5, 2012, at 1:50 PM, Reeted wrote: > On 08/29/12 21:35, Atchley, Scott wrote: >> Hi all, >> >> I am benchmarking a sockets based application and I want a sanity check on >> IPoIB performance expectations when using connected mode (65520 MTU)..... > > I have read that with newer cards the datagram (unconnected) mode is > faster at IPoIB than connected mode. Do you want to check?
I have read that the latency is lower (better) but the bandwidth is lower. Using datagram mode limits the MTU to 2044 and the throughput to ~3 Gb/s on these machines/cards. Connected mode at the same MTU performs roughly the same. The win in connected mode comes with larger MTUs. With a 9000 MTU, I see ~6 Gb/s. Pushing the MTU to 655120 (the maximum for ipoib), I can get ~13 Gb/s. > What benchmark program are you using? netperf with process binding (-T). I tune sysctl per the DOE FasterData specs: http://fasterdata.es.net/host-tuning/linux/ Scott-- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
