Lenoid Arsh wrote: Lenoid> Shirley, Lenoid> some additional information you may be interested:
Lenoid> According to our experience with the Voltaire IPoIB driver, Lenoid> splitting CQ harmed the throughput (we checked with the iperf Lenoid> application, UDP mode.) Splitting the the CQ caused more interrupts, Lenoid> context switches and CQ polls. Interesting results. I think some of Shirley's work reduced the number of interrupts on ehca so this is starting to sound like a one size does not fit all driver approach. I wonder what Pathscale see if they split the completion queues? Lenoid> Note, the case is rather different from OpenIB mthca, since Voltare Lenoid> IPoIB is based on the VAPI driver, Lenoid> where CQ completions are handled in a tasklet context, Lenoid> unlike mthca where CQ completions are handled in the HW interrupt Lenoid> context. Another question is what do we do about adapter specific code where each adapter type ( ehca, mthca, Voltare and Pathscale ) can all provide better performance if adapter specific code and tuning is required? Lenoid> NAPI gave us some improvement. I think NAPI should improve much more Lenoid> in mthca, with the HW interrupt CQ completions. However, I don't believe that NAPI can provide the same benefit for all the driver models listed above. It may help in overall interrupt handling, but there is probably a need for additional adapter/driver specific tuning. Some of these may end up requiring support in the OpenIB stack. There are many cases not covered by using Netperf, and netpipe that show improved performance. These cases are running multiple sockets per link/adapter, and the case where you have a larger machine where you have multiple adapters. I haven't seen any data recently on duplex traffic, only STREAM ( or unidirectional) either. Bernie King-Smith IBM Corporation Server Group Cluster System Performance [EMAIL PROTECTED] (845)433-8483 Tie. 293-8483 or wombat2 on NOTES "We are not responsible for the world we are born into, only for the world we leave when we die. So we have to accept what has gone before us and work to change the only thing we can, -- The Future." William Shatner _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general