Hello Leonid,
Leonid Arsh <[EMAIL PROTECTED]> wrote on 04/23/2006 06:38:00 AM:
> Shirley,
>
> some additional information you may be interested:
>
> According to our experience with the Voltaire IPoIB driver,
> splitting CQ harmed the throughput (we checked with the iperf
> application, UDP mode.) Splitting the the CQ caused more interrupts,
> context switches and CQ polls.
> Note, the case is rather different from OpenIB mthca, since Voltare
> IPoIB is based on the VAPI driver,
> where CQ completions are handled in a tasklet context,
> unlike mthca where CQ completions are handled in the HW interrupt
> context.
That expected because only one tasklet is allowed running across all cpus in the same time.
Have you tried to use other SOFTIRQ instead of TASKLET_SOFTIRQ?
My expectation is the performance will be better since there would be multiple
softirqs running simultaneously. If it's a simple change of your code, could you please try it?
I am thinking to split mthca CQ completion into HW interrupt and softirq context.
> NAPI gave us some improvement. I think NAPI should improve much more
> in mthca, with the HW interrupt CQ completions.
Yes, with the hardware interrupts are disabled.
It would be interesting to compare the completion CQ with NAPI and in softirq context.
It all depends on how you implement NAPI. If you only implement NAPI without
changing the sender, NAPI might not get better performance than softirq.
The benefit of NAPI, it has one dev->poll running across all cpus to prevent
packets out of order totally.
Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
_______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general