On Thu, 2006-02-02 at 17:45, Andi Kleen wrote: > There was already talk some time ago to make NAPI drivers use > the hardware mitigation again. The reason is when you have > a workload that runs below overload and doesn't quite > fill the queues and is a bit bursty, then NAPI tends to turn > on/off the NIC interrupts quite often.
In SGI's experience, all it takes to get into this state is an even workload and a sufficiently fast CPU. On Thu, 2006-02-02 at 17:49, David S. Miller wrote: > From: Andi Kleen <[EMAIL PROTECTED]> > Date: Thu, 2 Feb 2006 07:45:26 +0100 > > > Don't think it was ever implemented though. In the end we just > > eat the slowdown in that particular load. > > The tg3 driver uses the chip interrupt mitigation to help > deal with the SGI NUMA issues resulting from NAPI. The tg3 driver uses small hardcoded values for the RXCOL_TICKS and RXMAX_FRAMES registers, and allows "ethtool -C" to change them. SGI's solution is do is ship a script that uses ethtool at boot to tune rx-usecs, rx-frames, rx-usecs-irq, rx-frames-irq up from the defaults. This helps a lot, and we're very grateful ;-) But a scheme which used the interrupt mitigation hardware dynamically based on load could reduce the irq rate and CPU usage even further without compromising latency at low load. On Thu, 2006-02-02 at 17:51, Andi Kleen wrote: > On Thursday 02 February 2006 04:19, Greg Banks wrote: > > On Thu, 2006-02-02 at 14:13, David S. Miller wrote: > > > From: Greg Banks <[EMAIL PROTECTED]> > > > Multiple trips down through TCP, qdisc, and the driver for each > > NFS packet sent: > > Normally TSO was supposed to fix that. Sure, except that the last time SGI looked at TSO it was extremely flaky. I gather that's much better now, but TSO still has a very small size limit imposed by the stack (not the hardware). > I was playing with a design some time ago to let TCP batch > the lower level transactions even without that. The idea > was instead of calling down into IP and dev_queue_xmit et.al. > for each packet generated by TCP first generate a list of packets > in sendmsg/sendpage and then just hand down the list > through all layers into the driver. Cool. Wouldn't it mean rewriting the nontrivial qdiscs? Greg. -- Greg Banks, R&D Software Engineer, SGI Australian Software Group. I don't speak for SGI. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
