>-----Original Message----- >From: Gavin Hamill [mailto:[email protected]] >Sent: Monday, August 09, 2010 4:27 PM >To: Wyborny, Carolyn >Cc: [email protected] >Subject: RE: [E1000-devel] 5% dropped packets under high load >with e1000e > >On Mon, 2010-08-09 at 15:36 -0700, Wyborny, Carolyn wrote: > >> Hello, >> >> How do you have flow control configured for these ports? >Has this been >> a problem since you've set the system up or is this new? >If new, what >> has changed recently in your environment? > >Hullo. :) > >Flow control is not specifically configured; I use Cisco Cat >3750s so am >running with whatever they end up negotiating with the NICs: > >[39309953.090015] eth1: Link is Up 1000 Mbps Full Duplex, Flow Control: >None >[39309953.114324] eth0: Link is Up 1000 Mbps Full Duplex, Flow Control: >None >[39309953.358634] eth3: Link is Up 1000 Mbps Full Duplex, Flow Control: >None >[39309953.762084] eth2: Link is Up 1000 Mbps Full Duplex, Flow Control: >None > >Can you advise what the Cisco IOS config options (if any..) would be to >set flow control? 'show flowcontrol' tells me: > >Port Send FlowControl Receive FlowControl RxPause TxPause > admin oper admin oper >--------- -------- -------- -------- -------- ------- ------- >... >Gi1/0/17 Unsupp. Unsupp. off off 0 0 >Gi1/0/18 Unsupp. Unsupp. off off 0 0 >Gi1/0/19 Unsupp. Unsupp. off off 0 0 >Gi1/0/20 Unsupp. Unsupp. off off 0 0 > >Meanwhile, I took the opportunity to reload the e1000e driver and >specify InterruptThrottleRate=1,1,1,1 in the hopes that the increased >ceiling of 70k interrupts/sec would minimise the dropped packets: > >[39309887.935861] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k2 >[39309887.935861] e1000e: Copyright (c) 1999-2008 Intel Corporation. >[39309887.935861] ACPI: PCI Interrupt 0000:15:00.0[A] -> GSI 16 (level, >low) -> IRQ 16 >[39309887.935861] PCI: Setting latency timer of device 0000:15:00.0 to >64 >[39309887.963861] 0000:15:00.0: Interrupt Throttling Rate >(ints/sec) set >to dynamic mode >[39309888.115867] eth0: (PCI Express:2.5GB/s:Width x4) >00:23:7d:fb:df:1a >[39309888.115867] eth0: Intel(R) PRO/1000 Network Connection >[39309888.115942] eth0: MAC: 0, PHY: 4, PBA No: d51930-005 > >(... plus near-identical for eth1, 2, 3) > >We are now into the quiet time for our site, but even having allowed up >to 70k interrupts/sec with mode 1, the dropped-packet count is still >rising quickly: > >cor4:~# ifconfig eth3 >eth3 Link encap:Ethernet HWaddr 00:24:81:7b:04:ac > UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 > RX packets:17935406 errors:0 dropped:103128 >overruns:0 frame:0 > >And yet the overall system interrupts/sec is still only hovering around >20k: > >cor4:~# vmstat 1 >procs -----------memory---------- ---swap-- -----io---- -system-- >----cpu---- > r b swpd free buff cache si so bi bo in cs us sy id wa > 0 0 60 1291692 164312 346380 0 0 0 0 0 0 1 4 95 0 > 0 0 60 1291808 164312 346380 0 0 0 0 19582 856 0 7 92 0 > 0 0 60 1291312 164312 346380 0 0 0 0 19071 1028 0 7 93 0 > 0 0 60 1292752 164312 346380 0 0 0 0 19308 1034 1 8 91 0 > 0 0 60 1292048 164312 346380 0 0 0 0 20653 1085 0 9 91 0 > >If this is the case, would I be likely to see any benefit from setting >InterruptThrottleRate=0,0,0,0 ? > >=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > >Hang on... I just had a look in e1000e/param.c in the Debianised source >for 2.6.26 .. and found this.... > > case 1: > ndev_info(netdev, > "%s set to dynamic mode\n", > opt.name); > adapter->itr_setting = adapter->itr; > adapter->itr = 20000; > break; > case 3: > ndev_info(netdev, > "%s set to dynamic conservative >mode\n", > opt.name); > adapter->itr_setting = adapter->itr; > adapter->itr = 20000; > break; > >Huh? Both modes 1 + 3 set ITR to 20000? I'm no kernel programmer but >shouldn't it be 70000 for mode 1? > >gdh > > > > > Well, I think flow control isn't the problem. 8-) It can explain dropped packets though. Good question on the ITR setting. That source certainly doesn't look exactly right. I will find out for sure and let you know. I will also do some more checking around on possible causes for your symptoms.
Thanks, Carolyn ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
