Jesse Brandeburg wrote: > okay, are you getting NETDEV WATCHDOG messages?
I have seen only one, half a day after booting the new kernel: [41242.800042] ------------[ cut here ]------------ [41242.800061] WARNING: at /build/buildd-linux-2.6_2.6.37-1-i386-vxjyZA/linux-2.6-2.6.37/debian/build/source_i386_none/net/sched/sch_generic.c:258 dev_watchdog+0xec/0x17d() [41242.800067] Hardware name: PowerEdge 1850 [41242.800070] NETDEV WATCHDOG: eth2 (e1000): transmit queue 0 timed out [41242.800073] Modules linked in: binfmt_misc fuse ipmi_si ipmi_devintf ipmi_msghandler ide_generic ide_gd_mod ide_cd_mod ide_core radeon ttm drm_kms_helper drm i2c_algo_bit e752x_edac i2c_core power_supply edac_core video output tpm_tis tpm tpm_bios psmouse dcdbas evdev shpchp pcspkr processor serio_raw button thermal_sys pci_hotplug rng_core ext3 jbd mbcache sd_mod crc_t10dif uhci_hcd mptspi mptscsih mptbase scsi_transport_spi sg sr_mod cdrom ehci_hcd ata_generic usbcore ata_piix libata scsi_mod e1000 floppy nls_base [last unloaded: scsi_wait_scan] [41242.800135] Pid: 0, comm: kworker/0:1 Not tainted 2.6.37-1-amd64 #1 [41242.800139] Call Trace: [41242.800142] <IRQ> [<ffffffff81046ed4>] ? warn_slowpath_common+0x78/0x8c [41242.800158] [<ffffffff81046f87>] ? warn_slowpath_fmt+0x45/0x4a [41242.800166] [<ffffffff81015e62>] ? p4_pmu_enable_event+0x121/0x132 [41242.800171] [<ffffffff8127e1a0>] ? netif_tx_lock+0x3d/0x65 [41242.800175] [<ffffffff8127e2b4>] ? dev_watchdog+0xec/0x17d [41242.800181] [<ffffffff81015ea6>] ? p4_pmu_enable_all+0x33/0x46 [41242.800188] [<ffffffff81052ff8>] ? run_timer_softirq+0x1cc/0x298 [41242.800193] [<ffffffff8127e1c8>] ? dev_watchdog+0x0/0x17d [41242.800201] [<ffffffff81067971>] ? ktime_get+0x5f/0xb8 [41242.800208] [<ffffffff8104c9b9>] ? __do_softirq+0xcf/0x1b6 [41242.800215] [<ffffffff8100a91c>] ? call_softirq+0x1c/0x30 [41242.800220] [<ffffffff8100bef7>] ? do_softirq+0x3f/0x79 [41242.800226] [<ffffffff8104c851>] ? irq_exit+0x36/0x79 [41242.800234] [<ffffffff8102120e>] ? smp_apic_timer_interrupt+0x87/0x94 [41242.800240] [<ffffffff8100a3d3>] ? apic_timer_interrupt+0x13/0x20 [41242.800243] <EOI> [<ffffffff81010c71>] ? mwait_idle+0x81/0x8c [41242.800251] [<ffffffff81010c1e>] ? mwait_idle+0x2e/0x8c [41242.800256] [<ffffffff81008be9>] ? cpu_idle+0xb2/0x124 [41242.800265] [<ffffffff81319f93>] ? _raw_spin_unlock_irqrestore+0xb/0x11 [41242.800271] [<ffffffff8131334e>] ? start_secondary+0x1e5/0x1eb [41242.800276] ---[ end trace 4b5c047ab36314c4 ]--- > what does ethtool -S eth2 say, are there lots of tx_timeout there? What is "lots"? :-) I see 50 right now, after running for 3 days: # ethtool -S eth2 NIC statistics: rx_packets: 59592562 tx_packets: 91875040 rx_bytes: 12104735304 tx_bytes: 112256070528 rx_broadcast: 396143 tx_broadcast: 82 rx_multicast: 310754 tx_multicast: 315292 rx_errors: 0 tx_errors: 0 tx_dropped: 0 multicast: 310754 collisions: 0 rx_length_errors: 0 rx_over_errors: 0 rx_crc_errors: 0 rx_frame_errors: 0 rx_no_buffer_count: 0 rx_missed_errors: 1409 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 50 tx_restart_queue: 7688 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 12513512 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_long_byte_count: 12104735304 rx_csum_offload_good: 57600914 rx_csum_offload_errors: 373 alloc_rx_buff_failed: 0 tx_smbus: 0 rx_smbus: 0 dropped_smbus: 0 # uptime 08:51:21 up 3 days, 14:17, 3 users, load average: 0.79, 0.72, 0.72 > sometimes these can be false hangs. If you are truly getting reset > repeatedly like this every couple days or so, would you be willing to > run a driver that logged a bunch of info to syslog when the tx hang > occurs (it can keep the nic offline for a couple extra seconds while > dumping all the info) it doesn't have any run-time impact. I can't vouch for every single hang line, but I ssh into this machine to read mail and I notice these freezes quite frequently. Often several times a day. (I even got one just now while writing this mail.) I'll be happy to run a debug driver. -- Björn ------------------------------------------------------------------------------ Free Software Download: Index, Search & Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired