We have been having problems with VMware 5.5. VMware are pointing fingers at
our storage (EMC). EMC point at VMware and the HBA's. We swapped HBA's with a
different vendor. Issue still happens. Anyone seen errors like this before?
This guest is Redhat 6.5
ESX hardware is Dell.
Aug 20 02:15:52 server01 kernel: [sched_delayed] sched: RT throttling activated
Aug 20 02:16:01 server01 kernel: hrtimer: interrupt took 1318457321 ns
Aug 20 02:17:12 server01 kernel: BUG: soft lockup - CPU#0 stuck for 21s!
[watchdog/0:11]
Aug 20 02:17:12 server01 kernel: Modules linked in: autofs4 fuse vsock ipv6
ppdev coretemp hwmon kvm_intel kvm crc32c_intel ghash_clmulni_intel microcode
pcspkr vmxnet3 vmw_vmci i2c_piix4 sg shpchp acpi_memhotplug parport_pc parport
ext4 jbd2 mbcache sd_mod crc_t10dif sr_mod cdrom aesni_intel ablk_helper cryptd
lrw aes_x86_64 xts gf128mul floppy mptsas mptscsih mptbase scsi_transport_sas
pata_acpi ata_generic ata_piix vmwgfx ttm drm i2c_core dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: speedstep_lib]
Aug 20 02:17:12 server01 kernel: CPU 0
Aug 20 02:17:12 server01 kernel: Pid: 11, comm: watchdog/0 Not tainted
3.8.13-35.1.3.el6uek.x86_64 #2 VMware, Inc. VMware Virtual Platform/440BX
Desktop Reference Platform
Aug 20 02:17:12 server01 kernel: RIP: 0010:[<ffffffffa0261714>]
[<ffffffffa0261714>] vmxnet3_rq_rx_complete+0x364/0x870 [vmxnet3]
Aug 20 02:17:12 server01 kernel: RSP: 0000:ffff88013b003db0 EFLAGS: 00010282
Aug 20 02:17:12 server01 kernel: RAX: 0000000000000000 RBX: 0000000000000000
RCX: 000000000420a140
Aug 20 02:17:12 server01 kernel: RDX: 0000000000000000 RSI: 0000000000000286
RDI: ffff8800379938d0
Aug 20 02:17:12 server01 kernel: RBP: ffff88013b003e20 R08: ffff880134740800
R09: 0000000000c80042
Aug 20 02:17:12 server01 kernel: R10: 01010000c06c7a00 R11: 0000000000000042
R12: ffff88013b003d28
Aug 20 02:17:12 server01 kernel: R13: ffffffff8159ee9d R14: ffff88013b003e20
R15: ffff88013420bbc0
Aug 20 02:17:12 server01 kernel: FS: 0000000000000000(0000)
GS:ffff88013b000000(0000) knlGS:0000000000000000
Aug 20 02:17:12 server01 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Aug 20 02:17:12 server01 kernel: CR2: ffffffffff600400 CR3: 0000000137ee3000
CR4: 00000000000407f0
Aug 20 02:17:12 server01 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Aug 20 02:17:12 server01 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Aug 20 02:17:12 server01 kernel: Process watchdog/0 (pid: 11, threadinfo
ffff8801390d6000, task ffff8801390d42c0)
Aug 20 02:17:12 server01 kernel: Stack:
Aug 20 02:17:12 server01 kernel: 0000000000000001 00000000378707c0
ffff880134741020 ffffffff812a19b0
Aug 20 02:17:12 server01 kernel: 0000004005f23e20 ffff8800378707c0
ffff880134740800 0000009100000006
Aug 20 02:17:12 server01 kernel: ffff88013b003e00 ffff880134740800
ffff880134741020 0000000000000040
Aug 20 02:17:12 server01 kernel: Call Trace:
Aug 20 02:17:12 server01 kernel: <IRQ>
Aug 20 02:17:12 server01 kernel: [<ffffffff812a19b0>] ? swiotlb_map_sg+0x10/0x10
Aug 20 02:17:12 server01 kernel: [<ffffffffa0261e73>]
vmxnet3_poll_rx_only+0x43/0xc0 [vmxnet3]
Aug 20 02:17:12 server01 kernel: [<ffffffff814cc185>] net_rx_action+0x105/0x2b0
Aug 20 02:17:12 server01 kernel: [<ffffffff81066187>] __do_softirq+0xd7/0x240
Aug 20 02:17:12 server01 kernel: [<ffffffff810b7ea4>] ?
clockevents_program_event+0x74/0x120
Aug 20 02:17:12 server01 kernel: [<ffffffff81087d0e>] ?
hrtimer_interrupt+0x13e/0x240
Aug 20 02:17:12 server01 kernel: [<ffffffff8159f5dc>] call_softirq+0x1c/0x30
Aug 20 02:17:12 server01 kernel: [<ffffffff810174b5>] do_softirq+0x65/0xa0
Aug 20 02:17:12 server01 kernel: [<ffffffff81065f6d>] irq_exit+0xbd/0xe0
Aug 20 02:17:12 server01 kernel: [<ffffffff815a030e>]
smp_apic_timer_interrupt+0x6e/0x99
Aug 20 02:17:12 server01 kernel: [<ffffffff8159ee9d>]
apic_timer_interrupt+0x6d/0x80
Aug 20 02:17:12 server01 kernel: <EOI>
Aug 20 02:17:12 server01 kernel: [<ffffffff810a32ad>] ?
__dequeue_rt_entity+0x13d/0x2e0
Aug 20 02:17:12 server01 kernel: [<ffffffff81092cb7>] ?
finish_task_switch+0x67/0x120
Aug 20 02:17:12 server01 kernel: [<ffffffff81092cb1>] ?
finish_task_switch+0x61/0x120
Aug 20 02:17:12 server01 kernel: [<ffffffff8159410d>] __schedule+0x3fd/0x720
Aug 20 02:17:12 server01 kernel: [<ffffffff81594749>] schedule+0x29/0x70
Aug 20 02:17:12 server01 kernel: [<ffffffff8108bf05>]
smpboot_thread_fn+0xc5/0x180
Aug 20 02:17:12 server01 kernel: [<ffffffff8108be40>] ?
smpboot_create_threads+0x80/0x80
Aug 20 02:17:12 server01 kernel: [<ffffffff81082e7e>] kthread+0xce/0xe0
Aug 20 02:17:12 server01 kernel: [<ffffffff81082db0>] ?
kthread_freezable_should_stop+0x70/0x70
Aug 20 02:17:12 server01 kernel: [<ffffffff8159e1ac>] ret_from_fork+0x7c/0xb0
Aug 20 02:17:12 server01 kernel: [<ffffffff81082db0>] ?
kthread_freezable_should_stop+0x70/0x70
Aug 20 02:17:12 server01 kernel: Code: 80 7d 09 00 0f 88 91 04 00 00 48 8b 55
c0 48 8b 82 e8 14 00 00 66 83 b8 80 00 00 00 00 0f 89 01 02 00 00 4c 89 e7 e8
3c a3 26 e1 <48> c7 83 d8 00 00 00 00 00 00 00 45 31 c9 8b 4d cc 4c 89 f8 4d
Thanks
Craig
_______________________________________________
Tech mailing list
[email protected]
https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
