Besides the based on what I've heard this is low budget equipment and ProLiants aren't cheap. If I had to guess we are talking about Dells.
-- Sent from my HP Pre3
On Dec 4, 2013 18:36, David Sommerseth <sl+us...@lists.topphemmelig.net> wrote:
On 04/12/13 14:21, ~Stack~ wrote:> Greetings,
>
> I have a test system I use for testing deployments and when I am not
> using it, it runs Boinc. It is a Scientific Linux 6.4 fully updated box.
> Recently (last ~3 weeks) I have started getting the same kernel panic.
> Sometimes it will be multiple times in a single day and other times it
> will be days before the next one (it just had a 5 day uptime). But the
> kernel panic looks pretty much the same. It is a complaint about a hung
> task plus information about the ext4 file system. I have run the
> smartmon tool against both drives (2 drives setup in a hardware RAID
> mirror) and both drives checkout fine. I ran a fsck against the /
> partition and everything looked fine (on this text box there is only /
> and swap partitions). I even took out a drive at a time and had the same
> crashes (though this could be an indicator that both drives are bad). I
> am wondering if my RAID card is going bad.
>
> When the crash happens I still have the SSH prompt, however, I can only
> do basic things like navigating directories and sometimes reading files.
> Writing to a file seems to hang, using tab-autocomplete will frequently
> hang, running most programs (even `init 6` or `top`) will hang.
>
> It crashed again last night, and I am kind of stumped. I would greatly
> appreciate others thoughts and input on what the problem might be.
>
> Thanks!
> ~Stack~
>
> Dec 4 02:25:09 testbox kernel: INFO: task jbd2/cciss!c0d0:273 blocked
> for more than 120 seconds.
> Dec 4 02:25:09 testbox kernel: "echo 0 >
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Dec 4 02:25:09 testbox kernel: jbd2/cciss!c0 D 0000000000000000 0
> 273 2 0x00000000
> Dec 4 02:25:09 testbox kernel: ffff8802142cfb30 0000000000000046
> ffff8802138b5800 0000000000001000
> Dec 4 02:25:09 testbox kernel: ffff8802142cfaa0 ffffffff81012c59
> ffff8802142cfae0 ffffffff810a2431
> Dec 4 02:25:09 testbox kernel: ffff880214157058 ffff8802142cffd8
> 000000000000fb88 ffff880214157058
> Dec 4 02:25:09 testbox kernel: Call Trace:
> Dec 4 02:25:09 testbox kernel: [<ffffffff81012c59>] ? read_tsc+0x9/0x20
This looks like some locking issue to me, triggered by something around the
TSC timer.
This is either a buggy driver (most likely the ccsis driver) or a related
firmware (read the complete boot log carefully, look after firmware warnings).
Or it's a really unstable TSC clock source. Try switching from TSC to HPET
(or in really worst case acpi_pm). See this KB for some related info:
<https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_MRG/2/html/Realtime_Reference_Guide/chap-Realtime_Reference_Guide-Timestamping.html>
But my hunch tells me it's a driver related issue, with some bad locking.
There seems to be several filesystem operations happening on two or more CPU
cores in a certain order which seems to trigger a deadlock.
--
kind regards,
David Sommerseth
