Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-11-24 Thread Michael Grant
On Thu, Sep 11, 2008 at 11:56 AM, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: > On Thu, Sep 11, 2008 at 12:08:47PM +0200, Michael Grant wrote: >> On Thu, Sep 11, 2008 at 11:20 AM, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: >> > On Thu, Sep 11, 2008 at 10:38:36AM +0200, Michael Grant wrote: >> >> My

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-09-11 Thread Jeremy Chadwick
On Thu, Sep 11, 2008 at 12:08:47PM +0200, Michael Grant wrote: > On Thu, Sep 11, 2008 at 11:20 AM, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: > > On Thu, Sep 11, 2008 at 10:38:36AM +0200, Michael Grant wrote: > >> My box crashed again: > >> > >> panic: kmem_malloc(4096): kmem_map too small: 1073741

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-09-11 Thread Michael Grant
On Thu, Sep 11, 2008 at 11:20 AM, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: > On Thu, Sep 11, 2008 at 10:38:36AM +0200, Michael Grant wrote: >> My box crashed again: >> >> panic: kmem_malloc(4096): kmem_map too small: 1073741824 total allocated >> cpuid = 0 >> Uptime: 33d11h12m58s >> Dumping 3327

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-09-11 Thread Jeremy Chadwick
On Thu, Sep 11, 2008 at 10:38:36AM +0200, Michael Grant wrote: > My box crashed again: > > panic: kmem_malloc(4096): kmem_map too small: 1073741824 total allocated > cpuid = 0 > Uptime: 33d11h12m58s > Dumping 3327 MB (2 chunks) > chunk 0: 1MB (151 pages) ... ok > chunk 1: 3327MB (851568 pages)

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-09-11 Thread Michael Grant
My box crashed again: panic: kmem_malloc(4096): kmem_map too small: 1073741824 total allocated cpuid = 0 Uptime: 33d11h12m58s Dumping 3327 MB (2 chunks) chunk 0: 1MB (151 pages) ... ok chunk 1: 3327MB (851568 pages) <---hung here Still no valid dump. There is 4gig of physical memory in the

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread john
> I feel for you John, I've lost many nights sleep in the last couple weeks trying to understand why this production box was crashing. I was really surprised to see this start happening, normally my freebsd boxes have uptimes in terms of years, not hours. Thanks for the sentiment, at las

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread Kris Kennaway
Michael Grant wrote: I have been having what seems like similar panics. I too cannot manage to get a crash dump, neither classic style nor minidump. Nor can I get it to work with DDB, there seems to be a problem with DDB and my Geom mirror. They're not at all similar, please don't confuse the

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread Michael Grant
I have been having what seems like similar panics. I too cannot manage to get a crash dump, neither classic style nor minidump. Nor can I get it to work with DDB, there seems to be a problem with DDB and my Geom mirror. Kris recommended I up kmem_size which I have done (twice now) and since the

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread Chuck Swiger
On Jul 24, 2008, at 9:15 AM, John Sullivan wrote: Right, after trying for a number of days the system still just hung without letting me get either a dump or to interactively debug in the failed state, I reverted back to the Generic kernel, removed half the memory (2 of the 4 1GB sticks) and t

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread Kris Kennaway
John Sullivan wrote: Removing KDB_UNATTENDED from your kernel will allow you to interact with the debugger and obtain backtraces etc, which is useful when dumps are not being saved. Easier said than done, this cause a few panics - no dumps though ...g!! Still the same result ... the sys

RE: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-24 Thread John Sullivan
>> Removing KDB_UNATTENDED from your kernel will allow you >> to interact with the debugger and obtain backtraces etc, >> which is useful when dumps are not being saved. > > Easier said than done, this cause a few panics - no dumps > though ...g!! > > Still the same result ... the system

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread john
> OK, the first thing to do is disable bg fsck, then force a full fsck of all filesystems.  bg fsck does a poor job of fixing arbitrary filesystem corruption (it's not designed to do so, in fact), and you can get into a situation where corrupted filesystems cause further panics. Done, not

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread Kevin Oberman
> From: "John Sullivan" <[EMAIL PROTECTED]> > Date: Wed, 16 Jul 2008 09:38:26 +0100 > > > > Could be memory, but I'd also suggest looking at > > temperatures. I've had overheating systems produce lots of > > such errors. > > Temperature is fine - it never get's that hot here in the UK ;-) > Se

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread Kris Kennaway
Michael Grant wrote: On Wed, Jul 16, 2008 at 10:38 AM, John Sullivan <[EMAIL PROTECTED]> wrote: Could be memory, but I'd also suggest looking at temperatures. I've had overheating systems produce lots of such errors. Temperature is fine - it never get's that hot here in the UK ;-) Seriously, I

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread Michael Grant
On Wed, Jul 16, 2008 at 10:38 AM, John Sullivan <[EMAIL PROTECTED]> wrote: > >> Could be memory, but I'd also suggest looking at >> temperatures. I've had overheating systems produce lots of >> such errors. > > Temperature is fine - it never get's that hot here in the UK ;-) Seriously, > I put my

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread Kris Kennaway
John Sullivan wrote: John, a question, how is swap set up on your system? I was swapping to a file (a memory disk device /dev/md0). I was doing this because for some reason lost in ancient history, this machine was not set up with a real swap partition. Hence, no crash dump. Swap is a p

RE: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread John Sullivan
> John, a question, how is swap set up on your system? I was > swapping to a file (a memory disk device /dev/md0). I was > doing this because for some reason lost in ancient history, > this machine was not set up with a real swap partition. > Hence, no crash dump. Swap is a partition on t

RE: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-16 Thread John Sullivan
> Could be memory, but I'd also suggest looking at > temperatures. I've had overheating systems produce lots of > such errors. Temperature is fine - it never get's that hot here in the UK ;-) Seriously, I put my hand in the box, touched a few heat sync's, it is not running hot enough to cause

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kevin Oberman
> From: "John Sullivan" <[EMAIL PROTECTED]> > Date: Tue, 15 Jul 2008 10:58:19 +0100 > Sender: [EMAIL PROTECTED] > > I am experiencing 'random' reboots interspersed with panics whenever I put a > newly installed system under load (make index in > /usr/ports is enough). A sample panic is at the en

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kris Kennaway
[EMAIL PROTECTED] wrote: >> #9 0x8067d3ee in uma_zalloc_arg (zone=0xff00bfed07e0, udata=0x0, flags=-256) at /usr/src/sys/vm/uma_core.c:1835 From the frame #9, please do p *zone I am esp. interested in the value of the uz_ctor member. It seems that it becomes corrupted, it valu

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread john
> Do the "frame 9" before "p *zone". It's obvious now you say it ;-) You are indeed right: (kgdb) frame 9 #9  0x8067d3ee in uma_zalloc_arg (zone=0xff00bfed07e0, udata=0x0,     flags=-256) at /usr/src/sys/vm/uma_core.c:1835 1835    uma_dbg_alloc(zone

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kostik Belousov
On Tue, Jul 15, 2008 at 08:47:03PM +0100, [EMAIL PROTECTED] wrote: > > > >> #9  0x8067d3ee in uma_zalloc_arg (zone=0xff00bfed07e0, > >>udata=0x0, > >>flags=-256) at /usr/src/sys/vm/uma_core.c:1835 > >From the frame #9, please do > >p *zone > >I am esp. interested in the value of the

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kostik Belousov
On Tue, Jul 15, 2008 at 08:19:15PM +0100, [EMAIL PROTECTED] wrote: > > > > Please collect kgdb/ddb backtraces. > > kgdb backtrace: > > server251# kgdb -c /var/crash/vmcore.0 > kgdb: couldn't find a suitable kernel image > server251# kgdb /boot/kernel/kernel /var/crash/vmcore.0 > kgdb: kvm

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread john
>> #9  0x8067d3ee in uma_zalloc_arg (zone=0xff00bfed07e0, udata=0x0, flags=-256) at /usr/src/sys/vm/uma_core.c:1835 From the frame #9, please do p *zone I am esp. interested in the value of the uz_ctor member. It seems that it becomes corrupted, it value should be 0, as this see

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kris Kennaway
[EMAIL PROTECTED] wrote: (kgdb) backtrace #0 doadump () at pcpu.h:194 #1 0xff0004742440 in ?? () #2 0x80477699 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #3 0x80477a9d in panic (fmt=0x104 ) at /usr/src/sys/kern/kern_shutdown.c:563 #4 0x8

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread john
> Please collect kgdb/ddb backtraces. kgdb backtrace: server251# kgdb -c /var/crash/vmcore.0 kgdb: couldn't find a suitable kernel image server251# kgdb /boot/kernel/kernel /var/crash/vmcore.0 kgdb: kvm_read: invalid address (0xff00010e5468) [GDB will not be able to debug user-mode t

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Kris Kennaway
John Sullivan wrote: Can the system in question run memtest86+ successfully (no errors) for an hour? It would help diminish (but not entirely rule out) hardware (memory or chipset) issues. Sorry, forgot to mention, I ran memtest over night without any problem reported. I ran Fedora 9 for

RE: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread John Sullivan
> Can the system in question run memtest86+ successfully (no > errors) for an hour? It would help diminish (but not > entirely rule out) hardware (memory or chipset) issues. Sorry, forgot to mention, I ran memtest over night without any problem reported. I ran Fedora 9 for a month without a

Re: Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread Jeremy Chadwick
On Tue, Jul 15, 2008 at 10:58:19AM +0100, John Sullivan wrote: > I am experiencing 'random' reboots interspersed with panics whenever I put a > newly installed system under load (make index in > /usr/ports is enough). A sample panic is at the end of this email. > > I have updated to 7.0-RELEASE

Fresh 7.0 Install: Fatal Trap 12 panic when put under load

2008-07-15 Thread John Sullivan
I am experiencing 'random' reboots interspersed with panics whenever I put a newly installed system under load (make index in /usr/ports is enough). A sample panic is at the end of this email. I have updated to 7.0-RELEASE-p2 using the GENERIC amd64 kernel and it is still the same. The system