Re: panic during nfs operations in 4.8S on Dell 2650
On Fri, 8 Aug 2003, Andrew Kinney wrote: > On 8 Aug 2003, at 11:52, Mark Powell wrote: > > > #6 0xc0312ea3 in generic_bzero () > > FWIW, I think this is where the problem occurred. Probably tried > to zero a page that didn't exist because of a failed KVA allocation. > > We had several panics on one of our 4GB machines at the same > point. Our solution was to increase the KVA space to 2GB from > 1GB and rebuild the whole world with the new KVA setting. The > panics disappeared. Yep, that was it. Well I upped KVA_PAGES from the default of 260 (in LINT?) to 384 and rebuilt the kernel. Sysctl shows it now has plenty to spare when running the rsyncs. Why did you have to rebuild world when changing this and not just the kernel? Cheers. -- Mark Powell - UNIX System Administrator - The University of Salford Information Services Division, Clifford Whitworth Building, Salford University, Manchester, M5 4WT, UK. Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: panic during nfs operations in 4.8S on Dell 2650
On Tue, 12 Aug 2003, Andrew Kinney wrote: > Well, it's been awhile since I did this, but it seems like we were > having some trouble with some applications or system utilities. It > could have just been that we had some stuff out of synch on that > system since it had been upgraded from 4.5-RELEASE to 4.7- > RELEASE and then to 4.8-RELEASE. Ok. Many thanks for the pointer. Cheers. -- Mark Powell - UNIX System Administrator - The University of Salford Information Services Division, Clifford Whitworth Building, Salford University, Manchester, M5 4WT, UK. Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: panic during nfs operations in 4.8S on Dell 2650
On Fri, 8 Aug 2003, Mark Powell wrote: > On Thu, 7 Aug 2003, Kip Macy wrote: > > > Can you get a backtrace? > > Isn't that what I included at the bottom of my first message? I tried bumping nmbclusters up to 65536 from 32768. Still got a panic. Here's another backtrace of the latest panic: Script started on Fri Aug 8 11:49:19 2003 /var/crash # gdb -k kernel.debug.4 vmcore.4 GNU gdb 4.18 (FreeBSD) Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"...Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 2627 in elfstab_build_psymtabs Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 933 in fill_symbuf SMP 2 cpus IdlePTD at phsyical address 0x004ad000 initial pcb at physical address 0x003f8d00 panicstr: page fault panic messages: --- Fatal trap 12: page fault while in kernel mode mp_lock = 01000290; cpuid = 1; lapic.id = 0200 fault virtual address = 0x0 fault code = supervisor write, page not present instruction pointer = 0x8:0xc0312ea3 stack pointer = 0x10:0xff487ca0 frame pointer = 0x10:0xff487ccc code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 230 (rsync) interrupt mask = none <- SMP: XXX trap number = 12 panic: page fault mp_lock = 01000290; cpuid = 1; lapic.id = 0200 boot() called on cpu#1 syncing disks... 2 1 done Uptime: 1h1m1s dumping to dev #aacd/0x20001, offset 524636 dump 3839 3838 3837 3836 3835 3834 3833 3832 3831 3830 3829 3828 3827 3826 3825 3824 3823 3822 3821 3820 3819 3818 3817 3816 3815 3814 3813 3812 3811 3810 3809 3808 3807 3806 3805 3804 3803 3802 3801 3800 3799 3798 3797 3796 3795 3794 3793 3792 3791 3790 3789 3788 3787 3786 3785 3784 3783 3782 3781 3780 3779 3778 3777 3776 3775 3774 3773 3772 3771 3770 3769 3768 3767 3766 3765 3764 3763 3762 3761 3760 3759 3758 3757 3756 3755 3754 3753 3752 3751 3750 3749 3748 3747 3746 3745 3744 3743 3742 3741 3740 3739 3738 3737 3736 3735 3734 3733 3732 3731 3730 3729 3728 3727 3726 3725 3724 3723 3722 3721 3720 3719 3718 3717 3716 3715 3714 3713 3712 3711 3710 3709 3708 3707 3706 3705 3704 3703 3702 3701 3700 3699 3698 3697 3696 3695 3694 3693 3692 3691 3690 3689 3688 3687 3686 3685 3684 3683 3682 3681 3680 3679 3678 3677 3676 3675 3674 3673 3672 3671 3670 3669 3668 3667 3666 3665 3664 3663 3662 3661 3660 3659 3658 3657 3656 3655 3654 3653 3652 3651 3650 3649 3648 3647 3646 3645 3644 3643 3642 3641 3640 3639 3638 3637 3636 3635 3634 3633 3632 3631 3630 3629 3628 3627 3626 3625 3624 3623 3622 3621 3620 3619 3618 3617 3616 3615 3614 3613 3612 3611 3610 3609 3608 3607 3606 3605 3604 3603 3602 3601 3600 3599 3598 3597 3596 3595 3594 3593 3592 3591 3590 3589 3588 3587 3586 3585 3584 3583 3582 3581 3580 3579 3578 3577 3576 3575 3574 3573 3572 3571 3570 3569 3568 3567 3566 3565 3564 3563 3562 3561 3560 3559 3558 3557 3556 3555 3554 3553 3552 3551 3550 3549 3548 3547 3546 3545 3544 3543 3542 3541 3540 3539 3538 3537 3536 3535 3534 3533 3532 3531 3530 3529 3528 3527 3526 3525 3524 3523 3522 3521 3520 3519 3518 3517 3516 3515 3514 3513 3512 3511 3510 3509 3508 3507 3506 3505 3504 3503 3502 3501 3500 3499 3498 3497 3496 3495 3494 3493 3492 3491 3490 3489 3488 3487 3486 3485 3484 3483 3482 3481 3480 3479 3478 3477 3476 3475 3474 3473 3472 3471 3470 3469 3468 3467 3466 3465 3464 3463 3462 3461 3460 3459 3458 3457 3456 3455 3454 3453 3452 3451 3450 3449 3448 3447 3446 3445 3444 3443 3442 3441 3440 3439 3438 3437 3436 3435 3434 3433 3432 3431 3430 3429 3428 3427 3426 3425 3424 3423 3422 3421 3420 3419 3418 3417 3416 3415 3414 3413 3412 3411 3410 3409 3408 3407 3406 3405 3404 3403 3402 3401 3400 3399 3398 3397 3396 3395 3394 3393 3392 3391 3390 3389 3388 3387 3386 3385 3384 3383 3382 3381 3380 3379 3378 3377 3376 3375 3374 3373 3372 3371 3370 3369 3368 3367 3366 3365 3364 3363 3362 3361 3360 3359 3358 3357 3356 3355 3354 3353 3352 3351 3350 3349 3348 3347 3346 3345 3344 3343 3342 3341 3340 3339 3338 3337 3336 3335 3334 3332 3331 3330 3329 3328 3327 3326 3325 3324 3323 3322 3321 3320 3319 3318 3317 3316 3315 3314 3313 3312 3311 3310 3309 3308 3307 3306 3305 3304 3303 3302 3301 3300 3299 3298 3297 3296 3295 3294 3293 3292 3291 3290 3289 3288 3287 3286 3285 3284 3283 3282 3281 3280 3279 3278 3277 3276 3275 3274 3273 3272 3271 3270 3269 3268 3267 3266 3265 3264 3263 3262 3261 3260 3259 3258 3257 3256 3255 3254 3253
Re: panic during nfs operations in 4.8S on Dell 2650
On Thu, 7 Aug 2003, Mark Powell wrote: > We've recently got a couple of Dell Poweredge 2650's with 2x2.8GHz > Xeons, 4GB RAM, PERC 3/Di (aac) RAID controller. They are mounting a 700GB > fs over NFS from a NetAPP. They are connected to a Cisco 3550-12T gigabit > over copper switch. I tried them first on the intel em cards and they > panicked and also the internal bge adapters with the same result. > Thought everything was fine until I was rsyncing the POP3 mail stores > from the old machines onto these. Rsync runs for about an hour or so and > get's large. In the 300M-600M region the system will always panic. This > happens on both systems, so doesn't seem a hardware fault. This is a 4.8S kernel and world rebuilt as of today. -- Mark Powell - UNIX System Administrator - The University of Salford Information Services Division, Clifford Whitworth Building, Salford University, Manchester, M5 4WT, UK. Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: panic during nfs operations in 4.8S on Dell 2650
On Thu, 7 Aug 2003, Kip Macy wrote: > Can you get a backtrace? Isn't that what I included at the bottom of my first message? > Not knowing anything at this point, > bumping up the number of mbuf clusters *might* help. Doh. Will try that first. Been a long time since I had a panic :( Cheers. > > > -Kip > > > FYI: I'm not representing NetApp in any official capacity on this, > I just happen to have a vested interest in both OnTap and FreeBSD. > > > > On Thu, 7 Aug 2003, Mark Powell wrote: > > > On Thu, 7 Aug 2003, Mark Powell wrote: > > > We've recently got a couple of Dell Poweredge 2650's with 2x2.8GHz > > > Xeons, 4GB RAM, PERC 3/Di (aac) RAID controller. They are mounting a 700GB > > > fs over NFS from a NetAPP. They are connected to a Cisco 3550-12T gigabit > > > over copper switch. I tried them first on the intel em cards and they > > > panicked and also the internal bge adapters with the same result. > > > Thought everything was fine until I was rsyncing the POP3 mail stores > > > from the old machines onto these. Rsync runs for about an hour or so and > > > get's large. In the 300M-600M region the system will always panic. This > > > happens on both systems, so doesn't seem a hardware fault. > > > > This is a 4.8S kernel and world rebuilt as of today. > > > > -- > > Mark Powell - UNIX System Administrator - The University of Salford > > Information Services Division, Clifford Whitworth Building, > > Salford University, Manchester, M5 4WT, UK. > > Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key > > _______ > > [EMAIL PROTECTED] mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > > > > > -- Mark Powell - UNIX System Administrator - The University of Salford Information Services Division, Clifford Whitworth Building, Salford University, Manchester, M5 4WT, UK. Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
panic during nfs operations in 4.8S on Dell 2650
Hi,
We've recently got a couple of Dell Poweredge 2650's with 2x2.8GHz
Xeons, 4GB RAM, PERC 3/Di (aac) RAID controller. They are mounting a 700GB
fs over NFS from a NetAPP. They are connected to a Cisco 3550-12T gigabit
over copper switch. I tried them first on the intel em cards and they
panicked and also the internal bge adapters with the same result.
Thought everything was fine until I was rsyncing the POP3 mail stores
from the old machines onto these. Rsync runs for about an hour or so and
get's large. In the 300M-600M region the system will always panic. This
happens on both systems, so doesn't seem a hardware fault.
# gdb -k kernel.debug.3 vmcore.3
GNU gdb 4.18 (FreeBSD)
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-unknown-freebsd"...Deprecated bfd_read
called at
/usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c
line 2627 in elfstab_build_psymtabs
Deprecated bfd_read called at
/usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c
line 933 in fill_symbuf
SMP 2 cpus
IdlePTD at phsyical address 0x004ad000
initial pcb at physical address 0x003f8d00
panicstr: page fault
panic messages:
---
Fatal trap 12: page fault while in kernel mode
mp_lock = 01001186; cpuid = 1; lapic.id = 0200
fault virtual address = 0x0
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc0312ea3
stack pointer = 0x10:0xf946cc60
frame pointer = 0x10:0xf946cc94
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 330 (rsync)
interrupt mask = bio <- SMP: XXX
trap number = 12
panic: page fault
mp_lock = 01001186; cpuid = 1; lapic.id = 0200
boot() called on cpu#1
syncing disks... 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
giving up on 2 buffers
Uptime: 1h35m9s
...
(kgdb) where
#0
dumpsys ()
at
../../kern/kern_shutdown.c:487
#1 0xc01c45e7 in boot (howto=256) at ../../kern/kern_shutdown.c:316
#2 0xc01c4a59 in panic (fmt=0xc03781f9 "%s") at
../../kern/kern_shutdown.c:595
#3 0xc031458d in trap_fatal (frame=0xf946cc20, eva=0) at
../../i386/i386/trap.c:974
#4 0xc03141f9 in trap_pfault (frame=0xf946cc20, usermode=0, eva=0) at
../../i386/i386/trap.c:867
#5 0xc0313d53 in trap (frame={tf_fs = -869007336, tf_es = -1069547504,
tf_ds = 16, tf_edi = 0,
tf_esi = -8400896, tf_ebp = -112800620, tf_isp = -112800692, tf_ebx
= 0,
tf_edx = -1745141825, tf_ecx = 42, tf_eax = 0, tf_trapno = 12,
tf_err = 2,
tf_eip = -1070518621, tf_cs = 8, tf_eflags = 66050, tf_esp =
-113966560,
tf_ss = -1071700836}) at ../../i386/i386/trap.c:466
#6 0xc0312ea3 in generic_bzero ()
#7 0xc0240d90 in nfs_nget (mntp=0xcc621400, fhp=0xc62fe84c, fhsize=32,
npp=0xf946cd34)
at ../../nfs/nfs_node.c:143
#8 0xc026716f in nfs_lookup (ap=0xf946ce00) at ../../nfs/nfs_vnops.c:959
#9 0xc01f15e5 in lookup (ndp=0xf946ce7c) at vnode_if.h:52
#10 0xc01f10e0 in namei (ndp=0xf946ce7c) at ../../kern/vfs_lookup.c:153
#11 0xc01f7441 in lstat (p=0xf9350220, uap=0xf946cf80) at
../../kern/vfs_syscalls.c:1824
#12 0xc03148c9 in syscall2 (frame={tf_fs = -112852945, tf_es =
-1070464977, tf_ds = -869334993,
tf_edi = -1077951584, tf_esi = -1077953648, tf_ebp = -1077953744,
tf_isp = -112799788,
tf_ebx = -1077952624, tf_edx = 2, tf_ecx = -49, tf_eax = 190,
tf_trapno = 7, tf_err = 2,
tf_eip = 1745642828, tf_cs = 31, tf_eflags = 659, tf_esp =
-1077953772, tf_ss = 47})
at ../../i386/i386/trap.c:1175
#13 0xc03015fb in Xint0x80_syscall ()
cannot read proc at 0
Cheers.
--
Mark Powell - UNIX System Administrator - The University of Salford
Information Services Division, Clifford Whitworth Building,
Salford University, Manchester, M5 4WT, UK.
Tel: +44 161 295 5936 Fax: +44 161 295 5888 www.pgp.com for PGP key
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
