ServeRAID semi show stopper (was Re: lots of exclusive sleep mutex)
Hi, Here comes more helpful ServeRAID related warnings: Bad malloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 Bad uma_zalloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 ( ... repeats ... ) The ServeRAID adaptor is # grep ips /var/run/dmesg.boot ips0: IBM ServeRAID Adapter mem 0xf000-0xf3ff irq 24 at device 1.0 on pci4 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ips0: adapter type: ServeRAID 5i II (sarasota) ips0: logical drives: 1 ips0: Logical Drive 0: RAID5 sectors: 142192640, state OK ipsd0: Logical Drive on ips0 GEOM: create disk ipsd0 dp=0xc6bc1010 ipsd0: Logical Drive (69430MB) To be more specific, it's FreeBSD 5.2-BETA with hint.acpi.0.disabled=1, running on IBM x225. Everything is ok, except the ServeRAID warns :) Clive ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ServeRAID semi show stopper (was Re: lots of exclusive sleep mutex)
Clive Lin wrote: Hi, Here comes more helpful ServeRAID related warnings: Bad malloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 Bad uma_zalloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 ( ... repeats ... ) The ServeRAID adaptor is # grep ips /var/run/dmesg.boot ips0: IBM ServeRAID Adapter mem 0xf000-0xf3ff irq 24 at device 1.0 on pci4 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ips0: adapter type: ServeRAID 5i II (sarasota) ips0: logical drives: 1 ips0: Logical Drive 0: RAID5 sectors: 142192640, state OK ipsd0: Logical Drive on ips0 GEOM: create disk ipsd0 dp=0xc6bc1010 ipsd0: Logical Drive (69430MB) To be more specific, it's FreeBSD 5.2-BETA with hint.acpi.0.disabled=1, running on IBM x225. Everything is ok, except the ServeRAID warns :) The patch below will silence this warning. However, the real solution is a lot more complicated and will have to wait for after 5.2. Scott --- ips_commands.c 27 Nov 2003 08:37:36 - 1.7 +++ ips_commands.c 3 Dec 2003 17:26:28 - @@ -165,7 +165,7 @@ void ips_start_io_request(ips_softc_t *sc, struct bio *iobuf) { - if(ips_get_free_cmd(sc, ips_send_io_request, iobuf, 0)){ + if(ips_get_free_cmd(sc, ips_send_io_request, iobuf, IPS_NOWAIT_FLAG)){ device_printf(sc-dev, no mem for command slots!\n); iobuf-bio_flags |= BIO_ERROR; iobuf-bio_error = ENOMEM; ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ServeRAID semi show stopper (was Re: lots of exclusive sleep mutex)
On Wed, Dec 03, 2003 at 10:29:19AM -0700, Scott Long wrote: Clive Lin wrote: Hi, Here comes more helpful ServeRAID related warnings: Bad malloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 Bad uma_zalloc flags: 0 Stack backtrace: malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe1f31cbc) locked @ /usr/src/sys/geom/geom_io.c:355 ( ... repeats ... ) The patch below will silence this warning. However, the real solution is a lot more complicated and will have to wait for after 5.2. Scott --- ips_commands.c 27 Nov 2003 08:37:36 - 1.7 +++ ips_commands.c 3 Dec 2003 17:26:28 - @@ -165,7 +165,7 @@ void ips_start_io_request(ips_softc_t *sc, struct bio *iobuf) { - if(ips_get_free_cmd(sc, ips_send_io_request, iobuf, 0)){ + if(ips_get_free_cmd(sc, ips_send_io_request, iobuf, IPS_NOWAIT_FLAG)){ device_printf(sc-dev, no mem for command slots!\n); iobuf-bio_flags |= BIO_ERROR; iobuf-bio_error = ENOMEM; Hi, It helps a lot to reduce my serial console traffic :) Now only few warns below appears under heavy disk access: ipsd0: iobuf error 12 ips0: no mem for command slots! Clive ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: lots of exclusive sleep mutex
On 2003/10/31 10:09:04, YONETANI Tomokazu wrote: On 2003/10/30 00:16:47, Clive Lin wrote: On Sat, Oct 04, 2003 at 02:00:33AM +0800, Clive Lin wrote: Hi, I've seen lots of messages on rescent -CURRENT malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 Many of above are still seen on the latest current. malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 Perhaps it's a ServeRAID specific glitch? dmesg|grep ips ips0: IBM ServeRAID Adapter mem 0xf000-0xf3ff irq 16 at device 1.0 on pci4 ips0: logical drives: 1 ipsd0: Logical Drive on ips0 GEOM: create disk ipsd0 dp=0xc6b25310 ipsd0: Logical Drive (69430MB) Does this fix? Index: ips.c [snipped] Of course it doesn't. Hmm, I didn't realize that I've removed WITNESS* from the kernel config file months ago. Anyway, this is the backtrace of the above warnings from witness code(by manually setting a breakpoint inside witness_warning and issuing panic command from DDB): Script started on Fri Oct 31 12:47:51 2003 # gdb -k /usr/obj/home/source/freebsd/current/src/sys/MAIL/kernel.debug vmcore.0 GNU gdb 5.2.1 (FreeBSD) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as i386-undermydesk-freebsd... panic: from debugger panic messages: --- --- Reading symbols from /boot/kernel/acpi.ko...done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/usb.ko...done. Loaded symbols for /boot/kernel/usb.ko Reading symbols from /boot/kernel/nfsclient.ko...done. Loaded symbols for /boot/kernel/nfsclient.ko #0 doadump () at /home/source/freebsd/current/src/sys/kern/kern_shutdown.c:240 240 dumping++; (kgdb) bt #0 doadump () at /home/source/freebsd/current/src/sys/kern/kern_shutdown.c:240 #1 0xc0576346 in boot (howto=260) at /home/source/freebsd/current/src/sys/kern/kern_shutdown.c:372 #2 0xc0576677 in panic () at /home/source/freebsd/current/src/sys/kern/kern_shutdown.c:550 #3 0xc045d1c2 in db_panic () at /home/source/freebsd/current/src/sys/ddb/db_command.c:450 #4 0xc045d122 in db_command (last_cmdp=0xc073e460, cmd_table=0x0, aux_cmd_tablep=0xc0709cec, aux_cmd_tablep_end=0xc0709d04) at /home/source/freebsd/current/src/sys/ddb/db_command.c:346 #5 0xc045d265 in db_command_loop () at /home/source/freebsd/current/src/sys/ddb/db_command.c:472 #6 0xc0460265 in db_trap (type=3, code=0) at /home/source/freebsd/current/src/sys/ddb/db_trap.c:73 #7 0xc069765c in kdb_trap (type=3, code=0, regs=0xd77e5b3c) at /home/source/freebsd/current/src/sys/i386/i386/db_interface.c:171 #8 0xc06a7fe8 in trap (frame= {tf_fs = -1065877480, tf_es = -1065877488, tf_ds = -679608304, tf_edi = -1065836412, tf_esi = -1065836416, tf_ebp = -679584872, tf_isp = -679584920, tf_ebx = 0, tf_edx = 0, tf_ecx = -1056882688, tf_eax = 19, tf_trapno = 3, tf_err = 0, tf_eip = -1067865189, tf_cs = 8, tf_eflags = 514, tf_esp = -1066450495, tf_ss = -679584852}) at /home/source/freebsd/current/src/sys/i386/i386/trap.c:582 #9 0xc0699048 in calltrap () at {standard input}:102 #10 0xc0674b34 in uma_zalloc_arg (zone=0xc078a080, udata=0x0, flags=0) at /home/source/freebsd/current/src/sys/vm/uma_core.c:1385 #11 0xc056bd05 in malloc (size=0, type=0xc07235c0, flags=0) at /home/source/freebsd/current/src/sys/vm/uma.h:234 #12 0xc04e5d44 in ips_add_waiting_command (sc=0xc479f000, callback=0xc04e8050 ips_send_io_request, data=0xc7ea9a18, flags=0) at /home/source/freebsd/current/src/sys/dev/ips/ips.c:147 #13 0xc04e5f1e in ips_get_free_cmd (sc=0x0, callback=0xc04e8050 ips_send_io_request, data=0xc7ea9a18, flags=0) at /home/source/freebsd/current/src/sys/dev/ips/ips.c:222 #14 0xc04e813e in ips_start_io_request (sc=0xc7ea9a18, iobuf=0x0) at /home/source/freebsd/current/src/sys/dev/ips/ips_commands.c:168 #15 0xc04e7b62 in ipsd_strategy (iobuf=0x0) at /home/source/freebsd/current/src/sys/dev/ips/ips_disk.c:100 #16 0xc0546036 in g_disk_start (bp=0xc8798110) at /home/source/freebsd/current/src/sys/geom/geom_disk.c:272 ---Type return to continue, or q return to quit--- #17 0xc0547fcc in g_io_schedule_down (tp=0xc1d17000) at
Re: lots of exclusive sleep mutex
On 2003/10/30 00:16:47, Clive Lin wrote: On Sat, Oct 04, 2003 at 02:00:33AM +0800, Clive Lin wrote: Hi, I've seen lots of messages on rescent -CURRENT malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 Many of above are still seen on the latest current. malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 Perhaps it's a ServeRAID specific glitch? dmesg|grep ips ips0: IBM ServeRAID Adapter mem 0xf000-0xf3ff irq 16 at device 1.0 on pci4 ips0: logical drives: 1 ipsd0: Logical Drive on ips0 GEOM: create disk ipsd0 dp=0xc6b25310 ipsd0: Logical Drive (69430MB) Does this fix? Index: ips.c === RCS file: /home/cvs/freebsd/src/sys/dev/ips/ips.c,v retrieving revision 1.5 diff -u -6 -r1.5 ips.c --- ips.c 24 Aug 2003 17:49:13 - 1.5 +++ ips.c 30 Oct 2003 08:45:32 - @@ -147,12 +147,13 @@ waiter = malloc(sizeof(ips_wait_list_t), M_DEVBUF, memflags); if(!waiter) return ENOMEM; mask = splbio(); if(sc-state IPS_OFFLINE){ splx(mask); + free(waiter, M_DEVBUF); return EIO; } command = SLIST_FIRST(sc-free_cmd_list); if(command !(sc-state IPS_TIMEOUT)){ SLIST_REMOVE_HEAD(sc-free_cmd_list, next); (sc-used_commands)++; By the way, does this panic your machine? $ exec sh $ mkdir foo $ i=0; while :; do echo $i foo/$i; i=$(($i+1)); done Regards. -- YONETANI Tomokazu / Ergo-Brains Inc. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: lots of exclusive sleep mutex
On Sat, Oct 04, 2003 at 02:00:33AM +0800, Clive Lin wrote: Hi, I've seen lots of messages on rescent -CURRENT malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 Many of above are still seen on the latest current. malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:355 Perhaps it's a ServeRAID specific glitch? dmesg|grep ips ips0: IBM ServeRAID Adapter mem 0xf000-0xf3ff irq 16 at device 1.0 on pci4 ips0: logical drives: 1 ipsd0: Logical Drive on ips0 GEOM: create disk ipsd0 dp=0xc6b25310 ipsd0: Logical Drive (69430MB) Regards, Clive ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
lots of exclusive sleep mutex
Hi, I've seen lots of messages on rescent -CURRENT malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 malloc() of 16 with the following non-sleepable locks held: exclusive sleep mutex g_xdown r = 0 (0xe044eca8) locked @ /usr/src/sys/geom/geom_io.c:351 uname -av is FreeBSD x225.dmz 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Fri Oct 3 23:47:45 CST 2003 [EMAIL PROTECTED]:/d/obj/usr/src/sys/XEON5 i386 Clive ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]