I honestly have no clue. I have banged on my mfis as much as I could
and have never seen anything like this. I am doing some investigation
into this. If you find a way to repro this let me know please.
On Mon, Feb 12, 2007 at 01:12:56PM -0500, Alejandro Lozanoff wrote:
> I was planning on running bonnie tonight (This is a production server)
> to get some IO and see if i can reproduce the problem.
> We have 3 of this servers with OBSD 4.0 stable and the same RAID card
> running for about 2 months now, this is the first problem we
> encounter. If you have any idea of how to reproduce this i would be
> glad to test it in one of the servers that we are not using right now.
>
> Thanks for replying
> Alejandro.-
>
> Quoting Marco Peereboom <[EMAIL PROTECTED]>:
>
> >I have never seen this but I am very interested in this particular
> >instance. Apparently there is an issue with read ahead on mfi that I
> >have never seen before on OpenBSD but other OS' have run into.
> >
> >Is this reproducible? If so can you try to disable read ahead in
> >CTRL-R (bios)?
> >
> >Thanks,
> >/marco
> >
> >On Mon, Feb 12, 2007 at 11:31:26AM -0500, Alejandro Lozanoff wrote:
> >>Sorry for the message without body, im a little sleepy and hitted the
> >>wrong button... :p
> >>
> >>Ok,
> >>
> >>I had this problem last night on one of our shiny Dell PowerEdge 2950s
> >>with RAID 10 and SAS disks.
> >>For no reason it started screaming "sd0: not queued: error 5" on the
> >>console, the server didnt crash (no core, no trace) but was in an
> >>unresponsive state, i couldnt log on or anything. I was wondering if
> >>this error could be a hardware problem (RAID card maybe?).
> >>The logs dont show anything notorious, there were no heavy IO at the
> >>time of the problem. I'm kind of clueless at what could have been the
> >>problem, a quick search showed some old messages with SCSI cards
> >>problems only.
> >>
> >>This is 4.0 Stable as shown below.
> >>
> >>dmesg follows:
> >>
> >>OpenBSD 4.0-stable (GENERIC) #1: Mon Nov 27 16:23:49 GMT 2006
> >> [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
> >>cpu0: Intel(R) Xeon(TM) CPU 3.00GHz ("GenuineIntel" 686-class) 3 GHz
> >>cpu0:
> >>
> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
> >>
> H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,CNXT-ID,C
> >>X16
> >>cpu0: Enhanced SpeedStep disabled by BIOS
> >>real mem = 1072955392 (1047808K)
> >>avail mem = 970735616 (947984K)
> >>using 4256 buffers containing 53751808 bytes (52492K) of memory
> >>mainbus0 (root)
> >>bios0 at mainbus0: AT/286+(00) BIOS, date 06/21/06, BIOS32 rev. 0 @
> >>0xffe90, SMBIOS rev. 2.4 @ 0x3ffbc000 (62 entries)
> >>bios0: Dell Inc. PowerEdge 2950
> >>pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
> >>pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfade0/384 (22 entries)
> >>pcibios0: PCI Interrupt Router at 000:31:0 ("Intel 6321ESB LPC" rev 0x00)
> >>pcibios0: PCI bus #16 is the last bus
> >>bios0: ROM list: 0xc0000/0x9000! 0xc9000/0x1000 0xca000/0x1800
> >>0xcb800/0x5200 0xec000/0x4000!
> >>ipmi0 at mainbus0: version 2.0 interface KCS iobase 0xca8/8 spacing 4
> >>cpu0 at mainbus0
> >>pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
> >>pchb0 at pci0 dev 0 function 0 "Intel 5000X Host" rev 0x12
> >>ppb0 at pci0 dev 2 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci1 at ppb0 bus 6
> >>ppb1 at pci1 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
> >>pci2 at ppb1 bus 7
> >>ppb2 at pci2 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
> >>pci3 at ppb2 bus 8
> >>ppb3 at pci3 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc2
> >>pci4 at ppb3 bus 9
> >>bnx0 at pci4 dev 0 function 0 "Broadcom BCM5708" rev 0x11: irq 5,
> >>address 00:18:8b:72:c0:fd
> >>brgphy0 at bnx0 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 5
> >>ppb4 at pci2 dev 1 function 0 "Intel 6321ESB PCIE" rev 0x01
> >>pci5 at ppb4 bus 10
> >>ppb5 at pci1 dev 0 function 3 "Intel 6321ESB PCIE-PCIX" rev 0x01
> >>pci6 at ppb5 bus 11
> >>ppb6 at pci0 dev 3 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci7 at ppb6 bus 1
> >>ppb7 at pci7 dev 0 function 0 "Intel IOP333 PCIE-PCIX" rev 0x00
> >>pci8 at ppb7 bus 2
> >>mfi0 at pci8 dev 14 function 0 "Dell PERC 5" rev 0x00: irq 6
> >>mfi0: logical drives 1, version 5.0.1-0030, 256MB RAM
> >>scsibus0 at mfi0: 1 targets
> >>sd0 at scsibus0 targ 0 lun 0: <DELL, PERC 5/i, 1.00> SCSI3 0/direct fixed
> >>sd0: 278784MB, 278784 cyl, 64 head, 32 sec, 512 bytes/sec, 570949632 sec
> >>total
> >>ppb8 at pci7 dev 0 function 2 "Intel IOP333 PCIE-PCIX" rev 0x00
> >>pci9 at ppb8 bus 3
> >>ppb9 at pci0 dev 4 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci10 at ppb9 bus 12
> >>ppb10 at pci0 dev 5 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci11 at ppb10 bus 13
> >>ppb11 at pci0 dev 6 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci12 at ppb11 bus 14
> >>ppb12 at pci0 dev 7 function 0 "Intel 5000 PCIE" rev 0x12
> >>pci13 at ppb12 bus 15
> >>pchb1 at pci0 dev 16 function 0 "Intel 5000 Error Reporting" rev 0x12
> >>pchb2 at pci0 dev 16 function 1 "Intel 5000 Error Reporting" rev 0x12
> >>pchb3 at pci0 dev 16 function 2 "Intel 5000 Error Reporting" rev 0x12
> >>pchb4 at pci0 dev 17 function 0 "Intel 5000 Reserved" rev 0x12
> >>pchb5 at pci0 dev 19 function 0 "Intel 5000 Reserved" rev 0x12
> >>pchb6 at pci0 dev 21 function 0 "Intel 5000 FBD" rev 0x12
> >>pchb7 at pci0 dev 22 function 0 "Intel 5000 FBD" rev 0x12
> >>ppb13 at pci0 dev 28 function 0 "Intel 6321ESB PCIE" rev 0x09
> >>pci14 at ppb13 bus 4
> >>ppb14 at pci14 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc2
> >>pci15 at ppb14 bus 5
> >>bnx1 at pci15 dev 0 function 0 "Broadcom BCM5708" rev 0x11: irq 5,
> >>address 00:18:8b:72:c0:fb
> >>brgphy1 at bnx1 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 5
> >>uhci0 at pci0 dev 29 function 0 "Intel 6321ESB USB" rev 0x09: irq 11
> >>usb0 at uhci0: USB revision 1.0
> >>uhub0 at usb0
> >>uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
> >>uhub0: 2 ports with 2 removable, self powered
> >>uhci1 at pci0 dev 29 function 1 "Intel 6321ESB USB" rev 0x09: irq 10
> >>usb1 at uhci1: USB revision 1.0
> >>uhub1 at usb1
> >>uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1
> >>uhub1: 2 ports with 2 removable, self powered
> >>uhci2 at pci0 dev 29 function 2 "Intel 6321ESB USB" rev 0x09: irq 11
> >>usb2 at uhci2: USB revision 1.0
> >>uhub2 at usb2
> >>uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1
> >>uhub2: 2 ports with 2 removable, self powered
> >>ehci0 at pci0 dev 29 function 7 "Intel 6321ESB USB" rev 0x09: irq 11
> >>usb3 at ehci0: USB revision 2.0
> >>uhub3 at usb3
> >>uhub3: Intel EHCI root hub, rev 2.00/1.00, addr 1
> >>uhub3: 6 ports with 6 removable, self powered
> >>ppb15 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0xd9
> >>pci16 at ppb15 bus 16
> >>vga1 at pci16 dev 13 function 0 "ATI ES1000" rev 0x02
> >>wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
> >>wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> >>ichpcib0 at pci0 dev 31 function 0 "Intel 6321ESB LPC" rev 0x09: PM
> disabled
> >>pciide0 at pci0 dev 31 function 1 "Intel 6321ESB IDE" rev 0x09: DMA,
> >>channel 0 configured to compatibility, channel 1 configured to
> >>compatibility
> >>atapiscsi0 at pciide0 channel 0 drive 0
> >>scsibus1 at atapiscsi0: 2 targets
> >>cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, CD-ROM GCR-8240N, 1.10> SCSI0
> >>5/cdrom removable
> >>cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
> >>pciide0: channel 1 ignored (disabled)
> >>isa0 at ichpcib0
> >>isadma0 at isa0
> >>pckbc0 at isa0 port 0x60/5
> >>pckbd0 at pckbc0 (kbd slot)
> >>pckbc0: using irq 1 for kbd slot
> >>wskbd0 at pckbd0: console keyboard, using wsdisplay0
> >>pcppi0 at isa0 port 0x61
> >>midi0 at pcppi0: <PC speaker>
> >>spkr0 at pcppi0
> >>npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
> >>pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> >>pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
> >>biomask ffc5 netmask ffe5 ttymask ffe7
> >>pctr: user-level cycle counter enabled
> >>uhub4 at uhub3 port 5
> >>uhub4: Cypress Semiconductor USB2 Hub, rev 2.00/0.0b, addr 2
> >>uhub4: 4 ports with 4 removable, self powered, multiple transaction
> >>translators
> >>uhidev0 at uhub1 port 1 configuration 1 interface 0
> >>uhidev0: CHESEN PS2 to USB Converter, rev 1.10/0.10, addr 2, iclass 3/1
> >>ukbd0 at uhidev0: 8 modifier keys, 6 key codes
> >>wskbd1 at ukbd0 mux 1
> >>wskbd1: connecting to wsdisplay0
> >>uhidev1 at uhub1 port 1 configuration 1 interface 1
> >>uhidev1: CHESEN PS2 to USB Converter, rev 1.10/0.10, addr 2, iclass 3/1
> >>uhidev1: 3 report ids
> >>ums0 at uhidev1 reportid 1: 5 buttons and Z dir.
> >>wsmouse0 at ums0 mux 0
> >>uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
> >>uhid1 at uhidev1 reportid 3: input=3, output=0, feature=0
> >>dkcsum: sd0 matches BIOS drive 0x80
> >>root on sd0a
> >>rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
> >>WARNING: / was not properly unmounted
