Re: RELENG_8 / mpt / zpool Errors
The SAS 2008 chip (SAS 6G) is the one that the FreeBSD mps driver has problems with when used with port expanders. It's the older SAS 3G chip that works OK with FreeBSD I think. I had crossed some wires earlier on in our discussion. We do have a SAS 2008 chip in the system already, but it's not the one that's running the external disk boxes (and we are having no problem with those drives). The external disk boxes (the ones that are having the problems) are connect through an LSI SAS 3801E, which is handled by the mpt driver. The mpt driver is what's giving us trouble right now. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
Can you connect an enclosure to the SAS 2008 chip card you have now and test? I thought I had read about problems like yours with that chip on FreeBSD when using such enclosures. The SAS 2008 chip we have is built in to the motherboard; it does not have any external connectors. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
I don't know what's out there having chosen only one such card for home use. So you'll need to do your own research. Start with looking at any card with the right chip and then look for evidence that people have used said card with FreeBSD. LSI has the SAS 9200-8e, which is based on the SAS 2008 chipset, which the mps (not the mpt) driver claims to support. We are currently using another controller that uses the SAS 2008 chipset in this same box for the internal OS drives and have not had any problems with it. So, that makes me feel good. Does anyone have any recommendation for or against the 9200-8e? -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
I'm not sure what cards you can get now with the LSI 1068E chip. As LSI branded cards cost more, I went for for the LSI SAS2008 based Supermicro AOC-USAS2-L8i as I new I wasn't going to use port expanders. It could be hard to get cards with the older chip now (you might have to get something second hand). I'm not at all wedded to that card; any card that has two external ports will do for me. The bigger requirement is that it works well with FreeBSD 8.2 and eventually 9.0. If you had your pick of any (reasonably priced) card that had two external ports and would work in my configuration, which would it be? A port expander would be required and just a few older drives in the enclosure. A developer (of which I'm not) would need console access and ability to install new kernels, reboot etc. I can probably swing that, at least for a time, in 3 or 4 weeks. We have hardware on order that I can use to test with for a short while before it goes into production. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
If not, look at paying a small (say $1k) bounty to get it fixed. :) Unfortunately, our budget situation would not allow this sort of thing right now. :\ Buying hardware is a lot easier than paying for something like this. Helping to fix the driver is a thing that Tim is in the unusual situation to be able to do as he has access to the affected hardware. However, if paid work demands a quicker solution, I'm hoping they'll fund use of the SAS 3G card (cheap) whilst allowing Tim to occasionally work with the SAS 6G card to work on a final solution! I can most likely switch to a different card. Just to make sure I get the right one, do you have a part number of the card that you are recommending? As I mentioned earlier, I was looking around on the LSI site and they have lots of options, but none with a port configuration that I need (2 external ports only). I'd be happy to help with debugging the 6G problem. I could probably install that card into another box and give some developers access to it, but I'd have to rustle up some disks and port expanders to make it a fair test. Ideally we don't want Tim to have to move away from FreeBSD. We want him to help fix the driver. I've been a FreeBSD man since 2.2.5, and I won't be switching off any time soon. :) We have several other FreeBSD servers at this point - 5 to be exact, plus my workstation, and there's talk of adopting it more because of Soracle's abandonment of various projects. We're primarily using them for bulk storage - most of them are ZFS with multi-terabyte ZFS file systems - but I've got one that's a workhorse for web and other such things. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
Advice is: Use -STABLE and not -RELEASE Upgrade firmware Avoid port multipliers As I'm using cheap 4K green drives I had to use wdidle3.exe to fix the 8 second head parking and I also had to use gnop to force my ZFS pool to use 4K transfers. We're already using -STABLE; we update to RELENG_8 periodically. LSI's web site is a little bizarre when it comes to their downloads section. I searched their drivers section for my specific model number and got a bunch of firmware upgrades for other cards with different port configurations. We can't avoid port multipliers. Nobody makes a 32-port SAS/SATA controller. And anyhow, the hardware is already purchased so I need to figure out how to make it work. :) WDC's web site says that the wdidle3.exe utility you suggested is not for this drive; the site says it's only for WD1000FYPS-01ZKB0, WD7500AYPS-01ZKB0, and WD7501AYPS-01ZKB0. I'm not sure what you mean about using gnop to force 4K transfers. The mps driver has problems with the LSI SAS2008 when using port multipliers (which are in the enclosures). If you go to the previous SAS 3G version of that chip I think you'll be OK. When you say go to the previous SAS 3G version of that chip, do you mean buy an older version of that controller card? Or are you talking about downgrading, rather than upgrading, the firmware? There are newer LSI cards out there as well. Would upgrading the card help any? The LSI SAS 9205-8e seems to be a workable solution, and not too expensive. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RELENG_8 / mpt / zpool Errors
! -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RELENG_8 / mpt / zpool Errors
What are the drives exactly? You may have issues like TLER or frequent head parking. Are these SATA, SCSI or SAS and are port multipliers in use? root@bsd-03: dmesg | grep da6 da6 at mpt0 bus 0 scbus1 target 1 lun 0 da6: ATA WDC WD2003FYYS-0 1D01 Fixed Direct Access SCSI-5 device da6: 300.000MB/s transfers da6: Command Queueing enabled da6: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C) The drives are housed in two of the following enclosures: ses0 at mpt0 bus 0 scbus1 target 32 lun 0 ses0: PROMISE 3U-SAS-16-D BP 0107 Fixed Enclosure Services SCSI-5 device ses0: 300.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 SES Device Each enclosure is directly connected by a dedicated cable to the 2-port controller: mps0: LSI SAS2008 port 0xfc00-0xfcff mem 0xef5f-0xef5f,0xef58-0xef5b irq 44 at device 0.0 on pci1 mps0: Firmware: 07.15.04.00 mps0: IOCCapabilities: 185cScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR mps0: [ITHREAD] -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
FreeBSD 9 + ZFS + MPS
I'm seeing some odd behavior on FreeBSD 9 with ZFS and an MPS controller. For two or three weeks, this configuration was working like a champ. It's a Bacula storage daemon server and was getting considerable usage - our backup set is in the multiple-terabytes-per-week of data being written to the disk. However, during the last two weeks something has broken. Starting on about July 15th or so we started to loose the data zpool which is connected via the mps driver. The disks are an external array of 32 drives, etc 2TB. When I did a zpool status, I saw that some of the drives had dropped out of the array. Rebooting the server brought them back until the next heavy write time. On July 22nd I did a make world to see if there were any kernel updates that may have fixed the problem, but any updates that may have happened did not seem to help. I'm including the last entry in my dmesg.yesterday file below. Have I stumbled upon a bug? lock order reversal: 1st 0xfe0019cf6db8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:829 2nd 0xfe0019cf69f8 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2134 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x807 __lockmgr_args() at __lockmgr_args+0xd42 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x47 vget() at vget+0x7b devfs_allocv() at devfs_allocv+0x13f devfs_root() at devfs_root+0x4d vfs_donmount() at vfs_donmount+0x988 nmount() at nmount+0x63 syscallenter() at syscallenter+0x1aa syscall() at syscall+0x4c Xfast_syscall() at Xfast_syscall+0xdd --- syscall (378, FreeBSD ELF64, nmount), rip = 0x800ab4dfc, rsp = 0x7fffccc8, rbp = 0x801009048 --- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9
it is nice that you give 9 a chance, but there is LSI SAS2008 support in 8.2, too. I tried booting off an 8.2 release disc and it did not see the controller. I don't have a dmesg from booting off that disc so I can't offer much more detail other than what I have in my dmesg now: mps0: LSI SAS2008 port 0xfc00-0xfcff mem 0xef5f-0xef5f,0xef58-0xef5b irq 44 at device 0.0 on pci1 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9
More specifically, it's in 8.2-STABLE. mps driver was MFC'ed on Feb 18th, 2011. It didn't make it into 8.2-RELEASE. Try mfsBSD image from http://mfsbsd.vx.sk/ -- it may work better. That's what I used. I used the zpool 15 versions though; maybe the zpool 28 version had the new driver on it? At any rate, 9 is working for now, except for the net-snmp compilation problem which I got around by using the binary package. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
FreeBSD 9
Hi, I've installed FreeBSD 9 on a new server because 8.2 doesn't have support for the LSI SAS2008 controller. I've also built the system as a ZFS-root box, and I have to say that I'm quite happy with the disk performance: we're getting about 500MB/s write and 675MB/s read. All in all, I'm very happy with FreeBSD 9. I have noticed two snafus that I thought I'd send to the group just as feedback: 1. net-snmp fails to compile with the following error: /bin/sh ../../libtool --mode=compile cc -I../../include -I. -I../../agent -I../../agent/mibgroup -I../../snmplib -I/usr/include -O2 -pipe -fno-strict-aliasing -Ufreebsd9 -Dfreebsd9=freebsd9 -c -o mibII/tcpTable.lo mibII/tcpTable.c libtool: compile: cc -I../../include -I. -I../../agent -I../../agent/mibgroup -I../../snmplib -I/usr/include -O2 -pipe -fno-strict-aliasing -Ufreebsd9 -Dfreebsd9=freebsd9 -c mibII/tcpTable.c -fPIC -DPIC -o mibII/.libs/tcpTable.o mibII/tcpTable.c:94: error: field 'pcb' has incomplete type mibII/tcpTable.c: In function 'tcpTable_load': mibII/tcpTable.c:866: error: dereferencing pointer to incomplete type mibII/tcpTable.c:868: error: dereferencing pointer to incomplete type mibII/tcpTable.c:868: error: invalid application of 'sizeof' to incomplete type 'struct xinpgen' mibII/tcpTable.c:872: error: dereferencing pointer to incomplete type mibII/tcpTable.c:876: error: dereferencing pointer to incomplete type mibII/tcpTable.c:877: error: invalid application of 'sizeof' to incomplete type 'struct inpcb' mibII/tcpTable.c:881: error: dereferencing pointer to incomplete type 2. secondary IP addresses on network interfaces don't seem to be working In my rc.conf, I have: ifconfig_bce0=1.2.3.4 netmask 255.255.252.0 ifconfig_bce0_alias0=1.2.3.5 netmask 255.255.255.255 but when the machine reboots, it only gets its primary IP address. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9
I think you need something along the line of ifconfig_bce0_alias0=inet 1.2.3.5 netmask ..., notice the 'inet', since aliasN can be used for both inet and inet6. Got it, thanks! I assume that's also recommended for the primary interface as well? I've added the inet prefix to both lines and it is working. Thanks! -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9
Someone already filed a PR for this ( http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/158266 ) and I'm working on cleaning up the autoconf tests to work properly for $WORK so we can upgrade to 5.6.1.1. The problem is caused by the recent netinet / net content shuffling and the fact that the autoconf tests for net-snmp are broken (and a number of includes on files). Unfortunately the upstream maintainers used a sledgehammer approach for all of the BSDs to detect how headers were supposed to be #include'd, and there's a lot of namespace pollution involved. Thanks! I've just installed the binary package for now which is working. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tim Gustafsont...@soe.ucsc.edu Baskin School of Engineering 831-459-5354 UC Santa Cruz Baskin Engineering 317B -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
