Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Fri, Jun 26, 2009 at 4:11 AM, Eric D. Mudama wrote: > True. In $ per sequential GB/s, rotating rust still wins by far. > However, your comment about all flash being slower than rotating at > sequential writes was mistaken. Even at 10x the price, if you're > working with a dataset that needs random IO, the $ per IOP from flash > can be significantly greater than any amount of rust, and typically > with much lower power consumption to boot. > > Obviously the primary benefits of SSDs aren't in sequential > reads/writes, but they're not necessarilly complete dogs there either. > It's all about iops. HDD can do about 300 iops, SSD can get up to 10k+ iops. On sequential writes obviously low iops is not a problem - 300 x 128kB is 40MB. But for small packet random sync NFS traffic 300 * 32kb is hardly a 1MB/s. Nicholas ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Wed, Jun 24 at 18:43, Bob Friesenhahn wrote: On Wed, 24 Jun 2009, Eric D. Mudama wrote: The main purpose for using SSDs with ZFS is to reduce latencies for synchronous writes required by network file service and databases. In the "available 5 months ago" category, the Intel X25-E will write sequentially at ~170MB/s according to the datasheets. That is faster than most, if not all rotating media today. Sounds good. Is that is after the whole device has been re-written a few times or just when you first use it? Based on the various review sites, some tests experience a temporary performance decrease when performing sequential IO over the top of previously randomly written data, which resolves in some short time period. I am not convinced that simply writing the devices makes them slower. Actual performance will be workload specific, YMMV. How many of these devices do you own and use? I own two of them personally, and work with many every day. Seagate Cheetah drives can now support a sustained data rate of 204MB/second. That is with 600GB capacity rather than 64GB and at a similar price point (i.e. 10X less cost per GB). Or you can just RAID-0 a few cheaper rotating rust drives and achieve a huge sequential data rate. True. In $ per sequential GB/s, rotating rust still wins by far. However, your comment about all flash being slower than rotating at sequential writes was mistaken. Even at 10x the price, if you're working with a dataset that needs random IO, the $ per IOP from flash can be significantly greater than any amount of rust, and typically with much lower power consumption to boot. Obviously the primary benefits of SSDs aren't in sequential reads/writes, but they're not necessarilly complete dogs there either. --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Wed, 24 Jun 2009, Eric D. Mudama wrote: The main purpose for using SSDs with ZFS is to reduce latencies for synchronous writes required by network file service and databases. In the "available 5 months ago" category, the Intel X25-E will write sequentially at ~170MB/s according to the datasheets. That is faster than most, if not all rotating media today. Sounds good. Is that is after the whole device has been re-written a few times or just when you first use it? How many of these devices do you own and use? Seagate Cheetah drives can now support a sustained data rate of 204MB/second. That is with 600GB capacity rather than 64GB and at a similar price point (i.e. 10X less cost per GB). Or you can just RAID-0 a few cheaper rotating rust drives and achieve a huge sequential data rate. I see that the Intel X25-E claims a sequential read performance of 250 MB/s. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Wed, Jun 24 at 15:38, Bob Friesenhahn wrote: On Wed, 24 Jun 2009, Orvar Korvar wrote: I thought of exchanging my PCI card with a PCIe card variant instead to reach higher speeds. PCI-X is legacy. The problem with PCIe cards is that soon SSD drives will be common. A ZFS raid with SSD would need maybe PCIe x 16 or so, to reach max band width. The PCIe cards are all PCIe x 4 or something of today. I need a PCIe x 16 card to make it future proof for the SSD discs. Maybe the best bet would be to attach a SSD disc directly to a PCIe slot, to reach max transfer speed? Or wait for SATA 3? I dont know. I want to wait until SSD I don't think this is valid thinking because it assumes that write rates for SSDs are higher than for traditional hard drives. This assumption is not often correct. Maybe someday. SSDs offer much lower write latencies (no head seek!) but their bulk sequential data transfer properties are not yet better than hard drives. The main purpose for using SSDs with ZFS is to reduce latencies for synchronous writes required by network file service and databases. In the "available 5 months ago" category, the Intel X25-E will write sequentially at ~170MB/s according to the datasheets. That is faster than most, if not all rotating media today. --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Wed, 24 Jun 2009, Orvar Korvar wrote: I thought of exchanging my PCI card with a PCIe card variant instead to reach higher speeds. PCI-X is legacy. The problem with PCIe cards is that soon SSD drives will be common. A ZFS raid with SSD would need maybe PCIe x 16 or so, to reach max band width. The PCIe cards are all PCIe x 4 or something of today. I need a PCIe x 16 card to make it future proof for the SSD discs. Maybe the best bet would be to attach a SSD disc directly to a PCIe slot, to reach max transfer speed? Or wait for SATA 3? I dont know. I want to wait until SSD I don't think this is valid thinking because it assumes that write rates for SSDs are higher than for traditional hard drives. This assumption is not often correct. Maybe someday. SSDs offer much lower write latencies (no head seek!) but their bulk sequential data transfer properties are not yet better than hard drives. The main purpose for using SSDs with ZFS is to reduce latencies for synchronous writes required by network file service and databases. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Hey sbreden! :o) No, I havent tried to tinker with my drives. They have been functioning all the time. I suspect (can not remember) that each SATA slot in the card has a number attached to it? Can anyone confirm this? If I am right, OpenSolaris will say something about "disc 6 is broken" and on the card there is a number 6? Then you can identify the disc? I thought of exchanging my PCI card with a PCIe card variant instead to reach higher speeds. PCI-X is legacy. The problem with PCIe cards is that soon SSD drives will be common. A ZFS raid with SSD would need maybe PCIe x 16 or so, to reach max band width. The PCIe cards are all PCIe x 4 or something of today. I need a PCIe x 16 card to make it future proof for the SSD discs. Maybe the best bet would be to attach a SSD disc directly to a PCIe slot, to reach max transfer speed? Or wait for SATA 3? I dont know. I want to wait until SSD raids are tested out. Then I will buy an apropriate card capable of SSD raids. Maybe SSD discs should never be used in conjunction with a card, and always connect directly to the SATA port? Until I know more on this, my PCI card will be fine. 150MB/sec is ok for my personal needs. (My ZFS raid is connected to my Desktop PC. I dont have a server that is on 24/7 using power. I want to save power. Save the earth! :o) All my 5 ZFS raid discs are connected to one Molex. That molex has a power switch. So I just turn on the ZFS raid and copy all files I need to my system disc (which is 500GB) and then immediately reboot and turn off the ZFS raid. This way I only have one disc active, which I use as a cache. When my data are ready, I copy them to the ZFS raid and then shut down the power to the ZFS raid discs.) However I have a question. Which speed will I get with this solution. I have 2 SSD discs in a PCI slot => 150MB/sec. Now I add 1 SSD disc into a SATA slot and another SSD disc into another SATA slot. Then I have 5 disc in PCI => 150MB/sec 1 disc in SATA => 300MB/sec (I assume SATA reach 300MB/sec?) 1 disc in SATA => 300MB/sec. I connect all the 7 discs into one ZFS raid. Which speed will I get? Will I get 150 + 300 + 300MB/sec? Or will the PCI slot strangle the SATA ports? Or will the fastest speed "win" and I will only get 300MB/sec? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Mon, 22 Jun 2009 15:28:08 -0700 Carson Gaspar wrote: > James C. McPherson wrote: > > > Use raidctl(1m). For fwflash(1m), this is on the "future project" > > list purely because we've got much higher priority projects on the > > boil - if we couldn't use raidctl(1m) this would be higher up the > > list. > > Nice to see that raidctl can do that. Although I don't see a way to > flash the BIOS or fcode with raidctl... am I missing something, is it a > doc bug, or is it not possible? The man page intro mentions BIOS and > fcode, but the only option I can see is '-F' and it just says firmware... We include both bios and fcode in the definition of firmware. The manpage for raidctl(1m) also gives an example: Example 4 Updating Flash Images on the Controller The following command updates flash images on the controller 0: # raidctl -F lsi_image.fw 0 ... > > Did you try "cfgadm -lav" ? I was under the impression that the > > cfgadm(1m) manpage's examples section was sufficient to provide > > at least a starting point for a usable command line. > > > > If you don't believe that is the case, I'd appreciate you filing > > a bug against it (yes, we do like to get doc/manpage bugs) so that > > we can make the manpage better. > ... > > $ cfgadm -alv c0 c3 > > Ap_Id Receptacle Occupant Condition > > Information > > When Type Busy Phys_Id > > c0 connectedconfigured unknown > > unavailable scsi-bus n > > /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi > > c0::dsk/c0t4d0 connectedconfigured unknown > > ST3320620AS ST3320620AS > > unavailable disk n > > /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t4d0 > > That gives the same data as 'ls -l /dev/dsk/c0t4d0'. It does _not_ give > the LSI HBA port number. And given the plethora of device mapping > options in the LSI controller, you really want the real port numbers. I don't see why that makes a difference to you, and I'd be grateful if you'd clue me in on that. I only know of two device mapping options for the 1064/1068-based cards, which are "logical target id" and "SAS WWN". We use the "logical target id" method with mpt(7d). > As for the man page, for a basic "give me a list of devices" the man > page is overly complex and verbose, but sufficient. It's all the _other_ > options that are documented to exist, but without any specifics. It all > basically reads as "reserved for future use". There are other manpages referred to in the SEE ALSO section of the cfgadm(1m) manpage, just like with other manpages: SEE ALSO cfgadm_fp(1M), cfgadm_ib(1M), cfgadm_pci(1M),cfgadm_sbd(1M), cfgadm_scsi(1M), cfgadm_usb(1M), ifconfig(1M), mount(1M), prtdiag(1M), psradm(1M), syslogd(1M), config_admin(3CFGADM), getopt(3C),getsubopt(3C), isatty(3C), attributes(5), environ(5) What else are you thinking of as "reserved for future use" ? James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Erik Ableson wrote: The problem I had was with the single raid 0 volumes (miswrote RAID 1 on the original message) This is not a straight to disk connection and you'll have problems if you ever need to move disks around or move them to another controller. Would you mind explaining exactly what issues or problems you had ? I have moved disks around several controllers without problems. You must remember however to create the RAID 0 lun throught LSI's megaraid CLI tool and / or to clear any foreign config before the controller will expose the disk(s) to the OS. The only real problem that I can think of is that you cannot use the autoreplace functionality of recent ZFS versions with these controllers. I agree that the MD1000 with ZFS is a rocking, inexpensive setup (we have several!) but I'd recommend using a SAS card with a true JBOD mode for maximum flexibility and portability. If I remember correctly, I think we're using the Adaptec 3085. I've pulled 465MB/s write and 1GB/s read off the MD1000 filled with SATA drives. Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 21:18, Henrik Johansen wrote: Kyle McDonald wrote: Erik Ableson wrote: Just a side note on the PERC labelled cards: they don't have a JBOD mode so you _have_ to use hardware RAID. This may or may not be an issue in your configuration but it does mean that moving disks between controllers is no longer possible. The only way to do a pseudo JBOD is to create broken RAID 1 volumes which is not ideal. It won't even let you make single drive RAID 0 LUNs? That's a shame. We currently have 90+ disks that are created as single drive RAID 0 LUNs on several PERC 6/E (LSI 1078E chipset) controllers and used by ZFS. I can assure you that they work without any problems and perform very well indeed. In fact, the combination of PERC 6/E and MD1000 disk arrays has worked so well for us that we are going to double the number of disks during this fall. The lack of portability is disappointing. The trade-off though is battery backed cache if the card supports it. -Kyle Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 04:33, "Eric D. Mudama" > wrote: > On Mon, Jun 22 at 15:46, Miles Nordin wrote: >>> "edm" == Eric D Mudama writes: >> >> edm> We bought a Dell T610 as a fileserver, and it comes with an >> edm> LSI 1068E based board (PERC6/i SAS). >> >> which driver attaches to it? >> >> pciids.sourceforge.net says this is a 1078 board, not a 1068 board. >> >> please, be careful. There's too much confusion about these cards. > > Sorry, that may have been confusing. We have the cheapest storage > option on the T610, with no onboard cache. I guess it's called the > "Dell SAS6i/R" while they reserve the PERC name for the ones with > cache. I had understood that they were basically identical except for > the cache, but maybe not. > > Anyway, this adapter has worked great for us so far. > > > snippet of prtconf -D: > > > i86pc (driver name: rootnex) >pci, instance #0 (driver name: npe) >pci8086,3411, instance #6 (driver name: pcie_pci) >pci1028,1f10, instance #0 (driver name: mpt) >sd, instance #1 (driver name: sd) >sd, instance #6 (driver name: sd) >sd, instance #7 (driver name: sd) >sd, instance #2 (driver name: sd) >sd, instance #4 (driver name: sd) >sd, instance #5 (driver name: sd) > > > For this board the mpt driver is being used, and here's the prtconf > -pv info: > > > Node 0x1f >assigned-addresses: > 81020010..fc00..0100.83020014.. > df2ec000..4000.8302001c. > .df2f..0001 >reg: > 0002.....01020010....0100.03020014....4000.0302001c. > ...0001 >compatible: 'pciex1000,58.1028.1f10.8' + 'pciex1000,58.1028.1f10' > + 'pciex1000,58.8' + 'pciex1000,58' + 'pciexclass,01' + > 'pciexclass,0100' + 'pci1000,58.1028.1f10.8' + > 'pci1000,58.1028.1f10' + 'pci1028,1f10' + 'pci1000,58.8' + > 'pci1000,58' + 'pciclass, 01' + 'pciclass,0100' >model: 'SCSI bus controller' >power-consumption: 0001.0001 >devsel-speed: >interrupts: 0001 >subsystem-vendor-id: 1028 >subsystem-id: 1f10 >unit-address: '0' >class-code: 0001 >revision-id: 0008 >vendor-id: 1000 >device-id: 0058 >pcie-capid-pointer: 0068 >pcie-capid-reg: 0001 >name: 'pci1028,1f10' > > > --eric > > > -- > Eric D. Mudama > edmud...@mail.bounceswoosh.org > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/l
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
The problem I had was with the single raid 0 volumes (miswrote RAID 1 on the original message) This is not a straight to disk connection and you'll have problems if you ever need to move disks around or move them to another controller. I agree that the MD1000 with ZFS is a rocking, inexpensive setup (we have several!) but I'd recommend using a SAS card with a true JBOD mode for maximum flexibility and portability. If I remember correctly, I think we're using the Adaptec 3085. I've pulled 465MB/s write and 1GB/s read off the MD1000 filled with SATA drives. Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 21:18, Henrik Johansen wrote: Kyle McDonald wrote: Erik Ableson wrote: Just a side note on the PERC labelled cards: they don't have a JBOD mode so you _have_ to use hardware RAID. This may or may not be an issue in your configuration but it does mean that moving disks between controllers is no longer possible. The only way to do a pseudo JBOD is to create broken RAID 1 volumes which is not ideal. It won't even let you make single drive RAID 0 LUNs? That's a shame. We currently have 90+ disks that are created as single drive RAID 0 LUNs on several PERC 6/E (LSI 1078E chipset) controllers and used by ZFS. I can assure you that they work without any problems and perform very well indeed. In fact, the combination of PERC 6/E and MD1000 disk arrays has worked so well for us that we are going to double the number of disks during this fall. The lack of portability is disappointing. The trade-off though is battery backed cache if the card supports it. -Kyle Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 04:33, "Eric D. Mudama" > wrote: > On Mon, Jun 22 at 15:46, Miles Nordin wrote: >>> "edm" == Eric D Mudama writes: >> >> edm> We bought a Dell T610 as a fileserver, and it comes with an >> edm> LSI 1068E based board (PERC6/i SAS). >> >> which driver attaches to it? >> >> pciids.sourceforge.net says this is a 1078 board, not a 1068 board. >> >> please, be careful. There's too much confusion about these cards. > > Sorry, that may have been confusing. We have the cheapest storage > option on the T610, with no onboard cache. I guess it's called the > "Dell SAS6i/R" while they reserve the PERC name for the ones with > cache. I had understood that they were basically identical except for > the cache, but maybe not. > > Anyway, this adapter has worked great for us so far. > > > snippet of prtconf -D: > > > i86pc (driver name: rootnex) >pci, instance #0 (driver name: npe) >pci8086,3411, instance #6 (driver name: pcie_pci) >pci1028,1f10, instance #0 (driver name: mpt) >sd, instance #1 (driver name: sd) >sd, instance #6 (driver name: sd) >sd, instance #7 (driver name: sd) >sd, instance #2 (driver name: sd) >sd, instance #4 (driver name: sd) >sd, instance #5 (driver name: sd) > > > For this board the mpt driver is being used, and here's the prtconf > -pv info: > > > Node 0x1f >assigned-addresses: > 81020010..fc00..0100.83020014.. > df2ec000..4000.8302001c. > .df2f..0001 >reg: > 0002.....01020010....0100.03020014....4000.0302001c. > ...0001 >compatible: 'pciex1000,58.1028.1f10.8' + 'pciex1000,58.1028.1f10' > + 'pciex1000,58.8' + 'pciex1000,58' + 'pciexclass,01' + > 'pciexclass,0100' + 'pci1000,58.1028.1f10.8' + > 'pci1000,58.1028.1f10' + 'pci1028,1f10' + 'pci1000,58.8' + > 'pci1000,58' + 'pciclass, 01' + 'pciclass,0100' >model: 'SCSI bus controller' >power-consumption: 0001.0001 >devsel-speed: >interrupts: 0001 >subsystem-vendor-id: 1028 >subsystem-id: 1f10 >unit-address: '0' >class-code: 0001 >revision-id: 0008 >vendor-id: 1000 >device-id: 0058 >pcie-capid-pointer: 0068 >pcie-capid-reg: 0001 >name: 'pci1028,1f10' > > > --eric > > > -- > Eric D. Mudama > edmud...@mail.bounceswoosh.org > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Med venlig hilsen / Best Regards Henrik Johansen hen...@scannet.dk Tlf. 75 53 35 00 ScanNet Group A/S ScanNet _
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Erik Ableson wrote: Just a side note on the PERC labelled cards: they don't have a JBOD mode so you _have_ to use hardware RAID. This may or may not be an issue in your configuration but it does mean that moving disks between controllers is no longer possible. The only way to do a pseudo JBOD is to create broken RAID 1 volumes which is not ideal. It won't even let you make single drive RAID 0 LUNs? That's a shame. The lack of portability is disappointing. The trade-off though is battery backed cache if the card supports it. -Kyle Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 04:33, "Eric D. Mudama" wrote: > On Mon, Jun 22 at 15:46, Miles Nordin wrote: >>> "edm" == Eric D Mudama writes: >> >> edm> We bought a Dell T610 as a fileserver, and it comes with an >> edm> LSI 1068E based board (PERC6/i SAS). >> >> which driver attaches to it? >> >> pciids.sourceforge.net says this is a 1078 board, not a 1068 board. >> >> please, be careful. There's too much confusion about these cards. > > Sorry, that may have been confusing. We have the cheapest storage > option on the T610, with no onboard cache. I guess it's called the > "Dell SAS6i/R" while they reserve the PERC name for the ones with > cache. I had understood that they were basically identical except for > the cache, but maybe not. > > Anyway, this adapter has worked great for us so far. > > > snippet of prtconf -D: > > > i86pc (driver name: rootnex) >pci, instance #0 (driver name: npe) >pci8086,3411, instance #6 (driver name: pcie_pci) >pci1028,1f10, instance #0 (driver name: mpt) >sd, instance #1 (driver name: sd) >sd, instance #6 (driver name: sd) >sd, instance #7 (driver name: sd) >sd, instance #2 (driver name: sd) >sd, instance #4 (driver name: sd) >sd, instance #5 (driver name: sd) > > > For this board the mpt driver is being used, and here's the prtconf > -pv info: > > > Node 0x1f >assigned-addresses: > 81020010..fc00..0100.83020014.. > df2ec000..4000.8302001c. > .df2f..0001 >reg: > 0002.....01020010....0100.03020014....4000.0302001c. > ...0001 >compatible: 'pciex1000,58.1028.1f10.8' + 'pciex1000,58.1028.1f10' > + 'pciex1000,58.8' + 'pciex1000,58' + 'pciexclass,01' + > 'pciexclass,0100' + 'pci1000,58.1028.1f10.8' + > 'pci1000,58.1028.1f10' + 'pci1028,1f10' + 'pci1000,58.8' + > 'pci1000,58' + 'pciclass,01' + 'pciclass,0100' >model: 'SCSI bus controller' >power-consumption: 0001.0001 >devsel-speed: >interrupts: 0001 >subsystem-vendor-id: 1028 >subsystem-id: 1f10 >unit-address: '0' >class-code: 0001 >revision-id: 0008 >vendor-id: 1000 >device-id: 0058 >pcie-capid-pointer: 0068 >pcie-capid-reg: 0001 >name: 'pci1028,1f10' > > > --eric > > > -- > Eric D. Mudama > edmud...@mail.bounceswoosh.org > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Just a side note on the PERC labelled cards: they don't have a JBOD mode so you _have_ to use hardware RAID. This may or may not be an issue in your configuration but it does mean that moving disks between controllers is no longer possible. The only way to do a pseudo JBOD is to create broken RAID 1 volumes which is not ideal. Cordialement, Erik Ableson +33.6.80.83.58.28 Envoyé depuis mon iPhone On 23 juin 2009, at 04:33, "Eric D. Mudama" wrote: On Mon, Jun 22 at 15:46, Miles Nordin wrote: "edm" == Eric D Mudama writes: edm> We bought a Dell T610 as a fileserver, and it comes with an edm> LSI 1068E based board (PERC6/i SAS). which driver attaches to it? pciids.sourceforge.net says this is a 1078 board, not a 1068 board. please, be careful. There's too much confusion about these cards. Sorry, that may have been confusing. We have the cheapest storage option on the T610, with no onboard cache. I guess it's called the "Dell SAS6i/R" while they reserve the PERC name for the ones with cache. I had understood that they were basically identical except for the cache, but maybe not. Anyway, this adapter has worked great for us so far. snippet of prtconf -D: i86pc (driver name: rootnex) pci, instance #0 (driver name: npe) pci8086,3411, instance #6 (driver name: pcie_pci) pci1028,1f10, instance #0 (driver name: mpt) sd, instance #1 (driver name: sd) sd, instance #6 (driver name: sd) sd, instance #7 (driver name: sd) sd, instance #2 (driver name: sd) sd, instance #4 (driver name: sd) sd, instance #5 (driver name: sd) For this board the mpt driver is being used, and here's the prtconf -pv info: Node 0x1f assigned-addresses: 81020010..fc00..0100.83020014.. df2ec000..4000.8302001c. .df2f..0001 reg: 0002.....01020010....0100.03020014....4000.0302001c. ...0001 compatible: 'pciex1000,58.1028.1f10.8' + 'pciex1000,58.1028.1f10' + 'pciex1000,58.8' + 'pciex1000,58' + 'pciexclass,01' + 'pciexclass,0100' + 'pci1000,58.1028.1f10.8' + 'pci1000,58.1028.1f10' + 'pci1028,1f10' + 'pci1000,58.8' + 'pci1000,58' + 'pciclass,01' + 'pciclass,0100' model: 'SCSI bus controller' power-consumption: 0001.0001 devsel-speed: interrupts: 0001 subsystem-vendor-id: 1028 subsystem-id: 1f10 unit-address: '0' class-code: 0001 revision-id: 0008 vendor-id: 1000 device-id: 0058 pcie-capid-pointer: 0068 pcie-capid-reg: 0001 name: 'pci1028,1f10' --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Mon, Jun 22 at 15:46, Miles Nordin wrote: "edm" == Eric D Mudama writes: edm> We bought a Dell T610 as a fileserver, and it comes with an edm> LSI 1068E based board (PERC6/i SAS). which driver attaches to it? pciids.sourceforge.net says this is a 1078 board, not a 1068 board. please, be careful. There's too much confusion about these cards. Sorry, that may have been confusing. We have the cheapest storage option on the T610, with no onboard cache. I guess it's called the "Dell SAS6i/R" while they reserve the PERC name for the ones with cache. I had understood that they were basically identical except for the cache, but maybe not. Anyway, this adapter has worked great for us so far. snippet of prtconf -D: i86pc (driver name: rootnex) pci, instance #0 (driver name: npe) pci8086,3411, instance #6 (driver name: pcie_pci) pci1028,1f10, instance #0 (driver name: mpt) sd, instance #1 (driver name: sd) sd, instance #6 (driver name: sd) sd, instance #7 (driver name: sd) sd, instance #2 (driver name: sd) sd, instance #4 (driver name: sd) sd, instance #5 (driver name: sd) For this board the mpt driver is being used, and here's the prtconf -pv info: Node 0x1f assigned-addresses: 81020010..fc00..0100.83020014..df2ec000..4000.8302001c..df2f..0001 reg: 0002.....01020010....0100.03020014....4000.0302001c....0001 compatible: 'pciex1000,58.1028.1f10.8' + 'pciex1000,58.1028.1f10' + 'pciex1000,58.8' + 'pciex1000,58' + 'pciexclass,01' + 'pciexclass,0100' + 'pci1000,58.1028.1f10.8' + 'pci1000,58.1028.1f10' + 'pci1028,1f10' + 'pci1000,58.8' + 'pci1000,58' + 'pciclass,01' + 'pciclass,0100' model: 'SCSI bus controller' power-consumption: 0001.0001 devsel-speed: interrupts: 0001 subsystem-vendor-id: 1028 subsystem-id: 1f10 unit-address: '0' class-code: 0001 revision-id: 0008 vendor-id: 1000 device-id: 0058 pcie-capid-pointer: 0068 pcie-capid-reg: 0001 name: 'pci1028,1f10' --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
James C. McPherson wrote: Use raidctl(1m). For fwflash(1m), this is on the "future project" list purely because we've got much higher priority projects on the boil - if we couldn't use raidctl(1m) this would be higher up the list. Nice to see that raidctl can do that. Although I don't see a way to flash the BIOS or fcode with raidctl... am I missing something, is it a doc bug, or is it not possible? The man page intro mentions BIOS and fcode, but the only option I can see is '-F' and it just says firmware... Although both lsiutil and hd produce errors from mpt when trying to get SMART data (specifically "Command failed with IOCStatus = 0045 (Data Underrun)"). I haven't tried using the LSI provided drivers. Is "hd" a utility from the x4500 software suite? Yes. It's the only Sun provided tool I know of that will dump detailed SMART info (even on non-X45x0 hosts). Did you try "cfgadm -lav" ? I was under the impression that the cfgadm(1m) manpage's examples section was sufficient to provide at least a starting point for a usable command line. If you don't believe that is the case, I'd appreciate you filing a bug against it (yes, we do like to get doc/manpage bugs) so that we can make the manpage better. ... $ cfgadm -alv c0 c3 Ap_Id Receptacle Occupant Condition Information When Type Busy Phys_Id c0 connectedconfigured unknown unavailable scsi-bus n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi c0::dsk/c0t4d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t4d0 That gives the same data as 'ls -l /dev/dsk/c0t4d0'. It does _not_ give the LSI HBA port number. And given the plethora of device mapping options in the LSI controller, you really want the real port numbers. As for the man page, for a basic "give me a list of devices" the man page is overly complex and verbose, but sufficient. It's all the _other_ options that are documented to exist, but without any specifics. It all basically reads as "reserved for future use". -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Also, is anybody using the AOC-USAS-L8i? If so, what's your experience of it, and identifying drives and replacing failed drives with it? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Thanks guys, keep your experiences coming. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
> "edm" == Eric D Mudama writes: edm> We bought a Dell T610 as a fileserver, and it comes with an edm> LSI 1068E based board (PERC6/i SAS). which driver attaches to it? pciids.sourceforge.net says this is a 1078 board, not a 1068 board. please, be careful. There's too much confusion about these cards. pgpzNLVmqHXd4.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Mon, 22 Jun 2009 01:25:54 -0700 Carson Gaspar wrote: > James C. McPherson wrote: > > On Sun, 21 Jun 2009 19:01:31 -0700 > > > > As a member of the team which works on mpt(7d), I'm disappointed that\ > > you believe you need to use lsiutil to "fully access all the functionality" > > of the board. > > > > What gaps have you found in mpt(7d) and the standard OpenSolaris > > tools that lsiutil fixes for you? > > > > What is the "full functionality" that you believe is missing? > > How does one upgrade firmware without using lsiutil? Use raidctl(1m). For fwflash(1m), this is on the "future project" list purely because we've got much higher priority projects on the boil - if we couldn't use raidctl(1m) this would be higher up the list. > Or toggle controller LEDs to identify which board is which, or... > Feel free to read the lsiutil docs (bad though they are) - the PDF is > available from the LSI web site. LED stuff ... yeah, not such an easy thing to solve. I believe there has been a fair amount of effort gone into the generic FMA topology "parser" so that we can do this, but I do not know the status of the project. > Although both lsiutil and hd produce errors from mpt when trying to get > SMART data (specifically "Command failed with IOCStatus = 0045 (Data > Underrun)"). I haven't tried using the LSI provided drivers. Is "hd" a utility from the x4500 software suite? > >> As for identifying disks, you can just use lsiutil: > > > > ... or use cfgadm(1m) which has had this ability for many years. > > Great. Please provide a sample command line. Because the man page is > completely useless (no, really - try reading it). And no invocation _I_ > can find provides the same information. I can only assume it's one of > the "hardware specific" options, which are documented nowhere that I can > find. Did you try "cfgadm -lav" ? I was under the impression that the cfgadm(1m) manpage's examples section was sufficient to provide at least a starting point for a usable command line. If you don't believe that is the case, I'd appreciate you filing a bug against it (yes, we do like to get doc/manpage bugs) so that we can make the manpage better. > Note that my comments all releate to Solaris 10 u7 - it's certainly > possible that things are better in OpenSolaris. $ cfgadm -alv c0 c3 Ap_Id Receptacle Occupant Condition Information When Type Busy Phys_Id c0 connectedconfigured unknown unavailable scsi-bus n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi c0::dsk/c0t4d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t4d0 c0::dsk/c0t5d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t5d0 c0::dsk/c0t6d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t6d0 c0::dsk/c0t7d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c0t7d0 c3 connectedconfigured unknown unavailable scsi-bus n /devices/p...@79,0/pci10de,3...@a/pci1000,3...@0:scsi c3::dsk/c3t3d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@79,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c3t3d0 c3::dsk/c3t5d0 connectedconfigured unknownSAMSUNG HD321KJ unavailable disk n /devices/p...@79,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c3t5d0 c3::dsk/c3t6d0 connectedconfigured unknownWDC WD3200AAKS-00VYA0 unavailable disk n /devices/p...@79,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c3t6d0 c3::dsk/c3t7d0 connectedconfigured unknownST3320620AS ST3320620AS unavailable disk n /devices/p...@79,0/pci10de,3...@a/pci1000,3...@0:scsi::dsk/c3t7d0 That functionality has been in Solaris 10 since FCS. The manpage for cfgadm(1m) indicates that it was last changed in October 2004, which is a good 6 months prior to FCS of Solaris 10. If you don't like it, and don't tell us, how are we supposed to know that it needs improving? James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
James C. McPherson wrote: On Sun, 21 Jun 2009 19:01:31 -0700 As a member of the team which works on mpt(7d), I'm disappointed that\ you believe you need to use lsiutil to "fully access all the functionality" of the board. What gaps have you found in mpt(7d) and the standard OpenSolaris tools that lsiutil fixes for you? What is the "full functionality" that you believe is missing? How does one upgrade firmware without using lsiutil? Or toggle controller LEDs to identify which board is which, or... Feel free to read the lsiutil docs (bad though they are) - the PDF is available from the LSI web site. Although both lsiutil and hd produce errors from mpt when trying to get SMART data (specifically "Command failed with IOCStatus = 0045 (Data Underrun)"). I haven't tried using the LSI provided drivers. As for identifying disks, you can just use lsiutil: ... or use cfgadm(1m) which has had this ability for many years. Great. Please provide a sample command line. Because the man page is completely useless (no, really - try reading it). And no invocation _I_ can find provides the same information. I can only assume it's one of the "hardware specific" options, which are documented nowhere that I can find. Note that my comments all releate to Solaris 10 u7 - it's certainly possible that things are better in OpenSolaris. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Mon, Jun 22 at 12:05, Andre van Eyssen wrote: I'll add another vote for the LSI products. I have a four port PCI-X card in my V880, and the performance is good and the product is well behaved. The only caveats: 1. Make sure you upgrade the firmware ASAP 2. You may need to use lsiutil to fiddle the target mappings We bought a Dell T610 as a fileserver, and it comes with an LSI 1068E based board (PERC6/i SAS). Worked out of the box, no special drivers or anything to install, everything autodetected just fine. Hotplug works great too, I've yanked drives (Came with WD RE3 SASA devices) while the box was under load without issues, took ~5 seconds to timeout the device and give me full interactivity at the console. They then show right back up when hot plugged back in, and I can resilver without problems. --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Jorgen Lundman wrote: I only have a 32bit PCI bus in the Intel Atom 330 board, so I have no choice than to be "slower", but I can confirm that the Supermicro dac-sata-mv8 (SATA-1) card works just fine, and does display in cfgadm. (Hot-swapping is possible). I have been told aoc-sat2-mv8 does as well (SATA-II) but I have not personally tried it. Lund I have an AOC-SAT2-MV8 in an older Opteron-based system. It's a 2-socket, Opteron 252 system with 8GB of RAM, and PCI-X slots. It's one of the newer AOC cards with the latest Marvell chipset, and it works like a champ - very well, smooth, and I don't see any problems. Simple, out-of-the-box installation and works with no tinkering at all (with OS 2008.11 and 2009.05). That said, there's a couple of things you want to be aware of about the AOC: (1) it uses normal sata cables. This is really nice in terms of availability (you can get any length you want easily at any computer store), but it's a bit messy compared to the nice multi-lane ones. (2) It's a PCI-X card, and will run at 133Mhz. I have a second gigabit ethernet card in my motherboard, which limits the two PCI-X cards to 100Mhz.The down side is that with 8 drives and 2 gigabit ethernet interfaces, it's not hard to flood the PCI-X bus (which can pump 100Mhz x 64bit = 6400 Mbps max, but about 50% of that under real usage) (3) as a PCI-X card, it's a "two-third's" length, low-profile card. It will fit in any PCI-X slot you have. However, if you are trying to put it in a 32-bit PCI slot, be aware that it extends about 2 inches (50mm) beyond the back of the PCI slot. Make sure you have the proper clearances for such a card. Also, it's a 3.3v card (won't work in 5v slots). None of this should be a problem in any modern motherboard/case setup, only in really old stuff. (4) All the SATA connectors are on the end of the card, which means you'll need _at least_ another 1 inch (25mm) clearance to plug the cables in. As much as I like the card, these days, I'd chose the LSI PCI-E model. The PCI-E bus is just superior to PCI-X - you get much less bus contention which means it's easier to get full throughput from each card One more thing: I've found that the newest MLC-based SSDs with the newer "barefoot" SATA controller and 64MB or more of cache are more than suitable for use as Read cache, and they actually do OK as write-cache, too. Particularly for small business server machine (those that have have 8-12 data drives, total). And, these days, there's nice little funky dual-2.5 drives in a floppy form-factor things. http://www.addonics.com/products/mobile_rack/doubledrive.asp Example new SSD for Readzilla/Logzilla : http://www.ocztechnology.com/products/flash_drives/ocz_summit_series_sata_ii_2_5-ssd -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
I only have a 32bit PCI bus in the Intel Atom 330 board, so I have no choice than to be "slower", but I can confirm that the Supermicro dac-sata-mv8 (SATA-1) card works just fine, and does display in cfgadm. (Hot-swapping is possible). I have been told aoc-sat2-mv8 does as well (SATA-II) but I have not personally tried it. Lund -- Jorgen Lundman | Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo| +81 (0)90-5578-8500 (cell) Japan| +81 (0)3 -3375-1767 (home) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Sun, 21 Jun 2009, Carson Gaspar wrote: I'll chime in as a happy owner of the LSI SAS 3081E-R PCI-E board. It works just fine. You need to get "lsiutil" from the LSI web site to fully access all the functionality, and they cleverly hide the download link only under their FC HBAs on their support site, even though it works for everything. I'll add another vote for the LSI products. I have a four port PCI-X card in my V880, and the performance is good and the product is well behaved. The only caveats: 1. Make sure you upgrade the firmware ASAP 2. You may need to use lsiutil to fiddle the target mappings Andre. -- Andre van Eyssen. mail: an...@purplecow.org jabber: an...@interact.purplecow.org purplecow.org: UNIX for the masses http://www2.purplecow.org purplecow.org: PCOWpix http://pix.purplecow.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Sun, 21 Jun 2009 19:01:31 -0700 Carson Gaspar wrote: > I'll chime in as a happy owner of the LSI SAS 3081E-R PCI-E board. It > works just fine. You need to get "lsiutil" from the LSI web site to > fully access all the functionality, and they cleverly hide the download > link only under their FC HBAs on their support site, even though it > works for everything. As a member of the team which works on mpt(7d), I'm disappointed that\ you believe you need to use lsiutil to "fully access all the functionality" of the board. What gaps have you found in mpt(7d) and the standard OpenSolaris tools that lsiutil fixes for you? What is the "full functionality" that you believe is missing? > As for identifying disks, you can just use lsiutil: ... or use cfgadm(1m) which has had this ability for many years. > root:gandalf 0 # lsiutil -p 1 42 > > LSI Logic MPT Configuration Utility, Version 1.62, January 14, 2009 > > 1 MPT Port found > > Port Name Chip Vendor/Type/RevMPT Rev Firmware Rev IOC > 1. mpt0 LSI Logic SAS1068E B3 105 011a 0 > > mpt0 is /dev/cfg/c6 > > B___T___L Type Operating System Device Name > 0 0 0 Disk /dev/rdsk/c6t0d0s2 > 0 1 0 Disk /dev/rdsk/c6t1d0s2 > 0 2 0 Disk /dev/rdsk/c6t2d0s2 > 0 3 0 Disk /dev/rdsk/c6t3d0s2 > 0 4 0 Disk /dev/rdsk/c6t4d0s2 > 0 5 0 Disk /dev/rdsk/c6t5d0s2 > 0 6 0 Disk /dev/rdsk/c6t6d0s2 > 0 7 0 Disk /dev/rdsk/c6t7d0s2 You can get that information from use of cfgadm(1m). James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
I'll chime in as a happy owner of the LSI SAS 3081E-R PCI-E board. It works just fine. You need to get "lsiutil" from the LSI web site to fully access all the functionality, and they cleverly hide the download link only under their FC HBAs on their support site, even though it works for everything. As for identifying disks, you can just use lsiutil: root:gandalf 0 # lsiutil -p 1 42 LSI Logic MPT Configuration Utility, Version 1.62, January 14, 2009 1 MPT Port found Port Name Chip Vendor/Type/RevMPT Rev Firmware Rev IOC 1. mpt0 LSI Logic SAS1068E B3 105 011a 0 mpt0 is /dev/cfg/c6 B___T___L Type Operating System Device Name 0 0 0 Disk /dev/rdsk/c6t0d0s2 0 1 0 Disk /dev/rdsk/c6t1d0s2 0 2 0 Disk /dev/rdsk/c6t2d0s2 0 3 0 Disk /dev/rdsk/c6t3d0s2 0 4 0 Disk /dev/rdsk/c6t4d0s2 0 5 0 Disk /dev/rdsk/c6t5d0s2 0 6 0 Disk /dev/rdsk/c6t6d0s2 0 7 0 Disk /dev/rdsk/c6t7d0s2 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Mon 22/06/09 02:07 , roland no-re...@opensolaris.org sent: > just a side-question: > > >I folthis thread with much > interest. > what are these "*" for ? > > why is "followed" turned into "fol*" on this > board? It isn't a board, it's a mail list. All the forum does is bugger up the formatiing and threading of emails! -- Ian ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
Hey Kebabber, long time no hear! :) It's great to hear that you've had good experiences with the card. It's a great pity to have throughput drop from a potential 1GB/s to 150MB/s, but as most of my use of the NAS is across the network, and not local intra-NAS transfers, this should not be a problem. Of course, with a single GbE connection speeds are normally limited to around 50MB/s or so anyway... Tell me, have you had any drives fail and had to figure out how to identify the failed drive and replace it & resilver using the AOC-SAT2-MV8, or have you tried any experiments to test resilvering ? I'm just curious as to how easy it is to do this with this controller card. Like yourself, I was toying with the idea of upgrading and buying a shiny new mobo with dual 64-bit PCI-X slots and socket LGA1366 for Xeon 5500 series (Nehalem) processors -- the SuperMicro X8SAX here: http://www.supermicro.com/products/motherboard/Xeon3000/X58/X8SAX.cfm Then I added up the price of all the components and decided to try to make do with the existing kit and just do an upgrade. So I narrowed down possible SATA controllers to the above choices and I'm interested in people's experiences of using these controllers to help me decide. Seems like the cheapest and simplest choice will be the AOC-SAT2-MV8, and I just take a hit on the reduced speed -- but that won't be a big problem. However, as I have 2 x PCIe x16 slots available, if the AOC-USAS-L8i is reliable and doesn't have issues now with identifying drive ids, and supports JBOD mode, then it looks like the better choice. It is uses the more modern PCI Express (PCIe) interface, rather than the ageing PCI-X interface, fine as I'm sure it is. Simon -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Sun, 21 Jun 2009 14:07:49 PDT roland wrote: > just a side-question: > > >I folthis thread with much interest. > > what are these "*" for ? > > why is "followed" turned into "fol*" on this board? The text of my original message was: On Sun, 21 Jun 2009 06:35:50 PDT Simon Breden wrote: > If anyone can throw some light on these topics, I would be pleased to > hear from you. Thanks a lot. I follow this thread with much interest. Curious to see what'll come out of it. Does the change occur again? -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D + http://nagual.nl/ | nevada / OpenSolaris 2009.06 release + All that's really worth doing is what we do for others (Lewis Carrol) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
just a side-question: >I folthis thread with much interest. what are these "*" for ? why is "followed" turned into "fol*" on this board? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
I use the AOC-SAT2-MV8 in a ordinary PCI slot. The PCI slot maxes at 150MB/sec or so. That is the fastest you will get. That card works very good with Solaris/OpenSolaris. Detects automatically, etc. Ive heard though that it does not work with hot swapping discs - avoid this. However, In a PCI-X it will max at 1GB/sec. I have been thinkin about buying a server mobo (they have PCI-X) to get 1GB/sec. Or should I buy a PCIe card instead? I dont know. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
After checking some more sources, it seems that if I used the AOC-SAT2-MV8 with this motherboard, I would need to run it on the standard PCI slot. Here is the full listing of the motherboard's expansion slots: 2 x PCI Express x16 slot at x16, x8 speed 2 x PCI Express x1 3 x PCI 2.2 <<--- this one -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best controller card for 8 SATA drives ?
On Sun, 21 Jun 2009 06:35:50 PDT Simon Breden wrote: > If anyone can throw some light on these topics, I would be pleased to > hear from you. Thanks a lot. I follow this thread with much interest. Curious to see what'll come out of it. -- Dick Hoogendijk -- PGP/GnuPG key: 01D2433D + http://nagual.nl/ | nevada / OpenSolaris 2009.06 release + All that's really worth doing is what we do for others (Lewis Carrol) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Best controller card for 8 SATA drives ?
Hi, I'm trying to find out which controller card people here recommend that can drive 8 SATA hard drives and that would work with my Asus M2N-SLI Deluxe motherboard, which has following expansion slots: 2 x PCI Express x16 slot at x16, x8 speed (PCIe) The main requirements I have are: - drive 8 SATA drives - rock solid reliability with x86 OpenSolaris 2009.06 or SXCE - easy to identify failed drives and replace them (hot swap is not necessary but a bonus if supported) - I must be able to move disks with data from one controller to another of different brands (and back!), only doing zpool export and import, which implies the HBA must be able to run in JBOD-mode without storing or modify anything on the disks. And preferably, the drives must show up with the format command. - should support staggered spinup of drives preferably >From limited research I can see that at least the following 3 main >possibilities exist: 1. Supermicro AOC-SAT2-MV8 (PCI-X interface) (pure SATA) (~$100) 2. Supermicro AOC-USAS-L8i / AOC-USASLP-L8i (PCIe interface) (miniSAS to SATA cables) (~$100) 3. LSI SAS 3081E-R or other LSI 'MegaRAID' cards (PCIe interface) (miniSAS to SATA cables) (~$200+) 1. AOC-SAT2-MV8 : Again, from reading a bit, I can see that although the M2N-SLI Deluxe motherboard does not have a PCI-X slot, apparently it could take the AOC-SAT2-MV8 card in one of the PCIe slots, although the card would only run in 32-bit mode, instead of 64-bit mode, and would therefore run slower. 2. AOC-USAS-L8i : The AOC-USAS-L8i card looks possible too, again running in the PCIe slot, but the old threads I saw on this seem to talk about some device numbering issue which could make determining the right failed drive to pull out, a difficult task -- see here for more details: http://www.opensolaris.org/jive/thread.jspa?messageID=271751 http://www.opensolaris.org/jive/thread.jspa?threadID=46982&tstart=90 3. LSI SAS 3081E-R or other LSI 'MegaRAID' cards : http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3081er/index.html?remote=1&locale http://www.newegg.com/Product/Product.aspx?Item=N82E16816118100 This forum thread from DEC 2007 doesn't sound too good regarding drive numbering (for identifying failed drives etc), but the thread is 18 months old, and perhaps the issues may have been resolved now? Also I noticed an extra '-R' in the model number I found, but this might be an omission of the original forum poster -- see here: http://www.opensolaris.org/jive/thread.jspa?threadID=46982&tstart=90 I saw Ben Rockwood saying good things about the LSI MegaRAID cards, although the model he references supports only 4 internal and 4 external drives so is not what I want -- see here: http://opensolaris.org/jive/message.jspa?messageID=368445#368445 Perhaps there are better LSI MegaRAID cards that people know of and can recommend? Preferably not too expensive though, as it's for a home system :) If anyone can throw some light on these topics, I would be pleased to hear from you. Thanks a lot. Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
