Re: PCIe SATA HBA for ZFS on -STABLE

[Matt Thyer]

On 8 June 2011 20:55, Matt Thyer  wrote:

> On 7 June 2011 12:03, Matthew Dillon  wrote:
>
>>The absolute cheapest solution is to buy a Sil-3132 PCIe card
>>(providing 2 E-SATA ports), and then connect an external port
>> multiplier
>>to each port.  External port multiplier enclosures typically support
>>5 drives each so that would give you your 10 drives.
>>
>
> I've decided to avoid issues with port multiplication by going for a
> Supermicro AOC-USAS2-L8i and then to flash it to IT (as opposed to IR) mode
> to make it run as a standard non-RAID HBA.
>
> As I've got 8 x 2 TB drives for the ZFS raidz2 I'll put them all on the
> AOC-USAS2-L8i and save my onboard SATA-II ports for my 2 x 1TB drives for
> the FreeBSD O.S. and any eSATA use.
>
> Now the only remaining issue is whether to go with the Supermicro firmware
> or the generic Lsi Logic firmware as some have reported better performance
> with the version 9 Lsi Logic firmware.
>
> I'll report on my experiences (as I keep a record of the revision of my
> -STABLE build this should actually be useful!).
>
>
I could not be happier with the result.

As planned, I've put all 8 2TB Western Digital WD20EARS drives on the 8 lane
PCIe 2.0 AOC-USAS2-L8i (flashed with the latest SuperMicro firmware to
behave as an AOC-USAS2-L8e).

Many of you are now screaming "Why is he using the dreaded WD20EARS ?".  The
answer is that I bought the first 4 drives before I knew their issues and
later decided to continue with them once I knew how to mitigate their issues
by:


   - Using the DOS based WDIDLE3.EXE utility to change the default "park the
   heads after 8 seconds of idle time" to the maximum of 5 minutes
   - Avoiding alignment problems by putting ZFS on the whole disks (with no
   GPT or MBR partition table)
   - Convincing ZFS to use 4KiB transfers by creating the pool on top of
   4KiB block sized devices created with "gnop -S 4096"


And the resulting performance...

Previously I could sustain about 45 MiB/s writing from high powered Windows
7 machines (via Samba using asynchronous I/O) on to my old arrangement of 4
drives in raidz1 (using only the Intel on-board SATA ports on the H55
chipset motherboard).

I can now sustain 90 MiB/s over the network with the 8 drive raidz2 but
that's only because the Windows 7 machines can't feed the data fast enough.

I can see from "zpool iostat pool 1" that ZFS is idle for about 5-7 seconds
and then it will write at around 390 - 410 MB/s for about 3 seconds.

dd tests are:
> sudo dd if=/dev/zero of=/export/1G bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 3.218604 secs (333604841 bytes/sec)
> sudo dd if=/dev/random of=/export/1G bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 15.531615 secs (69132658 bytes/sec)

So I'm very happy that I can keep my home network users happy with the
limits now being due to my gigabit ethernet network (and I'm not going 10
GbE any time soon!).

This is on 8-STABLE at r220359 (~ 5th of April 2011).
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Matt Thyer]

On 7 June 2011 12:03, Matthew Dillon  wrote:

>The absolute cheapest solution is to buy a Sil-3132 PCIe card
>(providing 2 E-SATA ports), and then connect an external port multiplier
>to each port.  External port multiplier enclosures typically support
>5 drives each so that would give you your 10 drives.
>

I've decided to avoid issues with port multiplication by going for a
Supermicro AOC-USAS2-L8i and then to flash it to IT (as opposed to IR) mode
to make it run as a standard non-RAID HBA.

As I've got 8 x 2 TB drives for the ZFS raidz2 I'll put them all on the
AOC-USAS2-L8i and save my onboard SATA-II ports for my 2 x 1TB drives for
the FreeBSD O.S. and any eSATA use.

Now the only remaining issue is whether to go with the Supermicro firmware
or the generic Lsi Logic firmware as some have reported better performance
with the version 9 Lsi Logic firmware.

I'll report on my experiences (as I keep a record of the revision of my
-STABLE build this should actually be useful!).
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Alexander Motin]

On 07.06.2011 05:33, Matthew Dillon wrote:

 The absolute cheapest solution is to buy a Sil-3132 PCIe card
 (providing 2 E-SATA ports), and then connect an external port multiplier
 to each port.  External port multiplier enclosures typically support
 5 drives each so that would give you your 10 drives.

 Even the 3132 is a piss-ant little card it does support FIS-Based
 switching so performance will be very good... it will just be limited
 to SATA-II speeds is all.


SiI3132 is indeed good for it's price and it is quite good for random 
I/O. But at burst speeds it is limited lower then SATA-II. Even lower 
then PCIe 1.0 x1 it uses. IIRC I've seen about 150MB/s from one port and 
about 170MB/s from two.


If burst rate is important, SiI3124 chip is much better -- up to about 
900MB/s measured from 4 ports. The only issue is PCI-X interface: either 
motherboard with PCI-X needed, or card with PCIe x8 bridge (like these 
http://www.addonics.com/products/host_controller/adsa3gpx8-4e.asp), but 
last case is too expensive.


There are also much cheaper (~$50) PCIe x1 bridge SiI3124 cards 
(http://www.sybausa.com/productInfo.php?iid=537). They are not so fast 
-- about 200MB/s, but still more then SiI3132. And they still have 4 
SATA ports.



 For SSDs you want to directly connect the SSD to a mobo SATA port and
 then either mount the SSD in the case or mount it in a hot-swap gadget
 that you can screw into a PCI slot (it doesn't actually use the PCI
 connector, just the slot).  A SATA-III port with a SATA-III SSD really
 shines here and 400-500 MBytes/sec random read performance from a single
 SSD is possible, but it isn't an absolute requirement.  A SATA-II port
 will still work fine as long as you don't mind maxing out the bandwidth
 at 250 MBytes/sec.


Agree. Intel on-board ports rock! Recently I've built new system with 
two OCZ Vertex 3 SSDs connected to 6Gbps SATA ports on Intel Sandy 
Bridge class motherboard. UFS on top of graid RAID0 volume gives me 
about 950MB/s on both read and write!



 To get robust hot-swap enclosures you either need to go with SAS or you
 need to go with discrete SATA ports (no port multiplication), and the
 ports have to support hot-swap.  The best hot-swap support for an AHCI
 port is if the AHCI chipset supports cold-presence-detect (CPD), and
 again Mobo AHCI chipsets usually don't.  Hot-swap is a bit hit or miss
 without CPD because power savings modes can effectively prevent hot-swap
 detect from working properly.  Drive disconnects will always be detected
 but drive connects might not be.


I would say it depends. In some cases it is easier to detect hot-plug 
then hot-unplug, as device sends COMINIT that should wake up port even 
from power-save state. With ICH10, for example, I've managed to make 
both hot plug and unplug work even with power-management enabled: 
hot-plug via tracking COMINIT, unplug via it's CPD capability. Without 
PM it "just works". :)


--
Alexander Motin
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Matthew Dillon]

:I'm not on the -STABLE list so please reply to me.
:
:I'm using an Intel Core i3-530 on a Gigabyte H55M-D2H motherboard with 8 x
:2TB drives & 2 x 1TB drives.
:The plan is to have the 1 TB drives in a zmirror and the 8 in a raidz2.
:
:Now the Intel chipset has only 6 on board SATA II ports so ideally I'm
:looking for a non RAID SATA II HBA to give me 6 extra ports (4 min).
:Why 6 extra ?
:Well the case I'm using has 2 x eSATA ports so 6 would be ideal, 5 OK, and 4
:the minimum I need to do the job.
:
:So...
:
:What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
:using ZFS ?
:
:Not wanting to break the bank.
:Not interested in SATA III 6GB at this time... though it could be useful if
:I add an SSD for... (is it ZIL ?).
:Can this be added at any time ?
:
:The main issue is I need at least 10 ports total for all existing drives...
:ZIL would require 11 so ideally we are talking a 6 port HBA.

The absolute cheapest solution is to buy a Sil-3132 PCIe card
(providing 2 E-SATA ports), and then connect an external port multiplier
to each port.  External port multiplier enclosures typically support
5 drives each so that would give you your 10 drives.

Even the 3132 is a piss-ant little card it does support FIS-Based
switching so performance will be very good... it will just be limited
to SATA-II speeds is all.

Motherboard AHCI-based SATA ports typically do NOT have FIS-Based
switching support (this would be the FBSS capability flag when the AHCI
driver probes the chipset).  This means that while you can attach an
external port multiplier enclosure to mobo SATA ports (see later
on E-SATA vs SATA), read performance from multiple drives concurrently
will be horrible.  Write performance will still be decent due to drive
write caches despite being serialized.

On E-SATA vs SATA.  Essentially there are only two differences between
E-SATA and SATA.  One is the cable and connector format.  The other is
hot swap detection.  Most mobo SATA ports can be strung out to E-SATA
with an appropriate adapter.  High-end Intel ASUS mobos often come with
such adapters (this is why they usually don't sport an actual E-Sata
port on the backplane) and the BIOS has setup features to specify E-SATA
on a port-by-port basis.

--

For SSDs you want to directly connect the SSD to a mobo SATA port and
then either mount the SSD in the case or mount it in a hot-swap gadget
that you can screw into a PCI slot (it doesn't actually use the PCI
connector, just the slot).  A SATA-III port with a SATA-III SSD really
shines here and 400-500 MBytes/sec random read performance from a single
SSD is possible, but it isn't an absolute requirement.  A SATA-II port
will still work fine as long as you don't mind maxing out the bandwidth
at 250 MBytes/sec.

--

I can't help with any of the other questions.  Someone also suggested
the MPS driver for FreeBSD, with caveats.

I'll add a caveat on the port multiplier enclosures.  Nearly all such
enclosures use another SIL chipset internally and it works pretty well
EXCEPT that it isn't 100% dependable if you try to hot-swap drives in the
enclosure while other drives in the enclosure are active.  So with that
caveat, I recommend the port multiplier enclosure as the cheapest solution.

To get robust hot-swap enclosures you either need to go with SAS or you
need to go with discrete SATA ports (no port multiplication), and the
ports have to support hot-swap.  The best hot-swap support for an AHCI
port is if the AHCI chipset supports cold-presence-detect (CPD), and
again Mobo AHCI chipsets usually don't.  Hot-swap is a bit hit or miss
without CPD because power savings modes can effectively prevent hot-swap
detect from working properly.  Drive disconnects will always be detected
but drive connects might not be.

And even with discrete SATA ports the AHCI firmware on mobos does not
necessarily handle hot-swap properly.  For example my Intel-I7 ASUS mobo
will generate spurious interrupts and status on a DIFFERENT discrete
SATA port when I hot swap on some other discrete SATA port, in addition
to generating the status interrupt on the correct port.  So then it comes
down to the driver in the operating system properly handling the
spurious status and properly stopping and restarting pending commands
when necessary.   So, again, it is best for the machine to be idle before
attempting a hot-swap.

Lots of caveats.  Sorry... you can blame Intel for all the blasted issues
with AHCI and SATA.  Intel didn't produce a very good chipset spec and
vendors took all sorts of liberties.

-Matt
Matthew Dillon 


___
freebsd-stable@f

Re: PCIe SATA HBA for ZFS on -STABLE

[Matt Thyer]

On 1 June 2011 17:37, Jeremy Chadwick  wrote:

> On Wed, Jun 01, 2011 at 02:34:55PM +0800, TJ Varghese wrote:
> > On Tue, May 31, 2011 at 10:31 PM, Freddie Cash 
> wrote:
>

[snip]


> > > SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port
> HBAs
> > > using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3
> Gpbs
> > > SATA/SAS, using multi-lane cables (2 connectors on the card, each
> connector
> > > supports 4 SATA ports), hot-plug, hot-swap.
> > >
> > >
> > The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure
> if
> > it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
> > does 3Gbps and is supported by the mpt driver.
>
> Folks considering use of mps(4), which was committed to RELENG_8 roughly
> around 2011/02/18 (thus is not in 8.2-RELEASE), should read the below
> threads just in case.  Always good to be educated.  Of course, the
> mailing lists are usually filled with complaints rather than success
> stories, so the tone of my mail here will therefore sound negative; I
> don't mean it that way, I just ask that people "be aware".
>
> * 2011/04/29 -- mps driver instability under stable/8
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62507
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-May/thread.html#62518
>
> * 2011/04/27 -- MPS driver: force bus rescan after remove SAS cable
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62438
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62443
>
> * 2011/03/10 -- LSI SAS2008 performance with mps(4) driver
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-March/thread.html#61862
>

Those threads assure me that the SuperMicro AOC-USAS2-L8i with version 9
firmware and the mps(4) driver work very well as long as I'm running FreeBSD
9-CURRENT or 8-STABLE (not 8.2-RELEASE).  As I'm running -STABLE
I'm quite happy to give it a go.

To the OP (Matt Thyer):
>
> Sadly I don't have a recommendation for you, since you effectively want
> a 6-port SATA300 controller that's reliable, you're almost certainly
> going to be paying Big Bucks(tm) given the number of ports and your
> requirement that it be PCIe-based.  You state quite boldly "not wanting
> to break the bank", but what you're asking for almost certainly WILL
> break the bank.
>

Jeremy, I think you need to have another look at current prices.
I have now bought a SuperMicro AOC-USAS2-L8i on EBay from bakamuzko
with the cables I need for only $US 210.99 (I do know about the UIO
bracket).

For example, an "affordable" controller might be one driven by Silicon
> Image's SiI3124 chip -- four (4) SATA300 ports, but it's only hooked to
> PCI or PCI-X, not PCIe, which means you're susceptible to a much more
> severe bus bottleneck than with PCIe:
>

I defintely would not consider PCI for part of a ZFS array with 4 drives on
that one controller.



> I tend to avoid consumer-grade Marvell and JMicron SATA chipsets like
> the plague, however.  That's based on my experiences with them under
> Windows, where I would expect (truly) the drivers to be rock solid given
> the marketing demographic of the chips in question.
>

I've had the same bad experiences.

Good luck, and please let us know what controller you *do* end up going
> with and your experience with it!  Positives are as important as
> negatives.
>

I'll let you know how it works out.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Freddie Cash]

On Tue, May 31, 2011 at 11:34 PM, TJ Varghese  wrote:

> On Tue, May 31, 2011 at 10:31 PM, Freddie Cash  wrote:
>
>> On Tue, May 31, 2011 at 5:48 AM, Matt Thyer  wrote:
>>
>> > What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
>> > using ZFS ?
>> >
>> > Not wanting to break the bank.
>> > Not interested in SATA III 6GB at this time... though it could be useful
>> if
>> > I add an SSD for... (is it ZIL ?).
>> > Can this be added at any time ?
>> >
>> > The main issue is I need at least 10 ports total for all existing
>> drives...
>> > ZIL would require 11 so ideally we are talking a 6 port HBA.
>> >
>>
>> SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
>> using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
>> SATA/SAS, using multi-lane cables (2 connectors on the card, each
>> connector
>> supports 4 SATA ports), hot-plug, hot-swap.
>>
>>
> The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure if
> it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
> does 3Gbps and is supported by the mpt driver.
>
>
Oops, you're right.  We have the USAS, not the USAS2, using the mpt(4)
driver.

-- 
Freddie Cash
fjwc...@gmail.com
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Stefan Esser]

Am 01.06.2011 10:07, schrieb Jeremy Chadwick:
> Sadly I don't have a recommendation for you, since you effectively want
> a 6-port SATA300 controller that's reliable, you're almost certainly
> going to be paying Big Bucks(tm) given the number of ports and your
> requirement that it be PCIe-based.  You state quite boldly "not wanting
> to break the bank", but what you're asking for almost certainly WILL
> break the bank.
> 
> For example, an "affordable" controller might be one driven by Silicon
> Image's SiI3124 chip -- four (4) SATA300 ports, but it's only hooked to
> PCI or PCI-X, not PCIe, which means you're susceptible to a much more
> severe bus bottleneck than with PCIe:
> 
> http://www.siliconimage.com/products/family.aspx?id=3

FYI: There is at least one PCIe card with Sil3124 (with PCIe to PCI-X
bridge on the controller card):

http://www.leaf-computer.com/#24e

Price is 69 Euro plus 5 Euro shipping to Europe or North America (i.e.
some US$110 total).

Regards, STefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Daniel Kalchev]

On 01.06.11 09:34, TJ Varghese wrote:



SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
supports 4 SATA ports), hot-plug, hot-swap.

The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure if
it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
does 3Gbps and is supported by the mpt driver.



One should also bear in mind, that the 1068e based controllers 
(AOC-USAS-L8i) apparently have 2TB drive size limitation, therefore 
cannot be used with current 3TB (and who knows what capacities by the 
end of the year) drives. Otherwise, this is an well supported, high 
performance, stable and relatively cheap HBA to consider. The SAS2 (LSI 
2008 based) HBAs are also good, despite some firmware issues (mostly are 
related to use with SAS expanders) and do not have obvious limitations 
yet. These might be just a bit more expensive, but my Supermicro 
supplier advised delivery times for the older SAS (3Gbps) versions would 
be much longer.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Jeremy Chadwick]

On Wed, Jun 01, 2011 at 02:34:55PM +0800, TJ Varghese wrote:
> On Tue, May 31, 2011 at 10:31 PM, Freddie Cash  wrote:
> 
> > On Tue, May 31, 2011 at 5:48 AM, Matt Thyer  wrote:
> >
> > > What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
> > > using ZFS ?
> > >
> > > Not wanting to break the bank.
> > > Not interested in SATA III 6GB at this time... though it could be useful
> > if
> > > I add an SSD for... (is it ZIL ?).
> > > Can this be added at any time ?
> > >
> > > The main issue is I need at least 10 ports total for all existing
> > drives...
> > > ZIL would require 11 so ideally we are talking a 6 port HBA.
> > >
> >
> > SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
> > using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
> > SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
> > supports 4 SATA ports), hot-plug, hot-swap.
> >
> >
> The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure if
> it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
> does 3Gbps and is supported by the mpt driver.

Folks considering use of mps(4), which was committed to RELENG_8 roughly
around 2011/02/18 (thus is not in 8.2-RELEASE), should read the below
threads just in case.  Always good to be educated.  Of course, the
mailing lists are usually filled with complaints rather than success
stories, so the tone of my mail here will therefore sound negative; I
don't mean it that way, I just ask that people "be aware".

* 2011/04/29 -- mps driver instability under stable/8
  http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62507
  http://lists.freebsd.org/pipermail/freebsd-stable/2011-May/thread.html#62518

* 2011/04/27 -- MPS driver: force bus rescan after remove SAS cable
  http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62438
  http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/thread.html#62443

* 2011/03/10 -- LSI SAS2008 performance with mps(4) driver
  http://lists.freebsd.org/pipermail/freebsd-stable/2011-March/thread.html#61862

To the OP (Matt Thyer):

Sadly I don't have a recommendation for you, since you effectively want
a 6-port SATA300 controller that's reliable, you're almost certainly
going to be paying Big Bucks(tm) given the number of ports and your
requirement that it be PCIe-based.  You state quite boldly "not wanting
to break the bank", but what you're asking for almost certainly WILL
break the bank.

For example, an "affordable" controller might be one driven by Silicon
Image's SiI3124 chip -- four (4) SATA300 ports, but it's only hooked to
PCI or PCI-X, not PCIe, which means you're susceptible to a much more
severe bus bottleneck than with PCIe:

http://www.siliconimage.com/products/family.aspx?id=3

FreeBSD does have support for many Silicon Image chips via the siis(4)
driver, and support is quite good since SI provides mav@ with technical
documentation and support.  I commend SI for that; it's good to see
companies supporting developers, regardless of OS.

I tend to avoid consumer-grade Marvell and JMicron SATA chipsets like
the plague, however.  That's based on my experiences with them under
Windows, where I would expect (truly) the drivers to be rock solid given
the marketing demographic of the chips in question.

Be aware that SATA port multipliers (if someone recommends them to you
as a way of providing expansion) will also limit your I/O bottleneck,
especially when multiple drives are used over a single multiplier port.
E.g. 4 drives operating at 100MByte/sec (common read speed with
consumer-grade Caviar Black drives!) will saturate a SATA300 connection
easily.  One port per drive solves this dilemma of course, putting the
focus back on the PCI/PCI-X/PCIe bus as a bottleneck.

Anyway, you need to ask yourself what your requirements really are, or
what sort of monetary limitations you have, then make a decision based
on that.  Remember: a good, solid controller will probably be a one-time
purchase.  But also think about the future and if in 2-3 years you want
to go about buying another controller (likely costing more than whatever
it is you buy now).

Good luck, and please let us know what controller you *do* end up going
with and your experience with it!  Positives are as important as
negatives.

-- 
| Jeremy Chadwick   j...@parodius.com |
| Parodius Networking   http://www.parodius.com/ |
| UNIX Systems Administrator  Mountain View, CA, USA |
| Making life hard for others since 1977.   PGP 4BD6C0CB |

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[TJ Varghese]

On Tue, May 31, 2011 at 10:31 PM, Freddie Cash  wrote:

> On Tue, May 31, 2011 at 5:48 AM, Matt Thyer  wrote:
>
> > What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
> > using ZFS ?
> >
> > Not wanting to break the bank.
> > Not interested in SATA III 6GB at this time... though it could be useful
> if
> > I add an SSD for... (is it ZIL ?).
> > Can this be added at any time ?
> >
> > The main issue is I need at least 10 ports total for all existing
> drives...
> > ZIL would require 11 so ideally we are talking a 6 port HBA.
> >
>
> SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
> using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
> SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
> supports 4 SATA ports), hot-plug, hot-swap.
>
>
The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure if
it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
does 3Gbps and is supported by the mpt driver.


--
TJ
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Artem Belevich]

On Tue, May 31, 2011 at 7:31 AM, Freddie Cash  wrote:
> On Tue, May 31, 2011 at 5:48 AM, Matt Thyer  wrote:
>
>> What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
>> using ZFS ?
>>
>> Not wanting to break the bank.
>> Not interested in SATA III 6GB at this time... though it could be useful if
>> I add an SSD for... (is it ZIL ?).
>> Can this be added at any time ?
>>
>> The main issue is I need at least 10 ports total for all existing drives...
>> ZIL would require 11 so ideally we are talking a 6 port HBA.
>>
>
> SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
> using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
> SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
> supports 4 SATA ports), hot-plug, hot-swap.
>
> These are UIO cards, so the bracket that comes with it doesn't work with
> normal cases (the bracket is on the wrong side of the card; they're made for
> SuperMicro's UIO-based motherboards).  However, these are normal PCIe cards
> and work in any PCIe slot.  You either have to remove the bracket, or you
> can purchase separate brackets online.
>
> These cards are recommended on the zfs-discuss mailing list.  They are only
> ~$120 CDN at places like cdw.ca and newegg.ca.

+1 for LSI1068(e) controller + mpt driver. It's cheap and it works.
Those LSI controllers are often hiding behind other brands. SuperMicro
mentioned above is one. Intel would be another -- search for Intel
SASUC8I. Tyan also sells one as TYAN P3208SR. LSI-branded controllers
tend to be a bit more expensive than rebranded ones, though
functionality is the same and you can often cross-flash firmware.

Keep in mind that HBAs based on LSI1068(e) can't handle hard drives
larger than 2TB and will truncate larger drive capacity to 2TB.

As for the SSD, you may want to hook them up to on-board SATA ports.
In my not-very scientific benchmark Intel's X25-M SSD connected to
on-board SATA port on ICH10 was able to deliver ~20% more reads/sec
than the same SSD connected to LSI1068 based controller.

--Artem
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Freddie Cash]

On Tue, May 31, 2011 at 5:48 AM, Matt Thyer  wrote:

> What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
> using ZFS ?
>
> Not wanting to break the bank.
> Not interested in SATA III 6GB at this time... though it could be useful if
> I add an SSD for... (is it ZIL ?).
> Can this be added at any time ?
>
> The main issue is I need at least 10 ports total for all existing drives...
> ZIL would require 11 so ideally we are talking a 6 port HBA.
>

SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
supports 4 SATA ports), hot-plug, hot-swap.

These are UIO cards, so the bracket that comes with it doesn't work with
normal cases (the bracket is on the wrong side of the card; they're made for
SuperMicro's UIO-based motherboards).  However, these are normal PCIe cards
and work in any PCIe slot.  You either have to remove the bracket, or you
can purchase separate brackets online.

These cards are recommended on the zfs-discuss mailing list.  They are only
~$120 CDN at places like cdw.ca and newegg.ca.

-- 
Freddie Cash
fjwc...@gmail.com
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: PCIe SATA HBA for ZFS on -STABLE

[Steven Hartland]

Areca's work well. The ARC-1220 (8 ports) should do you, not the cheapest but
good support and performance.

   Regards
   Steve

- Original Message - 
From: "Matt Thyer" 

To: 
Sent: Tuesday, May 31, 2011 1:48 PM
Subject: PCIe SATA HBA for ZFS on -STABLE



I'm not on the -STABLE list so please reply to me.

I'm using an Intel Core i3-530 on a Gigabyte H55M-D2H motherboard with 8 x
2TB drives & 2 x 1TB drives.
The plan is to have the 1 TB drives in a zmirror and the 8 in a raidz2.

Now the Intel chipset has only 6 on board SATA II ports so ideally I'm
looking for a non RAID SATA II HBA to give me 6 extra ports (4 min).
Why 6 extra ?
Well the case I'm using has 2 x eSATA ports so 6 would be ideal, 5 OK, and 4
the minimum I need to do the job.

So...

What do people recommend for 8-STABLE as a PCIe SATA II HBA for someone
using ZFS ?

Not wanting to break the bank.
Not interested in SATA III 6GB at this time... though it could be useful if
I add an SSD for... (is it ZIL ?).
Can this be added at any time ?

The main issue is I need at least 10 ports total for all existing drives...
ZIL would require 11 so ideally we are talking a 6 port HBA.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"




This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 


In the event of misdirection, illegible or incomplete transmission please 
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"