On Dec 30, 2009, at 11:01 AM, Bob Friesenhahn wrote:
On Wed, 30 Dec 2009, Thomas Burgess wrote:
Just curious, but in your "ideal" situation, is it considered best
to use 1 controller for each vdev or user a different controler for
each device in the vdev (i'd guess the latter but ive been wrong
before)
From both a fault-tolerance standpoint, and performance standpoint,
it
is best to distribute the vdev devices across controllers. With
perfect distribution, you could pull a controller and it would be as
if one drive failed in each vdev (and you can still read your data).
If a controller messes up and sends bad data, then the damage is
more limited.
Don't confuse reliability with diversity. Reliability is highly
dependent
on the number of parts, fewer is better. Therefore the optimum number
of items for redundancy is 2. Hence, you'll see RAS guys recommending
mirrors as a standard practice :-)
In the case where you want to have high GB/$, you must trade off
something.
If you decide to go raidz[123] instead of mirroring, then it really
doesn't
matter how many controllers you have, 2 is the optimum number for
dependability. The reason I keep harping about this is because in the
bad
old days, there was a 1:1 relationship between a controller and a shared
bus (IDE, parallel SCSI). If you take out the shared bus, you take
out the
controller. Today, with point-to-point connections like SATA or SAS,
there
is no shared bus, so the disks are isolated from each other. In this
architecture,
there is a 1:many relationship between controllers and disks, with one
less
(big, important) failure mode -- the shared bus.
It is true that the controller itself is still a SPOF, but it is also
true that the
controller is more than an order of magnitude more reliable than the
disks.
If you do an availability analysis on such systems, the results will
show that
the controller doesn't matter -- worry about the disks.
To take this thought a step further, if you are so worried about a
controller
being the SPOF, then you will reject using Intel or AMD architectures
where there is only one bridge chip. In other words, you would reject
99+%
of the Intel and AMD systems available on the market. So, if you
accept a
high-volume Intel or AMD system, then you should also accept as few
controllers as possible -- which is a very reasonable and defensible
engineering decision.
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
