On Sep 9, 2006, at 1:32 AM, Frank Cusack wrote:
On September 7, 2006 12:25:47 PM -0700 "Anton B. Rang"
<[EMAIL PROTECTED]> wrote:
The bigger problem with system utilization for software RAID is the
cache, not the CPU cycles proper. Simply preparing to write 1 MB
of data
will flush half of a 2 MB L2 cache. This hurts overall system
performance
far more than the few microseconds that XORing the data takes.
Interesting. So does this in any way invalidate benchmarks
recently posted
here which showed raidz on jbod to outperform a zfs stripe on HW
raid5?
No. There are, in fact, two reasons why RAID-Z is likely to outperform
hardware RAID 5, at least in certain types of I/O benchmarks. First,
RAID-5 requires read-modify-write cycles when full stripes aren't being
written; and ZFS tends to issue small and pretty much random I/O (in my
experience), which is the worst case for RAID-5. Second, performing
RAID
on the main CPU is faster, or at least just as fast, as in hardware.
There are also cases where hardware RAID 5 will likely outperform ZFS.
One is when there is a large RAM cache (which is not being flushed by
ZFS -- one issue to be addressed is that the commands ZFS uses to
control
the write cache on plain disks tend to effectively disable the NVRAM
cache on hardware RAID controllers). Another is when the I/O bandwidth
being used is near the maximum capacity of the host channel, because
doing software RAID requires moving more data over this channel. (If
you have sufficient channels to dedicate one per disk, as is the case
with SATA, this doesn't come into play.) This is particularly
noticeable during reconstruction, since the channels are being used
both to transfer data & reconstruct it, where in a hardware RAID-5
box (of moderate cost, at least) they are typically overprovisioned.
A third is if the system CPU or memory bandwidth is heavily used by
your application; for instance, a database running under heavy load.
In this case, the added CPU, cache, and memory bandwidth of software
RAID will stress the application.
Ultimately, you do want to use your actual application as the
benchmark,
but certainly generic benchmarks should at least be helpful.
They're helpful in measuring what the benchmark measures. ;-) If the
benchmark measures how quickly you can get data from host RAM to disk,
which is typically the case, it won't tell you anything about how much
CPU was used in the process. Real applications, however, often care.
There's a reason why we use interrupt-driven controllers, even though
you get better performance of the I/O itself with polling. :-)
Anton
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
