Re: Re[2]: [zfs-discuss] Re: ZFS and Storage

[Jonathan Edwards]

On Jun 28, 2006, at 18:25, Erik Trimble wrote: On Wed, 2006-06-28 at 14:55 -0700, Jeff Bonwick wrote: Which is better - zfs raidz on hardware mirrors, or zfs mirror on hardware raid-5? The latter.  With a mirror of RAID-5 arrays, you get: (1) Self-healing data. (2) Tolerance of whole-array failure. (3) Tolerance of *at least* three disk failures. (4) More IOPs than raidz of hardware mirrors (see Roch's blog entry). (5) More convenient FRUs (the whole array becomes a FRU). Jeff Not that I disagree with the inital assessment, but a couple of corrections: (1)  Both give you this. (2)  ZFS RAIDZ on HW mirrors can also survive a complete HW mirror array failure. (3)  Both configs can survive AT LEAST 3 drive failures. RAIDZ of HW mirrors is slightly better at being able to survive 4+ drive failures, statistically speaking. Here's 10 options I can think of to summarize combinations of zfs with hw redundancy: #   ZFS     ARRAY HW        CAPACITY    COMMENTS --  ---     --------        --------    -------- 1   R0      R1              N/2         hw mirror - no zfs healing (XXX) 2   R0      R5              N-1         hw R5 - no zfs healing (XXX) 3   R1      2 x R0          N/2         flexible, redundant, good perf 4   R1      2 x R5          (N/2)-1     flexible, more redundant, decent perf 5   R1      1 x R5          (N-1)/2     parity and mirror on same drives (XXX) 6   RZ      R0              N-1         standard RAID-Z - no array RAID (XXX) 7   RZ      R1 (tray)       (N/2)-1     RAIDZ+1 8   RZ      R1 (drives)     (N/2)-1     RAID1+Z (highest redundancy) 9   RZ      2 x R5          N-3         triple parity calculations (XXX) 10  RZ      1 x R5          N-2         double parity calculations (XXX) If we eliminate the configs with no zfs healing, and the configs with double parity calculations (overworking the drives), I believe that configs 3 and 4 on a decent RAID array will probably perform similarly for most workloads.  Config 4 (as Jeff pointed out) will probably get you the best performance and redundancy utilizing both the arrays' strengths and zfs' strengths. If we optimize for performance we'd probably shy away from the RAID-Z options since we can't really dedicate channels and resources for the parity calculations and writes (roch's explanation is much better.)  But if we optimize for reliability config 8 would get you the highest overall redundancy. Other options not considered: - Double mirroring - capacity loss is too high for too little gain - RAID2/3/4/6/S - not commonly used - have their own flaw areas Jonathan Edwards (generic storage and filesystem engineer)

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss