On Thu, Jan 28, 2010 at 07:26:42AM -0800, Ed Fang wrote: > 4 x x6 vdevs in RaidZ1 configuration > 3 x x8 vdevs in RaidZ2 configuration
Another choice might be 2 x x12 vdevs in raidz2 configuration This gets you the space of the first, with the recovery properties of the second - at a cost in potential performance. Your workload (mostly streaming, not many parallel streams, large files) sounds like it might be one that can tolerate this cost, but care will be needed. Experiment and measure, if you can. 2 x x12 could also get you to raidz3, for extra safety, making the same performance tradeoff against 3x8 with constant space. I don't think this is a choice you're likely to want, but worth mentioning. > Obviously if a drive fails, it'll take a good several days to > resilver. The data is important but not critical. That's important information. > Using raidz1 > allows you one drive failure, but my understanding is that if the > zpool has four vdevs using raidz1, then any single vdev failure of > more than one drive may fail the entire zpool ???? Correct, as already discussed. However, there are actually two questions here, and your final decision depends on both: - how many vdevs of what type? - how many pools? Do you need all the space available in one common pool, or can your application distribute space and load between multiple resource containers? You probably have more degrees of trade-off freedom, even for the same choices of base vdevs. If space is more important to you, and losing 1/4 of your non-critical files on a second disk failure is a tolerable risk, you might consider 4 pools of 6-disk raidz1. Likewise, 3 pools of 8-disk raidz2 reduces the worst impact of a third disk failure to 1/3 of you data, and 2 pools of 12-disk vdevs to 1/2. > If that is the > case, then it sounds better to consider 3 x8 with raidz2. Others have recommended raidz2, and I agree with them, in general principle. All that said, for large files that will fill large blocks, I'm wary of raidz pools with an odd number of data disks, and prefer if possible, a power-of-two number of data disks (plus whatever redundancy level you choose). Raid-z striping can leave holes, and this seems like it may result in inefficencies, either in space, fragmentation or just extra work. I have not measured this, and it may be irrelevant or invisible, generally or in your workload. So, I would recommend raidz2 vdevs, either 3x8 or 2x12. Test and compare the performance under your workload and see if you can afford the cost of the extra space the wide stripes offer. Test the performance while scrubs and resilvers are going on as well as real workload. If 2x12 can carry this for you, go for it. Then choose whether to combine the vdevs into a big pool, or keep them separate. -- Dan.
pgpTn58UmlK25.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
