Many thanks to all who responded. I learned a lot from this thread! For now I have decided to make a 3 way mirror because of the read performance. I don't want to take a risk on an unmirrored drive.
Instead of replying to everyone separately I am following the Sun Managers system since I read that newsgroup occasionalliy also. Here's a summary of the responses. Jim Klimov wrote: > Well, you can use this drive as a separate "scratch area", as a separate > single-disk pool, without redundancy. You'd have a separate spindle for > some dedicated tasks with data you're okay with losing. I thought about that and I really don't like losing data. I also don't generate much temporary data so I love ZFS because it makes mirroring easy. On my other systems where I don't have ZFS I run hourly backups from drive to drive. Consumer drives are pretty good these days but you never know when one will fail. I had a failure recently on a Linux box and although I didn't lose data because I back up hourly it's still annoying to deal with. If I hadn't had another good drive with that data on it I would have lost critical data. > You can also make the rpool a three-way mirror which may increase read > speeds if you have enough concurrentcy. And when one drive breaks, your > rpool is still mirrored. I think that's the best suggestion. I didn't realize a 3 way mirror would help performance but you and several others said it does, so that's what I will do. Thanks for the suggestions, Jim. Roy pointed out a theoretical 50% read increase when adding the third drive. Thanks Roy! Edward Ned Harvey wrote: > In my benchmarking, I found 2-way mirror reads 1.97x the speed of a single > disk, and a 3-way mirror reads 2.91x a single disk. Always great having hard data to base a decision on! That helped me make my decision! Thanks Edward! Jim Klimov answered a question that came up based on comments that read performance was improved in a three way mirror: > Writes in a mirror are deemed to be not faster than the slowest disk - all > two or three drives must commit a block before it is considered written > (in sync write mode), likewise for TXG sync but with some optimization by > caching and write-coalescing. Thanks Jim! Good to know. Edward Ned Harvey pointed out "If you make it a 3-way mirror, your write performance will be unaffected, but your read performance will increase 50% over a 2-way mirror. All 3 drives can read different data simultaneously for the net effect of 3x a single disk read performance." Bob clarified the theoretical benefit of adding a third drive to a mirror by saying "I think that a read performance increase of (at most) 33.3% is more correct. You might obtain (at most) 50% over one disk by mirroring it. Zfs makes a random selection of which disk to read from in a mirror set so the improvement is not truely linear." Thanks guys, that makes sense. Daniel Carosone suggested keeping the extra drive around in case of a failure and in the meantime using an SSD in the 3rd SATA slot. He pointed out a few other options that could help with performance besides creating a 3 way mirror when he wrote: > Namely, leave the third drive on the shelf as a cold spare, and use the > third sata connector for an ssd, as L2ARC, ZIL or even possibly both > (which will affect selection of which device to use). That's not an option for me right now but I am planning to revisit SSD again when the consumer drives are reliable enough and don't have wear issues. Right now overall integrity and long service life are more important than absolute performance on this box, although since I have the integrity with the ZFS mirror I could add an SSD but I really don't want to deal with another failure as long as I don't have to. I do want additional performance if I can afford it, but not at the expense of possible data loss. Daniel also wrote: > L2ARC is likely to improve read latency (on average) even more than a > third submirror. ZIL will be unmirrored, but may improve writes at an > acceptable risk for development system. If this risk is acceptable, you > may wish to consider whether setting sync=disabled is also acceptable at > least for certain datasets. I don't know what L2ARC is, but I'll take a look on the net. I did hear about ZIL but don't understand it fully, but I figured spending 500G on ZIL would be unwise. By that I mean I understand ZIL doesn't require much storage but if I don't have an identical drive I can't add a drive or slice with less storage than the other drives in a mirror to that mirror, so I would be forced to waste a lot of storage to implement ZIL. > Finally, if you're considering spending money, can you increase the RAM > instead? If so, do that first. This mobo is maxed out at 4G, it's a socket 775 I bought a couple of years ago. I have always seen the benefits to more RAM and I agree with you it helps more than people generally believe. Next time I buy a new box I am hoping to go with 8 to 16G although on the specific box I am talking about here 4G is more than adequate since it is a development machine and I'm the only one on it. On my other boxes where I do run virtual machines and have a lot going on I would definitely benefit from more RAM. Thanks a lot Dan, great suggestions! Richard Elling reminded me that warm spares are a good idea. I had thought about that option too, but then I realized I may as well create a 3 way mirror than rather having the drive sitting in the box spinning unproductively since I didn't think the OS or mobo had the smarts to idle a warm spare. Jim Klimov responded to Richard's suggestion above by noting a bug in update 8 that prevents the system from booting when using hot spares. This is what convinced me to go with the 3 way mirror because I am running update 8 on this box and don't want to update to another version since I am not on support and I understand U8 is the last version that comes with Star Office. Thanks Jim, very important info here and you saved me from possible severe aggravation and hair pulling! Dan responded to earlier comments about hot v. cold spares with an insight that maybe the third port could be better used for something else given my request to improve performance. Thanks Dan, that was a helpful comment. Always helps to clarify what the real question is and make sure it's answered! Bill Sommerfeld also agreed with the idea of using an SSD, and he gave a suggestion of how to use the extra space. He wrote "I'd use the extra SATA port for an ssd, and use that ssd for some combination of boot/root, ZIL, and L2ARC." Bill also clarified the suggestion he made is addressed to my specific scenario, a small system with a mirror consisting of two drives. He cautioned us by saying "While slicing an ssd and using one slice for root, one slice for zil, and one slice for l2arc isn't optimal from a performance standpoint and won't scale up to a larger configuration, it is a noticeable improvement from a 2-disk mirror." Bill gave an example of how he had configured a setup like that in the past. He said "I used an 80G intel X25-M, with 1G for zil, with the rest split roughly 50:50 between root pool and l2arc for the data pool." Thank you very much Bill. I'll think about that in the future if I revisit the SSD drives for my use. Lastly, Fajar raised some of the issues that were on my mind in the recent past when I looked into SSD for my uses. He wrote "Cheap-ish sandforce-based MLC SSDs usually say they support 1 million write cycles, and that they have some kind of wear-leveling. How does this translates when it's used as L2ARC? Can we expect something like one year or three years lifetime when the pool is relatively busy?" I don't know the answer to that, but I share the concern. Thanks for your comments. To everyone who answered, thank you sincerely! I hope this summary is helpful to other people. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
