Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On 7/24/2010 8:12 PM, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Russ Price Good advice - ZFS can use quite a lot of CPU cycles. A low-end AMD quad-core is I know a lot of CPU cycles is a relative term. But I never notice CPU utilization, even under the heaviest loads I can generate. Note: I'm not generally using compression (which will require CPU) and I'm not using dedupe (which will require RAM). Still, I don't think it's fair or accurate to generalize and say ZFS will use a lot of CPU cycles, unless you're qualifying it specifically such as if you have compression enabled. I was wondering about that too because I've been seeing a lot of zfs builds with atom processors and haven't complained about cpu utilization given a small home server application. Sid question... I recently ran across this blog post that indicates raidz and raidz2 don't increase performance over single drive performance unlike raid5. The post was old so I was wondering if that was still true. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On 7/23/2010 3:39 AM, tomwaters wrote: There is alot there to reply to...but I will try and help... Re. TLER. Do not worry about TLER when using ZFS. ZFS will handle it either way and will NOT time out and drop the drive...it may wait a long time, but it will not time out and drop the drive - nor will it have an issue if you do enable TLER-ON (which sets time out to 7 seonds). I run both with TLER-ON (disks from an old mdadm raid array) and without (TLER-OFF) 1.5TB WDEADS. That's reassuring. I guess the only time the issue will come up is when the drive starts to develop errors. I can not speak for the WD EARS, but the WDEADS are fine in my home nas. I also run 1.5TB Samsung Green/Silencer series and Seagate 11's. Others swear by Hitachi. I would recommend the Samsung or Hitachi and not the new WD EARS which have that 4k sectors or whatever it is. I've been digging through the archives more and I'm starting to lean towards laptop drives. I found this discussion http://opensolaris.org/jive/message.jspa?messageID=468269#468269 and contacted someone else who has used 3 WD Scorpio Black drives in a 3 way mirror for a year without any problems. This is mostly going to be a backup server so I'm not too worried about performance. Re the CPU, do not go low power Atom etc, go a newish Core2 duo...the power differential at idle is bugger all and when you want to use the nas, ZFS will make good use of the CPU. Honestly, sit down and do the calculations on power savings of a low power cpu and you'll see it's better to just not have that 5th beer on a friday - you'll save more money that way and be MUCH happier with your nas performance. re. cards...I use and recommend these 8-Port SUPERMICRO AOC-USASLP-L8I UIO SAS. They are cheap on e-bay, just work and are fast. Use them. You do want alot of ram I use 8GB, but you can use 4. Ram is cheap, ZFS loves ram, just buy 8. IMHO (and that of the best practice guide) - you should mirror the rpool (o/s disk). Just buy 2 cheap laptop drives and when installing choose to mirror them. I can find some motherboards with 4-6 onboard sata ports. If I go with 2 USB flash drives for a mirrored rpool do you think that would be ok? Performance seems to be about the same as 5400 rpm laptop drives. I hope that helps. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
Using a new email client and didn't notice that I didn't reply to the list. Since it might be helpful to others here are the missing bits. On 7/21/2010 5:07 PM, Freddie Cash wrote: We use the 500 GB versions attached to 3Ware controllers (configured as Single Disk arrays). They work quite nicely. Thanks, your reply was very helpful. Did you use the TLER utility to change your Caviar drives? Have you installed any of the new Caviar drives that don't support changing TLER settings? I forgot to mention, this is for a home server. I want something to backup my other computers and provide some network storage for stuff that doesn't that doesn't get used often but gets used frequently enough or are too big that optical storage would be a pain. Something along the lines of this http://blogs.sun.com/mebius/entry/diy_home_nas_box_with2 or smaller if I could use the 2.5 Scorpio drives. I'm willing to sacrifice performance for power and heat savings. This is the only thread I found regarding zfs and scorpio drives http://mail.opensolaris.org/pipermail/opensolaris-discuss/2009-June/048287.html It sounds like your 3Ware controller might be catching the errors before it gets to zfs but if zfs had to handle it then it might have the problems others have reported. If that's the case, *why can't zfs do what the 3ware controller is doing?* Or does it and these reports about problems with consumer drives and TLER are overblown? A 20s delay every couple of weeks I can live with. With the mini-itx mobo I'm planning to use I'm limited to PCI cards and there isn't much selection. The SiL based cards are supported in OpenSolaris but I'm not sure if they're going to behave the same as your 3ware card. I found this quote here http://hardforum.com/showthread.php?t=1500505 which contradicts a lot of what I read regarding ZFS and consumer drives. Do i need to use TLER or RAID edition harddrives? No and if you use TLER you should disable it when using ZFS. TLER is only useful for mission-critical servers who cannot afford to be frozen for 10-60 seconds, and to cope with bad quality RAID controller that panic when a drive is not responding for multiple seconds because its performing recovery on some sector. Do not use TLER with ZFS! Instead, allow the drive to recover its errors. ZFS will wait, the wait time can be configured. You won't have broken RAID arrays, which is common with Windows-based FakeRAID arrays. And Freddie Cash replied : - hide quoted text - On 7/21/2010 5:07 PM, Freddie Cash wrote: We use the 500 GB versions attached to 3Ware controllers (configured as Single Disk arrays). They work quite nicely. Thanks, your reply was very helpful. Did you use the TLER utility to change your Caviar drives? Have you installed any of the new Caviar drives that don't support changing TLER settings? The wdtler utility doesn't work on the Caviar Green drives. We did use the wdidle3 utility, though, to disable the 8 second idle timeout. Haven't needed to use the wdtler utility on any of the Caviar Black or Caviar Blue drives. They don't cause any issues. - hide quoted text - I forgot to mention, this is for a home server. I want something to backup my other computers and provide some network storage for stuff that doesn't that doesn't get used often but gets used frequently enough or are too big that optical storage would be a pain. Something along the lines of this http://blogs.sun.com/mebius/entry/diy_home_nas_box_with2 or smaller if I could use the 2.5 Scorpio drives. I'm willing to sacrifice performance for power and heat savings. This is the only thread I found regarding zfs and scorpio drives http://mail.opensolaris.org/pipermail/opensolaris-discuss/2009-June/048287.html If you want low-power, then the Green drives will be fine. So long as you don't want super-fast throughput. These are 5900 RPM drives that can slow down to 3400 RPM or thereabouts. But you definitely want to play with wdidle3 to change the default head-parking idle timeout. Otherwise you'll burn through the 500,000 load-store cycles in a couple of months. - hide quoted text - It sounds like your 3Ware controller might be catching the errors before it gets to zfs but if zfs had to handle it then it might have the problems others have reported. If that's the case, why can't zfs do what the 3ware controller is doing? Or does it and these reports about problems with consumer drives and TLER are overblown? A20s delay every couple of weeks I can live with. No idea. I don't have any ZFS systems using non-RAID controllers yet. -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [?] - What is the recommended number of disks for a consumer PC with ZFS
You haven't stated what you intend to use your pc for and what your requirements are. Without that I don't see how anyone can come up with an optimal configuration. So... what do you plan to do with your pc? Do you want the fastest performance and don't care about anything else? use all SSDs and add them as single vdevs without redundancy but make sure you have a good external backup. Do you want really faster performance and care about uptime? Use all SSDs and add them in mirrored pairs. Do you need a lot of storage space and don't want to spend thousands on SSDs? use the included ssd for your boot drive and set up a data pool of 2x2 mirror. I could go on and on. because there are many different uses for consumer pcs. For most people, a single ssd and a single data drive will be good as long as they have external backup. Today's consumer level HDDs are pretty fast. They get slow as they fill up. If the drive is only half filled and defraged it will perform faster than if it was 90% full. Unfortunately zfs doesn't have a defrag utility yet so if performance is really a concern maybe short stroking it to half size might be good. Using the other half as a hot spare as you suggested seems interesting if the drive is in a redundant configuration. If you really need performance so bad that you're considering short stroking your drives just get more SSDs. The gateway pc you linked to comes with 1 80 GB SSD and 1 1TB HDD. In most configurations you'd install the OS on the SSD and use the HDD for data and that configuration will be good for most applications. I don't see the point of dual booting any more. If you want OpenSolaris to be your main OS but want to be able to run windows there are other options like Wine and virtualization like VirtualBox. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Confused about consumer drives and zfs can someone help?
I wanted to build a small back up (maybe also NAS) server using OpenSolaris and ZFS using consumer drives but after reading a number of threads and blogs I'm totally confused and was hoping I could get some questions answered since many people have been using consumer drives with zfs. When ZFS first was announced I remember reading that it's error correction could make consumer level disks suitable for RAID applications but that doesn't seem to be the case. I'm familiar and have had good experience with WD drives so I'll be mainly referring to them. 1. WD Caviar Black Can they be used with in raidz or mirrors? With the new models the firmware is locked and you can't change the TLER settings. When a drive detects an error it's going to hang a long time trying to correct the problem before it reports the error to ZFS right? What type of errors are we talking about, bad blocks or something more serious? Once it reports an error zfs marks the drive as offline? In the case of something like bad blocks, can the drive somehow be salvaged by marking off the bad blocks and reducing the size of the vdev or does it need to be replaced? Because of the TLER issue, does this mean that drives that would have been able to remain in service as just an ordinary stand alone drive would need to be replaced so that you can expect to replace consumer drives like the WD Caviar Blacks to not last as long using ZFS? 2. WD Caviar Greens * I was hoping to use low power drives like the WD Caviar Greens. In addition to the TLER issue they have an issue with too many load/unload cycles that would wear out the drive faster when used in raid configs including zfs. WD put out a utility that could increase the time to reduce the load/unload cycles (wdidle.exe). Does this still work with the caviar green drives? Even if it does work, changing the idle behavior is going to make it use more energy right? Are Caviar Green drives (the new ones anyway) completely unsuitable for a ZFS based back-up or NAS server? 3. WD Scorpio Blue/Black *** It seems like the greens are out so these might be a better low power option? These are the 2.5 SATA laptop drives. These seem like a great drive to use in a small backup server because of their size and power usage. If you don't need large capacity but want multiple drives for redundancy could these be used? Storage density isn't so bad considering you can fit 2 in the space of one 3.5 drive. It would be great if these drives could be used but I haven't been able to find much info using them with zfs. To be able to save money and space it would be nice to at least have the option of using 2 of these mirrored as the rpool. The 5400RPM Scorpio Blue drives with their low power requirements and low heat would make a nice home backup server if they could work with ZFS. I've read about people using them in a RAID0 config in their laptops so maybe there's hope? There's a 1TB Scorpio blue that uses Advanced Formatting (whatever that is) will that be a problem for ZFS? 4 of them can fit in 1 5.25 bay and cost around $700 for 3TB. 4. Consumer vs High End Controllers I read a something along the lines of 'you can use consumer drives with low end sata cards but the enterprise drives like the RE3 and Velociraptor need to be used with high end sata raid cards'. Can someone clarify this for me please? I was planning on using something like the Syba SY-PCI40010 (http://www.newegg.com/Product/Product.aspx?Item=N82E16816124028cm_re=Sil3124-_-16-124-028-_-Product) which uses a SiL3124 chipset. I'm not great with this type of hardware but my understanding is that even though it has RAID features it's a software raid card and when you don't use the RAID features it's just a plain SATA port multiplier. This really confused me because I thought any hardware or software raid card is just a sata port replicator when you're not letting the card do the RAID. In terms of performance, problems, reliability, drive life, what can I expect under the 4 variations: SiL Controller with caviar drives? Sil Controller with RE3 drives? High end raid controller with caviar drives? High end raid controller with RE3 drives? 5. Mirror vs raidz ** Can any of the issues with consumer drives be reduced using one type of vdev over the other? Will adding a seperate log or cache device help? For a backup server, which would you choose, 4 drives in raidz, 4 drives in raidz2, 3 drives in raidz with hs, 4 drives with 2 mirrored pairs? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
