Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
I've been looking at using consumer 2.5" drives also, I think the ones I've settled on are the hitachi 7K500 500 GB. These are 7200 rpm, I'm concerned the 5400's might be a little too low performance wise. The main reasons for hitachi were performance seems to be among the top 2 or 3 in the laptop drive segment, I've found hitachi to be pretty reliable, and perhaps most importantly is there is the hitachi feature tool, which allows you to disable the head unload feature. You don't need to set it on each reboot, plus it's persistent across reboots. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On Jul 24, 2010, at 5:37 PM, JavaWebDev wrote: > On 7/24/2010 8:12 PM, Edward Ned Harvey wrote: >>> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- >>> boun...@opensolaris.org] On Behalf Of Russ Price >>> >>> Good advice - ZFS can use quite a lot of CPU cycles. A low-end AMD >>> quad-core is >> I know "a lot of CPU cycles" is a relative term. But I never notice CPU >> utilization, even under the heaviest loads I can generate. Note: I'm not >> generally using compression (which will require CPU) and I'm not using >> dedupe (which will require RAM). >> >> Still, I don't think it's fair or accurate to generalize and say "ZFS will >> use a lot of CPU cycles," unless you're qualifying it specifically such as >> "if you have compression enabled." > > I was wondering about that too because I've been seeing a lot of zfs builds > with atom processors and haven't complained about cpu utilization given a > small home server application. > > Sid question... I recently ran across this blog post that indicates raidz and > raidz2 don't increase performance over single drive performance unlike raid5. > The post was old so I was wondering if that was still true. > http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance This model is for small, random read performance only. I'm perfectly happy to entertain models which can accurately predict performance when caches are present. But, I've never seen one that is accurate :-( -- richard -- ZFS and performance consulting http://www.RichardElling.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On 7/24/2010 8:12 PM, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Russ Price Good advice - ZFS can use quite a lot of CPU cycles. A low-end AMD quad-core is I know "a lot of CPU cycles" is a relative term. But I never notice CPU utilization, even under the heaviest loads I can generate. Note: I'm not generally using compression (which will require CPU) and I'm not using dedupe (which will require RAM). Still, I don't think it's fair or accurate to generalize and say "ZFS will use a lot of CPU cycles," unless you're qualifying it specifically such as "if you have compression enabled." I was wondering about that too because I've been seeing a lot of zfs builds with atom processors and haven't complained about cpu utilization given a small home server application. Sid question... I recently ran across this blog post that indicates raidz and raidz2 don't increase performance over single drive performance unlike raid5. The post was old so I was wondering if that was still true. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Russ Price > > Good advice - ZFS can use quite a lot of CPU cycles. A low-end AMD > quad-core is I know "a lot of CPU cycles" is a relative term. But I never notice CPU utilization, even under the heaviest loads I can generate. Note: I'm not generally using compression (which will require CPU) and I'm not using dedupe (which will require RAM). Still, I don't think it's fair or accurate to generalize and say "ZFS will use a lot of CPU cycles," unless you're qualifying it specifically such as "if you have compression enabled." ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On 07/23/2010 02:39 AM, tomwaters wrote: > Re the CPU, do not go low power Atom etc, go a newish > Core2 duo...the power differential at idle is bugger all > and when you want to use the nas, ZFS will make good use > of the CPU. Good advice - ZFS can use quite a lot of CPU cycles. A low-end AMD quad-core is another good choice here. Even better, the AMD chips support ECC RAM, though you must get a motherboard with a BIOS that supports it. I have an Athlon II X4 630 in my setup, and it has plenty of horsepower for an eight-disk RAIDZ2. > re. cards...I use and recommend these 8-Port SUPERMICRO > AOC-USASLP-L8I UIO SAS. They are cheap on e-bay, just work > and are fast. Use them. Keep in mind you'll have to modify the bracket to make the card fit (unless you're using a Supermicro UIO case). Another good alternative with the same chipset is the Intel SASUC8I. This is sold as a bare card, and you need to order the fan-out cables separately. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On 7/23/2010 3:39 AM, tomwaters wrote: There is alot there to reply to...but I will try and help... Re. TLER. Do not worry about TLER when using ZFS. ZFS will handle it either way and will NOT time out and drop the drive...it may wait a long time, but it will not time out and drop the drive - nor will it have an issue if you do enable TLER-ON (which sets time out to 7 seonds). I run both with TLER-ON (disks from an "old" mdadm raid array) and without (TLER-OFF) 1.5TB WDEADS. That's reassuring. I guess the only time the issue will come up is when the drive starts to develop errors. I can not speak for the WD EARS, but the WDEADS are fine in my home nas. I also run 1.5TB Samsung Green/Silencer series and Seagate 11's. Others swear by Hitachi. I would recommend the Samsung or Hitachi and not the new WD EARS which have that 4k sectors or whatever it is. I've been digging through the archives more and I'm starting to lean towards laptop drives. I found this discussion http://opensolaris.org/jive/message.jspa?messageID=468269#468269 and contacted someone else who has used 3 WD Scorpio Black drives in a 3 way mirror for a year without any problems. This is mostly going to be a backup server so I'm not too worried about performance. Re the CPU, do not go low power Atom etc, go a newish Core2 duo...the power differential at idle is bugger all and when you want to use the nas, ZFS will make good use of the CPU. Honestly, sit down and do the calculations on power savings of a low power cpu and you'll see it's better to just not have that 5th beer on a friday - you'll save more money that way and be MUCH happier with your nas performance. re. cards...I use and recommend these 8-Port SUPERMICRO AOC-USASLP-L8I UIO SAS. They are cheap on e-bay, just work and are fast. Use them. You do want alot of ram I use 8GB, but you can use 4. Ram is cheap, ZFS loves ram, just buy 8. IMHO (and that of the best practice guide) - you should mirror the rpool (o/s disk). Just buy 2 cheap laptop drives and when installing choose to "mirror" them. I can find some motherboards with 4-6 onboard sata ports. If I go with 2 USB flash drives for a mirrored rpool do you think that would be ok? Performance seems to be about the same as 5400 rpm laptop drives. I hope that helps. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
I've found the Seagate 7200.12 1tb drives and Hitachi 7k2000 2TB drives to be by far the best. I've read lots of horror stories about any WD drive with 4k sectorsit'sbest to stay away from them. I've also read plenty of people say that the green drives are terrible. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
There is alot there to reply to...but I will try and help... Re. TLER. Do not worry about TLER when using ZFS. ZFS will handle it either way and will NOT time out and drop the drive...it may wait a long time, but it will not time out and drop the drive - nor will it have an issue if you do enable TLER-ON (which sets time out to 7 seonds). I run both with TLER-ON (disks from an "old" mdadm raid array) and without (TLER-OFF) 1.5TB WDEADS. I can not speak for the WD EARS, but the WDEADS are fine in my home nas. I also run 1.5TB Samsung Green/Silencer series and Seagate 11's. Others swear by Hitachi. I would recommend the Samsung or Hitachi and not the new WD EARS which have that 4k sectors or whatever it is. Re the CPU, do not go low power Atom etc, go a newish Core2 duo...the power differential at idle is bugger all and when you want to use the nas, ZFS will make good use of the CPU. Honestly, sit down and do the calculations on power savings of a low power cpu and you'll see it's better to just not have that 5th beer on a friday - you'll save more money that way and be MUCH happier with your nas performance. re. cards...I use and recommend these 8-Port SUPERMICRO AOC-USASLP-L8I UIO SAS. They are cheap on e-bay, just work and are fast. Use them. You do want alot of ram I use 8GB, but you can use 4. Ram is cheap, ZFS loves ram, just buy 8. IMHO (and that of the best practice guide) - you should mirror the rpool (o/s disk). Just buy 2 cheap laptop drives and when installing choose to "mirror" them. I hope that helps. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
Using a new email client and didn't notice that I didn't reply to the list. Since it might be helpful to others here are the missing bits. On 7/21/2010 5:07 PM, Freddie Cash wrote: We use the 500 GB versions attached to 3Ware controllers (configured as Single Disk arrays). They work quite nicely. Thanks, your reply was very helpful. Did you use the TLER utility to change your Caviar drives? Have you installed any of the new Caviar drives that don't support changing TLER settings? I forgot to mention, this is for a home server. I want something to backup my other computers and provide some network storage for stuff that doesn't that doesn't get used often but gets used frequently enough or are too big that optical storage would be a pain. Something along the lines of this http://blogs.sun.com/mebius/entry/diy_home_nas_box_with2 or smaller if I could use the 2.5" Scorpio drives. I'm willing to sacrifice performance for power and heat savings. This is the only thread I found regarding zfs and scorpio drives http://mail.opensolaris.org/pipermail/opensolaris-discuss/2009-June/048287.html It sounds like your 3Ware controller might be catching the errors before it gets to zfs but if zfs had to handle it then it might have the problems others have reported. If that's the case, *why can't zfs do what the 3ware controller is doing?* Or does it and these reports about problems with consumer drives and TLER are overblown? A <20s delay every couple of weeks I can live with. With the mini-itx mobo I'm planning to use I'm limited to PCI cards and there isn't much selection. The SiL based cards are supported in OpenSolaris but I'm not sure if they're going to behave the same as your 3ware card. I found this quote here http://hardforum.com/showthread.php?t=1500505 which contradicts a lot of what I read regarding ZFS and consumer drives. "Do i need to use TLER or RAID edition harddrives? No and if you use TLER you should disable it when using ZFS. TLER is only useful for mission-critical servers who cannot afford to be frozen for 10-60 seconds, and to cope with bad quality RAID controller that panic when a drive is not responding for multiple seconds because its performing recovery on some sector. Do not use TLER with ZFS! Instead, allow the drive to recover its errors. ZFS will wait, the wait time can be configured. You won't have broken RAID arrays, which is common with Windows-based FakeRAID arrays." And Freddie Cash replied : - hide quoted text - > On 7/21/2010 5:07 PM, Freddie Cash wrote: > We use the 500 GB versions attached to 3Ware controllers (configured > as Single Disk arrays). They work quite nicely. > > Thanks, your reply was very helpful. > > Did you use the TLER utility to change your Caviar drives? Have you > installed any of the new Caviar drives that don't support changing TLER > settings? The wdtler utility doesn't work on the Caviar Green drives. We did use the wdidle3 utility, though, to "disable" the 8 second idle timeout. Haven't needed to use the wdtler utility on any of the Caviar Black or Caviar Blue drives. They don't cause any issues. - hide quoted text - > I forgot to mention, this is for a home server. I want something to backup > my other computers and provide some network storage for stuff that doesn't > that doesn't get used often but gets used frequently enough or are too big > that optical storage would be a pain. > > Something along the lines of this > http://blogs.sun.com/mebius/entry/diy_home_nas_box_with2 or smaller if I > could use the 2.5" Scorpio drives. I'm willing to sacrifice performance for > power and heat savings. This is the only thread I found regarding zfs and > scorpio drives > http://mail.opensolaris.org/pipermail/opensolaris-discuss/2009-June/048287.html If you want low-power, then the Green drives will be fine. So long as you don't want super-fast throughput. These are 5900 RPM drives that can slow down to 3400 RPM or thereabouts. But you definitely want to play with wdidle3 to change the default head-parking idle timeout. Otherwise you'll burn through the 500,000 load-store cycles in a couple of months. - hide quoted text - > It sounds like your 3Ware controller might be catching the errors before it > gets to zfs but if zfs had to handle it then it might have the problems > others have reported. If that's the case, why can't zfs do what the 3ware > controller is doing? Or does it and these reports about problems with > consumer drives and TLER are overblown? A<20s delay every couple of weeks I > can live with. No idea. I don't have any ZFS systems using non-RAID controllers yet. -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
> I wanted to build a small back up (maybe also NAS) server using A common question that I am trying to get answered (and have a few) here: http://www.opensolaris.org/jive/thread.jspa?threadID=102368&tstart=0 Rob -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?
On Wed, Jul 21, 2010 at 1:38 PM, JavaWebDev wrote: > 1. WD Caviar Black > > Can they be used with in raidz or mirrors? We use the 500 GB versions attached to 3Ware controllers (configured as Single Disk arrays). They work quite nicely. > With the new models the firmware is locked and you can't change the TLER > settings. When a drive detects an error it's going to hang a long time > trying to correct the problem before it reports the error to ZFS right? What > type of errors are we talking about, bad blocks or something more serious? Haven't noticed any issues. The RAID controller notes the odd (maybe 1 per fortnight) timeout, but then reconnects the drive without ZFS noticing. We have yet (2 years) to have a drive timeout and cause ZFS to offline the drive. > 2. WD Caviar Greens > * > I was hoping to use low power drives like the WD Caviar Greens. In addition > to the TLER issue they have an issue with too many load/unload cycles that > would wear out the drive faster when used in raid configs including zfs. WD > put out a utility that could increase the time to reduce the load/unload > cycles (wdidle.exe). Does this still work with the caviar green drives? Even > if it does work, changing the idle behavior is going to make it use more > energy right? > > Are Caviar Green drives (the new ones anyway) completely unsuitable for a > ZFS based back-up or NAS server? We use 8x 1.5 TB WD Green drives, since they were really inexpensive at the time, and looked good on paper. We've been regretting it ever since. I actively pray for these to die so that I can replace them with Seagate 1.5 TB drives. For a home setup where you don't care about disk throughput, these may be workable. Otherwise, avoid the entire Green/GP series of WD disks. They are, to put it nicely, crap. Don't ask for an elaboration unless you want a multiple page diatribe on the horrors of using these drives. Just don't use them outside of the home. We also use WD Caviar Blue 500 GB, and RE2/RE3 500 GB drives attached to 3Ware controllers without issues. Finally, we also use Seagate Barracuda 7200.11 500 GB and 1.5 TB drives with great success. As wel as Seagate ES.2 500 GB drives. Basically, anything other than the WD Green/GP drives works well. > 5. Mirror vs raidz > ** > Can any of the issues with consumer drives be reduced using one type of vdev > over the other? Will adding a seperate log or cache device help? An L2ARC will help with reads, especially if using raidz vdevs. > For a backup server, which would you choose, 4 drives in raidz, 4 drives in > raidz2, 3 drives in raidz with hs, 4 drives with 2 mirrored pairs? Only 4 drives? Do they still make servers that only use 4 drives? ;) It all depends on which you prefer, raw throughput or redundancy. For best performance, use 2x 2-drive mirrors. For best redundancy, use 1x 4-drive raidz2. For middle-of-the-road performance/redundancy, use 1x 4-drive raidz1. Note: newegg.ca has a sale on right now. WD Caviar Black 1 TB drives are only $85 CDN. -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Confused about consumer drives and zfs can someone help?
I wanted to build a small back up (maybe also NAS) server using OpenSolaris and ZFS using consumer drives but after reading a number of threads and blogs I'm totally confused and was hoping I could get some questions answered since many people have been using consumer drives with zfs. When ZFS first was announced I remember reading that it's error correction could make consumer level disks suitable for RAID applications but that doesn't seem to be the case. I'm familiar and have had good experience with WD drives so I'll be mainly referring to them. 1. WD Caviar Black Can they be used with in raidz or mirrors? With the new models the firmware is locked and you can't change the TLER settings. When a drive detects an error it's going to hang a long time trying to correct the problem before it reports the error to ZFS right? What type of errors are we talking about, bad blocks or something more serious? Once it reports an error zfs marks the drive as offline? In the case of something like bad blocks, can the drive somehow be salvaged by marking off the bad blocks and reducing the size of the vdev or does it need to be replaced? Because of the TLER issue, does this mean that drives that would have been able to remain in service as just an ordinary stand alone drive would need to be replaced so that you can expect to replace consumer drives like the WD Caviar Blacks to not last as long using ZFS? 2. WD Caviar Greens * I was hoping to use low power drives like the WD Caviar Greens. In addition to the TLER issue they have an issue with too many load/unload cycles that would wear out the drive faster when used in raid configs including zfs. WD put out a utility that could increase the time to reduce the load/unload cycles (wdidle.exe). Does this still work with the caviar green drives? Even if it does work, changing the idle behavior is going to make it use more energy right? Are Caviar Green drives (the new ones anyway) completely unsuitable for a ZFS based back-up or NAS server? 3. WD Scorpio Blue/Black *** It seems like the greens are out so these might be a better low power option? These are the 2.5" SATA laptop drives. These seem like a great drive to use in a small backup server because of their size and power usage. If you don't need large capacity but want multiple drives for redundancy could these be used? Storage density isn't so bad considering you can fit 2 in the space of one 3.5" drive. It would be great if these drives could be used but I haven't been able to find much info using them with zfs. To be able to save money and space it would be nice to at least have the option of using 2 of these mirrored as the rpool. The 5400RPM Scorpio Blue drives with their low power requirements and low heat would make a nice home backup server if they could work with ZFS. I've read about people using them in a RAID0 config in their laptops so maybe there's hope? There's a 1TB Scorpio blue that uses "Advanced Formatting" (whatever that is) will that be a problem for ZFS? 4 of them can fit in 1 5.25" bay and cost around $700 for 3TB. 4. Consumer vs High End Controllers I read a something along the lines of 'you can use consumer drives with low end sata cards but the enterprise drives like the RE3 and Velociraptor need to be used with high end sata raid cards'. Can someone clarify this for me please? I was planning on using something like the Syba SY-PCI40010 (http://www.newegg.com/Product/Product.aspx?Item=N82E16816124028&cm_re=Sil3124-_-16-124-028-_-Product) which uses a SiL3124 chipset. I'm not great with this type of hardware but my understanding is that even though it has RAID features it's a software raid card and when you don't use the RAID features it's just a plain SATA port multiplier. This really confused me because I thought any hardware or software raid card is just a sata port replicator when you're not letting the card do the RAID. In terms of performance, problems, reliability, drive life, what can I expect under the 4 variations: SiL Controller with caviar drives? Sil Controller with RE3 drives? High end raid controller with caviar drives? High end raid controller with RE3 drives? 5. Mirror vs raidz ** Can any of the issues with consumer drives be reduced using one type of vdev over the other? Will adding a seperate log or cache device help? For a backup server, which would you choose, 4 drives in raidz, 4 drives in raidz2, 3 drives in raidz with hs, 4 drives with 2 mirrored pairs? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
