adding on... On Apr 28, 2010, at 8:57 AM, Tomas Ögren wrote:
> On 28 April, 2010 - Eric D. Mudama sent me these 1,6K bytes: > >> On Wed, Apr 28 at 1:34, Tonmaus wrote: >>>> Zfs scrub needs to access all written data on all >>>> disks and is usually >>>> disk-seek or disk I/O bound so it is difficult to >>>> keep it from hogging >>>> the disk resources. A pool based on mirror devices >>>> will behave much >>>> more nicely while being scrubbed than one based on >>>> RAIDz2. >>> >>> Experience seconded entirely. I'd like to repeat that I think we >>> need more efficient load balancing functions in order to keep >>> housekeeping payload manageable. Detrimental side effects of scrub >>> should not be a decision point for choosing certain hardware or >>> redundancy concepts in my opinion. >> >> While there may be some possible optimizations, i'm sure everyone >> would love the random performance of mirror vdevs, combined with the >> redundancy of raidz3 and the space of a raidz1. However, as in all >> systems, there are tradeoffs. >> >> To scrub a long lived, full pool, you must read essentially every >> sector on every component device, and if you're going to do it in the >> order in which your transactions occurred, it'll wind up devolving to >> random IO eventually. >> >> You can choose to bias your workloads so that foreground IO takes >> priority over scrub, but then you've got the cases where people >> complain that their scrub takes too long. There may be knobs for >> individuals to use, but I don't think overall there's a magic answer. > > We have one system with a raidz2 of 8 SATA disks.. If we start a scrub, > then you can kiss any NFS performance goodbye.. A single mkdir or > creating a file can take 30 seconds.. Single write()s can take 5-30 > seconds.. Without the scrub, it's perfectly fine. Local performance > during scrub is fine. NFS performance becomes useless. > > This means we can't do a scrub, because doing so will basically disable > the NFS service for a day or three. If the scrub would be less agressive > and take a week to perform, it would probably not kill the performance > as bad.. Which OS release? Later versions of ZFS have a scrub/resilver throttle of sorts. There is not an exposed interface to manage the throttle and I doubt there is much (if any) community experience with using it in real-world situations. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
