Roy Sigurd Karlsbakk wrote:
Hi all
I've been doing a lot of testing with dedup and concluded it's not really ready
for production. If something fails, it can render the pool unuseless for hours
or maybe days, perhaps due to single-threded stuff in zfs. There is also very
little data available in the docs (though I've from what I've got on this list)
on how much memory one should have for deduping an xTiB dataset.
I think it was Richard a month or so ago that had a good post about
about how much space the Dedup Table entry would be (it was in some
discussion where I ask about it). I can't remember what it was (a
hundred bytes?) per DDT entry, but one had to remember that each entry
was for a slab, which can vary in size (512 bytes to 128k). So, there's
no good generic formula for X bytes in RAM per Y TB space. You can
compute a rough guess if you know what kind of data and the general
usage pattern is for the pool (basically, you need to take a stab at
how big you think the average slab size is). Also, remember that if
you have a /very/ good dedup ratio, then you will have a smaller DDT for
a given X size pool, vs a pool with poor dedup ratios.
Unfortunately, there's no magic bullet, though if you can dig up
Richard's post, you should be able to take a guess, and not be off more
than x2 or so.
Also, remember you only need to hold the DDT in L2ARC, not in actual
RAM, so buy that SSD, young man!
As far as failures, well, I can't speak to that specifically. Though, do
realize that not having sufficient L2ARC/RAM to hold the DDT does mean
that you spend an awful amount of time reading pool metadata, which
really hurts performance (not to mention can cripple deleting of any
sort...)
Does anyone know how the status is for dedup now? In 134 it doesn't work very
well, but is it better in ON140 etc?
Honestly, I don't see it being much different over the last couple of
builds. The limitations are still there, but given those ones, I've
found it works well.
--
Erik Trimble
Java System Support
Mailstop: usca22-123
Phone: x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
