There are a number of threads (this one[1] for example) that describe memory requirements for deduplication. They're pretty high.
I'm trying to get a better understanding... on our NetApps we use 4K block sizes with their post-process deduplication and get pretty good dedupe ratios for VM content. Using ZFS we are using 128K record sizes by default, which nets us less impressive savings... however, to drop to a 4K record size would theoretically require that we have nearly 40GB of memory for only 1TB of storage (based on 150 bytes per block for the DDT). This obviously becomes prohibitively higher for 10+ TB file systems. I will note that our NetApps are using only 2TB FlexVols, but would like to better understand ZFS's (apparently) higher memory requirements... or maybe I'm missing something entirely. Thanks, Ray [1] http://markmail.org/message/wile6kawka6qnjdw _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
