On Wed, May 25, 2011 at 2:23 PM, Edward Ned Harvey < opensolarisisdeadlongliveopensola...@nedharvey.com> wrote:
> I've finally returned to this dedup testing project, trying to get a handle > on why performance is so terrible. At the moment I'm re-running tests and > monitoring memory_throttle_count, to see if maybe that's what's causing the > limit. But while that's in progress and I'm still thinking... > > > > I assume the DDT tree must be stored on disk, in the regular pool, and each > entry is stored independently from each other entry, right? So whenever > you're performing new unique writes, that means you're creating new entries > in the tree, and every so often the tree will need to rebalance itself. By > any chance, are DDT entry creation treated as sync writes? If so, that > could be hurting me. For every new unique block written, there might be a > significant amount of small random writes taking place that are necessary to > support the actual data write. Anyone have any knowledge to share along > these lines? > > The DDT is a ZAP object, so it is an on-disk hashtable, free of O(log(n)) rebalancing operations. It is written asynchronously, from syncing context. That said, for each block written (unique or not), the DDT must be updated, which means reading and then writing the block that contains that dedup table entry, and the indirect blocks to get to it. With a reasonably large DDT, I would expect about 1 write to the DDT for every block written to the pool (or "written" but actually dedup'd). --matt
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
