On 06/ 9/10 05:35 PM, Pawel Jakub Dawidek wrote:
Hi.
Currently zfs send has -D flag, which allows to deduplicate blocks
in single stream.
I'm wondering if it would be possible not to send blocks in incremental
stream if we know they are part of the given dataset already and were
sent to the remote site with some earlier snapshots.
I know deduplication is pool-wide mechanism and the block might be part
of many different datasets. In my case I'd need to know that the block
I'm about to send is part of this particular dataset.
With the current ZFS design, is something like this even possible to
implement in some clean way or would there be a need for heavy
modifcations of ZFS internals?
It's not possible to implement unless we establish a bidirectional
communication between the sending and receiving side. The logic for
send-stream dedup is:
for (each block to be written to stream) {
get the block's checksum
lookup the block's checksum in the dedup-table
established for *this* stream generation
if (an entry in the DDT exists for this checksum)
send a "write-by-reference" block across the stream
(this contains a reference to a block send earlier in the
stream)
else {
add an entry for this block to the DDT
send the full block
}
}
Since the dedup table on sending side only knows about blocks already
send in the stream, we have no way of knowing whether a copy of the
block already exists on the other side, and even if we did know, we
wouldn't know where it was on the other side. The sending side would
have to have a copy of the other side's on-disk DDT to know whether a
write-by-reference could be used.
Lori
_______________________________________________
zfs-code mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/zfs-code
