On Thu, Jun 10, 2010 at 04:32:06PM -0700, Erik Trimble wrote: > On 6/10/2010 1:21 PM, Pawel Jakub Dawidek wrote: > > > >If we send incremental stream we can be sure that up to the previous > >snapshot we have the same data on the other side. I'm aware it doesn't > >mean the data has exactly the same checksum (eg. it can be compressed > >with different algorithm). But in theory, are we able to figure out that > >the given block we try to send is already part of the dataset's previous > >snapshot? I'm fine with discarding incremental stream on the remote site > >if it uses different compression algorithm or simply deduplication is > >turned off (bascially when there is no block matching stored checksum). > >But if I have identical configurations on both ends I'd like not to send > >the same block multiple times in multiple incremental streams > > No, you can't be sure. You can *assume* you sent the proper incremental > stream to the receiving host, but what if you didn't? Or it got deleted? > etc.
So for this to work, the following conditions have to be meet: 1. Pools configuration on both sides have to be identical - the same checksum algorithms, the same compression algorithms, etc. 2. No snapshots can be removed on remote site as we can lose block by doing this. 3. We have to have all datasets on the remote site, as it would be too expensive to find if the given block which exists in DDT is referenced by the given dataset. If I want to send a block and it exists in DDT with refcount > 1, I've no way to tell which datasets are referencing it besides from scanning all datasets (or at least my dataset). If those conditions are meet, I can safely send checksums of blocks with brith date from before the snapshot I'm sending. Am I right? -- Pawel Jakub Dawidek http://www.wheelsystems.com [email protected] http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am!
pgpp9IbzyYVRy.pgp
Description: PGP signature
_______________________________________________ zfs-code mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/zfs-code
