Re: [zfs-discuss] Question about ZFS Incremental Send/Receive

2009-04-29 Thread Darren J Moffat

Mattias Pantzare wrote:

O> I feel like I understand what tar is doing, but I'm curious about what is it

that ZFS is looking at that makes it a "successful" incremental send? That
is, not send the entire file again. Does it have to do with how the
application (tar in this example) does a file open, fopen(), and what mode
is used? i.e. open for read, open for write, open for append. Or is it
looking at a file system header, or checksum? I'm just trying to explain
some observed behavior we're seeing during our testing.

My proof of concept is to remote replicate these "container files", which
are created by a 3rd party application.


ZFS knows what blocks where written since the first snapshot was taken.

Filenames or type of open is not important.

If you open a file and rewrite all blocks in that file with the same
content all those block will be sent. If you rewrite 5 block only 5
blocks are sent (plus the meta data that where updated).


Providing the application doing this does exactly what you said and not 
what a lot of apps (particularly "editors") do which is write in a tmp 
file, unlink and rename.



The way it works is that all blocks have a time stamp. Block with a time
stamp newer that the first snapshot will be sent.


Not really a time stamp but a transaction group number.

--
Darren J Moffat
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Question about ZFS Incremental Send/Receive

2009-04-28 Thread Mattias Pantzare
O> I feel like I understand what tar is doing, but I'm curious about what is it
> that ZFS is looking at that makes it a "successful" incremental send? That
> is, not send the entire file again. Does it have to do with how the
> application (tar in this example) does a file open, fopen(), and what mode
> is used? i.e. open for read, open for write, open for append. Or is it
> looking at a file system header, or checksum? I'm just trying to explain
> some observed behavior we're seeing during our testing.
>
> My proof of concept is to remote replicate these "container files", which
> are created by a 3rd party application.

ZFS knows what blocks where written since the first snapshot was taken.

Filenames or type of open is not important.

If you open a file and rewrite all blocks in that file with the same
content all those block will be sent. If you rewrite 5 block only 5
blocks are sent (plus the meta data that where updated).

The way it works is that all blocks have a time stamp. Block with a time
stamp newer that the first snapshot will be sent.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss