Hello all, in IRC we had a discussion on how we could solve sending live subvolumes and how to send subvolumes without the need to administrate/keep old snapshots for incremental sends. One of the ideas was to introduce "sendshots", which are basically snapshots where no refs are counted for file data. This means, that when file data is changed in the sendshot origin, we do not consume extra space for two copies of the data. We would only have the metadata duplicated.
For the initial btrfs send we could do this: 1. Create a hidden read-only snapshot of the subvolume to send. Hidden means that it's not referenced by any subvolume. It is however still a normal snapshot (not a sendshot!). Hidden snapshots are not possible atm so we would have to implement that. This step allows us to send read-write subvolumes, because we have a freezed version of it. 2. Send this new snapshot. 3. When we're done with sending, create a "sendshot" from the snapshot and delete the invisible snapshot. As an alternative, we could convert the invisible snapshot to a sendshot...but not sure if that would be easy to implement. When we later do an incremental send we can do this: 1. Do the same as point 1. from above. 2. Determine which of the previous sendshots is the correct one for the incremental send. We could use some magic auto detection here or the user has to specify it by himself. 3. Use the hidden snapshot from 1. and the determined sendshot from 2. to find the incremental changes and do the send. 4. Do the same as point 3. from above. Every incremental send will add a new sendshot for later use. To avoid having millions of such sendshots after some time, btrfs-progs would need to delete old ones. That's something the user needs 100% control of, as only he knows which ones can be deleted. He could either delete them by hand or let btrfs send do that automatically with a parameter that for example says how much sendshots to keep. The above steps would already make the use of btrfs send/receive a bit easier. The next step would be to implement a network protocol that allows on-the-fly sending/receiving without piping to a file in-between. The protocol would allow the sending and receiving side to agree on the sendshot to use for the incremental send. It would also allow the sending side to do all the sendshot cleanups on its own, because it would know which state is present on the receiving side. What do you guys think? Problem is, I probably won't be able to implement this due to missing time for the rest of this year...going on a world trip and I don't know when I'm back :) So, if anyone wants too take this idea and implement it, feel free to do so :) Alex. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
