Re: Send/receive snapshot from/between backup
On 11/02/2016 05:13 PM, Piotr Pawłow wrote: > On 02.11.2016 15:23, René Bühlmann wrote: >> Origin: S2 S3 >> >> USB: S1 S2 >> >> SSH: S1 >> >> Transferring S3 to USB is no problem as S2 is on both btrfs drives. But >> how can I transfer S3 to SSH? > If I understand correctly how send / receive works, for the incremental > receive to work there must be a subvolume on the destination which has > "received uuid" equal to the uuid of parent choosen for the incremental > send. > >> I tried to transfer... >> >> 1. S3 from Origin to SSH -> does not work as there is no common snapshot. >> >> 2. S2 from USB to SSH -> did not work. > The "received uuid" of S1 on SSH is the uuid S1 had on Origin. The uuid > of S1 on USB is different, so when choosen as parent for the incremental > send it doesn't match. > >> 3. S1 from USB to Origin (such that there is a common snapshot with SSH) >> -> did not work. > There are no previously received subvolumes on Origin at all, so it > isn't going to work. > >> Is it correct that 1. would work if a common snapshot is present on >> Origin and SSH? > If there was a snapshot received from Origin that still exists on > Origin, then yes, you could use it as a clone source for incremental send. > >> Is it expected that 2. and 3. do not work? >> >> Is there some other way to achieve it? > I doubt you can do it without some "hacking" to fool btrfs receive. > > You would need a tool that can issue BTRFS_IOC_SET_RECEIVED_SUBVOL ioctl > to change the received uuid. Then you could: > > 1. Change received uuid of S1 on SSH to match S1 uuid on USB. > 2. Send incremental S1-S2 from USB to SSH. > 3. Change received uuid of S2 on SSH to match S2 on Origin. > 4. Send incremental S2-S3 from Origin to SSH. > > Regards > Thanks for all the input, I did successfully try this approach, could change the "received uuid" and then transfer a snapshot from a different source. So far so good. But: Due to a lot of errors during btrfs check on SSH, I decided to recreate the BTRFS filesystem on SSH still with the goal to not transfer all the data over the network. These were the steps: 1. Create a new btrfs (calling it SSH') 2. Full transfer S1 from SSH to SSH' 3. Incremental transfer S2 from USB to SSH' (S1 as parent) 4. Incremental transfer S3 from Origin to SSH' (S2 as parent) 5. Btrfs check SSH' 6. Used rsync (with checksum-diff) to verify that S3 on Origin and SSH' contain the same files. Step 2 did work and beside of a single checksum error on SSH the transfer completed without errors. Step 3 and 4 did work as well and surprisingly, I did not even had to update the "received uuid". They cant be full transfers as that would have taken months with my bandwidth. How can this be? Step 5 did not return any errors Step 6 did find a single file differing which is due to the checksum error on step 2. So, everything seems to be fine now, I just do not understand why this did work without any updating of the UUID. Do you have an explanation for that? In any case, thanks for your help. René -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Send/receive snapshot from/between backup
On 11/02/2016 03:49 PM, Hans van Kranenburg wrote: > On 11/02/2016 03:23 PM, René Bühlmann wrote: >> I have a system running on btrfs which is backed up to two (one local >> USB and one through SSH) external drives (also using btrfs) using >> send/receive (with the help of btrbk). For each backup a snapshot is >> created and transferred to the external drives. After some time, the >> snapshot on the origin is removed but kept on the two backups. >> >> Now my problem is, that the SSH-drive has not received any snapshots for >> some time such that there is no snapshot any more which is on the origin >> AND the SSH-drive. I would like to prevent transferring a full snapshot >> over the network and am now trying to work around it. >> >> Here the current situation (S? for a Snapshot): >> >> Origin: S2 S3 >> >> USB: S1 S2 >> >> SSH: S1 >> >> Transferring S3 to USB is no problem as S2 is on both btrfs drives. But >> how can I transfer S3 to SSH? > Since there is no snapshot left that is present on both drives, you > cannot use incremental send. > > btrbk does not remember what snapshots were last transferred or were > present on the missing disk, since the tool does not do any extra meta > administation on top of just having the snapshots on btrfs level. So > it's figuring out the relationships between each of them (combining it > with info from the remotes) again every time it runs. Yes this explains why 1. did not work. But why can't I transfer S1 from USB back to Origin as incremental to S2? And why can't I transfer S2 from USB to SSH? Is S1 not recognized as the same snapshot there? > >> I tried to transfer... >> >> 1. S3 from Origin to SSH -> does not work as there is no common snapshot. >> >> 2. S2 from USB to SSH -> did not work. >> >> 3. S1 from USB to Origin (such that there is a common snapshot with SSH) >> -> did not work. >> >> Is it correct that 1. would work if a common snapshot is present on >> Origin and SSH? >> >> Is it expected that 2. and 3. do not work? >> >> Is there some other way to achieve it? > At home, I do a similar thing, I periodically send/receive changes of a > filesystem to two external disks, also using btrbk. Since I don't want > to have both backup disks and the originating filesystem all online and > in the same geographical location at the same moment, I have the same > problem. > > What I did is just setting expiry the snapshots on the origin manually, > and keep the meta-administration in my head. I don't do it that often > anyway. > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Send/receive snapshot from/between backup
Hi all, I have a system running on btrfs which is backed up to two (one local USB and one through SSH) external drives (also using btrfs) using send/receive (with the help of btrbk). For each backup a snapshot is created and transferred to the external drives. After some time, the snapshot on the origin is removed but kept on the two backups. Now my problem is, that the SSH-drive has not received any snapshots for some time such that there is no snapshot any more which is on the origin AND the SSH-drive. I would like to prevent transferring a full snapshot over the network and am now trying to work around it. Here the current situation (S? for a Snapshot): Origin: S2 S3 USB: S1 S2 SSH: S1 Transferring S3 to USB is no problem as S2 is on both btrfs drives. But how can I transfer S3 to SSH? I tried to transfer... 1. S3 from Origin to SSH -> does not work as there is no common snapshot. 2. S2 from USB to SSH -> did not work. 3. S1 from USB to Origin (such that there is a common snapshot with SSH) -> did not work. Is it correct that 1. would work if a common snapshot is present on Origin and SSH? Is it expected that 2. and 3. do not work? Is there some other way to achieve it? Thanks for the help René -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
