Re: btrfs-image and btrfs send related queries
Sorry. Its typo I used original disk /dev/sdb where filesystem is created and seeing these errors. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
> I have /dev/sdb , /dev/sdc. Using wipefs -fa I cleared both devices and > created btrfs on /dev/sdb. Mounted and written some files and unmounted it. > > Then I ran btrfs-image /dev/sdc /img1.img and got the dump. It looks like you imaged the wrong device, that might clarify the IO errors later on > Once image created I again ran wipefs -fa /dev/sdb. > > Then I ran btrfs-image -r /img1.img /dev/sdc and mounted /dev/sdc. > > ls to dumped filesystem shows the file size as original and no file content. > I guess btrfs-image doesn't modify files stat so i feel it is showing size > as original. > > However running cat on some of files give i/o error > > qd67:/btrfs1 # cat shadow.h > cat: shadow.h: Input/output error > > These errors are not on all files on other files, since dump doesn't > contains any data it just prompts for cat as below. > > qd67:/btrfs1 # cat stab.h > qd67:/btrfs1 # cat stdio_ext.h > > Not sure why i/o errors are coming. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
Henk Slager gmail.com> writes: > > On Mon, Apr 18, 2016 at 4:26 PM, Roman Mamedov romanrm.net> wrote: > > On Mon, 18 Apr 2016 16:13:28 +0200 > > Henk Slager gmail.com> wrote: > > > >> (your email keeps ending up in gmail spam folder) > >> > >> On Mon, Apr 18, 2016 at 9:24 AM, sri yahoo.co.in> wrote: > >> > I tried btrfs-image and created image file and ran btrfs-image -r to a > >> > different disk. Once recovered and mounted, I can able to see data is > >> > not zeroed out as mentioned in btrfs-image man page. > >> > >> "different disk" you mention, that is important info. If you doe the > >> restore to a image file, that image file is sparse and all data blocks > >> are read as zeros. > >> > >> However, if you restore to a block device, then you can assume it just > >> writes the device blocks for metadata and leaves the rest untouched. > >> So trim whole device first or brute-force overwrite completely with > >> zeros. > >> > >> So maybe the man pages needs some correction / extra notes. > >> > >> > I tried on same machine. > > > > Does btrfs-image store/restore the FS UUID? If it does, then potentially both > > the source FS and the restored one were visible at the same time to the kernel > > with identical UUIDs, and maybe it was actually accessing/mounting the source > > one. > > Very good point! The UUID's are the same. I remember I used a VM to > separate the source FS from the restored FS > > Also, the assumption I made about restoring to a block device is not > correct: If you restore to a loopdev that has a file with all > non-zeros as backing-store, the files in the mounted restored FS are > read as zeros. > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > thank you for inputs. Actually I have tried in the following way. I have /dev/sdb , /dev/sdc. Using wipefs -fa I cleared both devices and cre= ated btrfs on /dev/sdb. Mounted and written some files and unmounted it. Then I ran btrfs-image /dev/sdc /img1.img and got the dump. Once image created I again ran wipefs -fa /dev/sdb. Then I ran btrfs-image -r /img1.img /dev/sdc and mounted /dev/sdc. ls to dumped filesystem shows the file size as original and no file content= . I guess btrfs-image doesn't modify files stat so i feel it is showing siz= e as original. However running cat on some of files give i/o error qd67:/btrfs1 # cat shadow.hcat: shadow.h: Input/output error These errors are not on all files on other files, since dump doesn't contai= ns any data it just prompts for cat as below. qd67:/btrfs1 # cat stab.hqd67:/btrfs1 # cat stdio_ext.h Not sure why i/o errors are coming. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
On Mon, Apr 18, 2016 at 4:26 PM, Roman Mamedov wrote: > On Mon, 18 Apr 2016 16:13:28 +0200 > Henk Slager wrote: > >> (your email keeps ending up in gmail spam folder) >> >> On Mon, Apr 18, 2016 at 9:24 AM, sri wrote: >> > I tried btrfs-image and created image file and ran btrfs-image -r to a >> > different disk. Once recovered and mounted, I can able to see data is >> > not zeroed out as mentioned in btrfs-image man page. >> >> "different disk" you mention, that is important info. If you doe the >> restore to a image file, that image file is sparse and all data blocks >> are read as zeros. >> >> However, if you restore to a block device, then you can assume it just >> writes the device blocks for metadata and leaves the rest untouched. >> So trim whole device first or brute-force overwrite completely with >> zeros. >> >> So maybe the man pages needs some correction / extra notes. >> >> > I tried on same machine. > > Does btrfs-image store/restore the FS UUID? If it does, then potentially both > the source FS and the restored one were visible at the same time to the kernel > with identical UUIDs, and maybe it was actually accessing/mounting the source > one. Very good point! The UUID's are the same. I remember I used a VM to separate the source FS from the restored FS Also, the assumption I made about restoring to a block device is not correct: If you restore to a loopdev that has a file with all non-zeros as backing-store, the files in the mounted restored FS are read as zeros. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
On Mon, 18 Apr 2016 16:13:28 +0200 Henk Slager wrote: > (your email keeps ending up in gmail spam folder) > > On Mon, Apr 18, 2016 at 9:24 AM, sri wrote: > > I tried btrfs-image and created image file and ran btrfs-image -r to a > > different disk. Once recovered and mounted, I can able to see data is > > not zeroed out as mentioned in btrfs-image man page. > > "different disk" you mention, that is important info. If you doe the > restore to a image file, that image file is sparse and all data blocks > are read as zeros. > > However, if you restore to a block device, then you can assume it just > writes the device blocks for metadata and leaves the rest untouched. > So trim whole device first or brute-force overwrite completely with > zeros. > > So maybe the man pages needs some correction / extra notes. > > > I tried on same machine. Does btrfs-image store/restore the FS UUID? If it does, then potentially both the source FS and the restored one were visible at the same time to the kernel with identical UUIDs, and maybe it was actually accessing/mounting the source one. -- With respect, Roman pgpBYIv_huIUt.pgp Description: OpenPGP digital signature
Re: btrfs-image and btrfs send related queries
(your email keeps ending up in gmail spam folder) On Mon, Apr 18, 2016 at 9:24 AM, sri wrote: > I tried btrfs-image and created image file and ran btrfs-image -r to a > different disk. Once recovered and mounted, I can able to see data is > not zeroed out as mentioned in btrfs-image man page. "different disk" you mention, that is important info. If you doe the restore to a image file, that image file is sparse and all data blocks are read as zeros. However, if you restore to a block device, then you can assume it just writes the device blocks for metadata and leaves the rest untouched. So trim whole device first or brute-force overwrite completely with zeros. So maybe the man pages needs some correction / extra notes. > I tried on same machine. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
I tried btrfs-image and created image file and ran btrfs-image -r to a different disk. Once recovered and mounted, I can able to see data is not zeroed out as mentioned in btrfs-image man page. I tried on same machine. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
On Fri, Apr 15, 2016 at 2:49 PM, Hugo Mills wrote: > On Fri, Apr 15, 2016 at 12:41:36PM +, sri wrote: >> Hi, >> >> I have couple of queries related to btrfs-image, btrfs send and with >> combination of two. >> 1) >> I would like to know if a btrfs source file system is spread across more >> than 1 disks, does btrfs-image require same number of disks to create >> empty file system without files content?? > >I don't _think_ you need as many devices as there were originally. Indeed, if you runbtrfs-image -r on a dump from a multi device fs, you get 1 big fs image. I once did that for a 4x4TB RAID10 fs (tools v4.3.x at that time I believe), resulting in a 17TB (sparse) file. I was expecting that option -m would create multiple files, however, scanning the source-code revealed that there are things not implemented (it resulted in a 34T file that was simply not a valid fs). Or I did something wrong or there is a bug. For just my case, it was much quicker to patch the kernel so that it worked with the 17T file. There are then still issues w.r.t. devices, but data is missing so anyhow only a limited set of tool actions or issues can be researched with such a generated image. But for a multi-TB fs, the data volume is acceptable (roughly in 1G or 10G order). I think it would make sense that the btrfs-image restore output can be split into multiple files, so that the multidevice aspects are better represented (or modelled). >> 2) would btrfs-image can be modified to keep only given subvolume foot >> print and related meta data to bring back file system live on destination >> disk? >> >>To elaborate more on this, Lets say I have 5 subvolumes on source btrfs >> and i run btrfs-image written to destination disk say /dev/sdd. In this >> process, can btrfs-image modified to just have only 1 subvolume and skipp >> other 4 subvolumes and write to destination i.. /dev/sdd so that when I >> mount /dev/sdd , I will have btrfs with only 1 subvolume with no data. > >For a first approximation, you could just drop any FS tree from the > image which wasn't the target one. After that, it turns into a > complicated accounting exercise to drop all of the back-refs to the > missing FS trees, and to drop all the extent records for the > non-shared data and the metadata for the missing FS trees. > >It's probably going to be complicated, and will basically involve > rewriting most of the image to avoid the metadata you didn't want. > >> 3) If 3 can successful, can btrfs-image further changed to include data of >> selected subvolume which gives files data also written to /dev/sdd which >> would be kind of a backup of a subvolume taken out of a btrfs file system >> which is having more than 1 subvolumes. > >If you're going to do all the hard work of (2), then (3) is a > reasonable logical(?) extension. > >On the other hand, what's wrong with simply using send/receive? It > gives you a data structure (a FAR-format send stream) which contains > everything you need to reconstruct a subvolume on a btrfs different > to the original. > >Hugo. > > -- > Hugo Mills | Mary had a little lamb > hugo@... carfax.org.uk | You've heard this tale before > http://carfax.org.uk/ | But did you know she passed her plate > PGP: E2AB1DE4 | And had a little more? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-image and btrfs send related queries
On Fri, Apr 15, 2016 at 12:41:36PM +, sri wrote: > Hi, > > I have couple of queries related to btrfs-image, btrfs send and with > combination of two. > 1) > I would like to know if a btrfs source file system is spread across more > than 1 disks, does btrfs-image require same number of disks to create > empty file system without files content?? I don't _think_ you need as many devices as there were originally. > 2) would btrfs-image can be modified to keep only given subvolume foot > print and related meta data to bring back file system live on destination > disk? > >To elaborate more on this, Lets say I have 5 subvolumes on source btrfs > and i run btrfs-image written to destination disk say /dev/sdd. In this > process, can btrfs-image modified to just have only 1 subvolume and skipp > other 4 subvolumes and write to destination i.. /dev/sdd so that when I > mount /dev/sdd , I will have btrfs with only 1 subvolume with no data. For a first approximation, you could just drop any FS tree from the image which wasn't the target one. After that, it turns into a complicated accounting exercise to drop all of the back-refs to the missing FS trees, and to drop all the extent records for the non-shared data and the metadata for the missing FS trees. It's probably going to be complicated, and will basically involve rewriting most of the image to avoid the metadata you didn't want. > 3) If 3 can successful, can btrfs-image further changed to include data of > selected subvolume which gives files data also written to /dev/sdd which > would be kind of a backup of a subvolume taken out of a btrfs file system > which is having more than 1 subvolumes. If you're going to do all the hard work of (2), then (3) is a reasonable logical(?) extension. On the other hand, what's wrong with simply using send/receive? It gives you a data structure (a FAR-format send stream) which contains everything you need to reconstruct a subvolume on a btrfs different to the original. Hugo. -- Hugo Mills | Mary had a little lamb hugo@... carfax.org.uk | You've heard this tale before http://carfax.org.uk/ | But did you know she passed her plate PGP: E2AB1DE4 | And had a little more? signature.asc Description: Digital signature