Re: moving btrfs subvolumes to new disk
On Wed, Mar 23, 2016 at 8:08 PM, Ryan Erato wrote: > Success! Using the same ISO you previously linked to, I ran 'btrfs > check --repair', did another check which actually revealed many new > issues, ran a repair again and after that successive checks showed no > signs of other issues. I was able to successfully send my 'home' > subvolume to the SSD and wow, huge difference with otherwise little > effort. > > Thanks to you and everybody else for helping me resolve this issue. Yay, good news! -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
Success! Using the same ISO you previously linked to, I ran 'btrfs check --repair', did another check which actually revealed many new issues, ran a repair again and after that successive checks showed no signs of other issues. I was able to successfully send my 'home' subvolume to the SSD and wow, huge difference with otherwise little effort. Thanks to you and everybody else for helping me resolve this issue. On Tue, Mar 22, 2016 at 8:34 PM, Chris Murphy wrote: > On Tue, Mar 22, 2016 at 7:40 PM, Ryan Erato wrote: >> Finally got around to running the suggested commands. Same error with >> the send, but not much output to help. The check operation did seem >> to reveal some potential issues. Here's the play-by-play along with >> the file output from check: >> >> [liveuser@localhost /]$ sudo btrfs check /dev/sda6 > >> /home/liveuser/btrfscheck.txt >> checking extents >> checking free space cache >> checking fs roots >> root 257 inode 13324701 errors 200, dir isize wrong >> root 258 inode 226392 errors 200, dir isize wrong >> root 258 inode 236055 errors 2000, link count wrong >> unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 >> errors 3, no dir item, no dir index >> root 258 inode 236273 errors 2000, link count wrong >> root 258 inode 236276 errors 2000, link count wrong >> unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 >> filetype 0 errors 3, no dir item, no dir index >> root 258 inode 236277 errors 2000, link count wrong >> unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 >> errors 3, no dir item, no dir index >> root 258 inode 240618 errors 2000, link count wrong >> unresolved ref dir 226392 index 115 namelen 10 name 89.log >> filetype 0 errors 3, no dir item, no dir index >> root 487 inode 13324701 errors 200, dir isize wrong >> root 488 inode 226392 errors 200, dir isize wrong >> root 488 inode 236055 errors 2000, link count wrong >> unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 >> errors 3, no dir item, no dir index >> root 488 inode 236273 errors 2000, link count wrong >> root 488 inode 236276 errors 2000, link count wrong >> unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 >> filetype 0 errors 3, no dir item, no dir index >> root 488 inode 236277 errors 2000, link count wrong >> unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 >> errors 3, no dir item, no dir index >> root 488 inode 240618 errors 2000, link count wrong >> unresolved ref dir 226392 index 115 namelen 10 name 89.log >> filetype 0 errors 3, no dir item, no dir index > > OK so now the question is if 'btrfs check --repair' can fix this, and > what version to use? 4.4.1 or 4.5.0? Based on the changelog, you can > probably use either version. And I think it should be safe. But, you > should still have backups seeing as you can mount the volume. > > >> >> [liveuser@localhost /]$ sudo mount /dev/sda6 /mnt/hdd >> >> [liveuser@localhost /]$ sudo btrfs send -vvv --no-data -f homesnap.btr >> /mnt/hdd/home/home.snap/ >> Mode NO_FILE_DATA enabled >> At subvol /mnt/hdd/home/home.snap/ >> ERROR: send ioctl failed with -2: No such file or directory > > OK so there's something off with the metadata and it's not going to do > a send as a result is what this sounds like to me. > > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
On Tue, Mar 22, 2016 at 7:40 PM, Ryan Erato wrote: > Finally got around to running the suggested commands. Same error with > the send, but not much output to help. The check operation did seem > to reveal some potential issues. Here's the play-by-play along with > the file output from check: > > [liveuser@localhost /]$ sudo btrfs check /dev/sda6 > > /home/liveuser/btrfscheck.txt > checking extents > checking free space cache > checking fs roots > root 257 inode 13324701 errors 200, dir isize wrong > root 258 inode 226392 errors 200, dir isize wrong > root 258 inode 236055 errors 2000, link count wrong > unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 > errors 3, no dir item, no dir index > root 258 inode 236273 errors 2000, link count wrong > root 258 inode 236276 errors 2000, link count wrong > unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 > filetype 0 errors 3, no dir item, no dir index > root 258 inode 236277 errors 2000, link count wrong > unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 > errors 3, no dir item, no dir index > root 258 inode 240618 errors 2000, link count wrong > unresolved ref dir 226392 index 115 namelen 10 name 89.log > filetype 0 errors 3, no dir item, no dir index > root 487 inode 13324701 errors 200, dir isize wrong > root 488 inode 226392 errors 200, dir isize wrong > root 488 inode 236055 errors 2000, link count wrong > unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 > errors 3, no dir item, no dir index > root 488 inode 236273 errors 2000, link count wrong > root 488 inode 236276 errors 2000, link count wrong > unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 > filetype 0 errors 3, no dir item, no dir index > root 488 inode 236277 errors 2000, link count wrong > unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 > errors 3, no dir item, no dir index > root 488 inode 240618 errors 2000, link count wrong > unresolved ref dir 226392 index 115 namelen 10 name 89.log > filetype 0 errors 3, no dir item, no dir index OK so now the question is if 'btrfs check --repair' can fix this, and what version to use? 4.4.1 or 4.5.0? Based on the changelog, you can probably use either version. And I think it should be safe. But, you should still have backups seeing as you can mount the volume. > > [liveuser@localhost /]$ sudo mount /dev/sda6 /mnt/hdd > > [liveuser@localhost /]$ sudo btrfs send -vvv --no-data -f homesnap.btr > /mnt/hdd/home/home.snap/ > Mode NO_FILE_DATA enabled > At subvol /mnt/hdd/home/home.snap/ > ERROR: send ioctl failed with -2: No such file or directory OK so there's something off with the metadata and it's not going to do a send as a result is what this sounds like to me. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
Finally got around to running the suggested commands. Same error with the send, but not much output to help. The check operation did seem to reveal some potential issues. Here's the play-by-play along with the file output from check: [liveuser@localhost /]$ sudo btrfs check /dev/sda6 > /home/liveuser/btrfscheck.txt checking extents checking free space cache checking fs roots root 257 inode 13324701 errors 200, dir isize wrong root 258 inode 226392 errors 200, dir isize wrong root 258 inode 236055 errors 2000, link count wrong unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 errors 3, no dir item, no dir index root 258 inode 236273 errors 2000, link count wrong root 258 inode 236276 errors 2000, link count wrong unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 filetype 0 errors 3, no dir item, no dir index root 258 inode 236277 errors 2000, link count wrong unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 errors 3, no dir item, no dir index root 258 inode 240618 errors 2000, link count wrong unresolved ref dir 226392 index 115 namelen 10 name 89.log filetype 0 errors 3, no dir item, no dir index root 487 inode 13324701 errors 200, dir isize wrong root 488 inode 226392 errors 200, dir isize wrong root 488 inode 236055 errors 2000, link count wrong unresolved ref dir 226392 index 35 namelen 7 name LOG.old filetype 0 errors 3, no dir item, no dir index root 488 inode 236273 errors 2000, link count wrong root 488 inode 236276 errors 2000, link count wrong unresolved ref dir 226392 index 39 namelen 15 name MANIFEST-15 filetype 0 errors 3, no dir item, no dir index root 488 inode 236277 errors 2000, link count wrong unresolved ref dir 226392 index 41 namelen 7 name CURRENT filetype 0 errors 3, no dir item, no dir index root 488 inode 240618 errors 2000, link count wrong unresolved ref dir 226392 index 115 namelen 10 name 89.log filetype 0 errors 3, no dir item, no dir index " btrfscheck.txt Checking filesystem on /dev/sda6 UUID: 6bb38bce-d824-4b9c-8b03-adad460c0f97 found 79333650514 bytes used err is 1 total csum bytes: 60906940 total tree bytes: 1530494976 total fs tree bytes: 1392934912 total extent tree bytes: 59506688 btree space waste bytes: 343045471 file data blocks allocated: 97137004544 referenced 85008084992 [liveuser@localhost /]$ sudo mount /dev/sda6 /mnt/hdd [liveuser@localhost /]$ sudo btrfs send -vvv --no-data -f homesnap.btr /mnt/hdd/home/home.snap/ Mode NO_FILE_DATA enabled At subvol /mnt/hdd/home/home.snap/ ERROR: send ioctl failed with -2: No such file or directory On Sun, Mar 20, 2016 at 10:42 PM, Chris Murphy wrote: > On Sun, Mar 20, 2016 at 10:34 PM, Ryan Erato wrote: > . >> >> Sending "home.snap" to "/mnt/ssd" results in the -2 error. What is >> peculiar, or possibly a red herring, is that it seems to fail at the >> same point each time, at 4.39GB in to the transfer. > > > > That's pretty suspicious. I didn't realize from the first description > that the command is doing something for a while before failing. I > thought it was failing immediately. > > Try this: > > btrfs send -vvv --no-data -f homesnap.btr home.snapshot > > That will write out metadata only to a file, no receive. See if the > error still happens and if the extra v gives more info. > > If it still fails with no more useful information then what I'd try > next is a btrfs check with the most recent btrfs-progs you can find. > If you're in need of a suggestion, this has btrfs-progs 4.4.1, I've > tested that it boots, it's got a published sha256 hash, and is served > over https. Yes, it's not even an alpha, but all you're doing is a > check, not a --repair, and no need to mount it (although that's > probably safe also, I've been doing it most of the weekend). > https://dl.fedoraproject.org/pub/alt/stage/24_Alpha-1.6/Workstation/x86_64/iso/ > dd that iso file to a USB stick, it will destroy all data on the > stick, and then boot the computer, and switch to tty2 (control-alt-f2) > to get to a shell. > > I think 'btrfs check > btrfscheck.txt' will output most of the results > to a text file. Often it misses the first few lines for whatever > reason. You can either 'fpaste ' and then note the URL and > post it here, or you can scp the file elsewhere, if you have wired > ethernet connected. > > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
On Sun, Mar 20, 2016 at 10:34 PM, Ryan Erato wrote: . > > Sending "home.snap" to "/mnt/ssd" results in the -2 error. What is > peculiar, or possibly a red herring, is that it seems to fail at the > same point each time, at 4.39GB in to the transfer. That's pretty suspicious. I didn't realize from the first description that the command is doing something for a while before failing. I thought it was failing immediately. Try this: btrfs send -vvv --no-data -f homesnap.btr home.snapshot That will write out metadata only to a file, no receive. See if the error still happens and if the extra v gives more info. If it still fails with no more useful information then what I'd try next is a btrfs check with the most recent btrfs-progs you can find. If you're in need of a suggestion, this has btrfs-progs 4.4.1, I've tested that it boots, it's got a published sha256 hash, and is served over https. Yes, it's not even an alpha, but all you're doing is a check, not a --repair, and no need to mount it (although that's probably safe also, I've been doing it most of the weekend). https://dl.fedoraproject.org/pub/alt/stage/24_Alpha-1.6/Workstation/x86_64/iso/ dd that iso file to a USB stick, it will destroy all data on the stick, and then boot the computer, and switch to tty2 (control-alt-f2) to get to a shell. I think 'btrfs check > btrfscheck.txt' will output most of the results to a text file. Often it misses the first few lines for whatever reason. You can either 'fpaste ' and then note the URL and post it here, or you can scp the file elsewhere, if you have wired ethernet connected. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
Here's an example of what I've been trying: " mount new ssd root / # mount /dev/sdb6 /mnt/ssd/ " snapshot ROOT sub-volume mounted at / root / # btrfs subvol snapshot -r / /ROOT.snap Create a readonly snapshot of '/' in '//ROOT.snap' root / # btrfs filesystem sync / FSSync '/' root / # btrfs subvol list / ID 257 gen 747906 top level 5 path ROOT ID 258 gen 747906 top level 257 path home ID 487 gen 747893 top level 257 path ROOT.snap root / # btrfs send /ROOT.snap/ | btrfs receive /mnt/ssd/ At subvol /ROOT.snap/ At subvol ROOT.snap " snapshot home subvolume at /home root / # btrfs subvol snapshot -r /home/ /home/home.snap Create a readonly snapshot of '/home/' in '/home/home.snap' root / # btrfs filesystem sync /home FSSync '/home' root / # btrfs subvol list / ID 257 gen 747944 top level 5 path ROOT ID 258 gen 747944 top level 257 path home ID 487 gen 747893 top level 257 path ROOT.snap ID 488 gen 747942 top level 258 path home/home.snap root / # btrfs send /home/home.snap/ | btrfs receive /mnt/ssd/ At subvol /home/home.snap/ At subvol home.snap ERROR: send ioctl failed with -2: No such file or directory ERROR: unexpected EOF in stream. root / # btrfs subvol list /mnt/ssd/ ID 257 gen 57 top level 5 path ROOT.snap ID 285 gen 62 top level 5 path home.snap I have also attempted to mount /dev/sdb6 with mount option "subvol=ROOT" like I do in my system fstab for the current drive. I have also tried saving the snapshot in "/" and "/home", with the same results. Sending "home.snap" to "/mnt/ssd" results in the -2 error. What is peculiar, or possibly a red herring, is that it seems to fail at the same point each time, at 4.39GB in to the transfer. Using verbose output on receive it seems to process the files in the same order as again, I see the same set of files just before it fails each and every time. I setup my system according to gentoo documentation, so I'm not sure if I did something wrong with the initial btrfs partition setup that is making this more difficult. On Sun, Mar 20, 2016 at 12:31 PM, Chris Murphy wrote: > On Sat, Mar 19, 2016 at 4:58 PM, Ryan Erato wrote: >> I'm having quite the time trying to move my current Gentoo install to >> an SSD. I first attempted Clonezilla, but that failed while cloning >> the btrfs partition. I then realized I could use btrfs send/receive. >> >> The partition has 2 subvolumes ROOT and ROOT/home. ROOT snapshots and >> sends without hitch, but /home consistently errors at the same point >> (from -vv output on btrfs send). btrfs send returns -2, no such file >> or directory, unexpected EOF in stream. > > What's the exact command you're using for btrfs send receive for ROOT and > home? > > >> >> # cat /etc/fstab >> /dev/sda4 /boot vfat noauto,noatime 1 2 >> /dev/sda6 / btrfs defaults,noatime,compress=lzo,autodefrag,subvol=ROOT 0 0 >> /dev/sda5 none swap sw 0 0 >> >> # btrfs su l / >> ID 257 gen 745235 top level 5 path ROOT >> ID 258 gen 745235 top level 257 path home >> ID 498 gen 744743 top level 5 path ROOT.snap >> ID 501 gen 745043 top level 258 path home/home.snap > > Nested subvolumes makes this more complicated in my opinion. You might > have better luck mounting subvolid=5 to /mnt and using a fully > qualified pathname for send. > > > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
I'm not an expert by any means, but I did a migration like this a few weeks ago. The most consistent recommendation on this mailing list is to use the newest kernels and btrfs-progs feasible. I did my migration using Fedora 24 live media, which at the time was kernel ~4.3. I see your btrfs-progs is a little old. Maybe the kernel is as well and has a bug. I know you tried a bunch of variations, but it would be helpful if you posted the commands you've tried. Perhaps the most obvious problem that could be occurring is trying to send a RW subvol, which is illegal. I can't tell if you made a RO snapshot first, but your success with ROOT indicates you're aware of this limitation. I'm guessing that there's some problem with the destination path that you're providing. The path should be the containing subvol where you want it to go, not the destination name (i.e. btrfs receive /var/media/backups/ not btrfs receive /var/media/backups/HOME). Here's some send/receive commands that I've successfully used recently. Maybe they'll point you in the right direction: pull a RO snapshot from a remote system: cd ~/ksp/backups ssh 192.168.0.122 "sudo btrfs send /home/justin/ksp/backups/2016-03-15-17:24" | sudo btrfs receive . mv 2016-03-15-17:24 ../current Note how I received into the backups directory and have no control over the initial subvol name. I mv it to the proper location and name afterwards. move a RO snapshot between local file system: cd /tmp/old-btrfs/ sudo btrfs subvol snap -r home home-snap cd /tmp/new-btrfs/ sudo btrfs send /tmp/old-btrfs/home-snap | sudo btrfs receive . sudo btrfs subvol snapshot home-snap home sudo btrfs subvol delete home-snap Fedora names it's /home subvol "home." You *shouldn't* need to mess around with advanced mounting options to get this to work. On Sun, Mar 20, 2016 at 11:52 AM, Ryan Erato wrote: > I do plan on physically replacing the current drive with the new one > and my fstab/boot comands use device. I never could get UUID or labels > to work, but that's another project. > > However, this still leaves me unable to take advantage of btrfs > features for implementing an incremental backup solution to another > disk. > Ryan Erato > C. 414.630.5295 > rer...@gmail.com > http://www.linkedin.com/in/ryanerato > > > On Sun, Mar 20, 2016 at 1:55 AM, Phantom Guy wrote: >> You tried to hard. >> >> Build the new media If you need encryption or meets devices. >> >> >> Then add the new media to the whole filesystem. >> >> >> Then remove the old media from the filesystem. >> >> >> # btrfs device add /dev/sdb1 /mountpoint >> >> # btrfs device del /dev/sda1 /mountpoint >> >> Then sync the filesystem and wait. >> >> Once the old device ID no longer part of the filesystem it's done. >> >> When the sync finishes, the /dev/sda1 device is nill. >> >> If you've designed your fstsb and boot commands correctly the label or UUID >> has migrated so as long as you've prepped the disk with grub of whatever >> then you can move the disks with impunity. >> >> You don't even need to be in any sort of maintenance mode. >> >> >>> >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
On Sat, Mar 19, 2016 at 4:58 PM, Ryan Erato wrote: > I'm having quite the time trying to move my current Gentoo install to > an SSD. I first attempted Clonezilla, but that failed while cloning > the btrfs partition. I then realized I could use btrfs send/receive. > > The partition has 2 subvolumes ROOT and ROOT/home. ROOT snapshots and > sends without hitch, but /home consistently errors at the same point > (from -vv output on btrfs send). btrfs send returns -2, no such file > or directory, unexpected EOF in stream. What's the exact command you're using for btrfs send receive for ROOT and home? > > # cat /etc/fstab > /dev/sda4 /boot vfat noauto,noatime 1 2 > /dev/sda6 / btrfs defaults,noatime,compress=lzo,autodefrag,subvol=ROOT 0 0 > /dev/sda5 none swap sw 0 0 > > # btrfs su l / > ID 257 gen 745235 top level 5 path ROOT > ID 258 gen 745235 top level 257 path home > ID 498 gen 744743 top level 5 path ROOT.snap > ID 501 gen 745043 top level 258 path home/home.snap Nested subvolumes makes this more complicated in my opinion. You might have better luck mounting subvolid=5 to /mnt and using a fully qualified pathname for send. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: moving btrfs subvolumes to new disk
I do plan on physically replacing the current drive with the new one and my fstab/boot comands use device. I never could get UUID or labels to work, but that's another project. However, this still leaves me unable to take advantage of btrfs features for implementing an incremental backup solution to another disk. Ryan Erato C. 414.630.5295 rer...@gmail.com http://www.linkedin.com/in/ryanerato On Sun, Mar 20, 2016 at 1:55 AM, Phantom Guy wrote: > You tried to hard. > > Build the new media If you need encryption or meets devices. > > > Then add the new media to the whole filesystem. > > > Then remove the old media from the filesystem. > > > # btrfs device add /dev/sdb1 /mountpoint > > # btrfs device del /dev/sda1 /mountpoint > > Then sync the filesystem and wait. > > Once the old device ID no longer part of the filesystem it's done. > > When the sync finishes, the /dev/sda1 device is nill. > > If you've designed your fstsb and boot commands correctly the label or UUID > has migrated so as long as you've prepped the disk with grub of whatever > then you can move the disks with impunity. > > You don't even need to be in any sort of maintenance mode. > > >> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
