Re: BTRFS not mountable, recover won't work
At 04/13/2017 07:45 PM, Malte Eggers wrote: On Mon, 2017-04-10 at 16:53 +0800, Qu Wenruo wrote: At 04/10/2017 04:37 PM, Malte Eggers wrote: On Mon, 2017-04-10 at 16:18 +0800, Qu Wenruo wrote: At 04/10/2017 01:17 AM, Malte Eggers wrote: Hi, After suspending and waking up my laptop with the external hard drive connected, I could no longer access the files on it. So I unmounted and remounted it, only to discover that I could no longer mount it. This is the error (mounting with usebackuproot, same error without): [891667.002861] BTRFS info (device dm-0): trying to use backup root at mount time [891667.002870] BTRFS info (device dm-0): disk space caching is enabled [891667.002876] BTRFS info (device dm-0): has skinny extents [891667.016395] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017181] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017194] BTRFS error (device dm-0): failed to recover balance: -5 What about trying skip_balance mount option to skip balance Tried that, same error [891667.078829] BTRFS error (device dm-0): open_ctree failed btrfs restore and btrfs-find-root fail like this (on both debian sid and fedora 25): parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 Ignoring transid failure Would you please paste the output of "btrfs-debug-tree -b 108806144 /dev/dm-0" ? volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1 This BUG_ON() means we can't find a corresponding chunk for given offset. "btrfs-debug-tree -t chunk" would help, if it executes without problem. btrfs-debug-tree produces the same error If "btrfs-debug-tree" can't even open the fs, then "btrfs inspect-internal dump-super -f /dev/dm-0" would help them. dump-super finishes as it appears without error: https://pastebin.c om/t i8xuuR5 How would I proceed from here? The obvious problem I found is that, system chunk at bytenr 0 seems invalid. The physical address of that chunk is devid 1 offset 0, which is not possible as 0~1M is reserved. According to the backtrace of btrfs-progs, it seems to be related to chunk tree corruption. Which btrfs chunk recovery may help. It's recommended to backup your system chunks and superblocks first. -- # Backup sys chunks dd if=/dev/dm-0 of=sys_chunk1_stripe_0 bs=1 count=8388608 skip=20971520 dd if=/dev/dm-0 of=sys_chunk1_stripe_1 bs=1 count=8388608 skip=29360128 dd if=/dev/dm-0 of=sys_chunk0_stripe_0 bs=1 count=4194304 skip=0 # Backup first superblock dd if=/dev/dm-0 of=superblock0 bs=1 count=4k skip=64k -- Then try chunk recovery -- btrfs rescue chunk-recover /dev/dm-0 -- It can be very slow since it may scan the full device. Thanks, Qu Thanks, Qu Thank you Is there any way to just rescue whatever can still be reconstructed? Cheers, Malte The problem is that, btrfs fails to build its chunk maps. Unlike traditional fs, btrfs uses dynamically allocated chunk maps, so if btrfs fails to build its maps, nothing can be read out. That's why we have special purposed chunk recover command, due to the importance of chunk maps. (Although it doesn't work as expected sometimes) Thanks, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
On Mon, 2017-04-10 at 16:53 +0800, Qu Wenruo wrote: > > At 04/10/2017 04:37 PM, Malte Eggers wrote: > > On Mon, 2017-04-10 at 16:18 +0800, Qu Wenruo wrote: > > > > > > At 04/10/2017 01:17 AM, Malte Eggers wrote: > > > > Hi, > > > > > > > > After suspending and waking up my laptop with the external hard > > > > drive > > > > connected, I could no longer access the files on it. So I > > > > unmounted > > > > and > > > > remounted it, only to discover that I could no longer mount it. > > > > > > > > > > > > This is the error (mounting with usebackuproot, same error > > > > without): > > > > > > > > [891667.002861] BTRFS info (device dm-0): trying to use backup > > > > root > > > > at > > > > mount time > > > > [891667.002870] BTRFS info (device dm-0): disk space caching is > > > > enabled > > > > [891667.002876] BTRFS info (device dm-0): has skinny extents > > > > [891667.016395] BTRFS error (device dm-0): parent transid > > > > verify > > > > failed > > > > on 108855296 wanted 32139 found 32104 > > > > [891667.017181] BTRFS error (device dm-0): parent transid > > > > verify > > > > failed > > > > on 108855296 wanted 32139 found 32104 > > > > [891667.017194] BTRFS error (device dm-0): failed to recover > > > > balance: > > > > -5 > > > > > > What about trying skip_balance mount option to skip balance > > > > Tried that, same error > > > > > > > [891667.078829] BTRFS error (device dm-0): open_ctree failed > > > > > > > > > > > > btrfs restore and btrfs-find-root fail like this (on both > > > > debian > > > > sid > > > > and fedora 25): > > > > > > > > parent transid verify failed on 108806144 wanted 32139 found > > > > 32104 > > > > parent transid verify failed on 108806144 wanted 32139 found > > > > 32104 > > > > parent transid verify failed on 108806144 wanted 32139 found > > > > 32104 > > > > parent transid verify failed on 108806144 wanted 32139 found > > > > 32104 > > > > Ignoring transid failure > > > > > > Would you please paste the output of "btrfs-debug-tree -b > > > 108806144 > > > /dev/dm-0" ? > > > > > > > volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, > > > > value > > > > 1 > > > > > > This BUG_ON() means we can't find a corresponding chunk for given > > > offset. > > > > > > "btrfs-debug-tree -t chunk" would help, if it executes without > > > problem. > > > > btrfs-debug-tree produces the same error > > > > > > If "btrfs-debug-tree" can't even open the fs, then "btrfs > > > inspect-internal dump-super -f /dev/dm-0" would help them. > > > > dump-super finishes as it appears without error: https://pastebin.c > > om/t > > i8xuuR5 > > How would I proceed from here? > > The obvious problem I found is that, system chunk at bytenr 0 seems > invalid. > The physical address of that chunk is devid 1 offset 0, which is not > possible as 0~1M is reserved. > > According to the backtrace of btrfs-progs, it seems to be related to > chunk tree corruption. > Which btrfs chunk recovery may help. > > It's recommended to backup your system chunks and superblocks first. > -- > # Backup sys chunks > dd if=/dev/dm-0 of=sys_chunk1_stripe_0 bs=1 count=8388608 > skip=20971520 > dd if=/dev/dm-0 of=sys_chunk1_stripe_1 bs=1 count=8388608 > skip=29360128 > dd if=/dev/dm-0 of=sys_chunk0_stripe_0 bs=1 count=4194304 skip=0 > # Backup first superblock > dd if=/dev/dm-0 of=superblock0 bs=1 count=4k skip=64k > -- > > Then try chunk recovery > -- > btrfs rescue chunk-recover /dev/dm-0 > -- > > It can be very slow since it may scan the full device. > > Thanks, > Qu > > > > Thanks, > > > Qu > > > > Thank you > > > > > Is there any way to just rescue whatever can still be reconstructed? Cheers, Malte -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
On Mon, 2017-04-10 at 16:53 +0800, Qu Wenruo wrote: > > Then try chunk recovery > -- > btrfs rescue chunk-recover /dev/dm-0 > -- > > It can be very slow since it may scan the full device. > > Thanks, > Qu > > > > Thanks, > > > Qu > > > > Thank you > > Thank you for your help this far After some two hours, it finally finished, but not with the result that I was hoping for: ~# btrfs rescue chunk-recover /dev/dm-0 Scanning: DONE in dev0 No valid btrfs found open with broken chunk error Chunk tree recovery failed -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
At 04/10/2017 04:37 PM, Malte Eggers wrote: On Mon, 2017-04-10 at 16:18 +0800, Qu Wenruo wrote: At 04/10/2017 01:17 AM, Malte Eggers wrote: Hi, After suspending and waking up my laptop with the external hard drive connected, I could no longer access the files on it. So I unmounted and remounted it, only to discover that I could no longer mount it. This is the error (mounting with usebackuproot, same error without): [891667.002861] BTRFS info (device dm-0): trying to use backup root at mount time [891667.002870] BTRFS info (device dm-0): disk space caching is enabled [891667.002876] BTRFS info (device dm-0): has skinny extents [891667.016395] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017181] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017194] BTRFS error (device dm-0): failed to recover balance: -5 What about trying skip_balance mount option to skip balance Tried that, same error [891667.078829] BTRFS error (device dm-0): open_ctree failed btrfs restore and btrfs-find-root fail like this (on both debian sid and fedora 25): parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 Ignoring transid failure Would you please paste the output of "btrfs-debug-tree -b 108806144 /dev/dm-0" ? volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1 This BUG_ON() means we can't find a corresponding chunk for given offset. "btrfs-debug-tree -t chunk" would help, if it executes without problem. btrfs-debug-tree produces the same error If "btrfs-debug-tree" can't even open the fs, then "btrfs inspect-internal dump-super -f /dev/dm-0" would help them. dump-super finishes as it appears without error: https://pastebin.com/t i8xuuR5 How would I proceed from here? The obvious problem I found is that, system chunk at bytenr 0 seems invalid. The physical address of that chunk is devid 1 offset 0, which is not possible as 0~1M is reserved. According to the backtrace of btrfs-progs, it seems to be related to chunk tree corruption. Which btrfs chunk recovery may help. It's recommended to backup your system chunks and superblocks first. -- # Backup sys chunks dd if=/dev/dm-0 of=sys_chunk1_stripe_0 bs=1 count=8388608 skip=20971520 dd if=/dev/dm-0 of=sys_chunk1_stripe_1 bs=1 count=8388608 skip=29360128 dd if=/dev/dm-0 of=sys_chunk0_stripe_0 bs=1 count=4194304 skip=0 # Backup first superblock dd if=/dev/dm-0 of=superblock0 bs=1 count=4k skip=64k -- Then try chunk recovery -- btrfs rescue chunk-recover /dev/dm-0 -- It can be very slow since it may scan the full device. Thanks, Qu Thanks, Qu Thank you -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
On Mon, 2017-04-10 at 16:18 +0800, Qu Wenruo wrote: > > At 04/10/2017 01:17 AM, Malte Eggers wrote: > > Hi, > > > > After suspending and waking up my laptop with the external hard > > drive > > connected, I could no longer access the files on it. So I unmounted > > and > > remounted it, only to discover that I could no longer mount it. > > > > > > This is the error (mounting with usebackuproot, same error > > without): > > > > [891667.002861] BTRFS info (device dm-0): trying to use backup root > > at > > mount time > > [891667.002870] BTRFS info (device dm-0): disk space caching is > > enabled > > [891667.002876] BTRFS info (device dm-0): has skinny extents > > [891667.016395] BTRFS error (device dm-0): parent transid verify > > failed > > on 108855296 wanted 32139 found 32104 > > [891667.017181] BTRFS error (device dm-0): parent transid verify > > failed > > on 108855296 wanted 32139 found 32104 > > [891667.017194] BTRFS error (device dm-0): failed to recover > > balance: > > -5 > > What about trying skip_balance mount option to skip balance Tried that, same error > > > [891667.078829] BTRFS error (device dm-0): open_ctree failed > > > > > > btrfs restore and btrfs-find-root fail like this (on both debian > > sid > > and fedora 25): > > > > parent transid verify failed on 108806144 wanted 32139 found 32104 > > parent transid verify failed on 108806144 wanted 32139 found 32104 > > parent transid verify failed on 108806144 wanted 32139 found 32104 > > parent transid verify failed on 108806144 wanted 32139 found 32104 > > Ignoring transid failure > > Would you please paste the output of "btrfs-debug-tree -b 108806144 > /dev/dm-0" ? > > > volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value > > 1 > > This BUG_ON() means we can't find a corresponding chunk for given > offset. > > "btrfs-debug-tree -t chunk" would help, if it executes without > problem. btrfs-debug-tree produces the same error > > If "btrfs-debug-tree" can't even open the fs, then "btrfs > inspect-internal dump-super -f /dev/dm-0" would help them. dump-super finishes as it appears without error: https://pastebin.com/t i8xuuR5 How would I proceed from here? > Thanks, > Qu Thank you -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
At 04/10/2017 01:17 AM, Malte Eggers wrote: Hi, After suspending and waking up my laptop with the external hard drive connected, I could no longer access the files on it. So I unmounted and remounted it, only to discover that I could no longer mount it. This is the error (mounting with usebackuproot, same error without): [891667.002861] BTRFS info (device dm-0): trying to use backup root at mount time [891667.002870] BTRFS info (device dm-0): disk space caching is enabled [891667.002876] BTRFS info (device dm-0): has skinny extents [891667.016395] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017181] BTRFS error (device dm-0): parent transid verify failed on 108855296 wanted 32139 found 32104 [891667.017194] BTRFS error (device dm-0): failed to recover balance: -5 What about trying skip_balance mount option to skip balance? [891667.078829] BTRFS error (device dm-0): open_ctree failed btrfs restore and btrfs-find-root fail like this (on both debian sid and fedora 25): parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 Ignoring transid failure Would you please paste the output of "btrfs-debug-tree -b 108806144 /dev/dm-0" ? volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1 This BUG_ON() means we can't find a corresponding chunk for given offset. "btrfs-debug-tree -t chunk" would help, if it executes without problem. If "btrfs-debug-tree" can't even open the fs, then "btrfs inspect-internal dump-super -f /dev/dm-0" would help them. Thanks, Qu btrfs(+0x530d5)[0x5634cfb180d5] btrfs(btrfs_chunk_readonly+0x5a)[0x5634cfb1a3b3] btrfs(btrfs_read_block_groups+0x1dc)[0x5634cfb0fb0c] btrfs(btrfs_setup_all_roots+0x336)[0x5634cfb0b158] btrfs(+0x4654c)[0x5634cfb0b54c] btrfs(open_ctree_fs_info+0xd7)[0x5634cfb0b673] btrfs(cmd_restore+0x339)[0x5634cfaf93dc] btrfs(main+0x12f)[0x5634cfadca6b] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f3f7161f2b1] btrfs(_start+0x2a)[0x5634cfadcaaa] Aborted How would I go about recovering my Data? Thanks, Malte -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
On Sun, 2017-04-09 at 15:25 -0600, Chris Murphy wrote: > On Sun, Apr 9, 2017 at 11:17 AM, Malte Eggers > wrote: > > Hi, > > > > After suspending and waking up my laptop with the external hard > > drive > > connected, I could no longer access the files on it. So I unmounted > > and > > remounted it, only to discover that I could no longer mount it. > > What kernel and btrfs-progs version? Anything older than 4.9 of > either > and I'd try something newer as a first step. This sounds to me like > something dropped the ball: either fs freeze didn't happen (unlikely) > or the drive lied about fsync succeeding, or FUA if supported, and > dropped the data once power was cut to the drive. I'd check if there > are firmware updates for the drive. > > Curious about: > > btrfs rescue super -v > btrfs check > Kernel version: 4.9.0-2-amd64 BTRFS version: v4.9.1 btrfs rescue super -v and btrfs check both result in the same error: parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 parent transid verify failed on 108806144 wanted 32139 found 32104 Ignoring transid failure volumes.c:1645: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1 btrfs(+0x530d5)[0x560ede5090d5] btrfs(btrfs_chunk_readonly+0x5a)[0x560ede50b3b3] btrfs(btrfs_read_block_groups+0x1dc)[0x560ede500b0c] btrfs(btrfs_setup_all_roots+0x336)[0x560ede4fc158] btrfs(+0x4654c)[0x560ede4fc54c] btrfs(open_ctree_fs_info+0xd7)[0x560ede4fc673] btrfs(open_ctree+0x47)[0x560ede4fc6f2] btrfs(btrfs_recover_superblocks+0x325)[0x560ede4ef051] btrfs(main+0x12f)[0x560ede4cda6b] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f5e9f69f2b1] btrfs(_start+0x2a)[0x560ede4cdaaa] Aborted I tried using the hard drive on my raspberry pi with raspbian and kernel version 4.4, it doesn't give me a backtrace, but instead this: https://pastebin.com/tUxSdysM Which doesn't look like what is described in https://btrfs.wiki.kernel. org/index.php/Restore and restoring specifying any of the blocks doesn't work either. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
Chris Murphy posted on Sun, 09 Apr 2017 15:25:18 -0600 as excerpted: > On Sun, Apr 9, 2017 at 11:17 AM, Malte Eggers > wrote: >> Hi, >> >> After suspending and waking up my laptop with the external hard drive >> connected, I could no longer access the files on it. So I unmounted and >> remounted it, only to discover that I could no longer mount it. This is the only message from the thread I'm seeing ATM, so I'll reply here. Agreed with C Murphy on the btrfs stuff, but my reply is on a different tangent. If your external hard drive is USB (not ESATA or other non-USB, couldn't tell from the context CM quoted, which may or may not have been the entire post), you are likely running into a known issue... with an available solution. =:^) FWIW I found out about this when I had a laptop with two USB-connected CompactFlash slots, one of which was designed for effectively permanent installation (the CF didn't stick out when inserted), the other for temporary use (it stuck out and was thus easier to remove, but also easy to damage either it or the slot if it was left inserted during transport). But the permanent one wouldn't stay mounted over suspend... until I found out about this and changed the kernel option. Looks like you might be hitting the same issue. The official Linux kernel documentation describing the problem and optional solution is: $KERNEL_DIR/Documentation/usb/persist.txt Read that first. But that only explains the problem and gives you a way to deal with it on a per-USB-device basis by writing to the appropriate sysfs file. There's also a kernel option that lets you change the default: CONFIG_USB_DEFAULT_PERSIST In the kernel config that's found under: Device Drivers USB support Support for Host-side USB Enable USB persist by default If you use a distro-built kernel, you may want to at least check whether they enabled that option or not. If you build your own kernel, you may of course set that option as you wish. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS not mountable, recover won't work
On Sun, Apr 9, 2017 at 11:17 AM, Malte Eggers wrote: > Hi, > > After suspending and waking up my laptop with the external hard drive > connected, I could no longer access the files on it. So I unmounted and > remounted it, only to discover that I could no longer mount it. What kernel and btrfs-progs version? Anything older than 4.9 of either and I'd try something newer as a first step. This sounds to me like something dropped the ball: either fs freeze didn't happen (unlikely) or the drive lied about fsync succeeding, or FUA if supported, and dropped the data once power was cut to the drive. I'd check if there are firmware updates for the drive. Curious about: btrfs rescue super -v btrfs check -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
