Hi again.
> Looks like the answer is no. The chunk tree really has to be correct
> first before anything else because it's central to doing all the
> logical to physical address translation. And if it's busted and can't
> be repaired then nothing else is likely to work or be repairable. It's
> that critical.
>
> > I already ran chunk-recover. It needs two days to finish. But I used
> > btrfs-tools version 4.14 and it failed.
>
> I'd have to go dig in git history to even know if there's been
> improvements in chunk recover since then. But I pretty much consider
> any file system's tool obsolete within a year. I think it's total
> nonsense that distributions are intentionally using old tools.
> >
> > root@hikitty:/mnt$ btrfs rescue chunk-recover /dev/sdf1
> > Scanning: DONE in dev0
> > checksum verify failed on 99593231630336 found E4E3BDB6 wanted 00000000
> > checksum verify failed on 99593231630336 found E4E3BDB6 wanted 00000000
> > checksum verify failed on 124762809384960 found E4E3BDB6 wanted 00000000
> > checksum verify failed on 124762809384960 found E4E3BDB6 wanted 00000000
> > checksum verify failed on 124762809384960 found E4E3BDB6 wanted 00000000
> > checksum verify failed on 124762809384960 found E4E3BDB6 wanted 00000000
> > bytenr mismatch, want=124762809384960, have=0
> > open with broken chunk error
> > Chunk tree recovery failed
> >
> > I could try again with a newer version. (?) Because with version 4.14
> > also btrfs restore failed.
>
> It is entirely possible that 5.11 fails exactly the same way because
> it's just too badly damaged for the current state of the recovery
> tools to deal with damage of this kind. But it's also possible it'll
> work. It's a coin toss unless someone else a lot more familiar with
> the restore code speaks up. But looking at just the summary change
> log, it looks like no work has happened in chunk recover for a while.
>
> https://btrfs.wiki.kernel.org/index.php/Changelog
So I ran another chunk-recover with btrfs-progs version 5.11. This is
part of the output. (The list doesn't allow me attach the whole output
to this mail (5 mb zipped). But if you let me know what's important I
can send that.)
root@hikitty:~$ nohup /root/install/btrfs-progs-5.11/btrfs -v rescue
chunk-recover /dev/sdi1 >
/transfer/sebroll/btrfs-rescue-chunk-recover.out.txt 2>&1 &
nohup: ignoring input
All Devices:
Device: id = 2, name = /dev/sdi1
Scanning: DONE in dev0
DEVICE SCAN RESULT:
Filesystem Information:
sectorsize: 4096
nodesize: 16384
tree root generation: 825256
chunk root generation: 825256
All Devices:
Device: id = 2, name = /dev/sdi1
All Block Groups:
Block Group: start = 49477515739136, len = 1073741824, flag = 1
Block Group: start = 49478589480960, len = 1073741824, flag = 1
(…)
Block Group: start = 141942960685056, len = 1073741824, flag = 1
Block Group: start = 141944034426880, len = 33554432, flag = 22
All Chunks:
Chunk: start = 49477515739136, len = 1073741824, type = 1,
num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 1048576
Chunk: start = 49478589480960, len = 1073741824, type = 1,
num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 1074790400
(…)
Chunk: start = 141942960685056, len = 1073741824, type = 1,
num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 75414325166080
Chunk: start = 141944034426880, len = 33554432, type = 22,
num_stripes = 2
Stripes list:
[ 0] Stripe: devid = 2, offset = 2034741805056
[ 1] Stripe: devid = 2, offset = 2034775359488
All Device Extents:
Device extent: devid = 2, start = 1048576, len = 1073741824,
chunk offset = 49477515739136
Device extent: devid = 2, start = 1074790400, len =
1073741824, chunk offset = 49478589480960
Device extent: devid = 2, start = 2148532224, len =
1073741824, chunk offset = 49479663222784
(…)
Device extent: devid = 2, start = 75412177682432, len =
1073741824, chunk offset = 141940813201408
Device extent: devid = 2, start = 75413251424256, len =
1073741824, chunk offset = 141941886943232
Device extent: devid = 2, start = 75414325166080, len =
1073741824, chunk offset = 141942960685056
CHECK RESULT:
Recoverable Chunks:
Chunk: start = 49477515739136, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 1048576
Block Group: start = 49477515739136, len = 1073741824, flag = 1
Device extent list:
[ 0]Device extent: devid = 2, start = 1048576, len =
1073741824, chunk offset = 49477515739136
Chunk: start = 49478589480960, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 1074790400
Block Group: start = 49478589480960, len = 1073741824, flag = 1
Device extent list:
[ 0]Device extent: devid = 2, start = 1074790400, len =
1073741824, chunk offset = 49478589480960
Chunk: start = 49479663222784, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 2148532224
Block Group: start = 49479663222784, len = 1073741824, flag = 1
Device extent list:
[ 0]Device extent: devid = 2, start = 2148532224, len =
1073741824, chunk offset = 49479663222784
(…)
Chunk: start = 141085812719616, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 74690623176704
No block group.
Device extent list:
[ 0]Device extent: devid = 2, start = 74690623176704, len =
1073741824, chunk offset = 141085812719616
Chunk: start = 141403740962816, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 73223891845120
No block group.
Device extent list:
[ 0]Device extent: devid = 2, start = 73223891845120, len =
1073741824, chunk offset = 141403740962816
Unrecoverable Chunks:
Chunk: start = 73953460617216, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 23810225995776
No block group.
No device extent.
Chunk: start = 75698291081216, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 25555056459776
No block group.
No device extent.
(…)
Chunk: start = 139435974852608, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 49594055524352
Block Group: start = 139435974852608, len = 1073741824, flag = 1
No device extent.
Chunk: start = 140101996773376checksum verify failed on
99593231630336 found 000000B6 wanted 00000000
checksum verify failed on 124762809384960 found 000000B6 wanted 00000000
checksum verify failed on 124762809384960 found 000000B6 wanted 00000000
checksum verify failed on 124762809384960 found 000000B6 wanted 00000000
bad tree block 124762809384960, bytenr mismatch, want=124762809384960, have=0
ERROR: failed to read block groups: Input/output error
open with broken chunk error
, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 58501817696256
Block Group: start = 140101996773376, len = 1073741824, flag = 1
No device extent.
Chunk: start = 140221215670272, len = 1073741824, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 58416992092160
Block Group: start = 140221215670272, len = 1073741824, flag = 1
No device extent.
(…)
Chunk: start = 141836593135616, len = 33554432, type = 22, num_stripes = 2
Stripes list:
[ 0] Stripe: devid = 2, offset = 2034741805056
[ 1] Stripe: devid = 2, offset = 2034775359488
No block group.
No device extent.
Chunk: start = 141269456125952, len = 33554432, type = 22, num_stripes = 0
Stripes list:
Block Group: start = 141269456125952, len = 33554432, flag = 22
No device extent.
Total Chunks: 72859
Recoverable: 68784
Unrecoverable: 4075
Orphan Block Groups:
Orphan Device Extents:
Chunk tree recovery failed
So. Is this good or bad? First, there are a lot of unrecoverable
chunks. And some of them are system chunks.
But my biggest issue seems to be that "bad tree block" error that is
showing up and which I'm getting any time I try to access the
file-system. This error also seems to prevent mounting and btrfs
check. Can I use the position given there to look up the corresponding
chunk? If so:
Block Group and Chunks around bad tree block 124762809384960:
Block Group: start = 124760809799680, len = 1073741824, flag = 1
Block Group: start = 124761883541504, len = 1073741824, flag = 24
Block Group: start = 124798390763520, len = 1073741824, flag =
1
Chunk: start = 124760809799680, len = 1073741824, type = 1,
num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 67164766732288
Chunk: start = 124761883541504, len = 1073741824, type = 24,
num_stripes = 2
Stripes list:
[ 0] Stripe: devid = 2, offset = 67165840474112
[ 1] Stripe: devid = 2, offset = 67166914215936
Chunk: start = 124777989668864, len = 1073741824, type = 1,
num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 2, offset = 67183020343296
The bad tree block seems to contain meta data. Can I somehow recover
this? It seems to be DUP (2 Stripes)…
> > > And you'll need to look for a snapshot name in there, find its bytenr,
> > > and let's first see if just using that works. If it doesn't then maybe
> > > combining it with the next most recent root tree will work.
> >
> > I am working backwards right now using btrfs restore -f in combination
> > with -t. So far no success.
>
> Yep. I think it comes down to the chunk tree needing to be reasonable
> first, before anything else is possible.
Now I tried every possible combination with btrfs restore.
Unfortunately I found no working solution to restore the data
undamaged. As mentioned before the files contain "holes" of 0x00.
These holes are exactly 4 MiB in size. Does this tell anything?
Any ideas how to proceed from here on? Accept the data loss and try to
repair the files as good as possible -- putting them together from
different saves (if available) is a real grind. Or professional data
recovery -- but wouldn't they face the same issue regarding the
chunk-tree?
Kind regards,
Sebastian
