Re: Force recalculation of a data block checksum
On 24.07.2016 07:07, Andrei Borzenkov wrote: Rewriting single block in question should do it. As you can read from raw device and have both physical and logical block address something like dd if=/dev/sdd1 skip=222940168 count=8 | dd of=/path/to/dysk/dysk.bin seek=189669768 conv=notrunc count=8 Thank you for the one-liner, I thought of doing it myself before, but when I tried to figure out which numbers mean what in scrub messages, I got a bit lost. However, this one-liner doesn't seem to work here; `dd` prints: #v+ 8+0 records in 8+0 records out 4096 bytes (4.1 kB) copied, 5.2857e-05 s, 77.5 MB/s dd: writing to ‘/media/liori/USB/dysk/dysk.bin’: Input/output error 1+0 records in 0+0 records out 0 bytes (0 B) copied, 0.00364222 s, 0.0 kB/s #v- I'm not sure what would this message mean in this case. Is btrfs reading the block before writing? Maybe some block blacklist stored somewhere (I see that the corruption counter in scrub messages go up even if it reports the same errors in subsequent runs)? If you have possibility to verify file content(s), this technique can be used to read over unreadable parts of file by combining multiple dd's from file and device. Some of the files stored in this `dysk/dysk.bin` disk image have some internal checksums (zip archives, etc.). That includes the ones hit by the checksum error. I'll try verifying these later. -- Tomasz Melcer -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
On Sat, Jul 23, 2016 at 10:32 PM, Tomasz Melcerwrote: > On 24.07.2016 04:16, Chris Murphy wrote: >> >> On Sat, Jul 23, 2016 at 8:03 PM, Tomasz Melcer wrote: >>> >>> How can I ask btrfs to recompute the checksum of a data block as it is >>> stored on the drive? >> >> >> Since btrfs-progs 3.17 'btrfs check --init-csum-tree' will create a >> whole new csum tree. > > > I can't make this command recalculate only checksum of that single file? The likelihood of the data block being wrong is very high. If the csum itself were wrong, the csum of the leaf it's stored in would catch it, except in the narrow edge case where the csum was computed wrong in memory, and then put into the leaf which is then checksummed correctly, i.e. the leaf csum says the leaf is good but it contains a bad csum for just one block. Possible but, pretty rare if there isn't a problem causing in-memory corruption. You'd see a lot more errors. A few bad sectors are exactly what this most likely is. The drive ECC is probably failing to correct the problem, but thinks it's right which why there's no read error from the drive. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
24.07.2016 07:32, Tomasz Melcer пишет: > On 24.07.2016 04:16, Chris Murphy wrote: >> On Sat, Jul 23, 2016 at 8:03 PM, Tomasz Melcerwrote: >>> How can I ask btrfs to recompute the checksum of a data block as it is >>> stored on the drive? >> >> Since btrfs-progs 3.17 'btrfs check --init-csum-tree' will create a >> whole new csum tree. > > I can't make this command recalculate only checksum of that single file? > > Rewriting single block in question should do it. As you can read from raw device and have both physical and logical block address something like dd if=/dev/sdd1 skip=222940168 count=8 | dd of=/path/to/dysk/dysk.bin seek=189669768 conv=notrunc count=8 If you have possibility to verify file content(s), this technique can be used to read over unreadable parts of file by combining multiple dd's from file and device. It probably makes sense to create (read-only) snapshot to make sure you have old content available. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
On 24.07.2016 04:16, Chris Murphy wrote: On Sat, Jul 23, 2016 at 8:03 PM, Tomasz Melcerwrote: How can I ask btrfs to recompute the checksum of a data block as it is stored on the drive? Since btrfs-progs 3.17 'btrfs check --init-csum-tree' will create a whole new csum tree. I can't make this command recalculate only checksum of that single file? -- Tomasz Melcer -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
The negative of --init-csum-tree is that it's going to work on all data extents. It's going to take a while. It's probably faster to extract the suspect file with restore onto another file system and check its integrity by other means (compare to backup, database integrity check, etc). -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
Tomasz Melcer posted on Sun, 24 Jul 2016 04:03:08 +0200 as excerpted: > How can I ask btrfs to recompute the checksum of a data block as it is > stored on the drive? I don't see any command doing an operation like > that, and I couldn't find anything on the topic on the internet. There's no "easy" btrfs method to recompute the checksum of an individual block (you'd have to do some deep debugging and hex editing to manually rewrite the correct sum), but there /is/ one that reinitiates and recalculates checksums for the entire filesystem. You don't mention what version of btrfs-progs you have, but you'll want a reasonably new one, as AFAIK the command is fairly new, and early versions of the command simply wiped the existing checksums without recalculating them. btrfs check --init-csum-tree ... as found in the btrfs-check manpage. Another alternative is using btrfs restore, again, see the btrfs-restore manpage. Normally this command is used to get what you can off of a filesystem damaged badly enough it won't mount, but you can also use it to scrape specific files off an unmounted filesystem, if the checksum's bad, for instance. In this case, you'd use the --path-regex option to restore only files matching that regex. Of course, for a TB-sized file, you'll need a separate filesystem (need not be btrfs, but will need to be a filesystem that can handle a file that size) with at least a TB free to write it to. I'd normally prefer the btrfs restore method, myself, particularly if the btrfs in question has other files that you don't want to risk in the process. If that was the only file on the filesystem, I'd still try the restore method first, at least if I had some place to write it to, because btrfs restore is read-only against the btrfs you're restoring from, and thus has no chance of causing further damage, while of course btrfs check --init-csum-tree writes to the filesystem in question and thus is a higher risk. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Force recalculation of a data block checksum
On Sat, Jul 23, 2016 at 8:03 PM, Tomasz Melcerwrote: > Hi, > > I've got a USB-connected HDD with a btrfs partition. The partition contains > a 1TB file, a disk image. The first `btrfs scrub` after writing that file > found 3 logical bad blocks that developed somewhere in the middle of that > file (logs below). > > The full area of the btrfs partition can be read without I/O error, so I > think there are two possible cases: either the data block was written > incorrectly or an incorrect checksum is stored. The first case is obviously > unrecoverable, but if it's the second case, fixing the problem should be as > simple as recomputing the checksum for what is already stored. > > How can I ask btrfs to recompute the checksum of a data block as it is > stored on the drive? I don't see any command doing an operation like that, > and I couldn't find anything on the topic on the internet. Since btrfs-progs 3.17 'btrfs check --init-csum-tree' will create a whole new csum tree. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Force recalculation of a data block checksum
Hi, I've got a USB-connected HDD with a btrfs partition. The partition contains a 1TB file, a disk image. The first `btrfs scrub` after writing that file found 3 logical bad blocks that developed somewhere in the middle of that file (logs below). The full area of the btrfs partition can be read without I/O error, so I think there are two possible cases: either the data block was written incorrectly or an incorrect checksum is stored. The first case is obviously unrecoverable, but if it's the second case, fixing the problem should be as simple as recomputing the checksum for what is already stored. How can I ask btrfs to recompute the checksum of a data block as it is stored on the drive? I don't see any command doing an operation like that, and I couldn't find anything on the topic on the internet. Thanks, Logs: #v+ [ 7702.964265] BTRFS warning (device sdd1): checksum error at logical 5473719291904 on dev /dev/sdd1, sector 222940168, root 5, inode 1245769, offset 97110921216, length 4096, links 1 (path: dysk/dysk.bin) [ 7702.964274] BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 17, gen 0 [ 7702.964278] BTRFS error (device sdd1): unable to fixup (regular) error at logical 5473719291904 on dev /dev/sdd1 […] [ 9588.625906] BTRFS warning (device sdd1): checksum error at logical 5241172611072 on dev /dev/sdd1, sector 691494312, root 5, inode 1245769, offset 310632271872, length 4096, links 1 (path: dysk/dysk.bin) [ 9588.625916] BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 18, gen 0 [ 9588.625920] BTRFS error (device sdd1): unable to fixup (regular) error at logical 5241172611072 on dev /dev/sdd1 […] [20545.302898] BTRFS warning (device sdd1): checksum error at logical 3991747321856 on dev /dev/sdd1, sector 3185809200, root 5, inode 1245769, offset 907925676032, length 4096, links 1 (path: dysk/dysk.bin) [20545.302908] BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 19, gen 0 [20545.302912] BTRFS error (device sdd1): unable to fixup (regular) error at logical 3991747321856 on dev /dev/sdd1 #v- -- Tomasz Melcer -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html