Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 02/04/2017 07:01 AM, Christoph Anton Mitterer wrote: Hey Qu On Fri, 2017-02-03 at 14:20 +0800, Qu Wenruo wrote: Great thanks for that! You're welcome. :) I also added missing error message output for other places I found, and updated the branch, the name remains as "lowmem_tests" Please try it. # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used, no error found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used, no error found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 => looks good this time :) Great! btw: In your commit messages, please change my email to cales...@scientia.net everywhere... I accidentally used my university address (christoph.anton.mitte...@lmu.de) sometimes when sending mail. Not a problem. I'll change the address and re-send the patchset to mail list. Thanks, Qu Cheers, Chris. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
Hey Qu On Fri, 2017-02-03 at 14:20 +0800, Qu Wenruo wrote: > Great thanks for that! You're welcome. :) > I also added missing error message output for other places I found, > and > updated the branch, the name remains as "lowmem_tests" > > Please try it. # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used, no error found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used, no error found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 => looks good this time :) btw: In your commit messages, please change my email to cales...@scientia.net everywhere... I accidentally used my university address (christoph.anton.mitte...@lmu.de) sometimes when sending mail. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 02/03/2017 01:08 PM, Christoph Anton Mitterer wrote: Hey Qu. I'm sending this off-list for privacy reasons of the contained file- names / hash sums. It doesn't contained anything particularly secret, nevertheless, please try to avoid spreading it too far around and delete it once you don't need it anymore :) On Thu, 2017-02-02 at 10:12 +0800, Qu Wenruo wrote: Would you please to check the latest lowmem_tests branch? https://github.com/adam900710/btrfs-progs/tree/lowmem_tests Invocation was as usual: # btrfs check --mode=lowmem /dev/nbd0 ; echo $? stdout/err output in the attachment Great thanks for that! Although I made a mistake in the debug patch, the output info is still good enough for me to catch the bug in inline extent check. I also added missing error message output for other places I found, and updated the branch, the name remains as "lowmem_tests" Please try it. Thanks, Qu Sorry for the extra trouble. No worries :) Cheers, Chris. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 02/02/2017 08:25 AM, Qu Wenruo wrote: Thanks for your test. At 02/02/2017 06:03 AM, Christoph Anton Mitterer wrote: On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote: https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Which is also rebased to latest v4.9.1. Same game as last time, applied to 4.9, no RW mount between the runs. [snipped] => same as before # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots ERROR: errors found in fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used, error(s) found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 => Unfortunately :-( Would you please to check the latest lowmem_tests branch? https://github.com/adam900710/btrfs-progs/tree/lowmem_tests This time I added several debug output to catch the whole path where the silent error happens, to help us to solve the problem. Sorry for the extra trouble. Thanks, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
Thanks for your test. At 02/02/2017 06:03 AM, Christoph Anton Mitterer wrote: On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote: https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Which is also rebased to latest v4.9.1. Same game as last time, applied to 4.9, no RW mount between the runs. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents 137 => would be nice if btrfs-progrs could give a message on why it failed, i.e. "not enough memory" or so. # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 => error still occurs *without* patch => increased VM memory here # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Wed, 2017-02-01 at 09:06 +0800, Qu Wenruo wrote: > https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes > > Which is also rebased to latest v4.9.1. Same game as last time, applied to 4.9, no RW mount between the runs. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents 137 => would be nice if btrfs-progrs could give a message on why it failed, i.e. "not enough memory" or so. # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 => error still occurs *without* patch => increased VM memory here # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is 0 total csum bytes: 7330834320 total tree bytes: 10902437888
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 01/30/2017 11:07 AM, Christoph Anton Mitterer wrote: On Sun, 2017-01-29 at 12:27 +0800, Qu Wenruo wrote: Sorry for the late reply, in Chinese New Year vacation. No worries... and happy new year then ;) Happy new year! I'll update the patchset soon to address it. Just tell me and I re-check. Updated, since I'm confident with this update, I'm not using the lowmem_tests branch, but lowmem_fixes branch. (lowmem_tests branch is now deleted in my repo) https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Which is also rebased to latest v4.9.1. Thanks, Qu Thanks again for your detailed output and patience, Thanks as well :) Chris. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Sun, 2017-01-29 at 12:27 +0800, Qu Wenruo wrote: > Sorry for the late reply, in Chinese New Year vacation. No worries... and happy new year then ;) > I'll update the patchset soon to address it. Just tell me and I re-check. > Thanks again for your detailed output and patience, Thanks as well :) Chris. smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On 01/27/2017 07:31 AM, Christoph Anton Mitterer wrote: On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote: Would you please try lowmem_tests branch of my repo? That branch contains a special debug output for the case you encountered, which should help to debug the case. pecial debug output for the case you encountered, which Here the output with your patches (again, not having applied the unnecessary fs-tests patches): In the output below I've replaced filenames with "[snip..snap]" and exchanged some of the xattr values. In case you should need their original values for testing, just tell me and I send them to you off-list. Sorry for the late reply, in Chinese New Year vacation. btrfs-progs v4.9 WITH patches: ** # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing, have: 36864, expect: 45056 ret: 0 Thanks a lot for your debug output! This shows the full reason of the problem! And it can be fixed easily now. Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 === fs tree leaf dump: slot: 136 === leaf 5960902508544 items 191 free space 60 generation 2775 owner 6031 fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719 chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633 item 136 key (277 EXTENT_DATA 524288) itemoff 8443 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6805694144512 nr 36864 extent data offset 0 nr 45056 ram 45056 extent compression 1 (zlib) I just forgot that, we must handle plain extent and compressed extent separately. For compressed case, we should not use extent offset and nr_bytes, but use the full extent. I'll update the patchset soon to address it. Thanks again for your detailed output and patience, Qu === csum tree dump === leaf 4983028940800 items 22 free space 6853 generation 3093 owner 7 fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719 chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633 item 0 key (EXTENT_CSUM EXTENT_CSUM 6805691457536) itemoff 16279 itemsize 4 extent csum item item 1 key (EXTENT_CSUM EXTENT_CSUM 6805691461632) itemoff 16275 itemsize 4 extent csum item item 2 key (EXTENT_CSUM EXTENT_CSUM 6805691465728) itemoff 16263 itemsize 12 extent csum item item 3 key (EXTENT_CSUM EXTENT_CSUM 6805691478016) itemoff 16231 itemsize 32 extent csum item item 4 key (EXTENT_CSUM EXTENT_CSUM 6805691510784) itemoff 16223 itemsize 8 extent csum item item 5 key (EXTENT_CSUM EXTENT_CSUM 6805691518976) itemoff 16199 itemsize 24 extent csum item item 6 key (EXTENT_CSUM EXTENT_CSUM 6805691543552) itemoff 16183 itemsize 16 extent csum item item 7 key (EXTENT_CSUM EXTENT_CSUM 6805691559936) itemoff 15159 itemsize 1024 extent csum item item 8 key (EXTENT_CSUM EXTENT_CSUM 6805692608512) itemoff 15151 itemsize 8 extent csum item item 9 key (EXTENT_CSUM EXTENT_CSUM 6805692616704) itemoff 15055 itemsize 96 extent csum item item 10 key (EXTENT_CSUM EXTENT_CSUM 6805692715008) itemoff 14903 itemsize 152 extent csum item item 11 key (EXTENT_CSUM EXTENT_CSUM 6805693132800) itemoff 14519 itemsize 384 extent csum item item 12 key (EXTENT_CSUM EXTENT_CSUM 6805693526016) itemoff 14391 itemsize 128 extent csum item item 13 key (EXTENT_CSUM EXTENT_CSUM 6805693919232) itemoff 14135 itemsize 256 extent csum item item 14 key (EXTENT_CSUM EXTENT_CSUM 6805694443520) itemoff 12727 itemsize 1408 extent csum item item 15 key (EXTENT_CSUM EXTENT_CSUM 6805695885312) itemoff 11811 itemsize 916 extent csum item item 16 key (EXTENT_CSUM EXTENT_CSUM 6805696823296) itemoff 11007 itemsize 804 extent csum item item 17 key (EXTENT_CSUM EXTENT_CSUM 6805697646592) itemoff 10211 itemsize 796 extent csum item item 18 key (EXTENT_CSUM EXTENT_CSUM 6805698461696) itemoff 9799 itemsize 412 extent csum item item 19 key (EXTENT_CSUM EXTENT_CSUM 6805698883584) itemoff 9259 itemsize 540 extent csum item item 20 key (EXTENT_CSUM EXTENT_CSUM 6805699436544) itemoff 8827 itemsize 432 extent csum item item 21 key (EXTENT_CSUM EXTENT_CSUM 6805699878912) itemoff 7403 itemsize 1424 extent csum item ERROR: errors found in fs roots found 7519512838144 bytes used, error(s) found total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote: > Would you please try lowmem_tests branch of my repo? > > That branch contains a special debug output for the case you > encountered, which should help to debug the case. > pecial debug output for the case you encountered, which Here the output with your patches (again, not having applied the unnecessary fs-tests patches): In the output below I've replaced filenames with "[snip..snap]" and exchanged some of the xattr values. In case you should need their original values for testing, just tell me and I send them to you off-list. btrfs-progs v4.9 WITH patches: ** # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots ERROR: root 6031 EXTENT_DATA[277 524288] datasum missing, have: 36864, expect: 45056 ret: 0 Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 === fs tree leaf dump: slot: 136 === leaf 5960902508544 items 191 free space 60 generation 2775 owner 6031 fs uuid 326d292d-f97b-43ca-b1e8-c722d3474719 chunk uuid 5da7e818-7f0b-43c1-b465-fdfaa52da633 item 0 key (274 EXTENT_DATA 7733248) itemoff 16230 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807724716032 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 1 key (274 EXTENT_DATA 7864320) itemoff 16177 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807724834816 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 2 key (274 EXTENT_DATA 7995392) itemoff 16124 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807724953600 nr 122880 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 3 key (274 EXTENT_DATA 8126464) itemoff 16071 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725076480 nr 122880 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 4 key (274 EXTENT_DATA 8257536) itemoff 16018 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725199360 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 5 key (274 EXTENT_DATA 8388608) itemoff 15965 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725318144 nr 114688 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 6 key (274 EXTENT_DATA 8519680) itemoff 15912 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725432832 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 7 key (274 EXTENT_DATA 8650752) itemoff 15859 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725551616 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 8 key (274 EXTENT_DATA 8781824) itemoff 15806 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725670400 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 9 key (274 EXTENT_DATA 8912896) itemoff 15753 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725789184 nr 122880 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 10 key (274 EXTENT_DATA 9043968) itemoff 15700 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807725912064 nr 114688 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 11 key (274 EXTENT_DATA 9175040) itemoff 15647 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807726026752 nr 114688 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 12 key (274 EXTENT_DATA 9306112) itemoff 15594 itemsize 53 generation 2775 type 1 (regular) extent data disk byte 6807726141440 nr 118784 extent data offset 0 nr 131072 ram 131072 extent compression 1 (zlib) item 13 key (274 EXTENT_DATA 9437184) itemoff 15541 itemsize 53 generation 2775 type 1 (regular) extent data disk byte
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Thu, 2017-01-26 at 11:10 +0800, Qu Wenruo wrote: > In fact, the result without patches is not really needed for current > stage. > > Feel free to skip them until the patched ones passed. > Which should save you some time. Well the idea is, that if I do further writes in the meantime (by adding new backup data), then things in the fs could change (I blindly assume) in such a way, that the false positive isn't triggered any more - not because a patch would finally have it fixed, but simply because things on the fs changed... That's why I repeated it always so far - just to see that the issues would be still there. > Would you please try lowmem_tests branch of my repo? > > That branch contains a special debug output for the case you > encountered, which should help to debug the case. > pecial debug output for the case you encountered, which Sure, tomorrow. Best wishes, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
Thanks for your patient and the test! At 01/26/2017 10:50 AM, Christoph Anton Mitterer wrote: On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote: https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Just finished trying your new patches. Same game as last time, applied to 4.9, no RW mount between the runs. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is 0 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated:
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote: > https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Just finished trying your new patches. Same game as last time, applied to 4.9, no RW mount between the runs. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is 0 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 btrfs-progs v4.9 WITH patches:
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Wed, 2017-01-25 at 12:16 +0800, Qu Wenruo wrote: > New patches are out now. > > Although I just updated > 0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch to > fix > all similar bugs. > > You could get it from github: > https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Sure, will take a while, though (hopefully get it done tomorrow) > Unfortunately, I didn't find the cause of the remaining error of > that > missing csum. > And considering the size of your fs, btrfs-image is not possible, so > I'm > afraid you need to test the patches every time it updates. No worries :-) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 01/25/2017 08:46 AM, Christoph Anton Mitterer wrote: On Wed, 2017-01-25 at 08:44 +0800, Qu Wenruo wrote: Thanks for the test, You're welcome... I'm happy if I can help :) Just tell me once you think you found something, and I'll repeat the testing. Cheers, Chr is. New patches are out now. Although I just updated 0001-btrfs-progs-lowmem-check-Fix-wrong-block-group-check.patch to fix all similar bugs. You could get it from github: https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Unfortunately, I didn't find the cause of the remaining error of that missing csum. And considering the size of your fs, btrfs-image is not possible, so I'm afraid you need to test the patches every time it updates. Sorry for that, Qu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
At 01/25/2017 12:54 AM, Christoph Anton Mitterer wrote: Hey Qu. I was giving your patches a try, again on the very same fs (which saw however writes in the meantime), from my initial report. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is 0 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 => so the fs would still show the symptoms Then, with no RW mount to the fs in between,
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
On Wed, 2017-01-25 at 08:44 +0800, Qu Wenruo wrote: > Thanks for the test, You're welcome... I'm happy if I can help :) Just tell me once you think you found something, and I'll repeat the testing. Cheers, Chr is. smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
Hey Qu. I was giving your patches a try, again on the very same fs (which saw however writes in the meantime), from my initial report. btrfs-progs v4.9 WITHOUT patch: *** # btrfs check /dev/nbd0 ; echo $? checking extents checking free space cache checking fs roots checking csums checking root refs Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is 0 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 0 # btrfs check --mode=lowmem /dev/nbd0 ; echo $? checking extents ERROR: block group[74117545984 1073741824] used 1073741824 but extent items used 0 ERROR: block group[239473786880 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[500393050112 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[581997428736 1073741824] used 1073741824 but extent items used 0 ERROR: block group[626557714432 1073741824] used 1073741824 but extent items used 0 ERROR: block group[668433645568 1073741824] used 1073741824 but extent items used 0 ERROR: block group[948680261632 1073741824] used 1073741824 but extent items used 0 ERROR: block group[982503129088 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1039411445760 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1054443831296 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1190809042944 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1279392743424 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1481256206336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[1620842643456 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[1914511032320 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3055361720320 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3216422993920 1073741824] used 1073741824 but extent items used 0 ERROR: block group[3670615785472 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3801612288000 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[3828455833600 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4250973241344 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4261710659584 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[4392707162112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4558063403008 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4607455526912 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4635372814336 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4640204652544 1073741824] used 1073741824 but extent items used 0 ERROR: block group[4642352136192 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[4681006841856 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5063795802112 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5171169984512 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5216267141120 1073741824] used 1073741824 but extent items used 1207959552 ERROR: block group[5290355326976 1073741824] used 1073741824 but extent items used 0 ERROR: block group[5445511020544 1073741824] used 1073741824 but extent items used 1074266112 ERROR: block group[6084387405824 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6104788500480 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6878956355584 1073741824] used 1073741824 but extent items used 0 ERROR: block group[6997067956224 1073741824] used 1073741824 but extent items used 0 ERROR: block group[7702516334592 1073741824] used 1073741824 but extent items used 0 ERROR: block group[8051482427392 1073741824] used 1073741824 but extent items used 1084751872 ERROR: block group[8116980678656 1073741824] used 1073741824 but extent items used 0 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/nbd0 UUID: 326d292d-f97b-43ca-b1e8-c722d3474719 found 7519512838144 bytes used err is -5 total csum bytes: 7330834320 total tree bytes: 10902437888 total fs tree bytes: 2019704832 total extent tree bytes: 1020149760 btree space waste bytes: 925714197 file data blocks allocated: 7509228494848 referenced 7630551511040 1 => so the fs would still show the symptoms Then, with no RW mount to the fs in between, 4.9 with the following of your patches:
[PATCH 0/9] Lowmem mode fsck fixes with fsck-tests framework update
Patches can be fetch from github:
https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes
Although there are near 10 patches, but they are all small.
No need to be scared. :)
Thanks for reports from Chris Murphy and Christoph Anton Mitterer,
several new bugs are exposed for lowmem mode fsck.
And one original mode bug, not fixed in this patchset.
The original mode bug is caused by fsck/006, which repairs doesn't
fix backrefs of a data extent, which lowmem mode detects it correctly.
1) Block group used space false alert
If a BLOCK_GROUP_ITEM or its first EXTENT/METADATA_ITEM is located at
the first slot of a leaf, search_slot() used by lowmem mode can
point to previous leaf, with path->slots[0] beyond valid leaf slots.
This makes us to read out uninitialized data, and can abort block
group used space check loop, causing a false alert.
Fix it with a test case image inside fsck-tests/020/extent-ref-cases
Reported by Christoph.
2) Partly written prealloc extent false alert
If a prealloc extent gets partily written, lowmem mode will report
prealloc extent shouldn't have csum.
Lowmem mode passed wrong variable to csum checking code, causing it
to check the whole range of the prealloc extent, making the bug
happens.
Fix it with a test case inside fsck-tests/020/extent-ref-cases.
Reported by Chirs Murphy And Christoph.
3) Extent item size false alert.
Under certain case, btrfs lowmem mode check reports data backref
lost.
It's because newly introduced extent item size check aborts normal
check routine.
It can happen if a data/metadata extent item has no inline ref.
Fix it, test case already submitted before and merged, but due to
fsck-tests framework bugs, it never get called for lowmem mode.
4) fsck-tests Lowmem mode override fixes
Allow lowmem mode override to get called for all tests, and allow
them all to pass lowmem mode except fsck-tests/006, which is a
original repair mode bug.
Lu Fengqi (1):
btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair
work
Qu Wenruo (8):
btrfs-progs: lowmem check: Fix wrong block group check when search
slot points to slot beyong leaf
btrfs-progs: fsck-test: Add test image for lowmem mode block group
false alert
btrfs-progs: fsck: Output verbose error when fsck found a bug
btrfs-progs: lowmem check: Fix false alert in checking data extent
pointing to prealloc extent
btrfs-progs: lowmem check: Fix extent item size false alert
btrfs-progs: tests: Move fsck-tests/015 to fuzz tests
btrfs-progs: fsck-tests: Make 013 compatible with lowmem mode
btrfs-progs: fsck-tests: Add new test case for partly written prealloc
extent
cmds-check.c | 80 -
tests/common | 8 +--
tests/common.local | 2 +-
tests/fsck-tests/013-extent-tree-rebuild/test.sh | 2 +-
.../block_group_item_false_alert.raw.xz| Bin 0 -> 47792 bytes
tests/fsck-tests/020-extent-ref-cases/test.sh | 30 ++--
.../images}/bko-97171-btrfs-image.raw.txt | 0
.../images}/bko-97171-btrfs-image.raw.xz | Bin
8 files changed, 95 insertions(+), 27 deletions(-)
create mode 100644
tests/fsck-tests/020-extent-ref-cases/block_group_item_false_alert.raw.xz
rename tests/{fsck-tests/015-check-bad-memory-access =>
fuzz-tests/images}/bko-97171-btrfs-image.raw.txt (100%)
rename tests/{fsck-tests/015-check-bad-memory-access =>
fuzz-tests/images}/bko-97171-btrfs-image.raw.xz (100%)
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
