On 2019/3/20 上午6:36, Piotr Balwierz wrote: > Hello BTRFS experts, > > I am reporting/confirming the same problem present in kernel 4.19.0 > (Debian Buster, x86_64). > To be specific: > > 0) The BTRFS partition used to work fine for 2 years until a power cut > today. It uses lzo compression. Until power cut kernel version was > probably 4.14.0 > > 1) BTRFS partition refused to mount during boot with the following > kernel and systemd errors: > > |Mar 19 15:10:52 mamut kernel: BTRFS error (device dm-5): open_ctree > failed Mar 19 15:10:52 mamut kernel: BTRFS info (device dm-5): use lzo > compression, level 0 Mar 19 15:10:52 mamut kernel: BTRFS info (device > dm-5): disk space caching is enabled Mar 19 15:10:52 mamut kernel: BTRFS > info (device dm-5): has skinny extents Mar 19 15:10:52 mamut systemd[1]: > mnt-storage.mount: Mount process exited, code=killed, status=15/TERM Mar > 19 15:10:52 mamut systemd[1]: mnt-storage.mount: Failed with result > 'timeout'. Mar 19 15:10:52 mamut systemd[1]: Failed to mount > /mnt/storage. Mar 19 15:10:52 mamut kernel: BTRFS error (device dm-5): > super_total_bytes 52798547820544 mismatch with fs_devices total_rw_bytes > 105597095641088 Mar 19 15:10:52 mamut kernel: BTRFS error (device dm-5): > failed to read chunk tree: -22 Mar 19 15:10:52 mamut kernel: BTRFS error > (device dm-5): open_ctree failed| > > 2)|Note that total_rw_bytes in kernel error message equals exactly twice > ||super_total_bytes|. super_total_bytes is the correct size of the > partition (52TB, 48TiB). It equals to the size reported by fdisk. > Something being counted twice? The partition is on top of a 8-fold > multipath to a fibre channel-attached disk array (in case it matters). > > 3) the first manual mount failed with a generic message (wrong fs type etc) > > 4) the first btrfs check (without --repair) fails with an IO error. I > used to have no IO errors on this device before. > > |# btrfs check /dev/mapper/fc_trunk-part3 Opening filesystem to check... > Checking filesystem on /dev/mapper/fc_trunk-part3 UUID: > 40a2e65b-f34a-4d33-946d-055d93fe7ffa [1/7] checking root items ERROR: > failed to repair root items: Input/output error| > > 5) the second btrfs check with --repair succeeds. I did *not* run btrfs > rescue fix-device-size. > > btrfs check --repair /dev/mapper/fc_trunk-part3 > enabling repair mode > Opening filesystem to check... > Checking filesystem on /dev/mapper/fc_trunk-part3 > UUID: 40a2e65b-f34a-4d33-946d-055d93fe7ffa > [1/7] checking root items > Fixed 0 roots. > [2/7] checking extents > No device size related problem found > [3/7] checking free space cache > cache and super generation don't match, space cache will be invalidated > [4/7] checking fs roots > [5/7] checking only csums items (without verifying data) > [6/7] checking root refs > [7/7] checking quota groups skipped (not enabled on this FS) > found 41629742850048 bytes used, no error found > total csum bytes: 40569609916 > total tree bytes: 76425707520 > total fs tree bytes: 15665889280 > total extent tree bytes: 16241786880 > btree space waste bytes: 4094701258 > file data blocks allocated: 54396052914176 > referenced 67747603632128
If btrfs-check can repair, then your fs is not seriously corrupted, which is a good news. > > 6) The second try of manual mount succeeded, but it took 2 minutes. Slow mount is related to extent tree size. > # time mount /mnt/storage > real 2m9.076s > user 0m0.005s > sys 0m2.753s > > 7) btrfs inspect-internal dump-super reports only the correct size: > > btrfs inspect-internal dump-super -fa /dev/mapper/fc_trunk-part3 | > grep total > total_bytes 52798547820544 > dev_item.total_bytes 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > total_bytes 52798547820544 > dev_item.total_bytes 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > total_bytes 52798547820544 > dev_item.total_bytes 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > backup_total_bytes: 52798547820544 > > > 8) I did not try a second reboot. > btrfs-progs v4.20.1 > > btrfs fi usage /mnt/storage > Overall: > Device size: 48.02TiB > Device allocated: 37.94TiB This explains the slow mount. Unless using the new BG_TREE feature I purposed, the slow mount can't really be solved. Thanks, Qu > Device unallocated: 10.08TiB > Device missing: 0.00B > Used: 37.93TiB > Free (estimated): 10.08TiB (min: 5.04TiB) > Data ratio: 1.00 > Metadata ratio: 2.00 > Global reserve: 512.00MiB (used: 0.00B) > > Data,single: Size:37.80TiB, Used:37.79TiB > /dev/mapper/fc_trunk-part3 37.80TiB > > Metadata,DUP: Size:73.00GiB, Used:71.17GiB > /dev/mapper/fc_trunk-part3 146.00GiB > > System,DUP: Size:8.00MiB, Used:4.09MiB > /dev/mapper/fc_trunk-part3 16.00MiB > > Unallocated: > /dev/mapper/fc_trunk-part3 10.08TiB > > > > Hope that it will be helpful, > Piotr J. Balwierz > > > > On 24/10/2017 20.02, Konstantin V. Gavrilenko wrote: >> The mentioning of the device scan scode and the fact the total_bytes >> is double made me try hashing the raid from the fstab. >> So i booted, run the "inspect-internal dump-super" that confirmed that >> it is in order. >> # grep -i total_bytes hashed-inspect-internal >> >> total_bytes 32004083023872 >> dev_item.total_bytes 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> total_bytes 32004083023872 >> dev_item.total_bytes 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> total_bytes 32004083023872 >> dev_item.total_bytes 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> backup_total_bytes: 32004083023872 >> >> then I unhashed the device in fstab, mounted it manually and it >> successfully mounted. >> >> # time mount /mnt/arh-backup1/ >> >> real 2m49.021s >> user 0m0.000s >> sys 0m1.244s >> >> >> >> With the unhashed device in the fstab, i rebooted and upon reboot I >> run mount >> >> time mount /mnt/arh-backup1/ >> mount: wrong fs type, bad option, bad superblock on /dev/sda, >> missing codepage or helper program, or other error >> >> In some cases useful info is found in syslog - try >> dmesg | tail or so. >> >> real 1m20.499s >> user 0m0.000s >> sys 0m0.045s >> >> that failed. I further waited for couple of minutes and run the mount >> again, and it mounted successfully. >> >> >> So it seems that because of the amount of time it takes mount, nearly >> 3 minutes, to mount the device, there is some sort of race condition, >> and two device scans are running at the same time, or something similar. >> I can say one thing for sure, it wasn't happening on 4.10 and I have >> only observed such behaviour on 4.12 and 4.13 >> >> p.s. the disk does not mount automatically upon boot, but can be >> mounted manually later >> >> >> # uptime >> 19:54:45 up 4 min, 1 user, load average: 0.30, 0.74, 0.39 >> >> # time mount /mnt/arh-backup1/ >> >> real 2m52.247s >> user 0m0.000s >> sys 0m1.246s >> >> >> Here is the dmesg extract. It seems that for some reason on 204th >> second the system return "open ctree failed" >> on 329 second, I started the mount manually. >> >> [ 204.389231] BTRFS error (device sda): open_ctree failed >> [ 329.234613] BTRFS info (device sda): force zlib compression >> [ 329.234618] BTRFS info (device sda): using free space tree >> [ 329.234620] BTRFS info (device sda): has skinny extents >> >> >> hope that helps and thanks for your help >> >> >> Yours sincerely, >> Konstantin V. Gavrilenko >> >> >> >> ----- Original Message ----- >> From: "Qu Wenruo" <quwenruo.bt...@gmx.com> >> To: "Konstantin V. Gavrilenko" <k.gavrile...@arhont.com> >> Cc: "Linux fs Btrfs" <linux-btrfs@vger.kernel.org> >> Sent: Tuesday, 24 October, 2017 3:44:21 PM >> Subject: Re: super_total_bytes 32004083023872 mismatch with fs_devices >> total_rw_bytes 64008166047744 >> >> >> >> On 2017年10月24日 19:44, Konstantin V. Gavrilenko wrote: >>> answers inline marked with KVG: >>> >>> Yours sincerely, >>> Konstantin V. Gavrilenko >>> >>> >>> >>> >>> ----- Original Message ----- >>> From: "Qu Wenruo" <quwenruo.bt...@gmx.com> >>> To: "Konstantin V. Gavrilenko" <k.gavrile...@arhont.com>, "Linux fs >>> Btrfs" <linux-btrfs@vger.kernel.org> >>> Sent: Tuesday, 24 October, 2017 11:37:56 AM >>> Subject: Re: super_total_bytes 32004083023872 mismatch with >>> fs_devices total_rw_bytes 64008166047744 >>> >>> >>> >>> On 2017年10月24日 17:20, Konstantin V. Gavrilenko wrote: >>>> Hi list, >>>> >>>> having installed the recent kernel version I am no longer able to >>>> mount the btrfs partition with compression on the first attempt. >>>> Previously on 4.10.0-37-generic everything was working fine, once I >>>> switched to 4.13.9-041309-generic I started getting the following >>>> error while trying to mount it with the same options >>>> "compress-force=zlib,space_cache=v2" >>>> >>>> [ 204.596381] BTRFS error (device sda): open_ctree failed >>>> [ 204.631895] BTRFS info (device sda): force zlib compression >>>> [ 204.631901] BTRFS info (device sda): using free space tree >>>> [ 204.631903] BTRFS info (device sda): has skinny extents >>>> [ 204.890145] BTRFS error (device sda): super_total_bytes >>>> 32004083023872 mismatch with fs_devices total_rw_bytes 64008166047744 >>>> [ 204.891276] BTRFS error (device sda): failed to read chunk tree: -22 >>>> [ 204.944333] BTRFS error (device sda): open_ctree failed >>> Such problem can be easily fixed with this branch: >>> https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev >>> >>> Use "btrfs rescue fix-device-size" should handle it well. >>> >>> But the problem is, normally the super_total_bytes should only be less >>> than 4K smaller than total device size. >>> >>> Unless something else went wrong, it should not have such large >>> difference. >>> >>> >>> KVG: Thanks, will try. >>> The drive was initially created as RAID5 with 4 x 8tb devices, and >>> later expanded to included another 8Tb. So now it is 5x8tb RAID5. >>> >>> >>>> For some reason, the super_total_bytes is exactly half of >>>> total_rw_bytes. >>>> >>>> >>>> however, if after unsuccessful first mount attempt, I mount it with >>>> minimum number of options "space_cache=v2" the partition mounts. >>>> Then I umount it, and mount normally, with full set of options >>>> "compress-force=zlib,space_cache=v2" it mounts without an error. >>>> I also observed the same error on 4.12.14-041214-generic >>>> Any ideas why this might be happening? >>> Would you please provide super dump by: >>> >>> # btrfs inspect-internal dump-super -fa /dev/sda >>> >>> (Although I don't think it will be very interesting since it can be >>> mounted later) >>> >>> >>> KVG: here you go >>> >>> # btrfs inspect-internal dump-super -fa /dev/sda >>> superblock: bytenr=65536, device=/dev/sda >>> --------------------------------------------------------- >>> csum_type 0 (crc32c) >>> csum_size 4 >>> csum 0xc4b95762 [match] >>> bytenr 65536 >>> flags 0x1 >>> ( WRITTEN ) >>> magic _BHRfS_M [match] >>> fsid 017b587a-3e88-4f08-8616-ec686f8f969f >>> label arh-backup >>> generation 145942 >>> root 17334200778752 >>> sys_array_size 129 >>> chunk_root_generation 145699 >>> root_level 1 >>> chunk_root 20971520 >>> chunk_root_level 1 >>> log_root 17334149644288 >>> log_root_transid 0 >>> log_root_level 0 >>> total_bytes 32004083023872 >> Here 32004083023872. >>> bytes_used 19719727349760 >>> sectorsize 4096 >>> nodesize 16384 >>> leafsize (deprecated) 16384 >>> stripesize 4096 >>> root_dir 6 >>> num_devices 1 >>> compat_flags 0x0 >>> compat_ro_flags 0x3 >>> ( FREE_SPACE_TREE | >>> FREE_SPACE_TREE_VALID ) >>> incompat_flags 0x169 >>> ( MIXED_BACKREF | >>> COMPRESS_LZO | >>> BIG_METADATA | >>> EXTENDED_IREF | >>> SKINNY_METADATA ) >>> cache_generation 7 >>> uuid_tree_generation 145942 >>> dev_item.uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> dev_item.fsid 017b587a-3e88-4f08-8616-ec686f8f969f [match] >>> dev_item.type 0 >>> dev_item.total_bytes 32004083023872 >> And here, 32004083023872 again. >> >> So it matches now. >> >> And considering the original report says it's twice the size, I wonder >> if it's something related to device scan. >> >> Like the same disk get counted twice, so it's reporting like that. >> >> If the problem happens again, would you please try run "btrfs device >> scan" and remount it with the same mount option? >> >> >>> dev_item.bytes_used 19832028266496 >>> dev_item.io_align 4096 >>> dev_item.io_width 4096 >>> dev_item.sector_size 4096 >>> dev_item.devid 1 >>> dev_item.dev_group 0 >>> dev_item.seek_speed 0 >>> dev_item.bandwidth 0 >>> dev_item.generation 0 >>> sys_chunk_array[2048]: >>> item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) >>> length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP >>> io_align 65536 io_width 65536 sector_size 4096 >>> num_stripes 2 sub_stripes 0 >>> stripe 0 devid 1 offset 20971520 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> stripe 1 devid 1 offset 29360128 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> backup_roots[4]: >>> backup 0: >>> backup_tree_root: 17334200778752 gen: >>> 145940 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177021952 gen: >>> 145940 level: 3 >>> backup_fs_root: 17334133260288 gen: >>> 145941 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 1: >>> backup_tree_root: 17334201057280 gen: >>> 145941 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145941 level: 3 >>> backup_fs_root: 17334133161984 gen: >>> 145942 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 2: >>> backup_tree_root: 17334200778752 gen: >>> 145942 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334184558592 gen: >>> 145942 level: 3 >>> backup_fs_root: 17334135275520 gen: >>> 145943 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 3: >>> backup_tree_root: 17334201057280 gen: >>> 145939 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145939 level: 3 >>> backup_fs_root: 17334131884032 gen: >>> 145940 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> >>> superblock: bytenr=67108864, device=/dev/sda >>> --------------------------------------------------------- >>> csum_type 0 (crc32c) >>> csum_size 4 >>> csum 0xd1c4187f [match] >>> bytenr 67108864 >>> flags 0x1 >>> ( WRITTEN ) >>> magic _BHRfS_M [match] >>> fsid 017b587a-3e88-4f08-8616-ec686f8f969f >>> label arh-backup >>> generation 145942 >>> root 17334200778752 >>> sys_array_size 129 >>> chunk_root_generation 145699 >>> root_level 1 >>> chunk_root 20971520 >>> chunk_root_level 1 >>> log_root 0 >>> log_root_transid 0 >>> log_root_level 0 >>> total_bytes 32004083023872 >>> bytes_used 19719727349760 >>> sectorsize 4096 >>> nodesize 16384 >>> leafsize (deprecated) 16384 >>> stripesize 4096 >>> root_dir 6 >>> num_devices 1 >>> compat_flags 0x0 >>> compat_ro_flags 0x3 >>> ( FREE_SPACE_TREE | >>> FREE_SPACE_TREE_VALID ) >>> incompat_flags 0x169 >>> ( MIXED_BACKREF | >>> COMPRESS_LZO | >>> BIG_METADATA | >>> EXTENDED_IREF | >>> SKINNY_METADATA ) >>> cache_generation 7 >>> uuid_tree_generation 145942 >>> dev_item.uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> dev_item.fsid 017b587a-3e88-4f08-8616-ec686f8f969f [match] >>> dev_item.type 0 >>> dev_item.total_bytes 32004083023872 >>> dev_item.bytes_used 19832028266496 >>> dev_item.io_align 4096 >>> dev_item.io_width 4096 >>> dev_item.sector_size 4096 >>> dev_item.devid 1 >>> dev_item.dev_group 0 >>> dev_item.seek_speed 0 >>> dev_item.bandwidth 0 >>> dev_item.generation 0 >>> sys_chunk_array[2048]: >>> item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) >>> length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP >>> io_align 65536 io_width 65536 sector_size 4096 >>> num_stripes 2 sub_stripes 0 >>> stripe 0 devid 1 offset 20971520 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> stripe 1 devid 1 offset 29360128 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> backup_roots[4]: >>> backup 0: >>> backup_tree_root: 17334200778752 gen: >>> 145940 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177021952 gen: >>> 145940 level: 3 >>> backup_fs_root: 17334133260288 gen: >>> 145941 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 1: >>> backup_tree_root: 17334201057280 gen: >>> 145941 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145941 level: 3 >>> backup_fs_root: 17334133161984 gen: >>> 145942 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 2: >>> backup_tree_root: 17334200778752 gen: >>> 145942 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334184558592 gen: >>> 145942 level: 3 >>> backup_fs_root: 17334133161984 gen: >>> 145942 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 3: >>> backup_tree_root: 17334201057280 gen: >>> 145939 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145939 level: 3 >>> backup_fs_root: 17334131884032 gen: >>> 145940 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> >>> superblock: bytenr=274877906944, device=/dev/sda >>> --------------------------------------------------------- >>> csum_type 0 (crc32c) >>> csum_size 4 >>> csum 0x2c434e4e [match] >>> bytenr 274877906944 >>> flags 0x1 >>> ( WRITTEN ) >>> magic _BHRfS_M [match] >>> fsid 017b587a-3e88-4f08-8616-ec686f8f969f >>> label arh-backup >>> generation 145942 >>> root 17334200778752 >>> sys_array_size 129 >>> chunk_root_generation 145699 >>> root_level 1 >>> chunk_root 20971520 >>> chunk_root_level 1 >>> log_root 0 >>> log_root_transid 0 >>> log_root_level 0 >>> total_bytes 32004083023872 >>> bytes_used 19719727349760 >>> sectorsize 4096 >>> nodesize 16384 >>> leafsize (deprecated) 16384 >>> stripesize 4096 >>> root_dir 6 >>> num_devices 1 >>> compat_flags 0x0 >>> compat_ro_flags 0x3 >>> ( FREE_SPACE_TREE | >>> FREE_SPACE_TREE_VALID ) >>> incompat_flags 0x169 >>> ( MIXED_BACKREF | >>> COMPRESS_LZO | >>> BIG_METADATA | >>> EXTENDED_IREF | >>> SKINNY_METADATA ) >>> cache_generation 7 >>> uuid_tree_generation 145942 >>> dev_item.uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> dev_item.fsid 017b587a-3e88-4f08-8616-ec686f8f969f [match] >>> dev_item.type 0 >>> dev_item.total_bytes 32004083023872 >>> dev_item.bytes_used 19832028266496 >>> dev_item.io_align 4096 >>> dev_item.io_width 4096 >>> dev_item.sector_size 4096 >>> dev_item.devid 1 >>> dev_item.dev_group 0 >>> dev_item.seek_speed 0 >>> dev_item.bandwidth 0 >>> dev_item.generation 0 >>> sys_chunk_array[2048]: >>> item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) >>> length 8388608 owner 2 stripe_len 65536 type SYSTEM|DUP >>> io_align 65536 io_width 65536 sector_size 4096 >>> num_stripes 2 sub_stripes 0 >>> stripe 0 devid 1 offset 20971520 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> stripe 1 devid 1 offset 29360128 >>> dev_uuid ee632cde-2139-4181-9d57-ff892768f2ef >>> backup_roots[4]: >>> backup 0: >>> backup_tree_root: 17334200778752 gen: >>> 145940 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177021952 gen: >>> 145940 level: 3 >>> backup_fs_root: 17334133260288 gen: >>> 145941 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 1: >>> backup_tree_root: 17334201057280 gen: >>> 145941 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145941 level: 3 >>> backup_fs_root: 17334133161984 gen: >>> 145942 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 2: >>> backup_tree_root: 17334200778752 gen: >>> 145942 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334184558592 gen: >>> 145942 level: 3 >>> backup_fs_root: 17334133161984 gen: >>> 145942 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> backup 3: >>> backup_tree_root: 17334201057280 gen: >>> 145939 level: 1 >>> backup_chunk_root: 20971520 gen: >>> 145699 level: 1 >>> backup_extent_root: 17334177988608 gen: >>> 145939 level: 3 >>> backup_fs_root: 17334131884032 gen: >>> 145940 level: 3 >>> backup_dev_root: 32391168 gen: >>> 145836 level: 1 >>> backup_csum_root: 17154410381312 gen: >>> 145925 level: 3 >>> backup_total_bytes: 32004083023872 >>> backup_bytes_used: 19719727349760 >>> backup_num_devices: 1 >>> >>> >>> >>> >>> >>> And device tree dump by: >>> # btrfs inspect-internal dump-tree -t dev /dev/sda >>> >>> KVG: Here you go again >>> >>> # btrfs inspect-internal dump-tree -t dev /dev/sda > >>> /tmp/inspect-internal-dump-tree >>> parent transid verify failed on 17334203744256 wanted 145954 found >>> 145956 >>> parent transid verify failed on 17334203744256 wanted 145954 found >>> 145956 >>> parent transid verify failed on 17334203744256 wanted 145954 found >>> 145956 >>> parent transid verify failed on 17334203744256 wanted 145954 found >>> 145956 >>> Ignoring transid failure >>> >>> # wc -l < >>> inspect-internal-dump-tree >>> >>> >>> 74760 >> That's beyond my expectation, but since the super block matches, I >> wonder it's related to device scan implemented wrongly or has some race. >> >> And in that case, dump-tree output is not that important. >> >> BTW, are you running dump-tree with device mounted? If so the transid >> verification failure may not be a big problem. >> >>> >>> # head -n 150 < >>> inspect-internal-dump-tree >>> >>> >>> [13:28:00] >>> btrfs-progs v4.13.3 >>> device tree key (DEV_TREE ROOT_ITEM 0) >>> node 32391168 level 1 items 88 free 405 generation 145836 owner 4 >>> fs uuid 017b587a-3e88-4f08-8616-ec686f8f969f >>> chunk uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> key (0 PERSISTENT_ITEM 1) block 32407552 (1978) gen 145836 >>> key (1 DEV_EXTENT 234113466368) block 231767769088 >>> (14145982) gen 274 >>> key (1 DEV_EXTENT 473557893120) block 484797186048 >>> (29589672) gen 378 >>> key (1 DEV_EXTENT 710854836224) block 861293805568 >>> (52569202) gen 482 >>> key (1 DEV_EXTENT 949225521152) block 1042388156416 >>> (63622324) gen 587 >>> key (1 DEV_EXTENT 1187596206080) block 1221328453632 >>> (74543973) gen 691 >>> key (1 DEV_EXTENT 1424893149184) block 1583808970752 >>> (96668028) gen 764 >>> key (1 DEV_EXTENT 1662190092288) block 1603743318016 >>> (97884724) gen 843 >>> key (1 DEV_EXTENT 1900560777216) block 2231483187200 >>> (136198925) gen 1017 >>> key (1 DEV_EXTENT 2138931462144) block 2241853407232 >>> (136831873) gen 1080 >>> key (1 DEV_EXTENT 2371933437952) block 2268266070016 >>> (138443974) gen 1138 >>> key (1 DEV_EXTENT 2610304122880) block 2311626964992 >>> (141090513) gen 1177 >>> key (1 DEV_EXTENT 2848674807808) block 2590371545088 >>> (158103732) gen 1263 >>> key (1 DEV_EXTENT 3088119234560) block 2837798567936 >>> (173205479) gen 1314 >>> key (1 DEV_EXTENT 3326489919488) block 3114321903616 >>> (190083124) gen 1372 >>> key (1 DEV_EXTENT 3564860604416) block 3591785086976 >>> (219225164) gen 1448 >>> key (1 DEV_EXTENT 3802157547520) block 3718121062400 >>> (226936100) gen 1524 >>> key (1 DEV_EXTENT 4040528232448) block 21927333150720 >>> (1338338205) gen 136290 >>> key (1 DEV_EXTENT 4278898917376) block 4133925863424 >>> (252314811) gen 1724 >>> key (1 DEV_EXTENT 4516195860480) block 22441652109312 >>> (1369729743) gen 133266 >>> key (1 DEV_EXTENT 4755640287232) block 14608468082688 >>> (891630132) gen 133130 >>> key (1 DEV_EXTENT 4992937230336) block 5180103491584 >>> (316168426) gen 2146 >>> key (1 DEV_EXTENT 5231307915264) block 20077315735552 >>> (1225422103) gen 131467 >>> key (1 DEV_EXTENT 5468604858368) block 5866555588608 >>> (358066137) gen 3067 >>> key (1 DEV_EXTENT 5705901801472) block 6367858622464 >>> (388663246) gen 3174 >>> key (1 DEV_EXTENT 5826160885760) block 6367858638848 >>> (388663247) gen 3174 >>> key (1 DEV_EXTENT 6064531570688) block 17709247496192 >>> (1080886688) gen 63205 >>> key (1 DEV_EXTENT 6303975997440) block 17459226411008 >>> (1065626612) gen 145661 >>> key (1 DEV_EXTENT 6500470751232) block 17459593674752 >>> (1065649028) gen 145669 >>> key (1 DEV_EXTENT 6737767694336) block 14608384458752 >>> (891625028) gen 132007 >>> key (1 DEV_EXTENT 6976138379264) block 15230725767168 >>> (929609727) gen 132017 >>> key (1 DEV_EXTENT 7136125911040) block 13657160859648 >>> (833566947) gen 137915 >>> key (1 DEV_EXTENT 7255311253504) block 13657457180672 >>> (833585033) gen 137917 >>> key (1 DEV_EXTENT 7492608196608) block 22441489580032 >>> (1369719823) gen 136294 >>> key (1 DEV_EXTENT 7730978881536) block 13854137253888 >>> (845589432) gen 138616 >>> key (1 DEV_EXTENT 7969349566464) block 7788285968384 >>> (475359251) gen 140545 >>> key (1 DEV_EXTENT 8207720251392) block 14113310818304 >>> (861408131) gen 127092 >>> key (1 DEV_EXTENT 8446090936320) block 8679398998016 >>> (529748474) gen 17985 >>> key (1 DEV_EXTENT 8684461621248) block 8916314619904 >>> (544208656) gen 18574 >>> key (1 DEV_EXTENT 8922832306176) block 29434636730368 >>> (1796547652) gen 144688 >>> key (1 DEV_EXTENT 9157981765632) block 6453626322944 >>> (393898091) gen 145220 >>> key (1 DEV_EXTENT 9361992712192) block 19459087679488 >>> (1187688457) gen 132049 >>> key (1 DEV_EXTENT 9600363397120) block 12893675651072 >>> (786967508) gen 116367 >>> key (1 DEV_EXTENT 9720622481408) block 22441517318144 >>> (1369721516) gen 95603 >>> key (1 DEV_EXTENT 9847860887552) block 22441466626048 >>> (1369718422) gen 117878 >>> key (1 DEV_EXTENT 10082473476096) block 22493359931392 >>> (1372885738) gen 117882 >>> key (1 DEV_EXTENT 10232797331456) block 13983139643392 >>> (853463113) gen 137923 >>> key (1 DEV_EXTENT 10471168016384) block 6959763996672 >>> (424790283) gen 105377 >>> key (1 DEV_EXTENT 10591427100672) block 10067009994752 >>> (614441528) gen 27773 >>> key (1 DEV_EXTENT 10830871527424) block 10072383062016 >>> (614769474) gen 28150 >>> key (1 DEV_EXTENT 11070315954176) block 10072413356032 >>> (614771323) gen 92253 >>> key (1 DEV_EXTENT 11309760380928) block 22493328687104 >>> (1372883831) gen 133268 >>> key (1 DEV_EXTENT 11548131065856) block 1042474778624 >>> (63627611) gen 133909 >>> key (1 DEV_EXTENT 11785428008960) block 15016548761600 >>> (916537400) gen 121951 >>> key (1 DEV_EXTENT 12024872435712) block 12656435298304 >>> (772487506) gen 32985 >>> key (1 DEV_EXTENT 12263243120640) block 12893629710336 >>> (786964704) gen 33745 >>> key (1 DEV_EXTENT 12500540063744) block 16644219928576 >>> (1015882564) gen 112366 >>> key (1 DEV_EXTENT 12738910748672) block 14703563259904 >>> (897434281) gen 145699 >>> key (1 DEV_EXTENT 12976207691776) block 14113684733952 >>> (861430953) gen 137926 >>> key (1 DEV_EXTENT 13214578376704) block 15230715084800 >>> (929609075) gen 116469 >>> key (1 DEV_EXTENT 13452949061632) block 12787035504640 >>> (780458710) gen 119919 >>> key (1 DEV_EXTENT 13690246004736) block 10067511001088 >>> (614472107) gen 119958 >>> key (1 DEV_EXTENT 13928616689664) block 14113506328576 >>> (861420064) gen 106290 >>> key (1 DEV_EXTENT 14166987374592) block 861654532096 >>> (52591219) gen 140596 >>> key (1 DEV_EXTENT 14404284317696) block 6959531130880 >>> (424776070) gen 103496 >>> key (1 DEV_EXTENT 14643728744448) block 484408541184 >>> (29565951) gen 103596 >>> key (1 DEV_EXTENT 14882099429376) block 205519962112 >>> (12543943) gen 137977 >>> key (1 DEV_EXTENT 15118322630656) block 13289788358656 >>> (811144309) gen 145698 >>> key (1 DEV_EXTENT 15356693315584) block 13289789358080 >>> (811144370) gen 145698 >>> key (1 DEV_EXTENT 15596137742336) block 15736233984000 >>> (960463500) gen 117936 >>> key (1 DEV_EXTENT 15833434685440) block 1221260197888 >>> (74539807) gen 138299 >>> key (1 DEV_EXTENT 16071805370368) block 6368249397248 >>> (388687097) gen 124384 >>> key (1 DEV_EXTENT 16310176055296) block 12787123699712 >>> (780464093) gen 138137 >>> key (1 DEV_EXTENT 16548546740224) block 9420764561408 >>> (574997837) gen 124415 >>> key (1 DEV_EXTENT 16787991166976) block 22441589932032 >>> (1369725948) gen 124583 >>> key (1 DEV_EXTENT 17025288110080) block 15016789147648 >>> (916552072) gen 144864 >>> key (1 DEV_EXTENT 17263658795008) block 17459171082240 >>> (1065623235) gen 144951 >>> key (1 DEV_EXTENT 17502029479936) block 673143095296 >>> (41085394) gen 143317 >>> key (1 DEV_EXTENT 17740400164864) block 19010757427200 >>> (1160324550) gen 141153 >>> key (1 DEV_EXTENT 17979844591616) block 10067131596800 >>> (614448950) gen 141191 >>> key (1 DEV_EXTENT 18217141534720) block 205212958720 >>> (12525205) gen 124623 >>> key (1 DEV_EXTENT 18455512219648) block 205287997440 >>> (12529785) gen 124626 >>> key (1 DEV_EXTENT 18692809162752) block 16703827378176 >>> (1019520714) gen 124744 >>> key (1 DEV_EXTENT 18931179847680) block 7268874698752 >>> (443656903) gen 124899 >>> key (1 DEV_EXTENT 19169550532608) block 14291177324544 >>> (872264241) gen 141256 >>> key (1 DEV_EXTENT 19407921217536) block 29134446510080 >>> (1778225495) gen 142791 >>> key (1 DEV_EXTENT 19646291902464) block 6453644148736 >>> (393899179) gen 143236 >>> key (1 DEV_EXTENT 19883588845568) block 15998310219776 >>> (976459364) gen 143270 >>> leaf 32407552 items 223 free space 12 generation 145836 owner 4 >>> leaf 32407552 flags 0x1(WRITTEN) backref revision 1 >>> fs uuid 017b587a-3e88-4f08-8616-ec686f8f969f >>> chunk uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> item 0 key (0 PERSISTENT_ITEM 1) itemoff 16243 itemsize 40 >>> persistent item objectid 0 offset 1 >>> device stats >>> write_errs 0 read_errs 0 flush_errs 0 >>> corruption_errs 0 generation 0 >>> item 1 key (1 DEV_EXTENT 20971520) itemoff 16195 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 20971520 length 8388608 >>> chunk_tree_uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> item 2 key (1 DEV_EXTENT 29360128) itemoff 16147 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 20971520 length 8388608 >>> chunk_tree_uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> item 3 key (1 DEV_EXTENT 37748736) itemoff 16099 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 29360128 length >>> 1073741824 >>> chunk_tree_uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> item 4 key (1 DEV_EXTENT 1111490560) itemoff 16051 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 29360128 length >>> 1073741824 >>> chunk_tree_uuid 5820b5e3-9610-4cf3-a37a-b638a9963084 >>> item 5 key (1 DEV_EXTENT 2185232384) itemoff 16003 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 16135487488 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 6 key (1 DEV_EXTENT 3258974208) itemoff 15955 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 17209229312 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 7 key (1 DEV_EXTENT 4332716032) itemoff 15907 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 18282971136 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 8 key (1 DEV_EXTENT 5406457856) itemoff 15859 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 19356712960 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 9 key (1 DEV_EXTENT 6480199680) itemoff 15811 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 20430454784 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 10 key (1 DEV_EXTENT 7553941504) itemoff 15763 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 21504196608 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 11 key (1 DEV_EXTENT 8627683328) itemoff 15715 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 22577938432 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 12 key (1 DEV_EXTENT 9701425152) itemoff 15667 itemsize 48 >>> dev extent chunk_tree 3 >>> chunk_objectid 256 chunk_offset 23651680256 length >>> 1073741824 >>> chunk_tree_uuid 00000000-0000-0000-0000-000000000000 >>> item 13 key (1 DEV_EXTENT 10775166976) itemoff 15619 >>> itemsize 48 >>> >>> KVG: and the complete file is attached to this message >>> >>> >>> >>> >>> Normally it should not be mountable after v4.6 kernel if >>> super_total_bytes mismatch, but I'm more interested in how it mounted >>> successfully. >>> >>> KVG: I've no idea why it mounts successfully when the compressions >>> flags are omitted. No indication in the dmesg. >> I think it's device scan code, maybe I can check it later. >> >> Considering your superblock is good, you won't encounter such problem >> any longer. >> >> And if it really happens again, please try "btrfs device scan" first. >> >> Thanks, >> Qu >> >>> >>> And BTW, are you using x86_64 kernel or x86 kernel? >>> I don't think it's related in your case, but some reports about 32bit >>> kernel has strange bugs are in mail list, so just in case. >>> >>> KVG: both kernels are 64bit. with 4.10.37 it mounts without a >>> problem, with newer kernels the problem exis >>> >>> [ 0.000000] Linux version 4.10.0-37-generic >>> (buildd@lgw01-amd64-037) (gcc version 5.4.0 20160609 (Ubuntu >>> 5.4.0-6ubuntu1~16.04.4) ) #41~16.04.1-Ubuntu SMP Fri Oct 6 22:42:59 >>> UTC 2017 (Ubuntu 4.10.0-37.41~16.04.1-generic 4.10.17) >>> >>> [ 0.000000] Linux version 4.13.9-041309-generic (kernel@tangerine) >>> (gcc version 7.2.0 (Ubuntu 7.2.0-8ubuntu3)) #201710211231 SMP Sat Oct >>> 21 16:32:44 UTC 2017 >>> >>> >>> >>> Thanks, >>> Qu >>> >>> >>>> >>>> >>>> System information >>>> >>>> distribution: Ubuntu 16.04 >>>> btrfs-progs v4.8.1 later upgraded to v4.13.3 >>>> >>>> # btrfs fi usage /mnt/backup >>>> Overall: >>>> Device size: 29.11TiB >>>> Device allocated: 18.04TiB >>>> Device unallocated: 11.07TiB >>>> Device missing: 0.00B >>>> Used: 17.99TiB >>>> Free (estimated): 11.12TiB (min: 5.58TiB) >>>> Data ratio: 1.00 >>>> Metadata ratio: 2.00 >>>> Global reserve: 512.00MiB (used: 0.00B) >>>> >>>> Data,single: Size:17.93TiB, Used:17.88TiB >>>> /dev/sda 17.93TiB >>>> >>>> Metadata,DUP: Size:53.50GiB, Used:51.78GiB >>>> /dev/sda 107.00GiB >>>> >>>> System,DUP: Size:8.00MiB, Used:2.30MiB >>>> /dev/sda 16.00MiB >>>> >>>> Unallocated: >>>> /dev/sda 11.07TiB >>>> >>>> >>>> >>>> >>>> Yours sincerely, >>>> Konstantin V. Gavrilenko >>>> >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe >>>> linux-btrfs" in >>>> the body of a message to majord...@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >> >> >
signature.asc
Description: OpenPGP digital signature
