[PATCH 39/42] btrfs: make free space cache size consistent across different PAGE_SIZE

2021-04-14 Thread Qu Wenruo
Currently free space cache inode size is determined by two factors: - block group size - PAGE_SIZE This means, for the same sized block group, with different PAGE_SIZE, it will result different inode size. This will not be a good thing for subpage support, so change the requirement for PAGE_SIZE

[PATCH v2] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-03-29 Thread Su Yue
repair FST using btrfs check. Above case failed at the second readonly check step. Test log said "cache and super generation don't match, space cache will be invalidated" which is printed by validate_free_space_cache(). If cache_generation of the superblock is not -1ULL, validate_f

Re: [PATCH RFC] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-02-17 Thread Su Yue
test failed for case 037-freespacetree-repair The test tries to corrupt FST, call btrfs check readonly then repair FST using btrfs check. Above case failed at the second readonly check steup. Test log said "cache and super generation don't match, space cache will be invalidated" w

Re: [PATCH RFC] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-02-17 Thread Boris Burkov
pacetree-repair > > The test tries to corrupt FST, call btrfs check readonly then repair FST > using btrfs check. Above case failed at the second readonly check steup. > Test log said "cache and super generation don't match, space cache will > be invalidated" which is

[PATCH RFC] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-02-15 Thread Su Yue
repair FST using btrfs check. Above case failed at the second readonly check steup. Test log said "cache and super generation don't match, space cache will be invalidated" which is printed by validate_free_space_cache(). If cache_generation of the superblock is not -1ULL, validate_f

Re: Space cache

2021-02-04 Thread Filipe Manana
On Wed, Feb 3, 2021 at 10:15 PM Martin Raiber wrote: > > Hi, > > I've been looking a bit into the btrfs space cache and came to following > conclusions. Please correct me if I'm wrong: > > 1. The space cache mount option only modifies how the space cache is &g

Space cache

2021-02-03 Thread Martin Raiber
Hi, I've been looking a bit into the btrfs space cache and came to following conclusions. Please correct me if I'm wrong: 1. The space cache mount option only modifies how the space cache is persisted and not the in-memory structures (hence why I have 2,3 GiB btrfs_free_space_bitmap

Re: [PATCH v3] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-25 Thread David Sterba
On Fri, Jan 22, 2021 at 07:07:45PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > After a sudden power failure we may end up with a space cache on disk that > is not valid and needs to be rebuilt from scratch. > > If that happens, during log replay when we attemp

Re: [PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Filipe Manana
On Fri, Jan 22, 2021 at 6:39 PM Josef Bacik wrote: > > On 1/22/21 12:56 PM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > After a sudden power failure we may end up with a space cache on disk that > > is not valid and needs to be rebuilt from scratch. &

Re: [PATCH v3] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 2:07 PM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at

Re: [PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 12:56 PM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at

[PATCH v3] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for

[PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for

Re: [PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Filipe Manana
On Fri, Jan 22, 2021 at 4:43 PM Josef Bacik wrote: > > On 1/22/21 10:28 AM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > After a sudden power failure we may end up with a space cache on disk that > > is not valid and needs to be rebuilt from scratch. &

Re: [PATCH 2/2] btrfs: fix log replay failure when space cache needs to be rebuilt

2021-01-22 Thread Josef Bacik
On 1/22/21 10:28 AM, fdman...@kernel.org wrote: From: Filipe Manana During log replay we first start by walking the log trees and pin the ranges for their extent buffers, through calls to the function btrfs_pin_extent_for_log_replay(). However if the space cache for a block group is invalid

Re: [PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 10:28 AM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at

[PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for

[PATCH 2/2] btrfs: fix log replay failure when space cache needs to be rebuilt

2021-01-22 Thread fdmanana
From: Filipe Manana During log replay we first start by walking the log trees and pin the ranges for their extent buffers, through calls to the function btrfs_pin_extent_for_log_replay(). However if the space cache for a block group is invalid and needs to be rebuilt, we can fail the log replay

Re: [PATCH] btrfs-progs: check: allow force v1 space cache cleanup even the fs has v2 space cache enabled

2021-01-14 Thread David Sterba
On Tue, Dec 29, 2020 at 09:34:51AM +, Stéphane Lesimple wrote: > December 29, 2020 1:32 AM, "Qu Wenruo" wrote: > > > There are cases where v1 free space cache is still left while user has > > already enabled v2 cache. > > > > In that case, we still

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-30 Thread Stéphane Lesimple
December 30, 2020 6:52 AM, "Qu Wenruo" wrote: > BTW, would you mind to dump the root tree? > > # btrfs ins dump-tree -t root > > I'm more interested in the half dropped v1 space cache. Currently > although they are sane (with proper inode items), I'm a

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
(2019-09-08 03:07:02) mtime 0.0 (1970-01-01 01:00:00) otime 0.0 (1970-01-01 01:00:00) item 27 key (51933 EXTENT_DATA 0) itemoff 9854 itemsize 53 generation 1517381 type 2 (prealloc) prealloc data disk byte 34626327621632 nr 262144 Got the point now. The type is preallocated, which means we haven

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
On 2020/12/29 上午8:38, Qu Wenruo wrote: In delete_v1_space_cache(), if we find a leaf whose owner is tree root, and we can't grab the free space cache inode, then we return -ENOENT. However this would make the caller, add_data_references(), to consider this as a critical error, and

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Stéphane Lesimple
ctime 1567904822.739884119 (2019-09-08 03:07:02) >> mtime 0.0 (1970-01-01 01:00:00) >> otime 0.0 (1970-01-01 01:00:00) >> item 27 key (51933 EXTENT_DATA 0) itemoff 9854 itemsize 53 >> generation 1517381 type 2 (prealloc) >> prealloc data disk byte 3462632762163

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
33 EXTENT_DATA 0) itemoff 9854 itemsize 53 generation 1517381 type 2 (prealloc) prealloc data disk byte 34626327621632 nr 262144 Got the point now. The type is preallocated, which means we haven't yet written space cache into it. But the code only checks the regular file extent (written, no

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Stéphane Lesimple
0600 links 1 uid 0 gid 0 rdev 0 >> sequence 116552 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC) >> atime 0.0 (1970-01-01 01:00:00) >> ctime 1567904822.739884119 (2019-09-08 03:07:02) >> mtime 0.0 (1970-01-01 01:00:00) >> otime 0.0 (1970-01-01 01:00:00) >&g

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
6327621632 nr 262144 Got the point now. The type is preallocated, which means we haven't yet written space cache into it. But the code only checks the regular file extent (written, not preallocated). So the proper fix would looks like this: diff --git a/fs/btrfs/relocation.c b/fs

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Stéphane Lesimple
root data bytenr 42678619897856 refs 1 item 33 key (52268 144 5) itemoff 8471 itemsize 21 item 34 key (52453 132 0) itemoff 8032 itemsize 439 root data bytenr 42705323196416 refs 1 item 35 key (52453 144 5) itemoff 8011 itemsize 21 item 36 key (58841 132 0) itemoff 7572 itemsize 439 root data bytenr 42677453111296 refs 1 item 37 key (58841 144 5) itemoff 7545 itemsize 27 BTRFS warning (device dm-10): leftover v1 space cache found, please use btrfs-check --clear-space-cache v1 to clean it up BTRFS info (device dm-10): balance: ended with status: -2 Regards, Stéphane.

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
oup_cache(leaf->fs_info, block_group, NULL, space_cache_ino); return ret; Thanks, Qu [ 451.026353] BTRFS warning (device dm-10): leftover v1 space cache found, please use btrfs-check --clear-space-cache v1 to clean it up [ 463.501781] BTRFS info (de

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Stéphane Lesimple
6327621632 [ 451.026353] BTRFS warning (device dm-10): leftover v1 space cache found, please use btrfs-check --clear-space-cache v1 to clean it up [ 463.501781] BTRFS info (device dm-10): balance: ended with status: -2 Regards, Stéphane.

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Qu Wenruo
On 2020/12/29 下午5:27, Stéphane Lesimple wrote: December 29, 2020 1:38 AM, "Qu Wenruo" wrote: In delete_v1_space_cache(), if we find a leaf whose owner is tree root, and we can't grab the free space cache inode, then we return -ENOENT. However this would

Re: [PATCH] btrfs-progs: check: allow force v1 space cache cleanup even the fs has v2 space cache enabled

2020-12-29 Thread Stéphane Lesimple
December 29, 2020 1:32 AM, "Qu Wenruo" wrote: > There are cases where v1 free space cache is still left while user has > already enabled v2 cache. > > In that case, we still want to force v1 space cache cleanup in > btrfs-check. > > This patch will remove the v

Re: [PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-29 Thread Stéphane Lesimple
December 29, 2020 1:38 AM, "Qu Wenruo" wrote: > In delete_v1_space_cache(), if we find a leaf whose owner is tree root, > and we can't grab the free space cache inode, then we return -ENOENT. > > However this would make the caller, add_data_references(), to consider

[PATCH] btrfs: relocation: output warning message for leftover v1 space cache before aborting current data balance

2020-12-28 Thread Qu Wenruo
In delete_v1_space_cache(), if we find a leaf whose owner is tree root, and we can't grab the free space cache inode, then we return -ENOENT. However this would make the caller, add_data_references(), to consider this as a critical error, and abort current data balance. This happens for fs

[PATCH] btrfs-progs: check: allow force v1 space cache cleanup even the fs has v2 space cache enabled

2020-12-28 Thread Qu Wenruo
There are cases where v1 free space cache is still left while user has already enabled v2 cache. In that case, we still want to force v1 space cache cleanup in btrfs-check. This patch will remove the v2 check if we're cleaning up v1 cache, allowing us to cleanup the leftover. Signed-off-b

Re: [PATCH v2] btrfs: free-space-cache: Fix error return code in __load_free_space_cache

2020-12-15 Thread David Sterba
On Mon, Dec 07, 2020 at 09:56:12PM +0800, Zhihao Cheng wrote: > Fix to return the error code(instead always 0) when memory allocating > failed in __load_free_space_cache(). > > This lacks the analysis of consequences, so there's only one caller and By "This lacks the analysis of consequences" I w

Re: [PATCH 1/2] btrfs: check page->mapping when loading free space cache

2019-10-15 Thread David Sterba
et_from_fork+0x1f/0x30 > > The page we were trying to drop had a page->private, but had no > page->mapping and so called drop_buffers, assuming that we had a > buffer_head on the page, and then panic'ed trying to deref 1, which is > our page->private for data pages. >

Re: [PATCH 1/2] btrfs: check page->mapping when loading free space cache

2019-09-27 Thread Nikolay Borisov
f/0x30 > > The page we were trying to drop had a page->private, but had no > page->mapping and so called drop_buffers, assuming that we had a > buffer_head on the page, and then panic'ed trying to deref 1, which is > our page->private for data pages. > > This is

Re: [PATCH 1/2] btrfs: check page->mapping when loading free space cache

2019-09-27 Thread Filipe Manana
_fork+0x1f/0x30 > > The page we were trying to drop had a page->private, but had no > page->mapping and so called drop_buffers, assuming that we had a > buffer_head on the page, and then panic'ed trying to deref 1, which is > our page->private for data pages. > >

[PATCH 1/2] btrfs: check page->mapping when loading free space cache

2019-09-24 Thread Josef Bacik
had a buffer_head on the page, and then panic'ed trying to deref 1, which is our page->private for data pages. This is happening because we're truncating the free space cache while we're trying to load the free space cache. This isn't supposed to happen, and I'll fix tha

Re: [PATCH] btrfs/048: fix test failure when fs mounted with v2 space cache option

2019-09-05 Thread Nikolay Borisov
cept when using the > v2 space cache mount option (MOUNT_OPTIONS="-o space_cache=v2"), since > the filesystem generation is 8 because creating a v2 space cache adds > an additional transaction commit. So update the test to not hardcode > specific generation numbers in its golden o

[PATCH] btrfs/048: fix test failure when fs mounted with v2 space cache option

2019-09-05 Thread fdmanana
From: Filipe Manana In order to check that the filesystem generation does not change after failure to set a property, the test expects a specific generation number of 7 in its golden output. That currently works except when using the v2 space cache mount option (MOUNT_OPTIONS="-o space_cac

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-08-11 Thread Chris Murphy
Is > > the lack of COW repairs with btrfs check solved? Can this file system > > be fixed? Maybe --init-extent-tree is worth a shot? > > > > > Latest btrfs check has solved the problem of bad CoW. In fact it's > solved in btrfs-progs v5.1 release. > > So,

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-08-11 Thread Qu Wenruo
Latest btrfs check has solved the problem of bad CoW. In fact it's solved in btrfs-progs v5.1 release. So, if you run btrfs check --clear-space-cache v1 again, it shouldn't cause transid mismatch problem any more. Although IIRC that fs already got transid error, thus enhanced

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-08-11 Thread Chris Murphy
On Sun, Mar 10, 2019 at 5:20 PM Chris Murphy wrote: > > On Sat, Mar 2, 2019 at 11:18 AM Chris Murphy wrote: > > > > Sending URL for dump-tree output offlist. Conversation should still be > > on-list. > > > Any more information required from me at this point? This file system has been on a shelf

[PATCH v4 03/13] btrfs: use btrfs_crc32c{,_final}() in for free space cache

2019-06-03 Thread Johannes Thumshirn
The CRC checksum in the free space cache is not dependant on the super block's csum_type field but always a CRC32C. So use btrfs_crc32c() and btrfs_crc32c_final() instead of btrfs_csum_data() and btrfs_csum_final() for computing these checksums. Signed-off-by: Johannes Thumshirn Review

[PATCH v3 03/13] btrfs: use btrfs_crc32c{,_final}() in for free space cache

2019-05-22 Thread Johannes Thumshirn
The CRC checksum in the free space cache is not dependant on the super block's csum_type field but always a CRC32C. So use btrfs_crc32c() and btrfs_crc32c_final() instead of btrfs_csum_data() and btrfs_csum_final() for computing these checksums. Signed-off-by: Johannes Thumshirn Review

[PATCH v2 03/13] btrfs: use btrfs_crc32c{,_final}() in for free space cache

2019-05-16 Thread Johannes Thumshirn
The CRC checksum in the free space cache is not dependant on the super block's csum_type field but always a CRC32C. So use btrfs_crc32c() and btrfs_crc32c_final() instead of btrfs_csum_data() and btrfs_csum_final() for computing these checksums. Signed-off-by: Johannes Thumshirn Review

Re: [PATCH 07/17] btrfs: use btrfs_crc32c{,_final}() in for free space cache

2019-05-10 Thread Nikolay Borisov
On 10.05.19 г. 14:15 ч., Johannes Thumshirn wrote: > The CRC checksum in the free space cache is not dependant on the super > block's csum_type field but always a CRC32C. > > So use btrfs_crc32c() and btrfs_crc32c_final() instead of btrfs_csum_data() > and btrfs_csum_f

[PATCH 07/17] btrfs: use btrfs_crc32c{,_final}() in for free space cache

2019-05-10 Thread Johannes Thumshirn
The CRC checksum in the free space cache is not dependant on the super block's csum_type field but always a CRC32C. So use btrfs_crc32c() and btrfs_crc32c_final() instead of btrfs_csum_data() and btrfs_csum_final() for computing these checksums. Signed-off-by: Johannes Thumshirn --- fs/

[PATCH v2 6/7] btrfs: check/original: Check and repair free space cache inode item

2019-03-31 Thread Qu Wenruo
Just like lowmem mode, also check and repair free space cache inode item. And since we don't really have a good timing/function to check free space chace inodes, we use the same common mode check_repair_free_space_inode() when iterating root tree. Signed-off-by: Qu Wenruo --- check/main.c

[PATCH v2 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-31 Thread Qu Wenruo
00:00) ctime 1553491158.189771625 (2019-03-25 13:19:18) mtime 0.0 (1970-01-01 08:00:00) otime 0.0 (1970-01-01 08:00:00) There is a report of such problem in the mail list. This patch will check and repair inode items of free space cache inodes in lowmem

[PATCH v2 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

2019-03-31 Thread Qu Wenruo
The image has one free space cache inode with invalid mode (0). item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 generation 30 transid 30 size 65536 nbytes 1507328 block group 0 mode 0 links 1 uid 0 gid 0 rdev 0 sequence 23 flags 0x1b

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

2019-03-31 Thread Qu Wenruo
On 2019/3/29 下午8:05, Nikolay Borisov wrote: > > > On 25.03.19 г. 10:22 ч., Qu Wenruo wrote: >> The image has one free space cache inode with invalid mode (0). >> item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 >> generation 30 t

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

2019-03-29 Thread Qu Wenruo
On 2019/3/29 下午8:05, Nikolay Borisov wrote: > > > On 25.03.19 г. 10:22 ч., Qu Wenruo wrote: >> The image has one free space cache inode with invalid mode (0). >> item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 >> generation 30 t

Re: [PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

2019-03-29 Thread Nikolay Borisov
On 25.03.19 г. 10:22 ч., Qu Wenruo wrote: > The image has one free space cache inode with invalid mode (0). > item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 > generation 30 transid 30 size 65536 nbytes 1507328 > block group 0 mode 0

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-29 Thread Nikolay Borisov
On 29.03.19 г. 13:02 ч., Qu Wenruo wrote: > [snip] >>> +/* >>> + * For free space inodes, we can't call check_inode_item() as free space >>> + * cache inode doesn't have INODE_REF. >>> + * We just check its inode mode. >>> + */ &

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-29 Thread Qu Wenruo
[snip] >> +/* >> + * For free space inodes, we can't call check_inode_item() as free space >> + * cache inode doesn't have INODE_REF. >> + * We just check its inode mode. >> + */ >> +int check_repair_free_space_inode(struct btrfs_fs_info *fs_

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-29 Thread Nikolay Borisov
report of such problem in the mail list. > > This patch will check and repair inode items of free space cache inodes in > lowmem mode. > > Since free space cache inodes doesn't have INODE_REF but still has 1 > link, we can't use check_inode_item() directly. > Instea

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-25 Thread Qu Wenruo
(1970-01-01 08:00:00) >> otime 0.0 (1970-01-01 08:00:00) >> >> There is a report of such problem in the mail list. >> >> This patch will check and repair inode items of free space cache >> inodes in >> lowmem mode. >> >> Since f

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-25 Thread Su Yue
inode items of free space cache inodes in lowmem mode. Since free space cache inodes doesn't have INODE_REF but still has 1 link, we can't use check_inode_item() directly. Instead we only check the inode mode, as that's the important part. The check and repair function: check_repair_f

[PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-25 Thread Qu Wenruo
00:00) ctime 1553491158.189771625 (2019-03-25 13:19:18) mtime 0.0 (1970-01-01 08:00:00) otime 0.0 (1970-01-01 08:00:00) There is a report of such problem in the mail list. This patch will check and repair inode items of free space cache inodes in lowmem

[PATCH 6/7] btrfs: check/original: Check and repair free space cache inode item

2019-03-25 Thread Qu Wenruo
Just like lowmem mode, also check and repair free space cache inode item. And since we don't really have a good timing/function to check free space chace inodes, we use the same lowmem mode check_repair_free_space_inode() when iterating root tree. Signed-off-by: Qu Wenruo --- check/main.c

[PATCH 7/7] btrfs: tests/fsck: Add test image for free space cache mode repair

2019-03-25 Thread Qu Wenruo
The image has one free space cache inode with invalid mode (0). item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160 generation 30 transid 30 size 65536 nbytes 1507328 block group 0 mode 0 links 1 uid 0 gid 0 rdev 0 sequence 23 flags 0x1b

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-10 Thread Chris Murphy
On Sat, Mar 2, 2019 at 11:18 AM Chris Murphy wrote: > > Sending URL for dump-tree output offlist. Conversation should still be > on-list. Any more information required from me at this point? -- Chris Murphy

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Qu Wenruo
717 >>> >>> That bug report includes the trace in user space; but I've also >>> processed the resulting coredump file with gdb and attached that to >>> the bug report. >>> >>> Before using --clear-space-cache v1, btrfs scrub and btrfs ch

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
Two more strange things: # btrfs check -r 930086912 /dev/mapper/sdd Opening filesystem to check... parent transid verify failed on 930086912 wanted 4514 found 4076 parent transid verify failed on 930086912 wanted 4514 found 4076 parent transid verify failed on 930086912 wanted 4514 found 4076 pare

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
OK this is screwy. The super has a totally different generation for the root tree than any of the backup roots. $ sudo btrfs rescue super -v /dev/mapper/sdd All Devices: Device: id = 1, name = /dev/mapper/sdc Device: id = 2, name = /dev/mapper/sdd Before Recovering: [All good supers]:

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
but I've also > > processed the resulting coredump file with gdb and attached that to > > the bug report. > > > > Before using --clear-space-cache v1, btrfs scrub and btrfs check came > > up clean no errors. Following the crash, I compiled brfsprogs 4.20.2 &g

Re: [bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Qu Wenruo
hat to > the bug report. > > Before using --clear-space-cache v1, btrfs scrub and btrfs check came > up clean no errors. Following the crash, I compiled brfsprogs 4.20.2 > and ran 'btrfs check' which finds corruption. > > I've taken no further action, and

[bug] btrfs check clear-space-cache v1 corrupted file system

2019-03-01 Thread Chris Murphy
Problem happens with btrfsprogs 4.19.1 https://bugzilla.kernel.org/show_bug.cgi?id=202717 That bug report includes the trace in user space; but I've also processed the resulting coredump file with gdb and attached that to the bug report. Before using --clear-space-cache v1, btrfs scru

[PATCH 10/42] btrfs: protect space cache inode alloc with nofs

2018-10-12 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik R

[PATCH 10/42] btrfs: protect space cache inode alloc with nofs

2018-10-11 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik R

Re: [PATCH 10/42] btrfs: protect space cache inode alloc with nofs

2018-10-01 Thread David Sterba
On Fri, Sep 28, 2018 at 07:17:49AM -0400, Josef Bacik wrote: > If we're allocating a new space cache inode it's likely going to be > under a transaction handle, so we need to use memalloc_nofs_save() in > order to avoid deadlocks, and more importantly lockdep messages that &

[PATCH 10/42] btrfs: protect space cache inode alloc with nofs

2018-09-28 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik ---

Re: [PATCH] btrfs-progs: Continue checking even we found something wrong in free space cache

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 07:43:27AM +0800, Qu Wenruo wrote: > > > On 2018/9/11 下午10:48, David Sterba wrote: > > On Tue, Sep 04, 2018 at 08:41:28PM +0800, Qu Wenruo wrote: > >> No need to abort checking, especially for RO check free space cache is > >> meaningless

Re: [PATCH] btrfs-progs: Continue checking even we found something wrong in free space cache

2018-09-11 Thread Qu Wenruo
On 2018/9/11 下午10:48, David Sterba wrote: > On Tue, Sep 04, 2018 at 08:41:28PM +0800, Qu Wenruo wrote: >> No need to abort checking, especially for RO check free space cache is >> meaningless, the errors in fs/extent tree is more interesting for >> developers. >> &

[PATCH 10/36] btrfs: protect space cache inode alloc with nofs

2018-09-11 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Reviewed-by: Omar Sandoval Signed-off-by: Josef Bacik ---

Re: [PATCH] btrfs-progs: Continue checking even we found something wrong in free space cache

2018-09-11 Thread David Sterba
On Tue, Sep 04, 2018 at 08:41:28PM +0800, Qu Wenruo wrote: > No need to abort checking, especially for RO check free space cache is > meaningless, the errors in fs/extent tree is more interesting for > developers. > > So continue checking even something in free space cache is wrong

[PATCH] btrfs-progs: Continue checking even we found something wrong in free space cache

2018-09-04 Thread Qu Wenruo
No need to abort checking, especially for RO check free space cache is meaningless, the errors in fs/extent tree is more interesting for developers. So continue checking even something in free space cache is wrong. Reported-by: Etienne Champetier Signed-off-by: Qu Wenruo --- check/main.c | 1

Re: [PATCH 09/35] btrfs: protect space cache inode alloc with nofs

2018-08-31 Thread Omar Sandoval
On Thu, Aug 30, 2018 at 01:41:59PM -0400, Josef Bacik wrote: > If we're allocating a new space cache inode it's likely going to be > under a transaction handle, so we need to use memalloc_nofs_save() in > order to avoid deadlocks, and more importantly lockdep messages that &

[PATCH 09/35] btrfs: protect space cache inode alloc with nofs

2018-08-30 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use memalloc_nofs_save() in order to avoid deadlocks, and more importantly lockdep messages that make xfstests fail. Signed-off-by: Josef Bacik --- fs/btrfs/free-space-c

Re: [PATCH 10/22] btrfs: alloc space cache inode with GFP_NOFS

2018-07-19 Thread Josef Bacik
On Thu, Jul 19, 2018 at 05:44:53PM +0200, David Sterba wrote: > On Thu, Jul 19, 2018 at 06:35:33PM +0300, Nikolay Borisov wrote: > > > > > > On 19.07.2018 17:49, Josef Bacik wrote: > > > If we're allocating a new space cache inode it's likely going to be

Re: [PATCH 10/22] btrfs: alloc space cache inode with GFP_NOFS

2018-07-19 Thread Josef Bacik
On Thu, Jul 19, 2018 at 06:35:33PM +0300, Nikolay Borisov wrote: > > > On 19.07.2018 17:49, Josef Bacik wrote: > > If we're allocating a new space cache inode it's likely going to be > > under a transaction handle, so we need to use GFP_NOFS to keep from > >

Re: [PATCH 10/22] btrfs: alloc space cache inode with GFP_NOFS

2018-07-19 Thread David Sterba
On Thu, Jul 19, 2018 at 06:35:33PM +0300, Nikolay Borisov wrote: > > > On 19.07.2018 17:49, Josef Bacik wrote: > > If we're allocating a new space cache inode it's likely going to be > > under a transaction handle, so we need to use GFP_NOFS to keep from > >

Re: [PATCH 10/22] btrfs: alloc space cache inode with GFP_NOFS

2018-07-19 Thread Nikolay Borisov
On 19.07.2018 17:49, Josef Bacik wrote: > If we're allocating a new space cache inode it's likely going to be > under a transaction handle, so we need to use GFP_NOFS to keep from > deadlocking. Otherwise GFP_KERNEL is fine. > > Signed-off-by: Josef Bacik > --- &g

[PATCH 10/22] btrfs: alloc space cache inode with GFP_NOFS

2018-07-19 Thread Josef Bacik
If we're allocating a new space cache inode it's likely going to be under a transaction handle, so we need to use GFP_NOFS to keep from deadlocking. Otherwise GFP_KERNEL is fine. Signed-off-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 5 + fs/btrfs/inode.c| 3 ++-

Re: [PATCH] btrfs-progs: free-space-cache: Don't panic when free space cache is corrupted

2018-07-02 Thread David Sterba
On Sun, Jul 01, 2018 at 10:45:26AM +0800, Qu Wenruo wrote: > In btrfs_add_free_space(), if the free space to be added is already > here, we trigger ASSERT() which is just another BUG_ON(). > > Let's remove such BUG_ON() at all. > > Reported-by: Lewis Diamond > Signed-off-by: Qu Wenruo Applied,

Re: [PATCH] btrfs-progs: free-space-cache: Don't panic when free space cache is corrupted

2018-07-02 Thread Nikolay Borisov
On 1.07.2018 05:45, Qu Wenruo wrote: > In btrfs_add_free_space(), if the free space to be added is already > here, we trigger ASSERT() which is just another BUG_ON(). > > Let's remove such BUG_ON() at all. > > Reported-by: Lewis Diamond > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Boris

[PATCH] btrfs-progs: free-space-cache: Don't panic when free space cache is corrupted

2018-06-30 Thread Qu Wenruo
In btrfs_add_free_space(), if the free space to be added is already here, we trigger ASSERT() which is just another BUG_ON(). Let's remove such BUG_ON() at all. Reported-by: Lewis Diamond Signed-off-by: Qu Wenruo --- free-space-cache.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)

[PATCH 1/5] libbtrfsutil: don't return free space cache inodes from deleted_subvolumes()

2018-03-29 Thread Omar Sandoval
From: Omar Sandoval Deleted free space cache inodes also get an orphan item in the root tree, but we shouldn't report those as deleted subvolumes. Deleted subvolumes will still have the root item, so we can just do an extra tree search. Reported-by: Tomohiro Misono Signed-off-by: Omar San

Re: [PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-09 Thread David Sterba
On Thu, Mar 08, 2018 at 03:02:31PM +0800, Qu Wenruo wrote: > When we found free space difference between free space cache and block > group item, we just discard this free space cache. > > Normally such difference is caused by btrfs_reserve_extent() called by > delalloc wh

Re: [PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-09 Thread David Sterba
On Fri, Mar 09, 2018 at 10:06:02AM +0200, Nikolay Borisov wrote: > >>> + * space cache has less free space, and both kernel just discard > >>> + * such cache. But if we find some case where free space cache > >>> + * has more free

Re: [PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-09 Thread Nikolay Borisov
On 9.03.2018 01:27, Qu Wenruo wrote: > > > On 2018年03月08日 22:05, Nikolay Borisov wrote: >> >> >> On 8.03.2018 09:02, Qu Wenruo wrote: >>> When we found free space difference between free space cache and block >>> group item, we just discard

Re: [PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-08 Thread Qu Wenruo
On 2018年03月08日 22:05, Nikolay Borisov wrote: > > > On 8.03.2018 09:02, Qu Wenruo wrote: >> When we found free space difference between free space cache and block >> group item, we just discard this free space cache. >> >> Normally such difference is caused by

Re: [PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-08 Thread Nikolay Borisov
On 8.03.2018 09:02, Qu Wenruo wrote: > When we found free space difference between free space cache and block > group item, we just discard this free space cache. > > Normally such difference is caused by btrfs_reserve_extent() called by > delalloc which is out of a transacti

[PATCH v2] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-07 Thread Qu Wenruo
When we found free space difference between free space cache and block group item, we just discard this free space cache. Normally such difference is caused by btrfs_reserve_extent() called by delalloc which is out of a transaction. And since all btrfs_release_extent() is called with a

[PATCH] btrfs-progs: free-space-cache: Enhance free space cache free space check

2018-03-07 Thread Qu Wenruo
When we found free space difference between free space cache and block group item, we just discard this free space cache. Normally such difference is caused by btrfs_reserve_extent() called by delalloc which is out of a transaction. And since all btrfs_release_extent() is called with a

Re: [PATCH 3/3] fstests: btrfs: Add test case to check v1 space cache corruption

2018-03-06 Thread Filipe Manana
ince ever) many reports of different >> types of corruptions. >> Which kind of corruption are you referring to? >> >>> although such corruption is pretty rare and almost impossible to >>> reproduce, with dm-log-writes we found it's highly related to v1 space

Re: [PATCH 3/3] fstests: btrfs: Add test case to check v1 space cache corruption

2018-03-06 Thread Filipe Manana
problems. > >> >>> although such corruption is pretty rare and almost impossible to >>> reproduce, with dm-log-writes we found it's highly related to v1 space >>> cache. >>> >>> Unlike journal based filesystems, btrfs completely rely on

Re: [PATCH 3/3] fstests: btrfs: Add test case to check v1 space cache corruption

2018-03-06 Thread Qu Wenruo
ind of corruption are you referring to? > >> although such corruption is pretty rare and almost impossible to >> reproduce, with dm-log-writes we found it's highly related to v1 space >> cache. >> >> Unlike journal based filesystems, btrfs completely rely on

  1   2   3   4   >