[PATCH 0/3] btrfs: tree-checker: Enhancement and fixes for new fuzzed image report

2019-03-07 Thread Qu Wenruo
This patchset can be fetched from: https://github.com/adam900710/linux/tree/tree_checker_enhancement The base branch write_time_tree_checker, the base commit is: commit cb3ba69132a83d5e87f243dece5db09a022b0be7 (github/write_time_tree_checker, write_time_tree_checker) Author: Qu Wenruo Date: Th

[PATCH 1/3] btrfs: tree-checker: Verify chunk items

2019-03-07 Thread Qu Wenruo
We already have btrfs_check_chunk_valid() to verify each chunk before tree-checker. Merge that function into tree-checker, and update its error message to be more readable. Old error message would be something like: BTRFS error (device dm-3): invalid chunk num_stipres: 0 New error message woul

[PATCH 3/3] btrfs: tree-checker: Fix NULL pointer access for corrupted chunk root

2019-03-07 Thread Qu Wenruo
[BUG] For a special crafted image, kernel can cause NULL pointer dereference like: BUG: unable to handle kernel NULL pointer dereference at 0024 #PF error: [normal kernel read fault] Oops: [#1] SMP PTI CPU: 0 PID: 146 Comm: kworker/u2:4 Not tainted 5.0.0-rc8+ #9 Workqueue

[PATCH 2/3] btrfs: tree-checker: Verify dev item

2019-03-07 Thread Qu Wenruo
[BUG] For fuzzed image whose DEV_ITEM has invalid total_bytes as 0, then kernel will just panic: BUG: unable to handle kernel NULL pointer dereference at 0098 #PF error: [normal kernel read fault] PGD 80022b2bd067 P4D 80022b2bd067 PUD 22b2bc067 PMD 0 Oops: [#1] SMP

Re: [PATCH] generic: add test for fsync after shrinking truncate and rename

2019-03-07 Thread Dave Chinner
On Thu, Mar 07, 2019 at 05:19:51PM -0600, Jayashree Mohan wrote: > Hi Amir, > > > I went back to look at similar fsync tests by Filipe: > > generic/{106,107,335,336,341,342,343,348,498,501,502,509,510,512} > > > > I found some alleged subtle mistakes about SOMC assumptions. > > > > generic/336 doe

Re: [PATCH] generic: add test for fsync after shrinking truncate and rename

2019-03-07 Thread Dave Chinner
On Thu, Mar 07, 2019 at 09:52:03AM +0200, Amir Goldstein wrote: > On Wed, Mar 6, 2019 at 11:48 PM Dave Chinner wrote: > > > > On Wed, Mar 06, 2019 at 09:51:23AM +0200, Amir Goldstein wrote: > > > On Wed, Mar 6, 2019 at 12:33 AM Dave Chinner wrote: > > > > > So the reason this is working is becaus

Re: [PATCH v2 0/3] btrfs: trace: Add trace events for extent_io_tree

2019-03-07 Thread Qu Wenruo
On 2019/3/8 上午12:32, David Sterba wrote: > On Fri, Mar 01, 2019 at 10:47:57AM +0800, Qu Wenruo wrote: >> - Allow NULL fs_info for TP_fast_assign_fsid() >> There is extent bits operation in selftest which is too deep to pass >> fs_info. And since it's in selftest, it shouldn't trigger trace >>

Re: [PATCH] generic: add test for fsync after shrinking truncate and rename

2019-03-07 Thread Jayashree Mohan
Hi Amir, > I went back to look at similar fsync tests by Filipe: > generic/{106,107,335,336,341,342,343,348,498,501,502,509,510,512} > > I found some alleged subtle mistakes about SOMC assumptions. > > generic/336 does: > touch $SCRATCH_MNT/a/foo > ln $SCRATCH_MNT/a/foo $SCRATCH_MNT/b/foo_link > t

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-03-07 Thread Zygo Blaxell
On Mon, Mar 04, 2019 at 04:34:39PM +0100, Christoph Anton Mitterer wrote: > Hey. > > > Thanks for your elaborate explanations :-) > > > On Fri, 2019-02-15 at 00:40 -0500, Zygo Blaxell wrote: > > The problem occurs only on reads. Data that is written to disk will > > be OK, and can be read corr

Re: [PATCH] btrfs: Turn an 'else if' into an 'else' in btrfs_uuid_tree_add

2019-03-07 Thread Nick Desaulniers
On Thu, Mar 7, 2019 at 9:54 AM David Sterba wrote: > > On Thu, Mar 07, 2019 at 09:35:15AM -0700, Nathan Chancellor wrote: > > When building with -Wsometimes-uninitialized, Clang warns: > > > > fs/btrfs/uuid-tree.c:129:13: warning: variable 'eb' is used uninitialized > > whenever 'if' condition is

Re: [PATCH] btrfs: Turn an 'else if' into an 'else' in btrfs_uuid_tree_add

2019-03-07 Thread David Sterba
On Thu, Mar 07, 2019 at 09:35:15AM -0700, Nathan Chancellor wrote: > When building with -Wsometimes-uninitialized, Clang warns: > > fs/btrfs/uuid-tree.c:129:13: warning: variable 'eb' is used uninitialized > whenever 'if' condition is false [-Wsometimes-uninitialized] > fs/btrfs/uuid-tree.c:129:1

Re: [GIT PULL] Btrfs updates for 5.1, part 1

2019-03-07 Thread pr-tracker-bot
The pull request you sent on Mon, 4 Mar 2019 20:20:53 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git > for-5.1-part1-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/b1e243957e9b3ba8e820fb8583bdf18e7c737aa2 Thank you! -- Deet-doot-dot, I

Re: [PATCH] btrfs: reduce kmap_atomic time for checksumming

2019-03-07 Thread David Sterba
On Thu, Mar 07, 2019 at 05:14:00PM +0100, Johannes Thumshirn wrote: > Since commit c40a3d38aff4 ("Btrfs: Compute and look up csums based on > sectorsized blocks") we do a kmap_atomic() on the contents of a bvec. > > kmap_atomic() in turn does a preempt_disable() and pagefault_disable(), > so we sh

Re: [PATCH v2 3/3] btrfs: trace: Add trace events for extent_io_tree

2019-03-07 Thread David Sterba
On Fri, Mar 01, 2019 at 10:48:00AM +0800, Qu Wenruo wrote: > +#define show_extent_io_tree_owner(owner) \ > + __print_symbolic(owner, \ > + { IO_TREE_FS_INFO_FREED_EXTENTS0, "FREED_EXTENTS0" }, \

Re: [PATCH v2 2/3] btrfs: Introduce extent_io_tree::owner to distinguish different io_trees

2019-03-07 Thread David Sterba
On Fri, Mar 01, 2019 at 10:47:59AM +0800, Qu Wenruo wrote: > +enum { > + IO_TREE_FS_INFO_FREED_EXTENTS0, > + IO_TREE_FS_INFO_FREED_EXTENTS1, > + IO_TREE_INODE_IO_TREE, > + IO_TREE_INODE_IO_FAILURE_TREE, > + IO_TREE_RELOC_BLOCKS, > + IO_TREE_TRANSACTION_DIRTY_PAGES, > + I

[PATCH] btrfs: Turn an 'else if' into an 'else' in btrfs_uuid_tree_add

2019-03-07 Thread Nathan Chancellor
When building with -Wsometimes-uninitialized, Clang warns: fs/btrfs/uuid-tree.c:129:13: warning: variable 'eb' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] fs/btrfs/uuid-tree.c:129:13: warning: variable 'offset' is used uninitialized whenever 'if' condition

Re: [PATCH v2 0/3] btrfs: trace: Add trace events for extent_io_tree

2019-03-07 Thread David Sterba
On Fri, Mar 01, 2019 at 10:47:57AM +0800, Qu Wenruo wrote: > - Allow NULL fs_info for TP_fast_assign_fsid() > There is extent bits operation in selftest which is too deep to pass > fs_info. And since it's in selftest, it shouldn't trigger trace > events. > But to be safe, we still need to c

[PATCH] btrfs: reduce kmap_atomic time for checksumming

2019-03-07 Thread Johannes Thumshirn
Since commit c40a3d38aff4 ("Btrfs: Compute and look up csums based on sectorsized blocks") we do a kmap_atomic() on the contents of a bvec. kmap_atomic() in turn does a preempt_disable() and pagefault_disable(), so we shouldn't map the data for too long. Reduce the time the bvec's page is mapped t

Re: [PATCH RFC 0/3] btrfs: Performance profiler support

2019-03-07 Thread David Sterba
On Thu, Mar 07, 2019 at 10:18:49PM +0800, Qu Wenruo wrote: > > Well, most of that is answered by 'figure out how to use tracepoints and > > perf for that'. > > > > If there were not a whole substystem, actively maintained and > > documented, implementing something like that might help, but that's

Re: [PATCH] btrfs: remove WARN_ON() in log_dir_items

2019-03-07 Thread David Sterba
On Wed, Mar 06, 2019 at 05:13:04PM -0500, Josef Bacik wrote: > When Filipe added the recursive directory logging stuff he specifically > didn't take the directory i_mutex for the children directories that we > need to log because of lockdep. This is generally fine, but can lead to > this WARN_ON()

[PATCH] btrfs: don't report readahead errors and don't update statistics

2019-03-07 Thread David Sterba
As readahead is an optimization, all errors are usually filtered out, but still properly handled when the real read call is done. The commit 5e9d398240b2 added REQ_RAHEAD to readpages() because that's only used for readahead (despite what one would expect from the callback name). This causes a flo

Re: [PATCH RFC 0/3] btrfs: Performance profiler support

2019-03-07 Thread Qu Wenruo
On 2019/3/7 下午10:02, David Sterba wrote: > On Wed, Mar 06, 2019 at 02:19:04PM +0800, Qu Wenruo wrote: >> This patchset can be fetched from github: >> https://github.com/adam900710/linux/tree/perf_tree_lock >> Which is based on v5.0-rc7 tag. >> >> Although we have ftrace/perf to do various perform

Re: [PATCH RFC 0/3] btrfs: Performance profiler support

2019-03-07 Thread David Sterba
On Wed, Mar 06, 2019 at 02:19:04PM +0800, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/linux/tree/perf_tree_lock > Which is based on v5.0-rc7 tag. > > Although we have ftrace/perf to do various performance analyse, under most > case the granularity i

[PATCH 4/4] btrfs-progs: disk-io: Try to find a best copy when reading tree blocks

2019-03-07 Thread Qu Wenruo
[BUG] If the first copy of a tree block has a bad key order, but the second copy is completely good, then "btrfs ins dump-tree -b " fails to print anything past the bad key: leaf 29786112 items 47 free space 983 generation 20 owner EXTENT_TREE leaf 29786112 flags 0x1(WRITTEN) backref revision 1

[PATCH 3/4] btrfs-progs: Move btrfs_num_copies() call out of the loop in read_tree_block()

2019-03-07 Thread Qu Wenruo
btrfs_num_copies really only needs to be called once, so move it out of the verification loop in read_tree_block(). Signed-off-by: Qu Wenruo --- disk-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/disk-io.c b/disk-io.c index 369592eb7b5c..1736d4a1f94b 100644 --- a/disk-

[PATCH 2/4] btrfs-progs: Use mirror_num start from 1 to avoid unnecessary retry

2019-03-07 Thread Qu Wenruo
[BUG] If the first copy of a tree block is corrupted but the other copy is good, btrfs-progs will report the error twice: checksum verify failed on 30556160 found 42A2DA71 wanted checksum verify failed on 30556160 found 42A2DA71 wanted While kernel only report it once, just a

[PATCH 1/4] btrfs-progs: Use @fs_info to replace @root for btrfs_check_leaf/node()

2019-03-07 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- check/main.c| 12 ++-- check/mode-lowmem.c | 8 ctree.c | 32 ctree.h | 8 4 files changed, 30 insertions(+), 30 deletions(-) diff --git a/check/main.c b/check/main.c index 7

Re: [PATCH] btrfs: remove WARN_ON() in log_dir_items

2019-03-07 Thread Filipe Manana
On Wed, Mar 6, 2019 at 10:15 PM Josef Bacik wrote: > > When Filipe added the recursive directory logging stuff he specifically > didn't take the directory i_mutex for the children directories that we > need to log because of lockdep. This is generally fine, but can lead to > this WARN_ON() trippi

Btrfs: cut down on loops through the allocator - a5e681d9bd641c4f0677e87d3a0c92a8f4f16293

2019-03-07 Thread Alex Lyakas
Hi Josef, This commit added the following code in find_free_extent: ret = do_chunk_alloc(trans, root, flags, CHUNK_ALLOC_FORCE); + + /* +* If we can't allocate a new chunk we've alrea