Re: fallocate does not prevent ENOSPC on write

2019-04-24 Thread Qu Wenruo
On 2019/4/23 下午7:33, David Sterba wrote: > On Tue, Apr 23, 2019 at 10:16:32AM +0800, Qu Wenruo wrote: >> On 2019/4/23 上午5:09, Jakob Unterwurzacher wrote: >>> I have a user who is reporting ENOSPC errors when running gocryptfs on >>> top of btrfs (ticket: https://github.com/rfjakob/gocryptfs/issue

Re: [PATCH v2] btrfs: fix filtering of scratch device in test case 048

2019-04-24 Thread Anand Jain
> On 25 Apr 2019, at 8:37 AM, fdman...@kernel.org wrote: > > From: Filipe Manana > > The recent commit 4529b20e1aa8f9 ("btrfs/048: amend property validation > cases"), does not properly filter the scratch device because the error > messages are sent to stderr and not to stdout, and the pipe fi

[PATCH v3] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Qu Wenruo
We have a user reporting BUG_ON() triggered in btrfs_set_item_key_safe(). Let's dump the leaf content before triggering BUG_ON() so that we can have some clue on what's going wrong. The output of tree locks should help us to debug such problem. Reviewed-by: Filip Manana Signed-off-by: Qu Wenruo

[PATCH v2] btrfs: fix filtering of scratch device in test case 048

2019-04-24 Thread fdmanana
From: Filipe Manana The recent commit 4529b20e1aa8f9 ("btrfs/048: amend property validation cases"), does not properly filter the scratch device because the error messages are sent to stderr and not to stdout, and the pipe filter only gets input from the stdout of the btrfs utility. We need to re

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

2019-04-24 Thread Qu Wenruo
On 2019/4/25 上午1:45, David Sterba wrote: > On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: >> Allowing error injection for btrfs_check_leaf_full() and >> btrfs_check_node() is useful to test the failure path of btrfs write >> time tree check. >> >> Signed-off-by: Qu Wenruo > > Does n

[PATCH] btrfs: fix filtering of scratch device in test case 048

2019-04-24 Thread fdmanana
From: Filipe Manana The recent commit 4529b20e1aa8f9 ("btrfs/048: amend property validation cases"), does not properly filter the scratch device because the error messages are sent to stderr and not to stdout, and the pipe filter only gets input from the stdout of the btrfs utility. We need to re

Re: [PATCH] Btrfs: improve performance on fsync of files with multiple hardlinks

2019-04-24 Thread David Sterba
On Wed, Apr 17, 2019 at 11:31:06AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Commit 41bd6067692382 ("Btrfs: fix fsync of files with multiple hard links > in new directories") introduced a path that makes fsync fallback to a full > transaction commit in order to avoid losing hard

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

2019-04-24 Thread David Sterba
On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: > Allowing error injection for btrfs_check_leaf_full() and > btrfs_check_node() is useful to test the failure path of btrfs write > time tree check. > > Signed-off-by: Qu Wenruo Added to misc-next, thanks.

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

2019-04-24 Thread David Sterba
On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: > Allowing error injection for btrfs_check_leaf_full() and > btrfs_check_node() is useful to test the failure path of btrfs write > time tree check. > > Signed-off-by: Qu Wenruo Does not compile: CC [M] fs/btrfs/tests/extent-map-test

Re: [PATCH] btrfs: Document btrfs_csum_one_bio

2019-04-24 Thread David Sterba
On Mon, Apr 22, 2019 at 04:07:31PM +0300, Nikolay Borisov wrote: > Signed-off-by: Nikolay Borisov Added to misc-next, thanks.

Re: [PATCH 1/2] btrfs: track odirect bytes in flight

2019-04-24 Thread David Sterba
On Fri, Apr 12, 2019 at 01:17:40PM +0300, Nikolay Borisov wrote: > > > On 10.04.19 г. 22:56 ч., Josef Bacik wrote: > > When diagnosing a slowdown of generic/224 I noticed we were wasting a > > lot of time in shrink_delalloc() despite all writes being O_DIRECT > > writes. O_DIRECT writes still ha

Re: [PATCH] btrfs: Make btrfs_finish_extent_commit() return void

2019-04-24 Thread David Sterba
On Tue, Apr 16, 2019 at 01:16:37PM +0800, Qu Wenruo wrote: > This function always return 0, and the only use case of @ret is to > output warning message for discard failure, which doesn't cause anything > wrong to the fs even discarding failed. > > There is no need to make it return int. Same com

Re: [PATCH v2] Btrfs: fix race between send and deduplication that lead to failures and crashes

2019-04-24 Thread David Sterba
On Mon, Apr 22, 2019 at 04:43:42PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Send operates on read only trees and expects them to never change while it > is using them. This is part of its initial design, and this expection is > due to two different reasons: > > 1) When it was

Re: [PATCH] Btrfs: send, flush dellaloc in order to avoid data loss

2019-04-24 Thread David Sterba
On Mon, Apr 15, 2019 at 09:29:36AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > When we set a subvolume to read-only mode we do not flush dellaloc for any > of its inodes (except if the filesystem is mounted with -o flushoncommit), > since it does not affect correctness for any sub

Re: [PATCH 00/10] cleanup btrfs_ioctl_setflags and btrfs_set_prop

2019-04-24 Thread David Sterba
On Sat, Apr 20, 2019 at 07:48:50PM +0800, Anand Jain wrote: > A set of pathes to clean up btrfs_ioctl_setflags(), btrfs_set_prop_trans() > and btrfs_set_prop(). So as to avoidi duplicate code in > btrfs_ioctl_setflags() and killing of btrfs_set_prop_trans(). > > Anand Jain (10): > btrfs: refacto

[PATCH v2 1/3] btrfs-progs: factor out super_block reading from load_and_dump_sb

2019-04-24 Thread Johannes Thumshirn
inspect-internal dump-superblock's load_and_dump_sb() already reads a super block from a file descriptor and places it into a 'struct btrfs_super_block'. For inspect-internal dump-csum we need this super block as well but don't care about printing it. Separate the read from the dump phase so we c

[PATCH v2 0/3] btrfs-progs: provide command to dump checksums

2019-04-24 Thread Johannes Thumshirn
Provide a command to dump checksums from 'btrfs inspect-internal'. This command does a lookup for the given file's extents via FIEMAP (which is handy as it already skips holes) and walks the checksum tree printing all checksums of an extent. It is tested against different layouts of files with and

[PATCH v2 3/3] btrfs-progs: completion: wire-up dump-csum

2019-04-24 Thread Johannes Thumshirn
Add the new 'dump-csum' command to inspect-internal. Signed-off-by: Johannes Thumshirn --- btrfs-completion | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-completion b/btrfs-completion index 6ae57d1b752b..a381036886f0 100644 --- a/btrfs-completion +++ b/btrfs-compl

[PATCH v2 2/3] btrfs-progs: add 'btrfs inspect-internal csum-dump' command

2019-04-24 Thread Johannes Thumshirn
Add a 'btrfs inspect-internal csum-dump' command to dump the on-disk checksums of a file. The dump command first uses the FIEMAP ioctl() to get a map of the file's extents and then uses the BTRFS_TREE_SEARCH_V2 ioctl() to get the checksums for these extents. Using FIEMAP instead of the BTRFS_TREE

Re: [PATCH v2] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Filipe Manana
On Wed, Apr 24, 2019 at 3:33 PM Qu Wenruo wrote: > > We have a user reporting BUG_ON() triggered in > btrfs_set_item_key_safe(). > > Let's dump the leaf content before triggering BUG_ON() so that we can > have some clue on what's going wrong. > The output of tree locks should help us to debug such

[PATCH v2] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Qu Wenruo
We have a user reporting BUG_ON() triggered in btrfs_set_item_key_safe(). Let's dump the leaf content before triggering BUG_ON() so that we can have some clue on what's going wrong. The output of tree locks should help us to debug such problem. Signed-off-by: Qu Wenruo --- changelog: v2: - Also

Re: [PATCH] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Qu Wenruo
On 2019/4/24 下午9:53, Filipe Manana wrote: > On Wed, Apr 24, 2019 at 1:57 PM Qu Wenruo wrote: >> >> >> >> On 2019/4/24 下午8:52, Nikolay Borisov wrote: >>> >>> >>> On 24.04.19 г. 15:48 ч., Qu Wenruo wrote: Let's dump the leaf content before triggering BUG_ON() so that we can have some cl

Re: [PATCH] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Filipe Manana
On Wed, Apr 24, 2019 at 1:57 PM Qu Wenruo wrote: > > > > On 2019/4/24 下午8:52, Nikolay Borisov wrote: > > > > > > On 24.04.19 г. 15:48 ч., Qu Wenruo wrote: > >> Let's dump the leaf content before triggering BUG_ON() so that we can > >> have some clue on what's going wrong. > >> The output of tree l

Re: [bug report] btrfs: get fs_info from block group in write_pinned_extent_entries

2019-04-24 Thread David Sterba
On Wed, Apr 24, 2019 at 12:52:54PM +0300, Dan Carpenter wrote: > Hello David Sterba, > > This is a semi-automatic email about new static checker warnings. > > The patch b4c7c5d50791: "btrfs: get fs_info from block group in > write_pinned_extent_entries" from Mar 20, 2019, leads to the > following

Re: [PATCH] btrfs: Fix off-by-one error btrfs_trim_free_extents

2019-04-24 Thread David Sterba
On Wed, Apr 24, 2019 at 11:46:55AM +0300, Nikolay Borisov wrote: > btrfs_trim_free_extents always caps the range it's going to trim based > on the size of the device. This happens if find_first_clear_extent_bit > detects that untrimmed range is past the last allocated range. Since it > doesn't have

Re: [PATCH] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Qu Wenruo
On 2019/4/24 下午8:52, Nikolay Borisov wrote: > > > On 24.04.19 г. 15:48 ч., Qu Wenruo wrote: >> Let's dump the leaf content before triggering BUG_ON() so that we can >> have some clue on what's going wrong. >> The output of tree locks should help us to debug such problem. >> > > The reason is som

Re: [PATCH] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Nikolay Borisov
On 24.04.19 г. 15:48 ч., Qu Wenruo wrote: > Let's dump the leaf content before triggering BUG_ON() so that we can > have some clue on what's going wrong. > The output of tree locks should help us to debug such problem. > The reason is some sort of in-memory corruption/confusion of the extent m

Re: kernel BUG at fs/btrfs/ctree.c:3230!

2019-04-24 Thread Qu Wenruo
On 2019/4/24 下午8:28, Gregory Malloff wrote: > > Hello, > > After 7 days, BTRFS crashed again with the same error: > > [Tue Apr 23 21:59:40 2019] [ cut here ] > [Tue Apr 23 21:59:40 2019] kernel BUG at fs/btrfs/ctree.c:3230! > [Tue Apr 23 21:59:40 2019] invalid opcod

[PATCH] btrfs: ctree: Dump the leaf before BUG_ON()

2019-04-24 Thread Qu Wenruo
We have a user reporting BUG_ON() triggered in btrfs_set_item_key_safe(). Let's dump the leaf content before triggering BUG_ON() so that we can have some clue on what's going wrong. The output of tree locks should help us to debug such problem. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.c | 2

kernel BUG at fs/btrfs/ctree.c:3230!

2019-04-24 Thread Gregory Malloff
Hello, After 7 days, BTRFS crashed again with the same error: [Tue Apr 23 21:59:40 2019] [ cut here ] [Tue Apr 23 21:59:40 2019] kernel BUG at fs/btrfs/ctree.c:3230! [Tue Apr 23 21:59:40 2019] invalid opcode: [#1] SMP PTI [Tue Apr 23 21:59:40 2019] CPU: 0 PID:

Re: [PATCH 1/2] btrfs: Simplify snapshot exclusion code

2019-04-24 Thread Filipe Manana
On Wed, Apr 24, 2019 at 10:49 AM Filipe Manana wrote: > > On Tue, Apr 23, 2019 at 12:43 PM Nikolay Borisov wrote: > > > > BTRFS sports a mechanism to provide exclusion when a snapshot is about > > to be created. This is implemented via btrfs_start_write_no_snapshotting > > et al. Currently the im

[bug report] btrfs: get fs_info from block group in write_pinned_extent_entries

2019-04-24 Thread Dan Carpenter
Hello David Sterba, This is a semi-automatic email about new static checker warnings. The patch b4c7c5d50791: "btrfs: get fs_info from block group in write_pinned_extent_entries" from Mar 20, 2019, leads to the following Smatch complaint: fs/btrfs/free-space-cache.c:1052 write_pinned_extent_

Re: fallocate does not prevent ENOSPC on write

2019-04-24 Thread Qu Wenruo
On 2019/4/24 下午5:28, Filipe Manana wrote: [snip] >>> So what's wrong with it? And how does it cause the ENOSPC? >> >> E.g. >> >> We have a 128Mb preallocated file extent. >> And assume the fs only have 128M free data space, meaning 0 remaining >> space at all. > > That's a contradicting sentence

Re: [PATCH 1/2] btrfs: Simplify snapshot exclusion code

2019-04-24 Thread Filipe Manana
On Tue, Apr 23, 2019 at 12:43 PM Nikolay Borisov wrote: > > BTRFS sports a mechanism to provide exclusion when a snapshot is about > to be created. This is implemented via btrfs_start_write_no_snapshotting > et al. Currently the implementation of that mechanism is some perverse > amalgamation of a

Re: fallocate does not prevent ENOSPC on write

2019-04-24 Thread Filipe Manana
On Wed, Apr 24, 2019 at 12:49 AM Qu Wenruo wrote: > > > > On 2019/4/23 下午10:50, Filipe Manana wrote: > > On Tue, Apr 23, 2019 at 1:14 PM Qu Wenruo wrote: > >> > >> > >> > >> On 2019/4/23 下午7:33, David Sterba wrote: > >>> On Tue, Apr 23, 2019 at 10:16:32AM +0800, Qu Wenruo wrote: > On 2019/4/

Re: [PATCH] Btrfs: prevent send failures and crashes due to concurrent relocation

2019-04-24 Thread Filipe Manana
On Wed, Apr 24, 2019 at 9:56 AM Nikolay Borisov wrote: > > > > On 22.04.19 г. 18:44 ч., fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Send always operates on read-only trees and always expected that while it > > is in progress, nothing changes in those trees. Due to that expectation

Re: [PATCH] Btrfs: prevent send failures and crashes due to concurrent relocation

2019-04-24 Thread Nikolay Borisov
On 22.04.19 г. 18:44 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > Send always operates on read-only trees and always expected that while it > is in progress, nothing changes in those trees. Due to that expectation > and the fact that send is a read-only operation, it operates on co

[PATCH] btrfs: Fix off-by-one error btrfs_trim_free_extents

2019-04-24 Thread Nikolay Borisov
btrfs_trim_free_extents always caps the range it's going to trim based on the size of the device. This happens if find_first_clear_extent_bit detects that untrimmed range is past the last allocated range. Since it doesn't have knowledge of the size of the device it just returns (u64)-1 for end. The

Re: reading/writing btrfs volume regularly freezes system

2019-04-24 Thread Qu Wenruo
On 2019/4/23 上午4:37, Nathan Dehnel wrote: > I have a raid10 volume that frequently locks up when I try to write to > it or delete things. Any command that touches it will hang (and can't > be killed) and I have to start a new ssh session to get into the > computer again. Nothing fixes it besides

btrfs-profiler: Error injection tool added

2019-04-24 Thread Qu Wenruo
Hi, btrfs-profiler (https://github.com/adam900710/btrfs-profiler) has added a new tool: inject.py. The tool is mostly a blend between bcc/tools/inject.py and Josef's debug-scripts/inject-error.py: - Variable length callchain (trigger) support This feature is mostly from bcc. With this tool, w

[PATCH] btrfs: tree-checker: Allow error injection for tree-checker

2019-04-24 Thread Qu Wenruo
Allowing error injection for btrfs_check_leaf_full() and btrfs_check_node() is useful to test the failure path of btrfs write time tree check. Signed-off-by: Qu Wenruo --- fs/btrfs/tree-checker.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-check