Exactly what is wrong with RAID5/6

I am trying to piece together the actual status of the RAID5/6 bit of BTRFS. The wiki refer to kernel 3.19 which was released in February 2015 so I assume that the information there is a tad outdated (the last update on the wiki page was July 2016)

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 04:12:03PM -0700, Marc MERLIN wrote: > Given that check --repair ran clean when I ran it yesterday after this first > happened, > and I then ran mount -o clear_cache , the cache got rebuilt, and I got the > problem again, > this is not looking good, seems like a

Re: Exactly what is wrong with RAID5/6

On Tue, Jun 20, 2017 at 5:25 PM, Hugo Mills wrote: > On Wed, Jun 21, 2017 at 12:57:19AM +0200, waxhead wrote: >> I am trying to piece together the actual status of the RAID5/6 bit of BTRFS. >> The wiki refer to kernel 3.19 which was released in February 2015 so >> I assume

Re: Exactly what is wrong with RAID5/6

On Wed, Jun 21, 2017 at 12:57:19AM +0200, waxhead wrote: > I am trying to piece together the actual status of the RAID5/6 bit of BTRFS. > The wiki refer to kernel 3.19 which was released in February 2015 so > I assume that the information there is a tad outdated (the last > update on the wiki page

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 5:12 PM, Marc MERLIN wrote: > I'm now going to remount this with nospace_cache to see if your guess about > space_cache was correct. > Other suggestions also welcome :) What results do you get with lowmem mode? It won't repair without additional

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 08:44:29AM -0700, Marc MERLIN wrote: > On Tue, Jun 20, 2017 at 03:36:01PM +, Hugo Mills wrote: > > > Thanks for having a look. Is it a bug, or is it a problem with my storage > > > subsystem? > > > >Well, I'd say it's probably a problem with some inconsistent data

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 09:31:42PM -0600, Chris Murphy wrote: > On Tue, Jun 20, 2017 at 5:12 PM, Marc MERLIN wrote: > > > I'm now going to remount this with nospace_cache to see if your guess about > > space_cache was correct. > > Other suggestions also welcome :) > > What

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 09:26:27PM -0600, Chris Murphy wrote: > Right now Btrfs isn't scalable if you have to repair it because large > volumes run into this problem; one of the reasons for the lowmem mode. > > It's a separate bug that it OOMs even with swap, I don't know why it > won't use that,

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 9:44 AM, Marc MERLIN wrote: > In the meantime, I ran into this again: > https://bugzilla.kernel.org/show_bug.cgi?id=195863 > btrfs check of a big filesystem kills the kernel due to OOM (but btrfs > userspace is not OOM killed) > > Is it achievable at

Re: Please help. Repair probably bitflip damage and suspected bug

> [Sun Jun 18 04:02:43 2017] BTRFS critical (device sdb2): corrupt node, bad key order: block=5123372711936, root=1, slot=82 >From the archives, most likely it's bad RAM. I see this system also uses XFS v4 file system, if it were made as XFS v5 using metadata csums you'd probably eventually run

Re: [PATCH] btrfs: use new block error code

On Mon, Jun 19, 2017 at 01:55:37PM +0300, Dan Carpenter wrote: > This function is supposed to return blk_status_t error codes now but > there was a stray -ENOMEM left behind. > > Fixes: 4e4cbee93d56 ("block: switch bios to blk_status_t") > Signed-off-by: Dan Carpenter

[PATCH] btrfs: add cond_resched to btrfs_qgroup_trace_leaf_items

From: Jeff Mahoney On an uncontended system, we can end up hitting soft lockups while doing replace_path. At the core, and frequently called is btrfs_qgroup_trace_leaf_items, so it makes sense to add a cond_resched there. Signed-off-by: Jeff Mahoney ---

Re: [PATCH v7 00/22] fs: enhanced writeback error reporting with errseq_t (pile #1)

On Tue, 2017-06-20 at 09:25 +1000, Stephen Rothwell wrote: > Hi Jeff, > > On Mon, 19 Jun 2017 12:23:46 -0400 Jeff Layton wrote: > > > > If there are no major objections to this set, I'd like to have > > linux-next start picking it up to get some wider testing. What's the > >

Re: [PATCH] btrfs: cleanup duplicate return value in insert_inline_extent

On 15.06.2017 20:09, David Sterba wrote: > The pattern when err is used for function exit and ret is used for > return values of callees is not used here. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov -- To unsubscribe from this list: send

Re: RFC: Compression - calculate entropy for data set

2017-06-20 1:09 GMT+03:00 Timofey Titovets : > Hi, for last several days i try work on entropy calculation that can > be usable in btrfs compression code (for detect bad compressible > data), > > I've implemented: > - avg meaning (Problems with accuracy) > - shannon entropy >

Re: [PATCH] btrfs: move fs_info::fs_frozen to the flags

On 15.06.2017 20:10, David Sterba wrote: > We can keep the state among the other fs_info flags, there's no reason > why fs_frozen would need to be separate. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov -- To unsubscribe from this list:

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

> error = filemap_write_and_wait_range(filp->f_mapping, start, end); > if (error) > - return error; > + goto out; > > /* >* There is no need to serialise calls to blkdev_issue_flush with > @@ -640,6 +640,10 @@ int blkdev_fsync(struct file *filp,

Re: [PATCH v7 11/22] fs: new infrastructure for writeback error handling and reporting

> @@ -393,6 +394,7 @@ struct address_space { > gfp_t gfp_mask; /* implicit gfp mask for > allocations */ > struct list_headprivate_list; /* ditto */ > void*private_data; /* ditto */ > + errseq_twb_err; >

Re: [PATCH v7 01/22] fs: remove call_fsync helper function

On Fri, Jun 16, 2017 at 03:34:06PM -0400, Jeff Layton wrote: > Requested-by: Christoph Hellwig > Signed-off-by: Jeff Layton Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs"

Re: [PATCH v7 11/22] fs: new infrastructure for writeback error handling and reporting

On Tue, 2017-06-20 at 05:34 -0700, Christoph Hellwig wrote: > > @@ -393,6 +394,7 @@ struct address_space { > > gfp_t gfp_mask; /* implicit gfp mask for > > allocations */ > > struct list_headprivate_list; /* ditto */ > > void

4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

My filesystem got remounted read only, and yet after a lengthy btrfs check --repair, it ran clean. Any idea what went wrong? [846332.992285] WARNING: CPU: 4 PID: 4095 at fs/btrfs/free-space-cache.c:1476 tree_insert_offset+0x78/0xb1 [846333.744721] BTRFS critical (device dm-1): unable to add free

Re: does using different uid/gid/forceuid/... mount options for different subvolumes work / does fuse.bindfs play nice with btrfs?

On Tue, Jun 20, 2017 at 04:35:48PM +0200, Alexander Peganz wrote: > Hello everyone, > > I intend to provide different "views" of the data stored on btrfs subvolumes. > e.g. mount a subvolume in location A rw; and ro in location B while > also overwriting uids, gids, and permissions. > In the past

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 07:39:16AM -0700, Marc MERLIN wrote: > My filesystem got remounted read only, and yet after a lengthy > btrfs check --repair, it ran clean. > > Any idea what went wrong? > [846332.992285] WARNING: CPU: 4 PID: 4095 at fs/btrfs/free-space-cache.c:1476 >

Re: [PATCH v7 01/22] fs: remove call_fsync helper function

On Fri 16-06-17 15:34:06, Jeff Layton wrote: > Requested-by: Christoph Hellwig > Signed-off-by: Jeff Layton Looks good. You can add: Reviewed-by: Jan Kara Honza > ---

[PATCH 00/13] use rbtrees for preliminary backrefs

This patch series attempts to improve the performance of backref searches by changing the prelim_refs implementation to use rbtrees instead of lists. This also aims to reduce the soft lockup occurences that can result when a backref search consumes too much cpu time. Test runs of btrfs/130 show

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 03:23:54PM +, Hugo Mills wrote: > On Tue, Jun 20, 2017 at 07:39:16AM -0700, Marc MERLIN wrote: > > My filesystem got remounted read only, and yet after a lengthy > > btrfs check --repair, it ran clean. > > > > Any idea what went wrong? > > [846332.992285] WARNING: CPU:

does using different uid/gid/forceuid/... mount options for different subvolumes work / does fuse.bindfs play nice with btrfs?

Hello everyone, I intend to provide different "views" of the data stored on btrfs subvolumes. e.g. mount a subvolume in location A rw; and ro in location B while also overwriting uids, gids, and permissions. In the past I have been using fuse.bindfs for this. Now I'm trying to find out if there

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 08:26:48AM -0700, Marc MERLIN wrote: > On Tue, Jun 20, 2017 at 03:23:54PM +, Hugo Mills wrote: > > On Tue, Jun 20, 2017 at 07:39:16AM -0700, Marc MERLIN wrote: > > > My filesystem got remounted read only, and yet after a lengthy > > > btrfs check --repair, it ran clean.

[PATCH 11/13] btrfs: add cond_resched() calls when resolving backrefs

Since backref resolution is CPU-intensive, the cond_resched calls should help alleviate soft lockup occurences. Signed-off-by: Edmund Nadolski Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 3 +++ 1 file changed, 3 insertions(+) diff --git

[PATCH 08/13] btrfs: convert prelimary reference tracking to use rbtrees

It's been known for a while that the use of multiple lists that are periodically merged was an algorithmic problem within btrfs. There are several workloads that don't complete in any reasonable amount of time (e.g. btrfs/130) and others that cause soft lockups. The solution is to use a pair of

[PATCH 02/13] btrfs: constify tracepoint arguments

From: Jeff Mahoney Tracepoint arguments are all read-only. If we mark the arguments as const, we're able to keep or convert those arguments to const where appropriate. Signed-off-by: Jeff Mahoney --- fs/btrfs/async-thread.c | 6 +-

[PATCH 01/13] btrfs: struct-funcs, constify readers

From: Jeff Mahoney We have reader helpers for most of the on-disk structures that use an extent_buffer and pointer as offset into the buffer that are read-only. We should mark them as const and, in turn, allow consumers of these interfaces to mark the buffers const as well. No

[PATCH 09/13] btrfs: add a node counter to each of the rbtrees

From: Jeff Mahoney This patch adds counters to each of the rbtrees so that we can tell how large they are growing for a given workload. These counters will be exported by tracepoints in the next patch. Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 6

[PATCH 04/13] btrfs: backref, add unode_aux_to_inode_list helper

From: Jeff Mahoney Replacing the double cast and ternary conditional with a helper makes the code easier on the eyes. Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git

[PATCH 07/13] btrfs: remove ref_tree implementation from backref.c

The BACKREF_FOUND_SHARED checking will be addressed in an upcoming patch. Signed-off-by: Edmund Nadolski Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 356 ++--- 1 file changed, 8 insertions(+), 348

[PATCH 05/13] btrfs: backref, cleanup __ namespace abuse

From: Jeff Mahoney We typically use __ to indicate a helper routine that shouldn't be called directly without understanding the proper context required to do so. We use static functions to indicate that a function is private to a particular C file. The backref code uses static

[PATCH 13/13] btrfs: clean up extraneous computations in add_delayed_refs

Repeating the same computation in multiple places is not necessary. Signed-off-by: Edmund Nadolski Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 30 +- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git

[PATCH 10/13] btrfs: backref, add tracepoints for prelim_ref insertion and merging

From: Jeff Mahoney This patch adds a tracepoint event for prelim_ref insertion and merging. For each, the ref being inserted or merged and the count of tree nodes is issued. Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 117

[PATCH 12/13] btrfs: allow backref search checks for shared extents

When called with a struct share_check, find_parent_nodes() will detect a shared extent and immediately return with BACKREF_SHARED_FOUND. Signed-off-by: Edmund Nadolski Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 165

[PATCH 03/13] btrfs: backref, constify some arguments

From: Jeff Mahoney This constifies a few buffers used in the backref code. Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 29 - 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/backref.c

[PATCH 06/13] btrfs: btrfs_check_shared should manage its own transaction

Signed-off-by: Edmund Nadolski Signed-off-by: Jeff Mahoney --- fs/btrfs/backref.c | 30 +++--- fs/btrfs/backref.h | 4 +--- fs/btrfs/extent_io.c | 22 +++--- 3 files changed, 23 insertions(+), 33 deletions(-) diff

Re: does using different uid/gid/forceuid/... mount options for different subvolumes work / does fuse.bindfs play nice with btrfs?

Ok, thanks for the clarification. Bindfs will continue to live on my machines then! Regards, Alexander On 20 June 2017 at 17:15, Hugo Mills wrote: > On Tue, Jun 20, 2017 at 04:35:48PM +0200, Alexander Peganz wrote: >> Hello everyone, >> >> I intend to provide different

Re: [PATCH v7 05/22] jbd2: don't clear and reset errors after waiting on writeback

On Fri 16-06-17 15:34:10, Jeff Layton wrote: > Resetting this flag is almost certainly racy, and will be problematic > with some coming changes. > > Make filemap_fdatawait_keep_errors return int, but not clear the flag(s). > Have jbd2 call it instead of filemap_fdatawait and don't attempt to >

Re: 4.11.3: BTRFS critical (device dm-1): unable to add free space :-17 => btrfs check --repair runs clean

On Tue, Jun 20, 2017 at 03:36:01PM +, Hugo Mills wrote: > > Thanks for having a look. Is it a bug, or is it a problem with my storage > > subsystem? > >Well, I'd say it's probably a problem with some inconsistent data > on the disk. How that data got there is another matter -- it may be >

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

On Tue, 2017-06-20 at 05:35 -0700, Christoph Hellwig wrote: > > error = filemap_write_and_wait_range(filp->f_mapping, start, end); > > if (error) > > - return error; > > + goto out; > > > > /* > > * There is no need to serialise calls to blkdev_issue_flush

Re: does using different uid/gid/forceuid/... mount options for different subvolumes work / does fuse.bindfs play nice with btrfs?

> I intend to provide different "views" of the data stored on > btrfs subvolumes. e.g. mount a subvolume in location A rw; > and ro in location B while also overwriting uids, gids, and > permissions. [ ... ] That's not how UNIX/Linux permissions and ACLs are supposed to work, perhaps you should