5.1.21: fs/btrfs/extent-tree.c:7100 __btrfs_free_extent+0x18b/0x921

This happened almost after a resume from suspend to disk. First corruption and read only I got a very long time. Could they be related? [26062.126505] [ cut here ] [26062.126524] WARNING: CPU: 7 PID: 12394 at fs/btrfs/extent-tree.c:7100 __btrfs_free_extent+0x18b/0x921 [26

Re: [PATCH 5/5] btrfs: document extent buffer locking

On 2019/10/18 上午3:39, David Sterba wrote: > Signed-off-by: David Sterba Great document. Some questions inlined below. > --- > fs/btrfs/locking.c | 110 +++-- > 1 file changed, 96 insertions(+), 14 deletions(-) > > diff --git a/fs/btrfs/locking.c b/fs/b

[PATCH 2/5] btrfs: set blocking_writers directly, no increment or decrement

The increment and decrement was inherited from previous version that used atomics, switched in commit 06297d8cefca ("btrfs: switch extent_buffer blocking_writers from atomic to int"). The only possible values are 0 and 1 so we can set them directly. The generated assembly (gcc 9.x) did the direct

[PATCH 5/5] btrfs: document extent buffer locking

Signed-off-by: David Sterba --- fs/btrfs/locking.c | 110 +++-- 1 file changed, 96 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c index e0e0430577aa..2a0e828b4470 100644 --- a/fs/btrfs/locking.c +++ b/fs/btrfs/locking.c

[PATCH 3/5] btrfs: access eb::blocking_writers according to ACCESS_ONCE policies

A nice writeup of the LKMM (Linux Kernel Memory Model) rules for access once policies can be found here https://lwn.net/Articles/799218/#Access-Marking%20Policies . The locked and unlocked access to eb::blocking_writers should be annotated accordingly, following this: Writes: - locked write must

[PATCH 4/5] btrfs: serialize blocking_writers updates

There's one potential memory ordering problem regarding eb::blocking_writers, although this hasn't been hit in practice. On TSO (total store ordering) architectures, the writes will always be seen in right order. In case the calls in btrfs_set_lock_blocking_write are seen in this (wrong) order on

[PATCH 0/5] Extent buffer locking and documentation

I've spent a lot of time staring at the locking code and speculating about all sorts of weird problems that could happen due to memory ordering or lost wakeups or if the custom locking is safe at all, also regarding the recent changes. Inevitably I found something but also wrote documentation. Ple

[PATCH 1/5] btrfs: merge blocking_writers branches in btrfs_tree_read_lock

There are two ifs that use eb::blocking_writers. As this is a variable modified inside and outside of locks, we could minimize number of accesses to avoid problems with getting different results at different times. The access here is locked so this can only race with btrfs_tree_unlock that sets bl

Evaluating File System Reliability on Solid State Drives

Since resiliency and recovery from corruption continues to be a hot topic, I figured I'd mention a paper/talk/slides from the recent Usenix ATC'19 conference: Evaluating File System Reliability on Solid State Drives https://www.usenix.org/conference/atc19/presentation/jaffer Maybe it provides

Re: [PATCH] Btrfs: check for the full sync flag while holding the inode lock during fsync

On Wed, Oct 16, 2019 at 04:28:52PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > We were checking for the full fsync flag in the inode before locking the > inode, which is racy, since at that that time it might not be set but > after we acquire the inode lock some other task set it.

Re: MD RAID 5/6 vs BTRFS RAID 5/6

On 17/10/2019 16:57, Chris Murphy wrote: > On Wed, Oct 16, 2019 at 10:07 PM Jon Ander MB > wrote: >> >> It would be interesting to know the pros and cons of this setup that >> you are suggesting vs zfs. >> +zfs detects and corrects bitrot ( >> http://www.zfsnas.com/2015/05/24/testing-bit-rot/ ) >

Re: [PATCH 00/15] Remove callback indirections in compression code

On Mon, Oct 14, 2019 at 02:22:21PM +0200, David Sterba wrote: > The series removes all per-compression algorithm callbacks and replaces > them with a switches. Lots of indirections are simplified and while it > looks nice from design POV, we don't need to over-engineer it as we have > only fixed kn

Re: [PATCH] Btrfs: fix qgroup double free after failure to reserve metadata for delalloc

On Tue, Oct 15, 2019 at 10:54:39AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > If we fail to reserve metadata for delalloc operations we end up releasing > the previously reserved qgroup amount twice, once explicitly under the > 'out_qgroup' label by calling btrfs_qgroup_free_meta

Re: [PATCH v2 0/3] btrfs llseek improvement, take 2

On Fri, Sep 27, 2019 at 01:23:15PM +0300, Nikolay Borisov wrote: > Here is v2 of the llseek improvements, main changes are: > > * Patch 1 - changed the locking scheme. I'm now using inode_lock_shared > since > holding the extent lock is not sufficient to prevent concurrent truncates. > > *

Re: [PATCH] btrfs-progs: drop unique uuid test for btrfstune -M

On Thu, Sep 12, 2019 at 08:45:50AM +0800, Anand Jain wrote: > In case of vm guest images copied from the golden image there is no > physical device or loop device or nbd device until its configured on > the host when required, so check for duplicate fsid at the time of > btrfstune -M is not convinc

Re: MD RAID 5/6 vs BTRFS RAID 5/6

On Wed, Oct 16, 2019 at 10:07 PM Jon Ander MB wrote: > > It would be interesting to know the pros and cons of this setup that > you are suggesting vs zfs. > +zfs detects and corrects bitrot ( > http://www.zfsnas.com/2015/05/24/testing-bit-rot/ ) > +zfs has working raid56 > -modules out of kernel f

Re: [PATCH RESEND] btrfs-progs: btrfstune -M|m drop test_uuid_unique

On Thu, Oct 17, 2019 at 09:03:00AM +0800, Anand Jain wrote: > Ping (4th time). Anand, the patch queue is big and ever growing, pinging can help to bring back the attention but for patches that have disgreements or unclear reasoning the turnaround time can be longer. Specifically, patches that hav

Re: [PATCH 00/15] Remove callback indirections in compression code

On 14.10.19 г. 15:22 ч., David Sterba wrote: > The series removes all per-compression algorithm callbacks and replaces > them with a switches. Lots of indirections are simplified and while it > looks nice from design POV, we don't need to over-engineer it as we have > only fixed known number of

Re: [PATCH] btrfs-progs: small fixes/cleanup in Documentation

On Thu, 17 Oct 2019 13:18:05 +0200 David Sterba wrote: > Well, for documentation patches and for progs it's not that strict and > I've applied many drive-by patches. My sign-off will be there and the > original author is usually mentioned as Author:, so the credit is > recorded. I'm fine with

Re: [PATCH 1/8] btrfs: Cleanup and simplify find_newest_super_backup

On Tue, Oct 15, 2019 at 06:42:17PM +0300, Nikolay Borisov wrote: > Backup roots are always written in a circular manner. By definition we > can only ever have 1 backup root whose generation equals to that of the > superblock. Hence, the 'if' in the for loop will trigger at most once. > This is suff

Re: [PATCH v2] tools/lib/traceevent, perf tools: Handle %pU format correctly

On Thu, 17 Oct 2019 13:54:23 +0200 David Sterba wrote: > > So if there's an interest for very compact printing loop, something like > > this produces the same output: > > > > for (i = 0; i < 8; i++) { > > printf("%02X", buf[i]); > > printf("%02X", buf[i]); > > Ok,

Re: [PATCH v2 0/2] btrfs: qgroup: Bug fixes for trace events

On Thu, Oct 17, 2019 at 10:38:35AM +0800, Qu Wenruo wrote: > Several stupid bugs exposed for qgroup related trace events. > Fix them. Queued for 5.4-rc, thanks.

Re: [PATCH 15/15] btrfs: compression: remove ops pointer from workspace_manager

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 14/15] btrfs: compression: inline free_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführe

Re: [PATCH 13/15] btrfs: compression: pass type to btrfs_put_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 12/15] btrfs: compression: inline alloc_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH v2] tools/lib/traceevent, perf tools: Handle %pU format correctly

On Thu, Oct 17, 2019 at 01:52:09PM +0200, David Sterba wrote: > On Thu, Oct 17, 2019 at 10:28:00AM +0800, Qu Wenruo wrote: > > [BUG] > > For btrfs related events, there is a field for fsid, but perf never > > parse it correctly. > > > > # perf trace -e btrfs:qgroup_meta_convert xfs_io -f -c "pwri

Re: [PATCH 11/15] btrfs: compression: pass type to btrfs_get_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH v2] tools/lib/traceevent, perf tools: Handle %pU format correctly

On Thu, Oct 17, 2019 at 10:28:00AM +0800, Qu Wenruo wrote: > [BUG] > For btrfs related events, there is a field for fsid, but perf never > parse it correctly. > > # perf trace -e btrfs:qgroup_meta_convert xfs_io -f -c "pwrite 0 4k" \ >/mnt/btrfs/file1 > 0.000 xfs_io/77915 btrfs:qgroup_me

Re: [PATCH 10/15] btrfs: compression: inline put_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 09/15] btrfs: compression: inline get_workspace

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 08/15] btrfs: compression: export alloc/free/get/put callbacks of all algos

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführe

Re: [PATCH 07/15] btrfs: compression: inline cleanup_workspace_manager

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführe

[PATCH 0/2] Minor tracepoints cleanups

David Sterba (2): btrfs: tracepoints: drop typecasts from printk btrfs: tracepoints: constify all pointers include/trace/events/btrfs.h | 80 ++-- 1 file changed, 39 insertions(+), 41 deletions(-) -- 2.23.0

[PATCH 1/2] btrfs: tracepoints: drop typecasts from printk

Remove typecasts from trace printk, adjust types and move typecast to the assignment if necessary. When assigning, the types are more obvious compared to matching the variables to the format strings. Signed-off-by: David Sterba --- include/trace/events/btrfs.h | 28 +---

[PATCH 2/2] btrfs: tracepoints: constify all pointers

We don't modify the data passed to tracepoints, some of the declarations are already const, add it to the rest. Signed-off-by: David Sterba --- include/trace/events/btrfs.h | 52 ++-- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/include/trace/ev

Re: [PATCH 02/15] btrfs: switch compression callbacks to direct calls

On Thu, Oct 17, 2019 at 11:42:53AM +0200, Johannes Thumshirn wrote: > On 14/10/2019 14:22, David Sterba wrote: > > +static int compression_decompress_bio(int type, struct list_head *ws, > > + struct compressed_bio *cb) > > +{ > > + switch (type) { > > + case BTRFS_COMPRESS_ZLIB: retur

Re: [PATCH] btrfs-progs: small fixes/cleanup in Documentation

On Thu, Oct 17, 2019 at 09:29:43AM +0300, Nikolay Borisov wrote: > > > On 17.10.19 г. 7:50 ч., Merlin Büge wrote: > > The removed paragraph in btrfs-man5.asciidoc says the same as the > > previous one. > > This patch cannot be applied without an SOB line. Otherwise the changes > look good. Wel

Re: [PATCH 06/15] btrfs: compression: let workspace manager cleanup take only the type

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 03/15] btrfs: compression: attach workspace manager to the ops

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 05/15] btrfs: compression: inline init_workspace_manager

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

Re: [PATCH 04/15] btrfs: compression: let workspace manager init take only the type

Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Ime

Re: [PATCH 02/15] btrfs: switch compression callbacks to direct calls

On 14/10/2019 14:22, David Sterba wrote: > +static int compression_decompress_bio(int type, struct list_head *ws, > + struct compressed_bio *cb) > +{ > + switch (type) { > + case BTRFS_COMPRESS_ZLIB: return zlib_decompress_bio(ws, cb); > + case BTRFS_COMPRESS_LZO: return lz

Re: [PATCH 01/15] btrfs: export compression and decompression callbacks

Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany (HRB 36809, AG Nürnberg) Geschäftsführer

[PATCH] btrfs: qgroup: Automatically remove qgroup item when dropping a subvolume

[BUG] When a subvolume is created, we automatically create a level 0 qgroup for it, but don't remove it when the subvolume is dropped. Although it's not a big deal, it can easily pollute the output of "btrfs qgroup show" and make it pretty annoying. [FIX] For btrfs_drop_snapshot(), if it's a vali

Re: [PATCH] btrfs-progs: warn users about the possible dangers of check --repair

On 16/10/2019 16:31, Nikolay Borisov wrote: [...] >> +printf("\tfsck successfully repair all types of filesystem >> corruption. Eg.\n"); >> +printf("\tsome other software or hardware bugs can fatally >> damage a volume.\n"); > > nit: The word 'other' here is redundant, no

Re: [PATCH] btrfs-progs: warn users about the possible dangers of check --repair

On 17/10/2019 08:40, Nikolay Borisov wrote: [...] >>  Agreed. -f will suffice (at least make it non-default) is a good fix. >>  But again as Qu pointed out our test cases will fail or old test case >>  with new progs will fail. > > They could be adjusted accordingly to always append the -f flag wh