Re: [PATCH v2 0/3] Misc volume patch set part2

2018-08-15 Thread Anand Jain
On 08/15/2018 08:20 PM, David Sterba wrote: On Fri, Aug 10, 2018 at 01:53:18PM +0800, Anand Jain wrote: The patch set (2/3 and 3/3) adds helper function to deduce the num_device (which is the number of the devices when device is mounted, this excludes the seed device however includes the

Re: [PATCH 1/2] btrfs: assert for num_devices below 0

2018-08-15 Thread Anand Jain
On 08/15/2018 08:11 PM, David Sterba wrote: On Fri, Aug 10, 2018 at 01:53:20PM +0800, Anand Jain wrote: In preparation to add helper function to deduce the num_devices with replace running, use assert instead of bug_on and warn_on. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 4

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

2018-08-15 Thread David Sterba
Hi, so I've merged the first bath to devel. There was quite some whitespace damage and style issues, but I fixed that as I went through the pathes. I've left out all patches that introduce new options, this is the hard part and single letter options cannot be added just randomly. That's what led

Re: [PATCH] Btrfs: remove duplicated btrfs_release_path in btrfs_unlink_subvol

2018-08-15 Thread Liu Bo
On Wed, Aug 15, 2018 at 4:48 AM, David Sterba wrote: > On Wed, Aug 15, 2018 at 10:52:56AM +0800, Liu Bo wrote: >> On Tue, Aug 14, 2018 at 12:46:00PM +0200, David Sterba wrote: >> > On Tue, Aug 14, 2018 at 10:47:09AM +0800, Liu Bo wrote: >> > > The btrfs_release_path() is just useless as path is

[PATCH 6/7] btrfs: Remove unnecessary locking code in qgroup_rescan_leaf

2018-08-15 Thread Nikolay Borisov
In qgroup_rescan_leaf a copy is made of the target leaf by calling btrfs_clone_extent_buffer. The latter allocates a new buffer and attaches a new set of pages and copies the content of the source buffer. The new scratch buffer is only used to iterate it's items, it's not published anywhere and

[PATCH 7/7] btrfs: Remove special handling of EXTENT_BUFFER_UNMAPPED while freeing

2018-08-15 Thread Nikolay Borisov
Now that the whole of btrfs code has been audited for eb reference count management it's time to remove the hunk in free_extent_buffer that essentially considered the condition "eb->ref == 2 && EXTENT_BUFFER_DUMMY" to equal eb->ref = 1. Also remove the last location which takes an extra reference

[PATCH 5/7] btrfs: Remove extra reference count bumps in btrfs_compare_trees

2018-08-15 Thread Nikolay Borisov
When the 2 comparison treees roots are initialised they are private to the function and already have referencecounts of 1 each. There is no need to further increment the reference count since the cloned buffers are already accessed via struct btrfs_path. Eventually the 2 paths used for comparison

[PATCH 3/7] btrfs: Remove redundant extent_buffer_get in get_old_root

2018-08-15 Thread Nikolay Borisov
get_old_root used used only by btrfs_search_old_slot to initialise the path structure. The old root is always a cloned buffer (either via alloc dummy or via btrfs_clone_extent_buffer) and its reference count is 2 - 1 from allocation 1 from extent_buffer_get call in get_old_root. This latter

[PATCH 2/7] btrfs: Remove needless locking in iterate_inode_extrefs

2018-08-15 Thread Nikolay Borisov
In iterate_inode_exrefs the eb is cloned via btrfs_clone_extent_buffer which creates a private extent buffer with the dummy flag set and ref count of 1. Then this buffer is locked for reading and its ref count is incremented by 1. Finally it's fed to the passed iterate_irefs_t function. The actual

[PATCH 4/7] btrfs: Remove extraneous extent_buffer_get from tree_mod_log_rewind

2018-08-15 Thread Nikolay Borisov
When a rewound buffer is created it already has a ref count of 1 and the dummy flag set. Then another ref is taken bumping the count to 2. Finally when this buffer is released from btrfs_release_path the extra reference is decremented by the special handling code in free_extent_buffer. However,

[PATCH 1/7] btrfs: Remove needless locking in iterate_inode_refs

2018-08-15 Thread Nikolay Borisov
In iterate_inode_refs the eb is cloned via btrfs_clone_extent_buffer which creates a private extent buffer with the dummy flag set and ref count of 1. Then this buffer is locked for reading and its ref count is incremented by 1. Finally it's fed to the passed iterate_irefs_t function. The actual

[PATCH 0/7] eb reference count cleanups

2018-08-15 Thread Nikolay Borisov
Here is a series which simplifies the way eb are used in EXTENT_BUFFER_UNMAPPED context. The end goal was to remove the special "if we have ref count of 2 and EXTENT_BUFFER_UNMAPPED flag then act as if this is the last ref and free the buffer" case. To enable this the first 6 patches modify call

Re: [PATCH v5] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-15 Thread Qu Wenruo
On 2018/8/9 下午3:05, Misono Tomohiro wrote: > When qgroup is on, subvolume deletion does not remove qgroup items > of the subvolume (qgroup info, limit, relation) from quota tree and > they need to get removed manually by "btrfs qgroup destroy". > > Since level 0 qgroup cannot be used/inherited

Re: [PATCH v5] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-15 Thread Qu Wenruo
On 2018/8/15 下午9:06, David Sterba wrote: > On Thu, Aug 09, 2018 at 04:05:36PM +0900, Misono Tomohiro wrote: >> When qgroup is on, subvolume deletion does not remove qgroup items >> of the subvolume (qgroup info, limit, relation) from quota tree and >> they need to get removed manually by "btrfs

Re: [PATCH] Btrfs: clean up scrub is_dev_replace parameter

2018-08-15 Thread David Sterba
On Tue, Aug 14, 2018 at 11:09:52AM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > struct scrub_ctx has an ->is_dev_replace member, so there's no point in > passing around is_dev_replace where sctx is available. > > Signed-off-by: Omar Sandoval Reviewed-by: David Sterba There's still

Re: [PATCH v5] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-15 Thread David Sterba
On Thu, Aug 09, 2018 at 04:05:36PM +0900, Misono Tomohiro wrote: > When qgroup is on, subvolume deletion does not remove qgroup items > of the subvolume (qgroup info, limit, relation) from quota tree and > they need to get removed manually by "btrfs qgroup destroy". > > Since level 0 qgroup

Deadlock between dedup and rename

2018-08-15 Thread Zygo Blaxell
Every month or two I hit a btrfs deadlock like this: dedup and rsync are both operating on the same file when the filesystem locked up. The deadlock happens at the moment when rsync renames its temporary file (the dedup dst file) to replace the old version of the file (the dedup src file).

Re: [PATCH v2 0/3] Misc volume patch set part2

2018-08-15 Thread David Sterba
On Fri, Aug 10, 2018 at 01:53:18PM +0800, Anand Jain wrote: > The patch set (2/3 and 3/3) adds helper function to deduce the num_device > (which is the number of the devices when device is mounted, this excludes > the seed device however includes the replacing target). We need to know > the

Re: [PATCH 1/2] btrfs: assert for num_devices below 0

2018-08-15 Thread David Sterba
On Fri, Aug 10, 2018 at 01:53:20PM +0800, Anand Jain wrote: > In preparation to add helper function to deduce the num_devices with > replace running, use assert instead of bug_on and warn_on. > > Signed-off-by: Anand Jain > --- > fs/btrfs/volumes.c | 4 ++-- > 1 file changed, 2 insertions(+), 2

Re: [PATCH] Btrfs: remove duplicated btrfs_release_path in btrfs_unlink_subvol

2018-08-15 Thread David Sterba
On Wed, Aug 15, 2018 at 10:52:56AM +0800, Liu Bo wrote: > On Tue, Aug 14, 2018 at 12:46:00PM +0200, David Sterba wrote: > > On Tue, Aug 14, 2018 at 10:47:09AM +0800, Liu Bo wrote: > > > The btrfs_release_path() is just useless as path is only used in error > > > handling. > > > > Where is it

Re: How to ensure that a snapshot is not corrupted?

2018-08-15 Thread Austin S. Hemmelgarn
On 2018-08-10 06:07, Cerem Cem ASLAN wrote: Original question is here: https://superuser.com/questions/1347843 How can we sure that a readonly snapshot is not corrupted due to a disk failure? Is the only way calculating the checksums one on another and store it for further examination, or does

Re: [PATCH 2/2] Btrfs: sync log after logging new name

2018-08-15 Thread David Sterba
On Wed, Aug 15, 2018 at 09:46:13AM +0100, Filipe Manana wrote: > >> > I had in mind was some leftover from locking, error handling or state > >> > that blocked umount of 011. > >> > >> Dave, are you sending this in for 4.19? I don't see it in your first > >> pull request. > > In another thread,

Re: [PATCH v2] Btrfs: correctly calculate item size used when item key collision happends

2018-08-15 Thread Nikolay Borisov
On 15.08.2018 11:42, ethanwu wrote: > Item key collision is allowed for some item types, like dir item and > inode refs, but the overall item size is limited by the leafsize. > > item size(ins_len) passed from btrfs_insert_empty_items to > btrfs_search_slot already contains size of btrfs_item.

Re: [PATCH 2/2] Btrfs: sync log after logging new name

2018-08-15 Thread Filipe Manana
On Tue, Aug 14, 2018 at 11:53 PM, David Sterba wrote: > On Tue, Aug 14, 2018 at 12:04:05PM -0700, Omar Sandoval wrote: >> On Mon, Jun 18, 2018 at 01:06:16PM +0200, David Sterba wrote: >> > On Fri, Jun 15, 2018 at 05:19:07PM +0100, Filipe Manana wrote: >> > > On Fri, Jun 15, 2018 at 4:54 PM, David

[PATCH v2] Btrfs: correctly calculate item size used when item key collision happends

2018-08-15 Thread ethanwu
Item key collision is allowed for some item types, like dir item and inode refs, but the overall item size is limited by the leafsize. item size(ins_len) passed from btrfs_insert_empty_items to btrfs_search_slot already contains size of btrfs_item. When btrfs_search_slot reaches leaf, we'll see

[PATCH 0/3] Refactor delayed refs processing loop

2018-08-15 Thread Nikolay Borisov
Here is a small series which aims to rectify the eye sore that delayed refs processing loop currently is. In fact, it's actually 2 loops in the guise of a single 'while' construct. All in all this should bring no functional changes and I've verified this with multiple xfstest runs with no

[PATCH 1/3] btrfs: Factor out ref head locking code in __btrfs_run_delayed_refs

2018-08-15 Thread Nikolay Borisov
This is in preparation to refactor the giant loop in __btrfs_run_delayed_refs. As a first step define a new function which implements acquiring a reference to a btrfs_delayed_refs_head and use it. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 54

[PATCH 2/3] btrfs: Factor out loop processing all refs of a head

2018-08-15 Thread Nikolay Borisov
This patch introduces a new helper encompassing the implicit inner loop in __btrfs_run_delayed_refs which processes all the refs for a given head. The code is mostly copy/paste, the only difference is that if we detect a newer reference then -EAGAIN is returned so that callers can react correctly.

[PATCH 3/3] btrfs: refactor __btrfs_run_delayed_refs loop

2018-08-15 Thread Nikolay Borisov
Refactor the delayed refs loop by using the newly introduced btrfs_run_delayed_refs_for_head function. This greatly simplifies __btrfs_run_delayed_refs and makes it more obvious what is happening. We now have 1 loop which iterates the existing delayed_heads and then each selected ref head is

Re: List of known BTRFS Raid 5/6 Bugs?

2018-08-15 Thread Menion
I needed to resume scrub two times after an unclear shutdown (I was cooking and using too much electricity) and two times after a manual cancel, because I wanted to watch a 4k movie and the array performances were not enough with scrub active. Each time I resumed it, I checked also the status, and

[PATCH] btrfs: Always check nocow for quota enabled case to make sure we won't reserve unnecessary data space

2018-08-15 Thread Qu Wenruo
Commit c6887cd11149 ("Btrfs: don't do nocow check unless we have to") makes nocow check less frequent to improve performance. However for quota enabled case, such optimization could lead to extra unnecessary data reservation, which results failure for test case like btrfs/153 in fstests. Fix it