Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-08 Thread David Sterba
st for the > > new snapshots inherigin the toplevel subvolume. Lockup. > > > > It gets stuck in a snapshot call with the follwin stacktrace > > > > [<0>] btrfs_tree_read_lock+0xf3/0x150 [btrfs] > > [<0>] btrfs_qgroup_trace_subtree+0x280/0x7b0 [btrfs] >

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-07 Thread Qu Wenruo
;> problem. >> >> Later patches will just keep tweaking qgroup to without affecting any >> other parts mostly. >> >> So I'm fine if you decide to pull it out for now. > > I've adapted a stress tests that unpacks a large tarball, snaphosts > every 20 seconds,

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-07 Thread David Sterba
ll it out for now. I've adapted a stress tests that unpacks a large tarball, snaphosts every 20 seconds, deletes a random snapshot every 50 seconds, deletes file from the original subvolume, now enhanced with qgroups just for the new snapshots inherigin the toplevel subvolume. Lockup. It gets stuck in

Re: [PATCH 2/8] btrfs: extent-tree: Open-code process_func in __btrfs_mod_ref

2018-12-07 Thread Nikolay Borisov
ned-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent-tree.c | 33 ++--- > 1 file changed, 18 insertions(+), 15 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index ea2c3d5220f0..ea68d288d761 100644 &g

[PATCH v2] Btrfs: use generic_remap_file_range_prep() for cloning and deduplication

2018-12-07 Thread fdmanana
From: Filipe Manana Since cloning and deduplication are no longer Btrfs specific operations, we now have generic code to handle parameter validation, compare file ranges used for deduplication, clear capabilities when cloning, etc. This change makes Btrfs use it, eliminating a lot of code

Re: [PATCH 05/10] btrfs: introduce delayed_refs_rsv

2018-12-07 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > From: Josef Bacik > > Traditionally we've had voodoo in btrfs to account for the space that > delayed refs may take up by having a global_block_rsv. This works most > of the time, except when it doesn't. We've had issues re

[PATCH] Btrfs: use generic_remap_file_range_prep() for cloning and deduplication

2018-12-07 Thread fdmanana
From: Filipe Manana Since cloning and deduplication are no longer Btrfs specific operations, we now have generic code to handle parameter validation, compare file ranges used for deduplication, clear capabilities when cloning, etc. This change makes Btrfs use it, eliminating a lot of code

[PATCH] Btrfs: scrub, move setup of nofs contexts higher in the stack

2018-12-07 Thread fdmanana
pplies on top of: Btrfs: fix deadlock with memory reclaim during scrub fs/btrfs/scrub.c | 34 ++ 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index bbd1b36f4918..f996f4064596 100644 --- a/fs/btrfs/scrub.c ++

Re: [PATCH 04/10] btrfs: only track ref_heads in delayed_ref_updates

2018-12-07 Thread Nikolay Borisov
thing along the lines of: In addition to using this number to limit the number of delayed refs run, a future patch is also going to use it to calculate the amount of space required for delayed refs space reservation. > > Reviewed-by: Nikolay Borisov > Signed-off-by: Josef Bacik >

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-07 Thread Nikolay Borisov
On 7.12.18 г. 9:09 ч., Nikolay Borisov wrote: > > > On 6.12.18 г. 19:54 ч., David Sterba wrote: >> On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: >>> >>> >>> On 3.12.18 г. 17:20 ч., Josef Bacik wrote: Now with the delayed_refs_rsv we can now know exactly how much

Re: HELP unmountable partition after btrfs balance to RAID0

2018-12-07 Thread Duncan
> A mount attempt of the array fails with following error code: > > btrfs recover yields roughly 1.6 out of 4 TB. [Just another btrfs user and list regular, not a dev. A dev may reply to your specific case, but meanwhile, for next time...] That shouldn't be a problem. Because

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
On 6.12.18 г. 19:54 ч., David Sterba wrote: > On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: >> >> >> On 3.12.18 г. 17:20 ч., Josef Bacik wrote: >>> Now with the delayed_refs_rsv we can now know exactly how much pending >>> delayed refs space we need. This means we can

Re: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges of "subvolume list/show"

2018-12-06 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 02:24:41PM +0900, Misono Tomohiro wrote: > Hello, > > This is basically the resend of > "[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the > root privileges of them" [1] > which I submitted in June.

Re: BTRFS RAID filesystem unmountable

2018-12-06 Thread Qu Wenruo
oblem is more serious, some chunk tree blocks are not even inside system chunk range, no wonder it fails to mount. To confirm it, you could run "btrfs ins dump-tree -b 17725903077376 " and paste the output. But I don't have any clue. My guess is some kernel problem related to new chu

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-06 Thread Qu Wenruo
On 2018/12/7 上午3:35, David Sterba wrote: > On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: >> On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: >>> This patchset can be fetched from github: >>> https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased >>> >>>

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-06 Thread David Sterba
On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > This patchset can be fetched from github: > > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased > > > > Which is based on v4.20-rc1. > > Thanks,

Re: [PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 12:12:21PM -0500, Josef Bacik wrote: > From: Josef Bacik > > With my delayed refs patches in place we started seeing a large amount > of aborts in __btrfs_free_extent > > BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-12-06 Thread David Sterba
s > > > being deleted, so mark the root when we are deleting it and make sure we > > > complain about it when it happens. > > > > > > Signed-off-by: Josef Bacik > > > --- > > > fs/btrfs/ctree.c | 3 +++ > > > fs/btrfs/ctree.h

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread David Sterba
On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: > > > On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > > Now with the delayed_refs_rsv we can now know exactly how much pending > > delayed refs space we need. This means we can drastically simplify > > IMO it will be helpful if

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
ond that amount then we > know it's time to commit the transaction and stop any more delayed refs > from being generated. > > Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov > --- > fs/btrfs/ctree.h | 2 +- > fs/btrfs/extent-tree.c | 48 ++--

Re: [PATCH 09/10] btrfs: don't run delayed refs in the end transaction logic

2018-12-06 Thread Nikolay Borisov
his problem solved. Rip out all of this code as > it is no longer needed. > > Signed-off-by: Josef Bacik > --- > fs/btrfs/transaction.c | 38 -- > 1 file changed, 38 deletions(-) > > diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transac

Re: [PATCH 0/3] btrfs: use offset_in_page and PAGE_ALIGNED

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 03:23:02PM +0100, Johannes Thumshirn wrote: > Use the offset_in_page() and PAGE_ALIGNED() macros instead of open-coding them > throughout btrfs. > > This series also includes a patch for 'make coccicheck' which is marked as an > RFC and I've CCed Julia in th

Re: [PATCH 06/10] btrfs: update may_commit_transaction to use the delayed refs rsv

2018-12-06 Thread Nikolay Borisov
ecision. > > Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov However, look below for one suggestion: > --- > fs/btrfs/extent-tree.c | 24 +++- > 1 file changed, 15 insertions(+), 9 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c

Re: [PATCH 02/10] btrfs: add cleanup_ref_head_accounting helper

2018-12-06 Thread Nikolay Borisov
f_cleanup mean that every time the last reference for a block was freed we were leaking bytes in total_bytes_pinned? Shouldn't this have lead to eventually total_bytes_pinned dominating the usage in a space_info ? Codewise lgtm: Reviewed-by: Nikolay Borisov > -

Re: [PATCH 01/10] btrfs: add btrfs_delete_ref_head helper

2018-12-06 Thread Nikolay Borisov
Borisov > --- > fs/btrfs/delayed-ref.c | 14 ++ > fs/btrfs/delayed-ref.h | 3 ++- > fs/btrfs/extent-tree.c | 22 +++--- > 3 files changed, 19 insertions(+), 20 deletions(-) > > diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-06 Thread Goldwyn Rodrigues
On 11:07 06/12, Johannes Thumshirn wrote: > On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > > This is a support for DAX in btrfs. I understand there have been > > previous attempts at it. However, I wanted to make sure copy-on-write > > (COW) works on dax as well. >

Re: [PATCH 07/10] dax: export functions for use with btrfs

2018-12-06 Thread Goldwyn Rodrigues
gt; needs first. This is not worth with btrfs. With non-page aligned I/O on btrfs, we need to copy the first/last page of the extents for CoW. So, we would end up using the exported functions anyways. Believe me, I have spent some time getting btrfs iomap compatible before giving up. The problems are btrfs needs

HELP unmountable partition after btrfs balance to RAID0

2018-12-06 Thread Thomas Mohr
Dear developers of BTRFS, we have a problem. We wanted to convert a file system to a RAID0 with two partitions. Unfortunately we had to reboot the server during the balance operation before it could complete. Now following happens: A mount attempt of the array fails with following error

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-06 Thread Johannes Thumshirn
On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > This is a support for DAX in btrfs. I understand there have been > previous attempts at it. However, I wanted to make sure copy-on-write > (COW) works on dax as well. > > Before I present this to the FS folks I wanted to run this thro

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Robert White
On 12/5/18 9:37 PM, Jeff Mahoney wrote: The high level idea that Jan Kara and I came up with in our conversation at Labs conf is pretty expensive.  We'd need to set a flag that pauses new page faults, set the WP bit on affected ranges, do the snapshot, commit, clear the flag, and wake up the

[PATCH 4/8] btrfs: delayed-ref: Use btrfs_ref to refactor btrfs_add_delayed_data_ref()

2018-12-05 Thread Qu Wenruo
Just like btrfs_add_delayed_tree_ref(), use btrfs_ref to refactor btrfs_add_delayed_data_ref(). Signed-off-by: Qu Wenruo --- fs/btrfs/delayed-ref.c | 19 +-- fs/btrfs/delayed-ref.h | 8 +++- fs/btrfs/extent-tree.c | 24 +++- 3 files changed, 27

[PATCH 8/8] btrfs: extent-tree: Use btrfs_ref to refactor btrfs_free_extent()

2018-12-05 Thread Qu Wenruo
Similar to btrfs_inc_extent_ref(), just use btrfs_ref to replace the long parameter list and the confusing @owner parameter. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 5 +--- fs/btrfs/extent-tree.c | 53 ++ fs/btrfs/file.c| 23

[PATCH 2/8] btrfs: extent-tree: Open-code process_func in __btrfs_mod_ref

2018-12-05 Thread Qu Wenruo
The process_func is never a function hook used anywhere else. Open code it to make later delayed ref refactor easier, so we can refactor btrfs_inc_extent_ref() and btrfs_free_extent() in different patches. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 33

[PATCH 7/8] btrfs: extent-tree: Use btrfs_ref to refactor btrfs_inc_extent_ref()

2018-12-05 Thread Qu Wenruo
Now we don't need to play the dirty game of reusing @owner for tree block level. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 6 ++--- fs/btrfs/extent-tree.c | 58 ++ fs/btrfs/file.c| 20 ++- fs/btrfs/inode.c | 10

[PATCH 5/8] btrfs: ref-verify: Use btrfs_ref to refactor btrfs_ref_tree_mod()

2018-12-05 Thread Qu Wenruo
It's a perfect match for btrfs_ref_tree_mod() to use btrfs_ref, as btrfs_ref describes a metadata/data reference update comprehensively. Now we have one less function use confusing owner/level trick. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 27 +++-- fs/btrfs/ref

[PATCH 3/8] btrfs: delayed-ref: Use btrfs_ref to refactor btrfs_add_delayed_tree_ref()

2018-12-05 Thread Qu Wenruo
-explaining without dirty level/owner trick, and provides the basis for later refactor. Signed-off-by: Qu Wenruo --- fs/btrfs/delayed-ref.c | 24 ++--- fs/btrfs/delayed-ref.h | 4 +--- fs/btrfs/extent-tree.c | 48 -- 3 files changed, 44 insertions

[PATCH 1/8] btrfs: delayed-ref: Introduce better documented delayed ref structures

2018-12-05 Thread Qu Wenruo
qgroup code, to record which tree is triggered this extent modification. This patch doesn't touch any code, but provides the basis for incoming refactors. Signed-off-by: Qu Wenruo --- fs/btrfs/delayed-ref.h | 109 + 1 file changed, 109 insertions(+) diff

[PATCH 6/8] btrfs: extent-tree: Use btrfs_ref to refactor add_pinned_bytes()

2018-12-05 Thread Qu Wenruo
Since add_pinned_bytes() only needs to know if the extent is metadata and if it's a chunk tree extent, btrfs_ref is a perfect match for it, as we don't need various owner/level trick to determine extent type. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 26 ++ 1

[PATCH 0/8] btrfs: Refactor delayed ref parameter list

2018-12-05 Thread Qu Wenruo
c_extent_ref(trans, ); This should improve the code readability and make later code easier to write. Qu Wenruo (8): btrfs: delayed-ref: Introduce better documented delayed ref structures btrfs: extent-tree: Open-code process_func in __btrfs_mod_ref btrfs: delayed-ref: Use btr

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Jeff Mahoney
On 12/5/18 7:28 AM, Goldwyn Rodrigues wrote: This is a support for DAX in btrfs. I understand there have been previous attempts at it. However, I wanted to make sure copy-on-write (COW) works on dax as well. Before I present this to the FS folks I wanted to run this through the btrfs. Even

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Jeff Mahoney
On 12/5/18 8:03 AM, Qu Wenruo wrote: On 2018/12/5 下午8:28, Goldwyn Rodrigues wrote: This is a support for DAX in btrfs. I understand there have been previous attempts at it. However, I wanted to make sure copy-on-write (COW) works on dax as well. Before I present this to the FS folks I wanted

Re: [PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-05 Thread Filipe Manana
On Wed, Dec 5, 2018 at 5:14 PM Josef Bacik wrote: > > From: Josef Bacik > > With my delayed refs patches in place we started seeing a large amount > of aborts in __btrfs_free_extent > > BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root > 35964

Re: btrfs progs always assume devid 1?

2018-12-05 Thread Austin S. Hemmelgarn
devices 1 FS bytes used 36.63GiB devid2 size 50.00GiB used 45.06GiB path /dev/mapper/vg-p1 And all the operations of btrfs-progs now fail to work in their default invocation, such as: # btrfs fi resize max . Resize '.' of 'max' ERROR: unable to resize '.': No such device [768813.414821

btrfs progs always assume devid 1?

2018-12-05 Thread Roman Mamedov
2 size 50.00GiB used 45.06GiB path /dev/mapper/vg-p1 And all the operations of btrfs-progs now fail to work in their default invocation, such as: # btrfs fi resize max . Resize '.' of 'max' ERROR: unable to resize '.': No such device [768813.414821] BTRFS info (device dm-5): resizer unable

[PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-05 Thread Josef Bacik
From: Josef Bacik With my delayed refs patches in place we started seeing a large amount of aborts in __btrfs_free_extent BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root 35964 owner 1 offset 0 Call Trace: ? btrfs_merge_delayed_refs+0xaf/0x340

Re: [RFC PATCH] btrfs: Remove __extent_readpages

2018-12-05 Thread Josef Bacik
On Mon, Dec 03, 2018 at 12:25:32PM +0200, Nikolay Borisov wrote: > When extent_readpages is called from the generic readahead code it first > builds a batch of 16 pages (which might or might not be consecutive, > depending on whether add_to_page_cache_lru failed) and submits them to >

Re: [PATCHv3] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-12-05 Thread David Sterba
On Wed, Nov 21, 2018 at 05:10:52PM +0200, Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >

Re: [PATCH 01/10] btrfs: create a mount option for dax

2018-12-05 Thread Adam Borowski
On Wed, Dec 05, 2018 at 02:43:03PM +0200, Nikolay Borisov wrote: > One question below though . > > > +++ b/fs/btrfs/super.c > > @@ -739,6 +741,17 @@ int btrfs_parse_options(struct btrfs_fs_info *info, > > char *options, > > c

Re: [PATCH 07/10] dax: export functions for use with btrfs

2018-12-05 Thread Christoph Hellwig
If you want to export these at all they have to be EXPORT_SYMBOL_GPL. But I'd really like to avoid seeing another duplicate DAX I/O path. Please try to adopt the existing iomap-based infrastructure for your needs first.

Re: [PATCH 1/3] btrfs: use offset_in_page instead of open-coding it

2018-12-05 Thread Nikolay Borisov
check. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/check-integrity.c | 12 +-- > fs/btrfs/compression.c | 2 +- > fs/btrfs/extent_io.c | 53 > +++++- > fs/btrfs/file.c

Re: [PATCH 2/3] btrfs: use PAGE_ALIGNED instead of open-coding it

2018-12-05 Thread Nikolay Borisov
(). > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/check-integrity.c | 8 > fs/btrfs/compression.c | 2 +- > fs/btrfs/inode.c | 2 +- > 3 files changed, 6 insertions(+), 6 deletions(-) > > diff --git a/fs/btrfs/

[PATCH 1/3] btrfs: use offset_in_page instead of open-coding it

2018-12-05 Thread Johannes Thumshirn
Constructs like 'var & (PAGE_SIZE - 1)' or 'var & ~PAGE_MASK' can denote an offset into a page. So replace them by the offset_in_page() macro instead of open-coding it if they're not used as an alignment check. Signed-off-by: Johannes Thumshirn --- fs/btrfs/check-integrit

[PATCH 0/3] btrfs: use offset_in_page and PAGE_ALIGNED

2018-12-05 Thread Johannes Thumshirn
Use the offset_in_page() and PAGE_ALIGNED() macros instead of open-coding them throughout btrfs. This series also includes a patch for 'make coccicheck' which is marked as an RFC and I've CCed Julia in the hoping to get input from her. Johannes Thumshirn (3): btrfs: use offset_in_page instead

[PATCH 2/3] btrfs: use PAGE_ALIGNED instead of open-coding it

2018-12-05 Thread Johannes Thumshirn
When using a 'var & (PAGE_SIZE - 1)' construct one is checking for a page alignment and thus should use the PAGE_ALIGNED() macro instead of open-coding it. Convert all open-coded occurrences of PAGE_ALIGNED(). Signed-off-by: Johannes Thumshirn --- fs/btrfs/check-integrity.c | 8

Btrfs progs release 4.19.1

2018-12-05 Thread David Sterba
Hi, btrfs-progs version 4.19.1 have been released. There are build fixes, minor update to libbtrfsutil and documentation updates. Changes since 4.19.1-rc1: fix typos in CHANGES Changes: * build fixes * big-endian builds fail due to bswap helper clashes * 'swap' macro is too generic

Re: [PATCH v2 07/13] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-05 Thread Qu Wenruo
On 2018/12/5 下午9:40, David Sterba wrote: > On Wed, Dec 05, 2018 at 02:40:12PM +0800, Qu Wenruo wrote: >> GCC 8.2.1 will report the following warning with "make W=1": >> >> ctree.c: In function 'btrfs_next_sibling_tree_block': >> ctree.c:2990:21: warning: 'slot' may be used uninitialized in

Re: [PATCH 07/10] dax: export functions for use with btrfs

2018-12-05 Thread Johannes Thumshirn
On 05/12/2018 13:28, Goldwyn Rodrigues wrote: [...] > -static void *grab_mapping_entry(struct xa_state *xas, > +void *grab_mapping_entry(struct xa_state *xas, > struct address_space *mapping, unsigned long size_flag) > { > unsigned long index = xas->xa_index; > @@ -531,6

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Adam Borowski
On Wed, Dec 05, 2018 at 06:28:25AM -0600, Goldwyn Rodrigues wrote: > This is a support for DAX in btrfs. Yay! > I understand there have been previous attempts at it. However, I wanted > to make sure copy-on-write (COW) works on dax as well. btrfs' usual use of CoW and DAX are t

Re: [PATCH 06/10] btrfs: dax write support

2018-12-05 Thread Johannes Thumshirn
+= done; > + if (iocb->ki_pos > i_size_read(inode)) > + i_size_write(inode, iocb->ki_pos); > + } > + > + btrfs_delalloc_release_extents(BTRFS_I(inode), count, false); > +out2: out? > + if (count - done > 0) > +

Re: [PATCH v2 07/13] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-05 Thread David Sterba
On Wed, Dec 05, 2018 at 02:40:12PM +0800, Qu Wenruo wrote: > GCC 8.2.1 will report the following warning with "make W=1": > > ctree.c: In function 'btrfs_next_sibling_tree_block': > ctree.c:2990:21: warning: 'slot' may be used uninitialized in this function > [-Wmaybe-uninitialized] >

Re: [PATCH 03/10] btrfs: dax: read zeros from holes

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/dax.c | 7 ++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/dax.c b/fs/btrfs/dax.c > index d614bf

Re: [PATCH 02/10] btrfs: basic dax read

2018-12-05 Thread Johannes Thumshirn
On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues Can you explain why we can't use th dax_iomap_rw() interface like XFS or EXT4? [...] > +static ssize_t em_dax_rw(struct inode *inode, struct extent_map *em, u64 pos, > +

Re: [PATCH 02/10] btrfs: basic dax read

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/Makefile | 1 + > fs/btrfs/ctree.h | 5 > fs/btrfs/dax.c| 68 > +++ &

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Qu Wenruo
On 2018/12/5 下午8:28, Goldwyn Rodrigues wrote: > This is a support for DAX in btrfs. I understand there have been > previous attempts at it. However, I wanted to make sure copy-on-write > (COW) works on dax as well. > > Before I present this to the FS folks I wanted to r

Re: [PATCH 01/10] btrfs: create a mount option for dax

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Also, set the inode->i_flags to S_DAX > > Signed-off-by: Goldwyn Rodrigues Reviewed-by: Nikolay Borisov One question below though . > --- > fs/btrfs/ctree.h | 1 + > fs/btrfs/ioc

Re: [PATCH 01/10] btrfs: create a mount option for dax

2018-12-05 Thread Johannes Thumshirn
On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Also, set the inode->i_flags to S_DAX > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/ctree.h | 1 + > fs/btrfs/ioctl.c | 5 - > fs/btrfs/super.c | 15 +++ >

[PATCH 10/10] btrfs: dax mmap write

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Create a page size extent and copy the contents of the original extent into the new one, and present to user space as the page to write. Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/dax.c | 39 +++ 1 file changed, 39 insertions

[PATCH 06/10] btrfs: dax write support

2018-12-05 Thread Goldwyn Rodrigues
that, keep a references of the first and last extent (if required) and then perform allocations. If the "pos" or "end" is not aligned, copy the data from first and last extent respectively. Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/ctree.h | 1 + fs

[PATCH 09/10] btrfs: dax support for cow_page/mmap_private and shared

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/dax.c | 25 - 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/dax.c b/fs/btrfs/dax.c index 88017f8799d1..6d68d39cc5da 100644 --- a/fs/btrfs/dax.c +++ b/fs/btrfs/dax.c

[PATCH 07/10] dax: export functions for use with btrfs

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues These functions are required for btrfs dax support. Signed-off-by: Goldwyn Rodrigues --- fs/dax.c| 35 --- include/linux/dax.h | 16 2 files changed, 40 insertions(+), 11 deletions(-) diff --git a/fs/dax.c b

[PATCH 03/10] btrfs: dax: read zeros from holes

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/dax.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/dax.c b/fs/btrfs/dax.c index d614bf73bf8e..5a297674adec 100644 --- a/fs/btrfs/dax.c +++ b/fs/btrfs/dax.c @@ -54,7 +54,12 @@ ssize_t

[PATCH 00/10] btrfs: Support for DAX devices

2018-12-05 Thread Goldwyn Rodrigues
This is a support for DAX in btrfs. I understand there have been previous attempts at it. However, I wanted to make sure copy-on-write (COW) works on dax as well. Before I present this to the FS folks I wanted to run this through the btrfs. Even though I wish, I cannot get it correct the first

[PATCH 01/10] btrfs: create a mount option for dax

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Also, set the inode->i_flags to S_DAX Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/ctree.h | 1 + fs/btrfs/ioctl.c | 5 - fs/btrfs/super.c | 15 +++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctre

[PATCH 02/10] btrfs: basic dax read

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/Makefile | 1 + fs/btrfs/ctree.h | 5 fs/btrfs/dax.c| 68 +++ fs/btrfs/file.c | 13 ++- 4 files changed, 86 insertions(+), 1 deletion(-) create mode

[PATCH 08/10] btrfs: dax add read mmap path

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/ctree.h | 1 + fs/btrfs/dax.c | 43 +++ fs/btrfs/file.c | 12 +++- 3 files changed, 55 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH 05/10] btrfs: Carve out btrfs_get_extent_map_write() out of btrfs_get_blocks_write()

2018-12-05 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues This makes btrfs_get_extent_map_write() independent of Direct I/O code. Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/inode.c | 40 +++- 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/fs

[PATCH v2] btrfs: Remove unnecessary code from __btrfs_rebalance

2018-12-05 Thread Nikolay Borisov
ving heavy use of the btree. Signed-off-by: Nikolay Borisov Suggested-by: Josef Bacik Reviewed-by: Josef Bacik --- Changes since v1: * Improved changelog by adding information about reduced runtimes and explaining where they would come from. I did measurements of btrfs balance with and without the

Re: BTRFS Mount Delay Time Graph

2018-12-04 Thread Nikolay Borisov
es around a minute to mount. >>> As a result of this, I suspect this will become an increasingly problem >>> for serious users of BTRFS as time goes on. I'm not complaining as I'm >>> not a contributor so I have no room to do so -- just shedding some light >>> on a

[PATCH v2 11/13] btrfs-progs: Introduce rescue.h to resolve missing-prototypes for chunk and super rescue

2018-12-04 Thread Qu Wenruo
0c0841ed..36e9e1277e40 100644 --- a/cmds-rescue.c +++ b/cmds-rescue.c @@ -26,15 +26,13 @@ #include "commands.h" #include "utils.h" #include "help.h" +#include "rescue.h" static const char * const rescue_cmd_group_usage[] = { "btrfs rescue [optio

[PATCH v2 03/13] btrfs-progs: Makefile.extrawarn: Don't warn on sign compare

2018-12-04 Thread Qu Wenruo
Under most case, we are just using 'int' for 'unsigned int', and doesn't care about the sign. The Wsign-compare is causing tons of false alerts. Suppressing it would make W=1 less noisy so we can focus on real problem, while still allow it in W=3 build. Signed-off-by: Qu Wenruo ---

[PATCH v2 09/13] btrfs-progs: Fix missing-prototypes warning caused by non-static functions

2018-12-04 Thread Qu Wenruo
Make the following functions static to avoid missing-prototypes warning: - btrfs.c::handle_special_globals() - check/mode-lowmem.c::repair_ternary_lowmem() - extent-tree.c::btrfs_search_overlap_extent() - free-space-tree.c::convert_free_space_to_bitmaps() -

[PATCH v2 12/13] btrfs-progs: Add utils.h include to solve missing-prototypes warning

2018-12-04 Thread Qu Wenruo
Prototypes for arg_strtou64() and lookup_path_rootid() are included in utils.c, resulting make W=1 warning for them. Just include that header to make W=1 happier. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- utils-lib.c | 1 + 1 file changed, 1 insertion(+) diff --git

[PATCH v2 04/13] btrfs-progs: Fix Wempty-body warning

2018-12-04 Thread Qu Wenruo
messages.h:49:24: warning: suggest braces around empty body in an 'if' statement [-Wempty-body] PRINT_TRACE_ON_ERROR;\ Just extra braces would solve the problem. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- messages.h | 15 ++- 1 file changed, 10

[PATCH v2 10/13] btrfs-progs: Move btrfs_check_nodesize to fsfeatures.c to fix missing-prototypes warning

2018-12-04 Thread Qu Wenruo
And fsfeatures.c is indeed a better location for that function. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- fsfeatures.c | 23 +++ utils.c | 23 --- 2 files changed, 23 insertions(+), 23 deletions(-) diff --git a/fsfeatures.c

[PATCH v2 08/13] btrfs-progs: Fix Wtype-limits warning

2018-12-04 Thread Qu Wenruo
The only hit is the following code: tlv_len = le16_to_cpu(tlv_hdr->tlv_len); if (tlv_type == 0 || tlv_type > BTRFS_SEND_A_MAX || tlv_len > BTRFS_SEND_BUF_SIZE) { error("invalid tlv in cmd tlv_type = %hu, tlv_len = %hu",

[PATCH v2 13/13] btrfs-progs: free-space-tree: Remove unsued function

2018-12-04 Thread Qu Wenruo
set_free_space_tree_thresholds() is never used, just remove it to solve the missing-prototypes warning from make W=1. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- free-space-tree.c | 29 - 1 file changed, 29 deletions(-) diff --git a/free-space-tree.c

[PATCH v2 07/13] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-04 Thread Qu Wenruo
GCC 8.2.1 will report the following warning with "make W=1": ctree.c: In function 'btrfs_next_sibling_tree_block': ctree.c:2990:21: warning: 'slot' may be used uninitialized in this function [-Wmaybe-uninitialized] path->slots[level] = slot; ~~~^~ The culprit is

[PATCH v2 05/13] btrfs-progs: Fix Wimplicit-fallthrough warning

2018-12-04 Thread Qu Wenruo
Although most fallthrough case is pretty obvious, we still need to teach the dumb compiler that it's an explicit fallthrough. Also reformat the code to use common indent. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- utils.c | 30 ++ 1 file changed, 22

[PATCH v2 06/13] btrfs-progs: Fix Wsuggest-attribute=format warning

2018-12-04 Thread Qu Wenruo
Add __attribute__ ((format (printf, 4, 0))) to fix the vprintf calling function. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- string-table.c | 1 + 1 file changed, 1 insertion(+) diff --git a/string-table.c b/string-table.c index 95833768960d..455285702d51 100644 ---

[PATCH v2 00/13] btrfs-progs: Make W=1 great (no "again")

2018-12-04 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/warning_fixes Which is based on v4.19 tag. This patchset will make "make W=1" reports no warning. This patch will first introduce fix to Makefile.extrawarn to make "cc-disable-warning"

[PATCH v2 01/13] btrfs-progs: Makefile.extrawarn: Import cc-disable-warning

2018-12-04 Thread Qu Wenruo
We imported cc-option but forgot to import cc-disable-warning. Fixes: b556a992c3ad ("btrfs-progs: build: allow to build with various compiler warnings") Signed-off-by: Qu Wenruo --- Makefile.extrawarn | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Makefile.e

[PATCH v2 02/13] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-12-04 Thread Qu Wenruo
From: Su Yanjun When using gcc8 + glibc 2.28.5 compiles utils.c, it complains as below: utils.c:852:45: warning: '%s' directive output may be truncated writing up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] snprintf(path, sizeof(path), "/dev/mapper/%s", name);

Re: [PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-04 Thread Qu Wenruo
he while loop for at least once, thus @slot should >> always be initialized. > > Harsh words for the compiler, and I say not deserved. The same code > pasted to kernel a built with the same version does not report the > warning, so it's apparently a missing annotation of BUG_ON in

Re: [PATCH 2/2] btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 05:22:19PM +0200, Nikolay Borisov wrote: > > @@ -3874,16 +3882,9 @@ int btrfs_scrub_dev(struct btrfs_fs_info *fs_info, > > u64 devid, u64 start, > > if (ret) { > > mutex_unlock(_info->scrub_lock); > >

Re: BTRFS Mount Delay Time Graph

2018-12-04 Thread Wilson, Ellis
pect this will become an increasingly problem >> for serious users of BTRFS as time goes on. I'm not complaining as I'm >> not a contributor so I have no room to do so -- just shedding some light >> on a problem that may deserve attention as filesystem sizes continue to >> grow.

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-12-04 Thread Josef Bacik
s the > > short (or long depending on how big the inode is) window where we think > > there are no more pending iputs when there really are some. > > > > Signed-off-by: Josef Bacik > > --- > > fs/btrfs/ctree.h | 4 +++- > > fs/btrfs/disk-i

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-12-04 Thread Josef Bacik
On Tue, Dec 04, 2018 at 11:21:14AM +0200, Nikolay Borisov wrote: > > > On 3.12.18 г. 18:06 ч., Josef Bacik wrote: > > The cleaner thread usually takes care of delayed iputs, with the > > exception of the btrfs_end_transaction_throttle path. The cleaner > > thread only gets woken up every 30

[Mount time bug bounty?] was: BTRFS Mount Delay Time Graph

2018-12-04 Thread Lionel Bouton
Le 03/12/2018 à 23:22, Hans van Kranenburg a écrit : > [...] > Yes, I think that's true. See btrfs_read_block_groups in extent-tree.c: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/btrfs/extent-tree.c#n9982 > > What the code is doin

Re: [PATCHv3] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-12-04 Thread Nikolay Borisov
On 21.11.18 г. 17:10 ч., Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >

Re: [PATCH 2/2] btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex

2018-12-04 Thread Nikolay Borisov
t the memory. And the device_list_mutex is held during superblock > commit, so this would cause a lockup. > > Move the alocation and initialization before any changes that require > the mutex. > > Signed-off-by: David Sterba > --- > fs/btrfs/scrub.c | 30 ++ >

  1   2   3   4   5   6   7   8   9   10   >