[RFC PATCH 01/37] block: introduce bio_init_fields() helper

2021-01-19 Thread Chaitanya Kulkarni
There are several places in the file-system, block layer, device drivers where struct bio members such as bdev, sector, private, end io callback, io priority, write hints are initialized where we can use a helper function. This pach introduces a helper function which we use in the block lyaer code

[RFC PATCH 02/37] fs: use bio_init_fields in block_dev

2021-01-19 Thread Chaitanya Kulkarni
Signed-off-by: Chaitanya Kulkarni --- fs/block_dev.c | 17 + 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 3e5b02f6606c..44b992976ee5 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -239,12 +239,9 @@ __blkdev_direct_IO_sim

[RFC PATCH 00/37] block: introduce bio_init_fields()

2021-01-19 Thread Chaitanya Kulkarni
Hi, This is a *compile only RFC* which adds a generic helper to initialize the various fields of the bio that is repeated all the places in file-systems, block layer, and drivers. The new helper allows callers to initialize various members such as bdev, sector, private, end io callback, io priori

Re: "bad tree block start" when trying to mount on ARM

2021-01-19 Thread Erik Jensen
On Mon, Jan 18, 2021 at 4:12 AM Erik Jensen wrote: > > The offending system is indeed ARMv7 (specifically a Marvell ARMADA® > 388), but I believe the Broadcom BCM2835 in my Raspberry Pi is > actually ARMv6 (with hardware float support). Using NBD, I have verified that I receive the same error whe

[PATCH 03/13] btrfs: Fix function description format

2021-01-19 Thread Nikolay Borisov
This fixes following W=1 warnings: fs/btrfs/file-item.c:27: warning: Cannot understand * @inode: the inode we want to update the disk_i_size for on line 27 - I thought it was a doc line fs/btrfs/file-item.c:65: warning: Cannot understand * @inode - the inode we're modifying on line 65 - I t

[PATCH 02/13] btrfs: Fix parameter description of btrfs_add_extent_mapping

2021-01-19 Thread Nikolay Borisov
This fixes the following compiler warnings: fs/btrfs/extent_map.c:601: warning: Function parameter or member 'fs_info' not described in 'btrfs_add_extent_mapping' fs/btrfs/extent_map.c:601: warning: Function parameter or member 'em_tree' not described in 'btrfs_add_extent_mapping' fs/btrfs/exten

[PATCH 06/13] btrfs: Document now parameter of peek_discard_list

2021-01-19 Thread Nikolay Borisov
Fixes fs/btrfs/discard.c:203: warning: Function parameter or member 'now' not described in 'peek_discard_list' Signed-off-by: Nikolay Borisov --- fs/btrfs/discard.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/discard.c b/fs/btrfs/discard.c index 2b8383d41144..bfe53eb4c1f3 10064

[PATCH 13/13] lib/zstd: Convert constants to defines

2021-01-19 Thread Nikolay Borisov
Those constants are really used internally by zstd and including linux/zstd.h into users results in the following warnings: In file included from fs/btrfs/zstd.c:19: ./include/linux/zstd.h:798:21: warning: ‘ZSTD_skippableHeaderSize’ defined but not used [-Wunused-const-variable=] 798 | static c

[PATCH 12/13] btrfs: Fix parameter description for functions in extent_io.c

2021-01-19 Thread Nikolay Borisov
This makes the file W=1 clean and fixes the following warnings: fs/btrfs/extent_io.c:414: warning: Function parameter or member 'tree' not described in '__etree_search' fs/btrfs/extent_io.c:414: warning: Function parameter or member 'offset' not described in '__etree_search' fs/btrfs/extent_io.c

[PATCH 09/13] btrfs: Document btrfs_check_shared parameters

2021-01-19 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov --- fs/btrfs/backref.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index ef71aba5bc15..eca255432a59 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -1503,6 +1503,12 @@ int btrfs_find_all_roots(struct

[PATCH 08/13] btrfs: Fix description format of fs_info parameter of btrfs_wait_on_delayed_iputs

2021-01-19 Thread Nikolay Borisov
Fixes fs/btrfs/inode.c:3101: warning: Function parameter or member 'fs_info' not described in 'btrfs_wait_on_delayed_iputs' Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 5906b4267

[PATCH 10/13] btrfs: Fix parameter description of btrfs_inode_rsv_release/btrfs_delalloc_release_space

2021-01-19 Thread Nikolay Borisov
Fixes following warnings: fs/btrfs/delalloc-space.c:205: warning: Function parameter or member 'inode' not described in 'btrfs_inode_rsv_release' fs/btrfs/delalloc-space.c:205: warning: Function parameter or member 'qgroup_free' not described in 'btrfs_inode_rsv_release' fs/btrfs/delalloc-space.

[PATCH 11/13] btrfs: Fix parameter description in space-info.c

2021-01-19 Thread Nikolay Borisov
With these fixes space-info.c is clearn for W=1 warnings, namely the following ones are fixed: fs/btrfs/space-info.c:575: warning: Function parameter or member 'fs_info' not described in 'may_commit_transaction' fs/btrfs/space-info.c:575: warning: Function parameter or member 'space_info' not de

[PATCH 07/13] btrfs: Document fs_info in btrfs_rmap_block

2021-01-19 Thread Nikolay Borisov
Fixes fs/btrfs/block-group.c:1570: warning: Function parameter or member 'fs_info' not described in 'btrfs_rmap_block' Signed-off-by: Nikolay Borisov --- fs/btrfs/block-group.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 0886e81e5540

[PATCH 05/13] btrfs: Improve parameter description for __btrfs_write_out_cache

2021-01-19 Thread Nikolay Borisov
Fixes following W=1 warnings: fs/btrfs/free-space-cache.c:1317: warning: Function parameter or member 'root' not described in '__btrfs_write_out_cache' fs/btrfs/free-space-cache.c:1317: warning: Function parameter or member 'inode' not described in '__btrfs_write_out_cache' fs/btrfs/free-space-ca

Re: [PATCH v4 03/18] btrfs: introduce the skeleton of btrfs_subpage structure

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 06:54:28AM +0800, Qu Wenruo wrote: > On 2021/1/19 上午6:46, David Sterba wrote: > > On Sat, Jan 16, 2021 at 03:15:18PM +0800, Qu Wenruo wrote: > >> + return; > >> + > >> + subpage = (struct btrfs_subpage *)detach_page_private(page); > >> + ASSERT(subpage); > >> + k

Re: [PATCH v4 03/18] btrfs: introduce the skeleton of btrfs_subpage structure

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 04:51:45PM +0100, David Sterba wrote: > On Tue, Jan 19, 2021 at 06:54:28AM +0800, Qu Wenruo wrote: > > On 2021/1/19 上午6:46, David Sterba wrote: > > > On Sat, Jan 16, 2021 at 03:15:18PM +0800, Qu Wenruo wrote: > > >> +return; > > >> + > > >> +subpage =

[PATCH 01/13] btrfs: Document modified paramater of add_extent_mapping

2021-01-19 Thread Nikolay Borisov
Fixes fs/btrfs/extent_map.c:399: warning: Function parameter or member 'modified' not described in 'add_extent_mapping' Signed-off-by: Nikolay Borisov --- fs/btrfs/extent_map.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c index bd6229fb2b6f.

[PATCH 04/13] btrfs: Fix paramter description in delayed-ref.c functions

2021-01-19 Thread Nikolay Borisov
This fixes the following warnings: fs/btrfs/delayed-ref.c:80: warning: Function parameter or member 'fs_info' not described in 'btrfs_delayed_refs_rsv_release' fs/btrfs/delayed-ref.c:80: warning: Function parameter or member 'nr' not described in 'btrfs_delayed_refs_rsv_release' fs/btrfs/delayed

Re: [RFC PATCH 00/37] block: introduce bio_init_fields()

2021-01-19 Thread Josef Bacik
On 1/19/21 12:05 AM, Chaitanya Kulkarni wrote: Hi, This is a *compile only RFC* which adds a generic helper to initialize the various fields of the bio that is repeated all the places in file-systems, block layer, and drivers. The new helper allows callers to initialize various members such as

Re: [PATCH v3 1/4] btrfs: add read_policy latency

2021-01-19 Thread Josef Bacik
On 1/11/21 4:41 AM, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait-time experienced by the read IOs through the individual device. This patch obtains the historical read IO stats from the kernel block layer and calculates its average. Examp

Re: [PATCH RFC 4/4] btrfs: introduce new read_policy round-robin

2021-01-19 Thread Josef Bacik
On 1/11/21 4:41 AM, Anand Jain wrote: Add round-robin read policy to route the read IO to the next device in the round-robin order. The chunk allocation and thus the stripe-index follows the order of free space available on devices. So to make the round-robin effective it shall follow the devid o

Re: [PATCH v3 3/4] btrfs: introduce new read_policy device

2021-01-19 Thread Josef Bacik
On 1/11/21 4:41 AM, Anand Jain wrote: Read-policy type 'device' and device flag 'read-preferred': The read-policy type device picks the device(s) flagged as read-preferred for reading stripes of type raid1, raid10, raid1c3 and raid1c4. A system might contain SSD, nvme, iscsi, or san lun, and wh

Re: [PATCH v4 08/18] btrfs: introduce helper for subpage uptodate status

2021-01-19 Thread David Sterba
On Sat, Jan 16, 2021 at 03:15:23PM +0800, Qu Wenruo wrote: > This patch introduce the following functions to handle btrfs subpage > uptodate status: > - btrfs_subpage_set_uptodate() > - btrfs_subpage_clear_uptodate() > - btrfs_subpage_test_uptodate() > Those helpers can only be called when the ra

Re: [PATCH v3 2/4] btrfs: introduce new device-state read_preferred

2021-01-19 Thread Josef Bacik
On 1/11/21 4:41 AM, Anand Jain wrote: This is a preparatory patch and introduces a new device flag 'read_preferred', RW-able using sysfs interface. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH v4 16/18] btrfs: introduce btrfs_subpage for data inodes

2021-01-19 Thread David Sterba
On Sat, Jan 16, 2021 at 03:15:31PM +0800, Qu Wenruo wrote: > -void set_page_extent_mapped(struct page *page) > +int __must_check set_page_extent_mapped(struct page *page) We're not using the __must_check, errors from such functions need to be handled by default so I've dropped the attribute.

Re: [PATCH 01/13] btrfs: Document modified paramater of add_extent_mapping

2021-01-19 Thread Josef Bacik
On 1/19/21 7:26 AM, Nikolay Borisov wrote: Fixes fs/btrfs/extent_map.c:399: warning: Function parameter or member 'modified' not described in 'add_extent_mapping' Signed-off-by: Nikolay Borisov Subject should be 'parameter'. Thanks, Josef

Re: [PATCH 04/13] btrfs: Fix paramter description in delayed-ref.c functions

2021-01-19 Thread Josef Bacik
On 1/19/21 7:26 AM, Nikolay Borisov wrote: This fixes the following warnings: Subject should read 'parameter', thanks, Josef

Re: [PATCH 01/13] btrfs: Document modified paramater of add_extent_mapping

2021-01-19 Thread Josef Bacik
On 1/19/21 7:26 AM, Nikolay Borisov wrote: Fixes fs/btrfs/extent_map.c:399: warning: Function parameter or member 'modified' not described in 'add_extent_mapping' Signed-off-by: Nikolay Borisov --- fs/btrfs/extent_map.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/extent_ma

Re: [PATCH v4 02/18] btrfs: merge PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK into PAGE_START_WRITEBACK

2021-01-19 Thread Josef Bacik
On 1/16/21 2:15 AM, Qu Wenruo wrote: PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK are two macros used in __process_pages_contig(), to inform the function to clear page dirty and then set page writeback. However page write back and dirty are two conflict status (at least for sector size == PAGE_SIZE c

Re: [PATCH v4 02/18] btrfs: merge PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK into PAGE_START_WRITEBACK

2021-01-19 Thread Josef Bacik
On 1/16/21 2:15 AM, Qu Wenruo wrote: PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK are two macros used in __process_pages_contig(), to inform the function to clear page dirty and then set page writeback. However page write back and dirty are two conflict status (at least for sector size == PAGE_SIZE c

Re: [PATCH v4 01/18] btrfs: update locked page dirty/writeback/error bits in __process_pages_contig()

2021-01-19 Thread Josef Bacik
On 1/16/21 2:15 AM, Qu Wenruo wrote: When __process_pages_contig() get called for extent_clear_unlock_delalloc(), if we hit the locked page, only Private2 bit is updated, but dirty/writeback/error bits are all skipped. There are several call sites call extent_clear_unlock_delalloc() with @locked

Re: [PATCH v4 04/18] btrfs: make attach_extent_buffer_page() to handle subpage case

2021-01-19 Thread Josef Bacik
On 1/16/21 2:15 AM, Qu Wenruo wrote: For subpage case, we need to allocate new memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to return int To indicate allocation failure - Prealloc btrfs_subpage structure for alloc_extent_buffer() We don't want to call

Cannot mount BTRFS filesystem following power cycle

2021-01-19 Thread Provost, Simon
Hi, We're having an issue with our BTRFS file system. Our systems are being shut down by cutting power. Sometimes, when the system gets powered on, the BTRFS partition fails to mount. System reports kernel bug fs/btrfs/extent-tree.c:1183. When that occurs, we're able to mount it using both ro an

[PATCH 00/13] Make btrfs W=1 clean

2021-01-19 Thread Nikolay Borisov
Hello, This patch series aims to fix all current warnings produced by compiling btrfs with W=1. My hopes are with these additions W=1 can become a default build options for btrfs. With this series applied misc-next currently produces 1 genuine warning for an unused variable: fs/btrfs/zoned.c: In

Re: "bad tree block start" when trying to mount on ARM

2021-01-19 Thread Erik Jensen
On Mon, Jan 18, 2021 at 9:22 PM Erik Jensen wrote: > > On Mon, Jan 18, 2021 at 4:12 AM Erik Jensen wrote: > > > > The offending system is indeed ARMv7 (specifically a Marvell ARMADA® > > 388), but I believe the Broadcom BCM2835 in my Raspberry Pi is > > actually ARMv6 (with hardware float support

Re: [PATCH v4 04/18] btrfs: make attach_extent_buffer_page() to handle subpage case

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 04:54:28PM -0500, Josef Bacik wrote: > On 1/16/21 2:15 AM, Qu Wenruo wrote: > > +/* For rare cases where we need to pre-allocate a btrfs_subpage structure > > */ > > +static inline int btrfs_alloc_subpage(struct btrfs_fs_info *fs_info, > > +

Re: [PATCH 03/13] btrfs: Fix function description format

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 02:26:39PM +0200, Nikolay Borisov wrote: > This fixes following W=1 warnings: > > fs/btrfs/file-item.c:27: warning: Cannot understand * @inode: the inode we > want to update the disk_i_size for > on line 27 - I thought it was a doc line > fs/btrfs/file-item.c:65: warnin

Re: [PATCH 06/13] btrfs: Document now parameter of peek_discard_list

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 02:26:42PM +0200, Nikolay Borisov wrote: > Fixes fs/btrfs/discard.c:203: warning: Function parameter or member 'now' not > described in 'peek_discard_list' > > Signed-off-by: Nikolay Borisov > --- > fs/btrfs/discard.c | 1 + > 1 file changed, 1 insertion(+) > > diff --g

Re: [PATCH 00/13] Make btrfs W=1 clean

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 02:26:36PM +0200, Nikolay Borisov wrote: > Hello, > > This patch series aims to fix all current warnings produced by compiling btrfs > with W=1. My hopes are with these additions W=1 can become a default build > options for btrfs. Great, I like that, some of the warnigs ge

Re: [PATCH 07/13] btrfs: Document fs_info in btrfs_rmap_block

2021-01-19 Thread David Sterba
On Tue, Jan 19, 2021 at 02:26:43PM +0200, Nikolay Borisov wrote: > Fixes fs/btrfs/block-group.c:1570: warning: Function parameter or member > 'fs_info' not described in 'btrfs_rmap_block' > > Signed-off-by: Nikolay Borisov > --- > fs/btrfs/block-group.c | 1 + > 1 file changed, 1 insertion(+) >

Re: [RFC PATCH 00/37] block: introduce bio_init_fields()

2021-01-19 Thread Mike Snitzer
On Tue, Jan 19 2021 at 12:05am -0500, Chaitanya Kulkarni wrote: > Hi, > > This is a *compile only RFC* which adds a generic helper to initialize > the various fields of the bio that is repeated all the places in > file-systems, block layer, and drivers. > > The new helper allows callers to init

Re: [RFC][PATCH V5] btrfs: preferred_metadata: preferred device for metadata

2021-01-19 Thread Zygo Blaxell
On Sun, Jan 17, 2021 at 07:54:30PM +0100, Goffredo Baroncelli wrote: > > Hi all, > > This is an RFC; I wrote this patch because I find the idea interesting > even though it adds more complication to the chunk allocator. > > The basic idea is to store the metadata chunk in the fasters disks. > Th

Re: [RFC PATCH 04/37] btrfs: use bio_init_fields in volumes

2021-01-19 Thread Nikolay Borisov
On 19.01.21 г. 7:05 ч., Chaitanya Kulkarni wrote: > Signed-off-by: Chaitanya Kulkarni > --- > fs/btrfs/volumes.c | 4 +--- > 1 file changed, 1 insertion(+), 3 deletions(-) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index ee086fc56c30..836167212252 100644 > --- a/fs/btrfs/volum

Re: [PATCH v4 03/18] btrfs: introduce the skeleton of btrfs_subpage structure

2021-01-19 Thread Qu Wenruo
On 2021/1/20 上午12:06, David Sterba wrote: On Tue, Jan 19, 2021 at 04:51:45PM +0100, David Sterba wrote: On Tue, Jan 19, 2021 at 06:54:28AM +0800, Qu Wenruo wrote: On 2021/1/19 上午6:46, David Sterba wrote: On Sat, Jan 16, 2021 at 03:15:18PM +0800, Qu Wenruo wrote: + return; + +

Re: [PATCH v4 04/18] btrfs: make attach_extent_buffer_page() to handle subpage case

2021-01-19 Thread Qu Wenruo
On 2021/1/20 上午5:54, Josef Bacik wrote: On 1/16/21 2:15 AM, Qu Wenruo wrote: For subpage case, we need to allocate new memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to return int    To indicate allocation failure - Prealloc btrfs_subpage structure for allo

Re: [PATCH v3 1/4] btrfs: add read_policy latency

2021-01-19 Thread Anand Jain
On 20/1/21 3:36 am, Josef Bacik wrote: On 1/11/21 4:41 AM, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait-time experienced by the read IOs through the individual device. This patch obtains the historical read IO stats from the kernel block

Re: [PATCH RFC 4/4] btrfs: introduce new read_policy round-robin

2021-01-19 Thread Anand Jain
On 20/1/21 3:41 am, Josef Bacik wrote: On 1/11/21 4:41 AM, Anand Jain wrote: Add round-robin read policy to route the read IO to the next device in the round-robin order. The chunk allocation and thus the stripe-index follows the order of free space available on devices. So to make the round-ro

Re: received uuid not set btrfs send/receive

2021-01-19 Thread Anders Halman
Sorry for answering so late. I compiled a more recent (v5.9) version of btrfs-progs first. The unpacking/receiving took about 2 days on the raspberry. $ nohup btrfs receive -v -f splitter . & $ tail -n1 nohup.out ERROR: short read from stream: expected 49233 read 44997 thanks to your -v suggest

Re: [RFC PATCH 00/37] block: introduce bio_init_fields()

2021-01-19 Thread Chaitanya Kulkarni
On 1/18/21 21:06, Chaitanya Kulkarni wrote: > Hi, > > This is a *compile only RFC* which adds a generic helper to initialize > the various fields of the bio that is repeated all the places in > file-systems, block layer, and drivers. > > The new helper allows callers to initialize various members s

[PATCH v4 1/3] btrfs: add read_policy latency

2021-01-19 Thread Anand Jain
The read policy type latency routes the read IO based on the historical average wait-time experienced by the read IOs through the individual device. This patch obtains the historical read IO stats from the kernel block layer and calculates its average. Example usage: echo "latency" > /sys/fs/btrf

[PATCH v4 0/3] btrfs: read_policy types latency, device and round-robin

2021-01-19 Thread Anand Jain
v4: Add rb from Josef in patch 1 and 3. In patch 1/3, use fs_info instead of device->fs_devices->fs_info. Drop round-robin policy because my workload (fio random) shows no performance gains due to fewer merges at the block layer. v3: The block layer commit 0d02129e76ed (block: merge struct block_

[PATCH v4 3/3] btrfs: introduce new read_policy device

2021-01-19 Thread Anand Jain
Read-policy type 'device' and device flag 'read-preferred': The read-policy type device picks the device(s) flagged as read-preferred for reading stripes of type raid1, raid10, raid1c3 and raid1c4. A system might contain SSD, nvme, iscsi, or san lun, and which are all a non-rotational device, so

[PATCH v4 2/3] btrfs: introduce new device-state read_preferred

2021-01-19 Thread Anand Jain
This is a preparatory patch and introduces a new device flag 'read_preferred', RW-able using sysfs interface. Signed-off-by: Anand Jain --- v4: - v2: C style fixes. Drop space in between '! test_bit' and extra lines after it. fs/btrfs/sysfs.c | 53 +