Re: [PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-09 Thread Anand Jain
On 10/02/2021 04:30, Michal Rostecki wrote: From: Michal Rostecki This patch series adds a new raid1 read policy - roundrobin. For each request, it selects the mirror which has lower load than queue depth. Load is defined as the number of inflight requests + a penalty value (if the scheduled r

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
This is an active (but idle) system.journal file. That is, it's open but not being written to. I did a sync right before this: https://pastebin.com/jHh5tfpe And then: btrfs fi defrag -l 8M system.journal https://pastebin.com/Kq1GjJuh Looks like most of it was a no op. So it seems btrfs in this

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-02-09 Thread Anand Jain
On 10/02/2021 05:12, Michal Rostecki wrote: On Thu, Feb 04, 2021 at 08:30:01PM +0800, Anand Jain wrote: Hi Michal, Did you get any chance to run the evaluation with this patchset? Thanks, Anand Hi Anand, Yes, I tested your policies now. Sorry for late response. For the singlethreade

Re: "bad tree block start" when trying to mount on ARM

2021-02-09 Thread Qu Wenruo
On 2021/2/6 上午9:57, Erik Jensen wrote: On Wed, Feb 3, 2021 at 10:16 PM Erik Jensen wrote: On Sun, Jan 31, 2021 at 9:50 PM Su Yue wrote: On Mon 01 Feb 2021 at 10:35, Qu Wenruo wrote: On 2021/1/29 下午2:39, Erik Jensen wrote: On Mon, Jan 25, 2021 at 8:54 PM Erik Jensen wrote: On Wed, Jan 2

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-09 Thread Michał Mirosław
On Tue, Feb 09, 2021 at 09:30:40PM +0100, Michal Rostecki wrote: [...] > For the array with 3 HDDs, not adding any penalty resulted in 409MiB/s > (429MB/s) performance. Adding the penalty value 1 resulted in a > performance drop to 404MiB/s (424MB/s). Increasing the value towards 10 > was making th

Re: [PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-09 Thread Michał Mirosław
On Tue, Feb 09, 2021 at 09:30:38PM +0100, Michal Rostecki wrote: > From: Michal Rostecki > > Add the btrfs_check_mixed() function which checks if the filesystem has > the mixed type of devices (non-rotational and rotational). This > information is going to be used in roundrobin raid1 read policy.

Re: [PATCH] btrfs-progs: remove workaround for setting capabilities in the receive command

2021-02-09 Thread Su Yue
On Tue, Feb 9, 2021 at 7:53 PM wrote: > > From: Filipe Manana > > We had a few bugs on the kernel side of send/receive where capabilities > ended up being lost after receiving a send stream. They all stem from the > fact that the kernel used to send all xattrs before issuing the chown > command,

Re: [PATCH u-boot] fs: btrfs: do not fail when offset of a ROOT_ITEM is not -1

2021-02-09 Thread Qu Wenruo
On 2021/2/10 上午9:05, Marek Behun wrote: On Wed, 10 Feb 2021 08:09:14 +0800 Qu Wenruo wrote: On 2021/2/10 上午1:33, Marek Behún wrote: When the btrfs_read_fs_root() function is searching a ROOT_ITEM with location key offset other than -1, it currently fails via BUG_ON. The offset can have ot

Re: [PATCH u-boot] fs: btrfs: do not fail when offset of a ROOT_ITEM is not -1

2021-02-09 Thread Marek Behun
On Wed, 10 Feb 2021 08:09:14 +0800 Qu Wenruo wrote: > On 2021/2/10 上午1:33, Marek Behún wrote: > > When the btrfs_read_fs_root() function is searching a ROOT_ITEM with > > location key offset other than -1, it currently fails via BUG_ON. > > > > The offset can have other value than -1, though. Thi

Re: [PATCH u-boot 1/2] fs: btrfs: skip xattrs in directory listing

2021-02-09 Thread Qu Wenruo
On 2021/2/10 上午2:05, Marek Behún wrote: Skip xattrs in directory listing. U-Boot filesystem drivers do not list xattrs. Signed-off-by: Marek Behún Cc: David Sterba Cc: Qu Wenruo Cc: Tom Rini Reviewed-by: Qu Wenruo Thanks, Qu --- fs/btrfs/btrfs.c | 5 - 1 file changed, 4 insert

Re: [PATCH u-boot 2/2] fs: btrfs: change directory list output to be aligned as before

2021-02-09 Thread Qu Wenruo
On 2021/2/10 上午2:05, Marek Behún wrote: Since commit 325dd1f642dd ("fs: btrfs: Use btrfs_iter_dir() to ...") when btrfs is listing a directory, the output is not aligned: 15 Wed Sep 09 13:20:03 2020 boot.scr -> @/boot/boot.scr 0 Tue Feb 02 12:42:09 2021 @

Re: [PATCH u-boot] fs: btrfs: do not fail when offset of a ROOT_ITEM is not -1

2021-02-09 Thread Qu Wenruo
On 2021/2/10 上午1:33, Marek Behún wrote: When the btrfs_read_fs_root() function is searching a ROOT_ITEM with location key offset other than -1, it currently fails via BUG_ON. The offset can have other value than -1, though. This can happen for example if a subvolume is renamed: $ btrfs su

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
On Tue, Feb 9, 2021 at 11:13 AM Goffredo Baroncelli wrote: > > On 2/9/21 1:42 AM, Chris Murphy wrote: > > Perhaps. Attach strace to journald before --rotate, and then --rotate > > > > https://pastebin.com/UGihfCG9 > > I looked to this strace. > > in line 115: it is called a ioctl() > in line 123:

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-02-09 Thread Michal Rostecki
On Thu, Feb 04, 2021 at 08:30:01PM +0800, Anand Jain wrote: > > Hi Michal, > > Did you get any chance to run the evaluation with this patchset? > > Thanks, Anand > Hi Anand, Yes, I tested your policies now. Sorry for late response. For the singlethreaded test: [global] name=btrfs-raid1

[PATCH RFC 1/6] btrfs: Add inflight BIO request counter

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add a per-CPU inflight BIO counter to btrfs_device which stores the number of requests currently processed by the device. This information is going to be used in roundrobin raid1 read policy. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 11 +-- fs/btrfs

[PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add the btrfs_check_mixed() function which checks if the filesystem has the mixed type of devices (non-rotational and rotational). This information is going to be used in roundrobin raid1 read policy. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 44

[PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki This patch series adds a new raid1 read policy - roundrobin. For each request, it selects the mirror which has lower load than queue depth. Load is defined as the number of inflight requests + a penalty value (if the scheduled request is not local to the last processed requ

[PATCH RFC 2/6] btrfs: Store the last device I/O offset

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add an atomic field which stores the physical offset of the last I/O operation scheduled to the device. This information is going to be used to measure the locality of I/O requests. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 4 fs/btrfs/volumes.h | 1 +

[PATCH RFC 3/6] btrfs: Add stripe_physical function

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Move the calculation of the physical address for a stripe to the new function - stripe_physical(). It can be used by raid1 read policies to calculate the offset and select mirrors based on I/O locality. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 22 ++

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Chris Murphy
On Tue, Feb 9, 2021 at 12:45 PM Goffredo Baroncelli wrote: > > On 2/9/21 8:01 PM, Chris Murphy wrote: > > On Tue, Feb 9, 2021 at 11:13 AM Goffredo Baroncelli > > wrote: > >> > >> On 2/9/21 1:42 AM, Chris Murphy wrote: > >>> Perhaps. Attach strace to journald before --rotate, and then --rotate >

[PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add a new raid1 read policy `roundrobin`. For each read request, it selects the mirror which has lower load than queue depth and it starts iterating from the last used mirror (by the current CPU). Load is defined as the number of inflight requests + a potential penalty value

[PATCH RFC 5/6] btrfs: sysfs: Add directory for read policies

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Before this change, raid1 read policy could be selected by using the /sys/fs/btrfs/[fsid]/read_policy file. Change it to /sys/fs/btrfs/[fsid]/read_policies/policy. The motivation behing creating the read_policies directory is that the next changes and new read policies are

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Goffredo Baroncelli
On 2/9/21 8:01 PM, Chris Murphy wrote: On Tue, Feb 9, 2021 at 11:13 AM Goffredo Baroncelli wrote: On 2/9/21 1:42 AM, Chris Murphy wrote: Perhaps. Attach strace to journald before --rotate, and then --rotate https://pastebin.com/UGihfCG9 I looked to this strace. in line 115: it is called a

Re: [PATCH 1/2] btrfs-progs: corrupt generic item data with btrfs-corrupt-block

2021-02-09 Thread Boris Burkov
On Tue, Feb 09, 2021 at 06:22:47AM +, Sidong Yang wrote: > On Thu, Feb 04, 2021 at 12:09:31PM -0800, Boris Burkov wrote: > > Hi Boris, I have a question for this code. > > > btrfs-corrupt-block already has a mix of generic and specific corruption > > options, but currently lacks the capacity

Re: [PATCH 2/5] btrfs: initial fsverity support

2021-02-09 Thread Boris Burkov
On Fri, Feb 05, 2021 at 10:06:07AM +0200, Nikolay Borisov wrote: > > > On 5.02.21 г. 1:21 ч., Boris Burkov wrote: > > From: Chris Mason > > > > Add support for fsverity in btrfs. To support the generic interface in > > fs/verity, we add two new item types in the fs tree for inodes with > > veri

Re: is BTRFS_IOC_DEFRAG behavior optimal?

2021-02-09 Thread Goffredo Baroncelli
On 2/9/21 1:42 AM, Chris Murphy wrote: Perhaps. Attach strace to journald before --rotate, and then --rotate https://pastebin.com/UGihfCG9 I looked to this strace. in line 115: it is called a ioctl() in line 123: it is called a ioctl() However the two descriptors for which the defrag is invo

[PATCH u-boot 2/2] fs: btrfs: change directory list output to be aligned as before

2021-02-09 Thread Marek Behún
Since commit 325dd1f642dd ("fs: btrfs: Use btrfs_iter_dir() to ...") when btrfs is listing a directory, the output is not aligned: 15 Wed Sep 09 13:20:03 2020 boot.scr -> @/boot/boot.scr 0 Tue Feb 02 12:42:09 2021 @ 108 Tue Feb 02 12:54:04 2021 1.info Return

[PATCH u-boot 1/2] fs: btrfs: skip xattrs in directory listing

2021-02-09 Thread Marek Behún
Skip xattrs in directory listing. U-Boot filesystem drivers do not list xattrs. Signed-off-by: Marek Behún Cc: David Sterba Cc: Qu Wenruo Cc: Tom Rini --- fs/btrfs/btrfs.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/btrfs.c b/fs/btrfs/btrfs.c index 346b2c4

[PATCH u-boot] fs: btrfs: do not fail when offset of a ROOT_ITEM is not -1

2021-02-09 Thread Marek Behún
When the btrfs_read_fs_root() function is searching a ROOT_ITEM with location key offset other than -1, it currently fails via BUG_ON. The offset can have other value than -1, though. This can happen for example if a subvolume is renamed: $ btrfs subvolume create X && sync Create subvolume '.

[PATCH btrfs-progs] btrfs-progs: do not fail when offset of a ROOT_ITEM is not -1

2021-02-09 Thread Marek Behún
When the btrfs_read_fs_root() function is searching a ROOT_ITEM with location key offset other than -1, it currently fails via BUG_ON. The offset can have other value than -1, though. This can happen for example if a subvolume is renamed: $ btrfs subvolume create X && sync Create subvolume '.

[PATCH] btrfs-progs: remove workaround for setting capabilities in the receive command

2021-02-09 Thread fdmanana
From: Filipe Manana We had a few bugs on the kernel side of send/receive where capabilities ended up being lost after receiving a send stream. They all stem from the fact that the kernel used to send all xattrs before issuing the chown command, and the later clears any existing capabilities in a

Re: [PATCH 5/7] fsdax: Dedup file range to use a compare function

2021-02-09 Thread Ruan Shiyang
On 2021/2/9 下午5:34, Christoph Hellwig wrote: On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: The dax dedupe comparison need the iomap_ops pointer as argument, so my understanding is that we don't modify the argument list of generic_remap_file_range_prep(), but move its code into

Re: [PATCH 5/7] fsdax: Dedup file range to use a compare function

2021-02-09 Thread Christoph Hellwig
On Tue, Feb 09, 2021 at 05:15:13PM +0800, Ruan Shiyang wrote: > The dax dedupe comparison need the iomap_ops pointer as argument, so my > understanding is that we don't modify the argument list of > generic_remap_file_range_prep(), but move its code into > __generic_remap_file_range_prep() whose

Re: [PATCH 5/7] fsdax: Dedup file range to use a compare function

2021-02-09 Thread Ruan Shiyang
On 2021/2/8 下午11:19, Christoph Hellwig wrote: On Mon, Feb 08, 2021 at 01:09:22AM +0800, Shiyang Ruan wrote: With dax we cannot deal with readpage() etc. So, we create a funciton callback to perform the file data comparison and pass s/funciton/function/g +#define MIN(a, b) (((a) < (b)) ? (