Re: [PATCH v2 1/3] btrfs: block-group: Refactor btrfs_read_block_groups()

2019-10-09 Thread Anand Jain
goto error; - } - Otherwise looks good. Reviewed-by: Anand Jain Thanks, Anand

Re: [PATCH v2 2/3] btrfs: disk-io: Remove unnecessary check before freeing chunk root

2019-10-09 Thread Anand Jain
On 8/10/19 12:49 PM, Qu Wenruo wrote: In free_root_pointers(), free_root_extent_buffers() has checked if the @root parameter is NULL. So there is no need checking chunk_root before freeing it. um.. Signed-off-by: Qu Wenruo --- fs/btrfs/disk-io.c | 3 +-- 1 file changed, 1 insertion(+), 2

[PATCH] btrfs: use bool argument in free_root_pointers()

2019-10-09 Thread Anand Jain
We don't need int argument bool shall do in free_root_pointers(). And rename the argument as it confused two people. Signed-off-by: Anand Jain --- fs/btrfs/disk-io.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c

Re: [PATCH v3 2/3] btrfs: block-group: Refactor btrfs_read_block_groups()

2019-10-09 Thread Anand Jain
() to replace open-coded feature check Signed-off-by: Qu Wenruo Reviewed-by: Johannes Thumshirn Reviewed-by: Anand Jain --- fs/btrfs/block-group.c | 215 + 1 file changed, 108 insertions(+), 107 deletions(-) diff --git a/fs/btrfs/block-group.c b/fs

Re: [PATCH v3 1/3] btrfs: block-group: Fix a memory leak due to missing btrfs_put_block_group()

2019-10-09 Thread Anand Jain
() to prevent memory leak. Fixes: 49303381f19a ("Btrfs: bail out if block group has different mixed flag") Signed-off-by: Qu Wenruo Reviewed-by: Anand Jain

[PATCH RESEND] btrfs-progs: btrfstune -M|m drop test_uuid_unique

2019-10-10 Thread Anand Jain
e changing the fsid of an unmounted image, so its ok to leave it to the users choice if the fsid is not unique, so that the image can be sent back the system where it was used with that fsid. So this patch drops the check test_uuid_unique() in btrfstune -M|m. Thanks. Signed-off-by: Anand Jain --

Re: [PATCH] btrfs: ioctl: Try to use btrfs_fs_info instead of *file

2019-10-10 Thread Anand Jain
igned-off-by: Marcos Paulo de Souza Thanks for the fix. Reviewed-by: Anand Jain

Re: [PATCH v3 3/3] btrfs: Introduce new incompat feature, BG_TREE, to speed up mount time

2019-10-14 Thread Anand Jain
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 044981cf6df9..1c5728e6a660 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1570,6 +1570,8 @@ struct btrfs_root *btrfs_get_fs_root(struct btrfs_fs_info *fs_info, if (location->objectid == BTRFS_FREE_SPACE_TREE_OB

Re: [PATCH v3] btrfs-progs: add verbose option to btrfs device scan

2019-10-14 Thread Anand Jain
On 10/14/19 11:24 PM, David Sterba wrote: On Tue, Oct 08, 2019 at 10:53:37AM +0800, Anand Jain wrote: On 10/8/19 1:41 AM, David Sterba wrote: On Wed, Oct 02, 2019 at 12:11:52PM +0800, Anand Jain wrote: To help debug device scan issues, add verbose option to btrfs device scan. The common

Re: [PATCH RESEND] btrfs-progs: btrfstune -M|m drop test_uuid_unique

2019-10-16 Thread Anand Jain
author there is no reasonable explanation why this check is there in the first place until now. IMO this patch is good integrate. Please let me know if otherwise. Thanks, Anand On 10/10/19 3:53 PM, Anand Jain wrote: It's common to copy/snapshot an OS image to run another instance of the

Re: [PATCH] btrfs-progs: warn users about the possible dangers of check --repair

2019-10-16 Thread Anand Jain
On 10/16/19 10:31 PM, Nikolay Borisov wrote: On 16.10.19 г. 17:05 ч., Johannes Thumshirn wrote: The manual page of btrfsck clearly states 'btrfs check --repair' is a dangerous operation. Although this warning is in place users do not read the manual page and/or are used to the behaviour of fs

Re: [PATCH v2 1/7] btrfs-progs: Refactor excluded extent functions to use fs_info

2019-10-16 Thread Anand Jain
On 10/8/19 12:49 PM, Qu Wenruo wrote: The following functions are just using @root to reach fs_info: - exclude_super_stripes - free_excluded_extents - add_excluded_extent Refactor them to use fs_info directly. Signed-off-by: Qu Wenruo Reviewed-by: Anand Jain

Re: [PATCH v2 2/7] btrfs-progs: Refactor btrfs_read_block_groups()

2019-10-16 Thread Anand Jain
On 10/8/19 12:49 PM, Qu Wenruo wrote: This patch does the following refactor: - Refactor parameter from @root to @fs_info - Refactor the large loop body into another function Now we have a helper function, read_one_block_group(), to handle block group cache and space info related routine.

Re: [PATCH v2 7/7] btrfs-progs: btrfstune: Allow to enable bg-tree feature offline

2019-10-16 Thread Anand Jain
Depending on the size of the FS the convert may take longer, further fatal error (power loss; pid kill) may leave the FS in a state where the bg items are in both extent-tree and bg-tree. The lessons which lead to the implementation of metadata_uuid fsid suggests, for conversions its bette

Re: [PATCH v2 3/7] btrfs-progs: Enable read-write ability for 'bg_tree' feature

2019-10-16 Thread Anand Jain
uo looks good. two nit below. Reviewed-by: Anand Jain --- ctree.h | 11 +++- disk-io.c | 20 +++ extent-tree.c | 152 +- 3 files changed, 168 insertions(+), 15 deletions(-) diff --git a/ctree.h b/ctree.h index 2899de358613..c2

Re: [PATCH v2 2/7] btrfs-progs: Refactor btrfs_read_block_groups()

2019-10-16 Thread Anand Jain
On 10/17/19 12:33 PM, Qu Wenruo wrote: On 2019/10/17 上午11:23, Anand Jain wrote: On 10/8/19 12:49 PM, Qu Wenruo wrote: This patch does the following refactor: - Refactor parameter from @root to @fs_info - Refactor the large loop body into another function    Now we have a helper function

Re: [PATCH] btrfs-progs: drop unique uuid test for btrfstune -M

2019-10-18 Thread Anand Jain
On 10/18/19 12:32 AM, David Sterba wrote: On Thu, Sep 12, 2019 at 08:45:50AM +0800, Anand Jain wrote: In case of vm guest images copied from the golden image there is no physical device or loop device or nbd device until its configured on the host when required, so check for duplicate fsid at

[RFC PATCH 03/14] btrfs-progs: migrate filesystem defragment to global verbose

2019-10-21 Thread Anand Jain
btrfs filesystem deframent already supports local sub-command verbose option, enable the same when the global verbose option is set. And as well make sure the same remains enabled at the local level. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/filesystem.c | 25

[RFC PATCH 00/14] btrfs-progs: global-verbose option

2019-10-21 Thread Anand Jain
logical-resolve --help -v verbose mode [2] btrfs send --help -v|--verbose enable verbose output to stderr, each occurrence of btrfs receive --help -v increase verbosity about performed action Anand Jain (14): btrfs-progs: add global verbose helper

[RFC PATCH 02/14] btrfs-progs: migrate subvolume delete to global verbose

2019-10-21 Thread Anand Jain
btrfs subvolume delete already supports verbose at the sub-command level, this patch restores same verbose which can be either enabled by the sub-command or from the top level command. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/subvolume.c | 35

[RFC PATCH 10/14] btrfs-progs: migrate restore to global verbose

2019-10-21 Thread Anand Jain
Command btrfs restore provides local verbose option, this patch makes it enable-able by using the global --verbose option as well. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/restore.c | 69 ++ 1 file changed, 31

[RFC PATCH 05/14] btrfs-progs: migrate balance status to global verbose

2019-10-21 Thread Anand Jain
Make sure top level verbose option can enable the blalance status subcommand's verbose option. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/balance.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/cmds/balance.c b/cmds/balance.c index 7e84efd

[RFC PATCH 08/14] btrfs-progs: migrate rescue super-recover to global verbose

2019-10-21 Thread Anand Jain
Now with this patch 'btrfs rescue super-recover' can show verbose output either by the top level --verbose option or by the sub-command -v option. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/rescue.c | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) di

[RFC PATCH 06/14] btrfs-progs: fix help, show long option in balance start and status

2019-10-21 Thread Anand Jain
btrfs balance start|status support both short and long option -v|--verbose however failed to show it in its --help. This patch fixes the --help. Signed-off-by: Anand Jain --- cmds/balance.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cmds/balance.c b/cmds/balance.c

[RFC PATCH 01/14] btrfs-progs: add global verbose helper functions

2019-10-21 Thread Anand Jain
. Suggested-by: David Sterba Signed-off-by: Anand Jain --- btrfs.c | 12 ++-- common/help.h | 8 common/messages.c | 19 +++ common/messages.h | 5 + 4 files changed, 42 insertions(+), 2 deletions(-) diff --git a/btrfs.c b/btrfs.c index 72dad6fb3983

[RFC PATCH 04/14] btrfs-progs: migrate btrfs balance start to global verbose

2019-10-21 Thread Anand Jain
Make sure the sub command balance start calls verbose when the global verbose is set and vise versa. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/balance.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/cmds/balance.c b/cmds

[RFC PATCH 07/14] btrfs-progs: migrate rescue chunk-recover to global verbose

2019-10-21 Thread Anand Jain
Now with this patch the btrfs rescue chunk-recover can display verbose output either at the sub-command level or at the top level. For example 'btrfs --verbose rescue chunk-recover <>' or 'btrfs rescue chunk-recover -v <>'. Suggested-by: David Sterba Signed-off-by

[RFC PATCH 09/14] btrfs-progs: restore: delete unreachable code

2019-10-21 Thread Anand Jain
Maximum value of %verbose is 1 when %verbose is enabled using 'btrfs restore -v ', and the code under the condition %verbose > 1 is never reached. So delete them. Signed-off-by: Anand Jain --- cmds/restore.c | 17 - 1 file changed, 4 insertions(+), 13 deletions(-)

[RFC PATCH 14/14] btrfs-progs: enable verbose for btrfs device scan

2019-10-21 Thread Anand Jain
Enable verbose output for the device scan only through the global verbose option. For example: ./btrfs --verbose device scan Scanning for Btrfs filesystems registered: /dev/sda1 registered: /dev/sda2 registered: /dev/sda3 registered: /dev/sda5 registered: /dev/sda6 Signed-off-by: Anand Jain

[RFC PATCH 12/14] btrfs-progs: migrate inspect-internal logical-resolve to global verbose

2019-10-21 Thread Anand Jain
Command btrfs inspect-internal logical-resolve provides local verbose option this patch makes it enable-able by using the global --verbose option. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/inspect.c | 25 +++-- 1 file changed, 11 insertions(+), 14

[RFC PATCH 11/14] btrfs-progs: migrate inspect-internal inode-resolve to global verbose

2019-10-21 Thread Anand Jain
Command btrfs inspect-internal inode-resolve provides verbose option at the sub-command level, this patch makes it enable-able by using the global --verbose option. Suggested-by: David Sterba Signed-off-by: Anand Jain --- cmds/inspect.c | 15 --- 1 file changed, 8 insertions(+), 7

[RFC PATCH 13/14] btrfs-progs: refactor btrfs_scan_devices() to accept verbose argument

2019-10-21 Thread Anand Jain
n option %verbose to btrfs_scan_devices(). Signed-off-by: Anand Jain --- cmds/device.c| 2 +- cmds/filesystem.c| 2 +- common/device-scan.c | 4 +++- common/device-scan.h | 2 +- common/utils.c | 2 +- disk-io.c| 2 +- 6 files changed, 8 insertions(+), 6 deletions(-

Re: [PATCH v3] btrfs-progs: add verbose option to btrfs device scan

2019-10-21 Thread Anand Jain
On 10/21/19 9:43 PM, David Sterba wrote: On Tue, Oct 15, 2019 at 11:29:34AM +0800, Anand Jain wrote: I was thinking there might be some common code between the sub-commands in btrfs-progs now or in future, and if the printf() due to verbose is required in one sub-command and the

Re: [RFC PATCH 00/14] btrfs-progs: global-verbose option

2019-10-21 Thread Anand Jain
On 10/22/19 12:12 AM, David Sterba wrote: On Mon, Oct 21, 2019 at 06:01:08PM +0800, Anand Jain wrote: This patch set brings --verbose option to the top level btrfs command, such as 'btrfs --verbose'. With this we don't have to add or remember verbose option at the sub-comm

Re: [PATCH 2/2] fstest: btrfs/197: test for alien devices

2019-10-22 Thread Anand Jain
On 10/18/19 5:13 PM, Eryu Guan wrote: On Mon, Oct 07, 2019 at 05:41:01PM +0800, Anand Jain wrote: Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians no btrfs magic string. Signed-off-by: Anand Jain --- tests/btrfs/197 | 79

Re: [PATCH 1/2] fstest: btrfs/196: test for alien btrfs-devices

2019-10-22 Thread Anand Jain
On 10/18/19 5:10 PM, Eryu Guan wrote: On Mon, Oct 07, 2019 at 05:41:00PM +0800, Anand Jain wrote: Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians someother btrfs. Signed-off-by: Anand Jain --- tests/btrfs/196 | 77

[PATCH v2 1/2] fstest: btrfs/197: test for alien btrfs-devices

2019-10-22 Thread Anand Jain
Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians someother btrfs. Signed-off-by: Anand Jain --- v2: Comments updated with the required kernel patch to pass this test. Use spare device instead of test_mnt to create an alien btrfs device

[PATCH v2 2/2] fstest: btrfs/198: test for alien devices

2019-10-22 Thread Anand Jain
Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians no btrfs magic string. Signed-off-by: Anand Jain --- v2: Comments added to list kernel patch required to pass the test case Fix copy and paste error in the comment tests/btrfs/198 | 79

Re: feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Anand Jain
I agree, I sent patches for it in 2017. VFS version. https://patchwork.kernel.org/patch/9745295/ btrfs version: https://patchwork.kernel.org/patch/9745295/ There wasn't response on btrfs-v2-patch. This is not the first time that I am writing patches ahead of users asking for it,

Re: feature request, explicit mount and unmount kernel messages

2019-10-22 Thread Anand Jain
On 10/22/19 5:55 PM, Anand Jain wrote:  I agree, I sent patches for it in 2017.  VFS version.    https://patchwork.kernel.org/patch/9745295/  btrfs version:    https://patchwork.kernel.org/patch/9745295/ David, I mean, do you think I should revive the above btrfs patch? Thanks, Anand

Re: [PATCH v2 1/2] btrfs: prevent NULL pointer dereference in extent_io_tree_panic()

2021-01-03 Thread Anand Jain
ter to extent_io_tree_panic() directly. Let it panic as expected at least. Reviewed-by: Anand Jain Thanks. Fixes: 05912a3c04eb ("btrfs: drop extent_io_ops::tree_fs_info callback") Signed-off-by: Su Yue

Re: [PATCH v2 2/2] btrfs: tree-checker: check if chunk item end oveflows

2021-01-04 Thread Anand Jain
On 3/1/21 5:28 pm, Su Yue wrote: while mounting the poc image user-provided, kernel panics due to the invalid chunk item whose end is less than start. [ 66.387422] loop: module loaded [ 66.389773] loop0: detected capacit

Re: [PATCH v2 2/2] btrfs: tree-checker: check if chunk item end oveflows

2021-01-04 Thread Anand Jain
nk_err(leaf, chunk, logical,     "invalid chunk stripe length: %llu", So this is a system chunk? It is not so evident from the trace above. It's a chunk item located in chunk tree leaf, not in system chunk array. Tree checker checks chunk items in both locations. Ok. Thanks. Reviewed-by: Anand Jain

Re: Raid1 of a slow hdd and a fast(er) SSD, howto to prioritize the SSD?

2021-01-05 Thread Anand Jain
­I have put a SSD and a slow laptop HDD in btrfs raid. This was a bad idea, my system does not feel responsive. When i load a program, dstat shows half of the program is loaded from the SSD, and the rest from the slow hard drive. The drive speeds are evolving. NVME introduced lower latency

[PATCH] btrfs: fixup read_policy latency

2021-01-05 Thread Anand Jain
patch fixes it. This fixup patch must be rolled into its base patch [1]. [1] [PATCH v2 1/4] btrfs: add read_policy latency I will include these changes in the base patch after the review comments. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2 insertions(+), 2

[bug] scrub on low disk space leads to Transaction aborted

2021-01-09 Thread Anand Jain
I ran scrub when disk space was low on a btrfs with some raid1 csum errors, on a system with kernel 5.11.0-rc2+. It lead to transaction aborted and rdonly FS. [ 2365.314375] [ cut here ] [ 2365.314385] BTRFS: Transaction aborted (error -28) [ 2365.314470] WARNING: CPU

Re: [bug] scrub on low disk space leads to Transaction aborted

2021-01-10 Thread Anand Jain
On 11/1/21 8:38 am, Qu Wenruo wrote: On 2021/1/9 下午4:53, Anand Jain wrote:   I ran scrub when disk space was low on a btrfs with some raid1 csum   errors, on a system with kernel 5.11.0-rc2+. It lead to transaction   aborted and rdonly FS. Would you please provide `btrfs fi usage

[PATCH v3 1/4] btrfs: add read_policy latency

2021-01-11 Thread Anand Jain
> /sys/fs/btrfs/$uuid/read_policy Signed-off-by: Anand Jain --- v3: The block layer commit 0d02129e76ed (block: merge struct block_device and struct hd_struct) has changed the first argument in the function part_stat_read_all() in 5.11-rc1. So the compilation will fail. This patch

[PATCH v3 0/4] btrfs: read_policy types latency, device and round-robin

2021-01-11 Thread Anand Jain
cost of calculating the latency per IO. diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index d3023879bdf6..72ec633e9063 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5665,6 +5665,12 @@ static int find_live_mirror(struct btrfs_fs_info *fs_info, fs_info->fs_devices->read_policy = BTRFS_

[PATCH v3 2/4] btrfs: introduce new device-state read_preferred

2021-01-11 Thread Anand Jain
This is a preparatory patch and introduces a new device flag 'read_preferred', RW-able using sysfs interface. Signed-off-by: Anand Jain --- v2: C style fixes. Drop space in between '! test_bit' and extra lines after it. fs/

[PATCH v3 3/4] btrfs: introduce new read_policy device

2021-01-11 Thread Anand Jain
iostat -zy 1 | egrep 'sdb|sdc' (from another terminal) sdc 49.00 40048.00 0.00 40048 0 Signed-off-by: Anand Jain --- v1:- v2:- v3: update the change log fs/btrfs/sysfs.c | 3 ++- fs/btrfs/volumes.c | 22 ++ fs/btrfs/vo

[PATCH RFC 4/4] btrfs: introduce new read_policy round-robin

2021-01-11 Thread Anand Jain
. Signed-off-by: Anand Jain -- RFC because: Provides terrible performance with the fio tests. I am not yet sure if there is any io workload or a block layer tuning that shall make this policy better. As of now just an experimental patch. fs/btrfs/sysfs.c | 2 +- fs/btrfs/volumes.c | 50

Re: [PATCH] btrfs: fixup read_policy latency

2021-01-11 Thread Anand Jain
On 8/1/21 9:57 pm, David Sterba wrote: On Wed, Jan 06, 2021 at 03:08:15PM +0800, Anand Jain wrote: In the meantime, since I have sent the base patch as below [1], the block layer commit 0d02129e76ed (block: merge struct block_device and struct hd_struct) has changed the first argument in the

[bug] Balance fails due to ENOSPC

2021-01-12 Thread Anand Jain
Balance fails due to ENOSPC on 5.11.0-rc2+. [ 2460.219094] BTRFS info (device sdb6): balance: start -f -dconvert=single -mconvert=single -sconvert=single [ 2460.219224] BTRFS info (device sdb6): relocating block group 194572713984 flags data [ 2460.241299] [ cut here ]-

[PATCH RFC] btrfs: no need to transition to rdonly for ENOSPC error

2021-01-12 Thread Anand Jain
In the current kernel both scrub and balance fails due to ENOSPC, however there is no reason that it should be transitioned to the RDONLY and making free spaces difficult. Signed-off-by: Anand Jain --- fs/btrfs/super.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/super.c b/fs

Re: [PATCH RFC] btrfs: no need to transition to rdonly for ENOSPC error

2021-01-13 Thread Anand Jain
On 12/1/21 10:41 pm, Josef Bacik wrote: On 1/12/21 5:34 AM, Anand Jain wrote: In the current kernel both scrub and balance fails due to ENOSPC, however there is no reason that it should be transitioned to the RDONLY and making free spaces difficult. Signed-off-by: Anand Jain ---   fs/btrfs

Re: btrfs doesn't understand a drive moving from one sata port to another during suspend

2021-01-17 Thread Anand Jain
I wish there are btrfs unmount logs or ctree-close logs so that we can know if the device was closed during the suspend. Those patches aren't merged yet. So here if I guess it correctly, the suspend does not seems to unmount the filesystem, which means the device open is never closed. And, i

Re: [PATCH v12 08/41] btrfs: allow zoned mode on non-zoned block devices

2021-01-18 Thread Anand Jain
On 18/1/21 10:15 pm, Naohiro Aota wrote: On Fri, Jan 15, 2021 at 05:07:26PM -0500, Josef Bacik wrote: On 1/15/21 1:53 AM, Naohiro Aota wrote: From: Johannes Thumshirn Run zoned btrfs mode on non-zoned devices. This is done by "slicing up" the block-device into static sized chunks and fake a c

Re: [PATCH v3 1/4] btrfs: add read_policy latency

2021-01-19 Thread Anand Jain
On 20/1/21 3:36 am, Josef Bacik wrote: On 1/11/21 4:41 AM, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait-time experienced by the read IOs through the individual device. This patch obtains the historical read IO stats from the kernel block

Re: [PATCH RFC 4/4] btrfs: introduce new read_policy round-robin

2021-01-19 Thread Anand Jain
On 20/1/21 3:41 am, Josef Bacik wrote: On 1/11/21 4:41 AM, Anand Jain wrote: Add round-robin read policy to route the read IO to the next device in the round-robin order. The chunk allocation and thus the stripe-index follows the order of free space available on devices. So to make the round

[PATCH v4 1/3] btrfs: add read_policy latency

2021-01-19 Thread Anand Jain
> /sys/fs/btrfs/$uuid/read_policy Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- v4: For btrfs_debug_rl() use fs_info instead of device->fs_devices->fs_info. v3: The block layer commit 0d02129e76ed (block: merge struct block_device and struct hd_struct) has changed the fi

[PATCH v4 0/3] btrfs: read_policy types latency, device and round-robin

2021-01-19 Thread Anand Jain
pts which were useful for testing. ---8< Set latency policy on the btrfs mounted at /mnt Usage example: $ readpolicyset /mnt latency Anand Jain (3): btrfs: add read_policy latency btrfs: introduce new device-state read_preferred btrfs: i

[PATCH v4 3/3] btrfs: introduce new read_policy device

2021-01-19 Thread Anand Jain
iostat -zy 1 | egrep 'sdb|sdc' (from another terminal) sdc 49.00 40048.00 0.00 40048 0 Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- v4: add Josef rb. v3: update the change log. v2: - rfc->v1: - fs/btrfs/sysfs.c | 3 ++- fs/b

[PATCH v4 2/3] btrfs: introduce new device-state read_preferred

2021-01-19 Thread Anand Jain
This is a preparatory patch and introduces a new device flag 'read_preferred', RW-able using sysfs interface. Signed-off-by: Anand Jain --- v4: - v2: C style fixes. Drop space in between '! test_bit' and extra lines after it. fs/

Re: [PATCH v3 1/4] btrfs: add read_policy latency

2021-01-20 Thread Anand Jain
Hi Anand, I tested this policy with fio and dstat. It performs overall really well. On my raid1c3 array with two HDDs and one SSD (which is the last device), I'm getting the following results. Michal, Thank you for verifying. More below... With direct=0: Run status group 0 (all jobs

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-01-21 Thread Anand Jain
On 20/1/21 8:14 pm, David Sterba wrote: On Tue, Jan 19, 2021 at 11:52:05PM -0800, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait-time experienced by the read IOs through the individual device. This patch obtains the historical read IO

Re: [PATCH v4 2/3] btrfs: introduce new device-state read_preferred

2021-01-21 Thread Anand Jain
On 20/1/21 3:52 pm, Anand Jain wrote: This is a preparatory patch and introduces a new device flag 'read_preferred', RW-able using sysfs interface. Signed-off-by: Anand Jain > --- v4: - There is rb from Josef for this patch in v3. Could you please add it? Thanks, Anand

Re: [PATCH] btrfs: remove redundant NULL check

2021-01-21 Thread Anand Jain
if (x) - kvfree(x); + kvfree(x); Reviewed-by: Anand Jain Thanks, Anand return 0; }

Re: [PATCH v3 1/4] btrfs: add read_policy latency

2021-01-21 Thread Anand Jain
On 20/1/21 9:54 pm, Michal Rostecki wrote: On Wed, Jan 20, 2021 at 08:30:56PM +0800, Anand Jain wrote: I ran fio tests again, now with dstat in an another window. I don't notice any such stalls, the read numbers went continuous until fio finished. Could you please check with the

Re: [PATCH v4 0/3, full-cover-letter] btrfs: read_policy types latency, device and round-robin

2021-01-21 Thread Anand Jain
uct btrfs_fs_info *fs_info, fs_info->fs_devices->read_policy = BTRFS_READ_POLICY_PID; fallthrough; case BTRFS_READ_POLICY_PID: + /* + * Just to factor in the cost of calculating the avg wait using + * iostat, call btrfs_find_best_stripe() here for the PID policy + * and drop its results o

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-01-22 Thread Anand Jain
On 22/1/21 1:52 am, David Sterba wrote: On Thu, Jan 21, 2021 at 06:10:36PM +0800, Anand Jain wrote: On 20/1/21 8:14 pm, David Sterba wrote: On Tue, Jan 19, 2021 at 11:52:05PM -0800, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-01-29 Thread Anand Jain
500m is really small data size for such measurement I reran the read policy tests with some changes in the fio command options. Mainly to measure IOPS throughput and latency on the filesystem with latency-policy and pid-policy. Each of these tests was run for 3 iterations and the best and wo

Re: [PATCH v14 03/42] btrfs: defer loading zone info after opening trees

2021-01-30 Thread Anand Jain
y: Naohiro Aota Patches are already in for-next. My apologies for the delay in review here and in the following patches how many ever I could. Reviewed-by: Anand Jain Thanks. --- fs/btrfs/disk-io.c | 13 + fs/btrfs/volumes.c | 4 fs/btrfs/zone

Re: [PATCH v14 04/42] btrfs: use regular SB location on emulated zoned mode

2021-01-30 Thread Anand Jain
he SB locations are at a conventional zone on emulated zoned mode. Reviewed-by: Josef Bacik Signed-off-by: Naohiro Aota Makes sense to me. Reviewed-by: Anand Jain Thanks. --- fs/btrfs/zoned.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/zoned.c b/fs/

Re: [PATCH v14 05/42] btrfs: release path before calling into btrfs_load_block_group_zone_info

2021-01-30 Thread Anand Jain
Thumshirn Reviewed-by: Josef Bacik Looks good to me. Reviewed-by: Anand Jain Thanks. --- fs/btrfs/block-group.c | 39 ++- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 763a3671b7af

Re: [PATCH v14 06/42] btrfs: do not load fs_info->zoned from incompat flag

2021-01-30 Thread Anand Jain
rrect zone_size as well. Once the correct zone_size is read from the device, we can rely on the zoned flag in fs_info as well to determine if the filesystem is running in zoned mode. Signed-off-by: Johannes Thumshirn Reviewed-by: Josef Bacik Looks good. Reviewed-by: Anand Jain T

Re: [PATCH v14 07/42] btrfs: disallow fitrim in ZONED mode

2021-01-30 Thread Anand Jain
space cache (but, only for fitrim) or scanning the extent tree at fitrim time. But, for now, disallow fitrim in ZONED mode. Signed-off-by: Naohiro Aota Reviewed-by: Josef Bacik Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/ioctl.c | 8 1 file changed, 8 insertions(+) diff

Re: [PATCH v14 08/42] btrfs: allow zoned mode on non-zoned block devices

2021-01-30 Thread Anand Jain
On 1/26/2021 10:24 AM, Naohiro Aota wrote: From: Johannes Thumshirn Run zoned btrfs mode on non-zoned devices. This is done by "slicing up" the block-device into static sized chunks and fake a conventional zone on each of them. The emulated zone size is determined from the size of device extent

Re: [PATCH v14 09/42] btrfs: implement zoned chunk allocator

2021-01-31 Thread Anand Jain
. Also, it checks that a region allocation is not overlapping any of the super block zones, and ensures the region is empty. Reviewed-by: Josef Bacik Signed-off-by: Naohiro Aota Reviewed-by: Anand Jain A nit below. --- fs/btrfs/volumes.c | 169

Re: [PATCH v14 18/42] btrfs: reset zones of unused block groups

2021-01-31 Thread Anand Jain
Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/block-group.c | 8 ++-- fs/btrfs/extent-tree.c | 17 - fs/btrfs/zoned.h | 16 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c

Re: [PATCH v14 17/42] btrfs: enable to mount ZONED incompat flag

2021-01-31 Thread Anand Jain
On 1/26/2021 10:24 AM, Naohiro Aota wrote: This final patch adds the ZONED incompat flag to BTRFS_FEATURE_INCOMPAT_SUPP and enables btrfs to mount ZONED flagged file system. Signed-off-by: Naohiro Aota Reviewed-by: Josef Bacik Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/ctree.h

Re: [PATCH v14 08/42] btrfs: allow zoned mode on non-zoned block devices

2021-02-01 Thread Anand Jain
of device extent. This is mainly aimed at testing parts of the zoned mode, i.e. the zoned chunk allocator, on regular block devices. Signed-off-by: Johannes Thumshirn Signed-off-by: Naohiro Aota Reviewed-by: Josef Bacik Reviewed-by: Anand Jain Thanks, Anand

[bug report] Unable to handle kernel paging request

2021-02-02 Thread Anand Jain
Qu, fstests ran fine on an aarch64 kvm with this patch set. Further, I was running few hand tests as below, and it fails with - Unable to handle kernel paging. Test case looks something like.. On x86_64 create btrfs on a file 11g copy /usr into /test-mnt stops at enospc set compressio

Re: [bug report] Unable to handle kernel paging request

2021-02-02 Thread Anand Jain
On 2/2/2021 6:23 PM, Qu Wenruo wrote: On 2021/2/2 下午5:21, Anand Jain wrote: Qu,   fstests ran fine on an aarch64 kvm with this patch set. Do you mean subpage patchset? With 4K sector size? No way it can run fine... No . fstests ran with sectorsize == pagesize == 64k. These aren&#

Re: [bug report] Unable to handle kernel paging request

2021-02-02 Thread Anand Jain
+0.001075] ---[ end trace d4f31b4f11a947b7 ]--- [ +14.775765] note: kworker/u64:1[5754] exited with preempt_count 1 Thanks, Anand On 2/2/2021 7:28 PM, Anand Jain wrote: On 2/2/2021 6:23 PM, Qu Wenruo wrote: On 2021/2/2 下午5:21, Anand Jain wrote: Qu,   fstests ran

Re: [PATCH v14 12/42] btrfs: calculate allocation offset for conventional zones

2021-02-02 Thread Anand Jain
offset to 0, anyway. Information about how are the WP of conventional zones used is missing here. Reviewed-by: Anand Jain Thanks. Signed-off-by: Naohiro Aota --- fs/btrfs/block-group.c | 4 +- fs/btrfs/zoned.c | 99 +++--- fs/btrfs/zoned.h

Re: [PATCH v14 12/42] btrfs: calculate allocation offset for conventional zones

2021-02-02 Thread Anand Jain
On 2/3/2021 2:10 PM, Damien Le Moal wrote: On 2021/02/03 14:22, Anand Jain wrote: On 1/26/2021 10:24 AM, Naohiro Aota wrote: Conventional zones do not have a write pointer, so we cannot use it to determine the allocation offset if a block group contains a conventional zone. But instead, we

Re: [PATCH 1/4] btrfs: avoid checking for RO block group twice during nocow writeback

2021-02-03 Thread Anand Jain
that protects it (specially since it is a spin lock and not a read-write lock). That may make a noticeable difference on very large filesystems, with thousands of allocated block groups. Signed-off-by: Filipe Manana Reviewed-by: Anand Jain Thanks. --- fs/btrfs/inode.c | 4 +--- 1 file

Re: [PATCH 2/4] btrfs: fix race between writes to swap files and scrub

2021-02-04 Thread Anand Jain
and not a read/write lock) on very large filesystems, with several thousands of allocated block groups. Fixes: ed46ff3d42378 ("Btrfs: support swap files") Signed-off-by: Filipe Manana I am not sure about the optimization of direct IO part, but as such changes looks good. Reviewed

Re: [PATCH 0/2] btrfs: add proper subpage compress read support

2021-02-04 Thread Anand Jain
Nice. Works well. Thanks, Anand On 2/4/2021 3:03 PM, Qu Wenruo wrote: During the long time subpage development, I forgot to properly check compression code after just one compression read success during early development. It turns out that, with current RO support, the compression read nee

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-02-04 Thread Anand Jain
Hi Michal, Did you get any chance to run the evaluation with this patchset? Thanks, Anand On 1/30/2021 9:08 AM, Anand Jain wrote: 500m is really small data size for such measurement I reran the read policy tests with some changes in the fio command options. Mainly to measure IOPS

Re: [PATCH 2/4] btrfs: fix race between writes to swap files and scrub

2021-02-04 Thread Anand Jain
On 2/4/2021 6:11 PM, Filipe Manana wrote: On Thu, Feb 4, 2021 at 8:48 AM Anand Jain wrote: On 2/3/2021 7:17 PM, fdman...@kernel.org wrote: From: Filipe Manana When we active a swap file, at btrfs_swap_activate(), we acquire the exclusive operation lock to prevent the physical location of

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-02-09 Thread Anand Jain
On 10/02/2021 05:12, Michal Rostecki wrote: On Thu, Feb 04, 2021 at 08:30:01PM +0800, Anand Jain wrote: Hi Michal, Did you get any chance to run the evaluation with this patchset? Thanks, Anand Hi Anand, Yes, I tested your policies now. Sorry for late response. For the

Re: [PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-09 Thread Anand Jain
On 10/02/2021 04:30, Michal Rostecki wrote: From: Michal Rostecki This patch series adds a new raid1 read policy - roundrobin. For each request, it selects the mirror which has lower load than queue depth. Load is defined as the number of inflight requests + a penalty value (if the scheduled r

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-10 Thread Anand Jain
On 10/02/2021 04:30, Michal Rostecki wrote: From: Michal Rostecki Add a new raid1 read policy `roundrobin`. For each read request, it selects the mirror which has lower load than queue depth and it starts iterating from the last used mirror (by the current CPU). Load is defined as the number of

Re: [PATCH v2 1/3] btrfs: avoid checking for RO block group twice during nocow writeback

2021-02-10 Thread Anand Jain
. Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/inode.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 04cd95899ac8..76a0151ef05a 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1657,9 +1657,6 @@ static noinline

Re: [PATCH v2 2/3] btrfs: fix race between writes to swap files and scrub

2021-02-10 Thread Anand Jain
- this is the same approach we currently use for balance. Fixes: ed46ff3d42378 ("Btrfs: support swap files") Signed-off-by: Filipe Manana Looks good. Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/block-group.c | 33 - fs/btrfs/block-gr

Re: [PATCH v2 3/3] btrfs: fix race between swap file activation and snapshot creation

2021-02-10 Thread Anand Jain
snapshot lock during swap file activation before locking the extent range, as that is the order in which we lock these during buffered writes. Fixes: ed46ff3d42378 ("Btrfs: support swap files") Signed-off-by: Filipe Manana Reviewed-by: Anand Jain Thanks, Anan

[PATCH 2/5] btrfs: btrfs_extent_readonly() change return type to bool

2021-02-10 Thread Anand Jain
btrfs_extent_readonly() checks if the bg is readonly, the bool return type will suffice its need. Signed-off-by: Anand Jain --- fs/btrfs/inode.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2ed7d736e39a..ebb2b8e3a71c 100644

[PATCH 0/5] cleanups btrfs_extent_readonly() and scrub, part1

2021-02-10 Thread Anand Jain
This patchset makes btrfs_extent_readonly() static and changes the return type to bool. And drops few unwanted function declarations in the scrub.c. Thanks. Anand Anand Jain (5): btrfs: make btrfs_extent_readonly() static btrfs: btrfs_extent_readonly() change return type to bool btrfs

<    5   6   7   8   9   10   11   12   13   14   >