[PATCH 8/8] btrfs: Remove redundant assignment of tgt_device->commit_total_bytes

2019-05-14 Thread Nikolay Borisov
This is already done in btrfs_init_dev_replace_tgtdev which is the first phase of device replace, called before doing scrub. During that time exclusive lock is held. Additionally btrfs_fs_device::commit_total_bytes is always set based on the size of the underlying block device which shouldn't chang

Re: [PATCH 1/8] btrfs: Don't opencode sync_blockdev in btrfs_init_dev_replace_tgtdev

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

[PATCH 3/8] btrfs: Remove impossible WARN_ON

2019-05-14 Thread Nikolay Borisov
This WARN_ON can never trigger because src_device cannot be null. btrfs_find_device_by_devspec always returns either an error or a valid pointer to the device. Just remove it. Signed-off-by: Nikolay Borisov --- fs/btrfs/dev-replace.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/de

[PATCH 4/8] btrfs: Ensure btrfs_init_dev_replace_tgtdev sees up to date values

2019-05-14 Thread Nikolay Borisov
btrfs_init_dev_replace_tgtdev reads certain values from the source device (such as commit_total_bytes) which are updated during transaction commit. Currently this function is called before committing any pending transaction, leading to possibly reading outdated values. Fix this by moving the functi

[PATCH 5/8] btrfs: Streamline replace sem unlock in btrfs_dev_replace_start

2019-05-14 Thread Nikolay Borisov
There are only 2 branches which goto leave label with need_unlock set to true. Essentially need_unlock is used as a substitute for directly calling up_write. Since the branches needing this are only 2 and their context is not that big it's more clear to just call up_write where required. No functio

[PATCH 2/8] btrfs: Reduce critical section in btrfs_init_dev_replace_tgtdev

2019-05-14 Thread Nikolay Borisov
There is no point in holding btrfs_fs_devices::device_list_mutex while initialising fields of the not-yet-published device. Instead, hold the mutex only when the newly initialised device is being published. I think holding device_list_mutex here is redundant altogether, because at this point BTRFS_

[PATCH 1/8] btrfs: Don't opencode sync_blockdev in btrfs_init_dev_replace_tgtdev

2019-05-14 Thread Nikolay Borisov
Using sync_blockdev makes it plain obvious what's happening. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/dev-replace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 55c15f31d00d..c738ba460682 10064

[PATCH 0/8] Misc improvements to dev-replace code

2019-05-14 Thread Nikolay Borisov
While fixing the failing ASSERT in device replace finishing procedure I also found several oddities/bugs. Here is the resultant pile. First 3 patches are a couple simple cleanups. Patch 4 fixes a real bug since btrfs_init_dev_replace_tgtdev accesses values which might be updated by transaction c

[PATCH 6/8] btrfs: Explicitly reserve space for devreplace item

2019-05-14 Thread Nikolay Borisov
Part of device replace involves writing an item to the device root containing information about pending replace operations. Currently space for this item is not being explicitly reserved so this works thanks to presence of global reserve. While not fatal it's not good practice. Let's be explicit ab

[PATCH 7/8] btrfs: Ensure replaced device doesn't have pending chunk allocation

2019-05-14 Thread Nikolay Borisov
Recent FITRIM work, namely bbbf7243d62d ("btrfs: combine device update operations during transaction commit") combined the way certain operations are recoded in a transaction. As a result an ASSERT was added in dev_replace_finish to ensure the new code works correctly. Unfortunately I got reports t

Re: [PATCH 3/8] btrfs: Remove impossible WARN_ON

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

Re: [PATCH 2/8] btrfs: Reduce critical section in btrfs_init_dev_replace_tgtdev

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

Re: [PATCH 14/17] btrfs: directly call into crypto framework for checsumming

2019-05-14 Thread Johannes Thumshirn
On Fri, May 10, 2019 at 01:45:47PM +, Chris Mason wrote: > I'm a little concerned about about btrfs_csum_data() and > btrfs_csum_final() below. We're using two different > SHASH_DESC_ON_STACK() and then overwriting internals (shash_desc_ctx()) > with the assumption that whatever we're doing

Re: [PATCH 5/8] btrfs: Streamline replace sem unlock in btrfs_dev_replace_start

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

Re: [PATCH 6/8] btrfs: Explicitly reserve space for devreplace item

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

Re: [PATCH 8/8] btrfs: Remove redundant assignment of tgt_device->commit_total_bytes

2019-05-14 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnb

[PATCH v3 3/3] btrfs-progs: completion: wire-up dump-csum

2019-05-14 Thread Johannes Thumshirn
Add the new 'dump-csum' command to inspect-internal. Signed-off-by: Johannes Thumshirn --- btrfs-completion | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-completion b/btrfs-completion index 6ae57d1b752b..a381036886f0 100644 --- a/btrfs-completion +++ b/btrfs-compl

[PATCH v3 0/3] btrfs-progs: provide command to dump checksums

2019-05-14 Thread Johannes Thumshirn
Provide a command to dump checksums from 'btrfs inspect-internal'. This command does a lookup for the given file's extents via FIEMAP (which is handy as it already skips holes) and walks the checksum tree printing all checksums of an extent. It is tested against different layouts of files with and

[PATCH v3 2/3] btrfs-progs: add 'btrfs inspect-internal csum-dump' command

2019-05-14 Thread Johannes Thumshirn
Add a 'btrfs inspect-internal csum-dump' command to dump the on-disk checksums of a file. The dump command first uses the FIEMAP ioctl() to get a map of the file's extents and then uses the BTRFS_TREE_SEARCH_V2 ioctl() to get the checksums for these extents. Using FIEMAP instead of the BTRFS_TREE

[PATCH v3 1/3] btrfs-progs: factor out super_block reading from load_and_dump_sb

2019-05-14 Thread Johannes Thumshirn
inspect-internal dump-superblock's load_and_dump_sb() already reads a super block from a file descriptor and places it into a 'struct btrfs_super_block'. For inspect-internal dump-csum we need this super block as well but don't care about printing it. Separate the read from the dump phase so we c

[PATCH v3.1 1/3] btrfs-progs: factor out super_block reading from load_and_dump_sb

2019-05-14 Thread Johannes Thumshirn
inspect-internal dump-superblock's load_and_dump_sb() already reads a super block from a file descriptor and places it into a 'struct btrfs_super_block'. For inspect-internal dump-csum we need this super block as well but don't care about printing it. Separate the read from the dump phase so we c

Re: [PATCH v3.1 1/3] btrfs-progs: factor out super_block reading from load_and_dump_sb

2019-05-14 Thread Nikolay Borisov
On 14.05.19 г. 16:58 ч., Johannes Thumshirn wrote: > inspect-internal dump-superblock's load_and_dump_sb() already reads a > super block from a file descriptor and places it into a 'struct > btrfs_super_block'. > > For inspect-internal dump-csum we need this super block as well but don't > care

Re: [PATCH] btrfs: extent-tree: Fix a bug that btrfs is unable to add pinned bytes

2019-05-14 Thread David Sterba
On Fri, May 10, 2019 at 12:45:05PM +0800, Qu Wenruo wrote: > Commit ddf30cf03fb5 ("btrfs: extent-tree: Use btrfs_ref to refactor > add_pinned_bytes()") refactored add_pinned_bytes(), but during that > refactor, there are two callers which add the pinned bytes instead > of subtracting. > > That ref

Re: [PATCH V9] Btrfs: enhance raid1/10 balance heuristic

2019-05-14 Thread Steven Davies
On 2019-05-13 19:52, David Sterba wrote: I'd rather see a generic mechanism that would spread the load than doing static checks if the device is or is not rotational. Though this is a significant factor, it's not 100% reliable, eg. NBD or iscsi or other layers between the physical device and wha

Re: [PATCH] btrfs: extent-tree: Fix a bug that btrfs is unable to add pinned bytes

2019-05-14 Thread Qu Wenruo
On 2019/5/15 上午1:29, David Sterba wrote: > On Fri, May 10, 2019 at 12:45:05PM +0800, Qu Wenruo wrote: >> Commit ddf30cf03fb5 ("btrfs: extent-tree: Use btrfs_ref to refactor >> add_pinned_bytes()") refactored add_pinned_bytes(), but during that >> refactor, there are two callers which add the pinn

[PATCH] btrfs: extent-tree: Refactor add_pinned_bytes() to add|sub_pinned_bytes()

2019-05-14 Thread Qu Wenruo
Instead of using @sign to determine whether we're adding or subtracting. Even it only has 3 callers, it's still (and in fact already caused problem in the past) confusing to use. Refactor add_pinned_bytes() to add_pinned_bytes() and sub_pinned_bytes() to explicitly show what we're doing. Signed-o

Re: [PATCH 17/17] btrfs: add sha256 as another checksum algorithm

2019-05-14 Thread Jeff Mahoney
On 5/13/19 8:54 AM, David Sterba wrote: > On Mon, May 13, 2019 at 09:11:14AM +0200, Johannes Thumshirn wrote: >> On Fri, May 10, 2019 at 03:30:36PM +0300, Nikolay Borisov wrote: >> [...] >>> >>> nit: Might be a good idea to turn that into an enum for self-documenting >>> purposes. Perhaps in a diff