Re: [PATCH v2] Btrfs: heuristic replace heap sort with radix sort

2017-12-04 Thread Timofey Titovets
2017-12-05 0:24 GMT+03:00 Timofey Titovets : > 2017-12-04 23:47 GMT+03:00 David Sterba : >> On Mon, Dec 04, 2017 at 12:30:33AM +0300, Timofey Titovets wrote: >>> Slowest part of heuristic for now is kernel heap sort() >>> It's can take up to 55% of runtime on sorting bucket items. >>> >>> As sortin

[PATCH] btrfs: Handle btrfs_set_extent_delalloc failure

2017-12-04 Thread Nikolay Borisov
This function was introduced by 247e743cbe6e ("Btrfs: Use async helpers to deal with pages that have been improperly dirtied") and it didn't do any error handling then. This function might very well fail in ENOMEM situation, yet it's not handled, this could lead to inconsistent state. So let's hand

Re: [PATCH] btrfs: fix inconsistency during missing device rejoin

2017-12-04 Thread Anand Jain
On 12/05/2017 10:18 AM, Misono, Tomohiro wrote: On 2017/12/04 16:19, Anand Jain wrote: When device is missing its not necessary that btrfs_device::name is null or the path is different when it reappears. Its possible that device can go missing after its been scanned where neither of btrfs_de

Re: [PATCH] btrfs: fix inconsistency during missing device rejoin

2017-12-04 Thread Misono, Tomohiro
On 2017/12/04 16:19, Anand Jain wrote: > When device is missing its not necessary that btrfs_device::name is null > or the path is different when it reappears. Its possible that device can > go missing after its been scanned where neither of > btrfs_device::name == NULL OR btrfs_device::name != r

[PATCH] Btrfs: remove useless wait in lock_stripe_add

2017-12-04 Thread Liu Bo
The defined wait is not used anywhere. Signed-off-by: Liu Bo --- fs/btrfs/raid56.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 24a6222..62bcda2 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -670,7 +670,6 @@ static noinline int lock_s

Re: [PATCH] btrfs: add helper for device path or missing

2017-12-04 Thread Anand Jain
On 12/05/2017 03:31 AM, David Sterba wrote: On Tue, Nov 28, 2017 at 10:43:10AM +0800, Anand Jain wrote: This patch creates a helper function to get either the rcu device path or missing. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 22 +++--- 1 file changed, 11 i

Re: [PATCH v2 0/5] define BTRFS_DEV_STATE

2017-12-04 Thread Anand Jain
On 12/05/2017 04:28 AM, David Sterba wrote: On Mon, Dec 04, 2017 at 12:54:51PM +0800, Anand Jain wrote: As of now device properties and states are being represented as int variable. So clean that up using bitwise operations. Also patches in the ML such as device failed state needs this cleanup

[PATCH 2/3] Btrfs: do not merge rbios if their fail stripe index are not identical

2017-12-04 Thread Liu Bo
Since fail stripe index in rbio would be used to decide which algorithm reconstruction would be run, we cannot merge rbios if their's fail striped index are different, otherwise, one of the two reconstructions would fail. Signed-off-by: Liu Bo --- fs/btrfs/raid56.c | 9 + 1 file changed,

[PATCH 3/3] Btrfs: make raid6 rebuild retry more

2017-12-04 Thread Liu Bo
There is a scenario that can end up with rebuild process failing to return good content, i.e. suppose that all disks can be read without problems and if the content that was read out doesn't match its checksum, currently for raid6 btrfs at most retries twice, - the 1st retry is to rebuild with all

[PATCH 1/3] Btrfs: remove redundant check in rbio_can_merge

2017-12-04 Thread Liu Bo
Given the above ' if (last->operation != cur->operation) return 0; ', it's guaranteed that two operations are same. Signed-off-by: Liu Bo --- fs/btrfs/raid56.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 24a6222..c

[PATCH 0/3] Btrfs: loop retry on raid6 read failures

2017-12-04 Thread Liu Bo
Patch 1 is a simple cleanup. Patch 2 fixes a bug in raid56 rbio merging code. Patch 3 fixes a bug in raid6 reconstruction process which can end up read failure when it can rebuild up good data. Liu Bo (3): Btrfs: remove redundant check in rbio_can_merge Btrfs: do not merge rbios if their fail

[PATCH] fstests: btrfs: reproduce a read failure on raid6 setup

2017-12-04 Thread Liu Bo
This test case is to reproduce a bug of raid6 reconstruction process. The kernel fix are Btrfs: do not merge rbios if their fail stripe index are not identical Btrfs: make raid6 rebuild retry more Signed-off-by: Liu Bo --- tests/btrfs/155 | 119 ++

Re: [PATCH v4] Btrfs: compress_file_range() change page dirty status once

2017-12-04 Thread David Sterba
On Tue, Oct 24, 2017 at 01:29:48AM +0300, Timofey Titovets wrote: > We need to call extent_range_clear_dirty_for_io() > on compression range to prevent application from changing > page content, while pages compressing. > > extent_range_clear_dirty_for_io() run on each loop iteration, > "(end - sta

Re: [PATCH v2] Btrfs: heuristic replace heap sort with radix sort

2017-12-04 Thread Timofey Titovets
2017-12-04 23:47 GMT+03:00 David Sterba : > On Mon, Dec 04, 2017 at 12:30:33AM +0300, Timofey Titovets wrote: >> Slowest part of heuristic for now is kernel heap sort() >> It's can take up to 55% of runtime on sorting bucket items. >> >> As sorting will always call on most data sets to get correctl

Re: [PATCH v2] Btrfs: heuristic replace heap sort with radix sort

2017-12-04 Thread David Sterba
On Mon, Dec 04, 2017 at 12:30:33AM +0300, Timofey Titovets wrote: > Slowest part of heuristic for now is kernel heap sort() > It's can take up to 55% of runtime on sorting bucket items. > > As sorting will always call on most data sets to get correctly > byte_core_set_size, the only way to speed u

Re: [PATCH 0/7] retry write on error

2017-12-04 Thread Edmund Nadolski
On 11/30/2017 01:22 PM, Liu Bo wrote: >>> If the underlying protocal doesn't support retry and there >>> are some transient errors happening somewhere in our IO >>> stack, we'd like to give an extra chance for IO. > Anyway, this is for a corner case, not for everyone, I think I need to

Re: [PATCH v2] Btrfs: heuristic replace heap sort with radix sort

2017-12-04 Thread David Sterba
On Mon, Dec 04, 2017 at 12:30:33AM +0300, Timofey Titovets wrote: > Slowest part of heuristic for now is kernel heap sort() > It's can take up to 55% of runtime on sorting bucket items. > > As sorting will always call on most data sets to get correctly > byte_core_set_size, the only way to speed u

Re: [PATCH] btrfs: fix missing error return in btrfs_drop_snapshot

2017-12-04 Thread David Sterba
On Mon, Dec 04, 2017 at 01:11:45PM -0500, je...@suse.com wrote: > From: Jeff Mahoney > > If btrfs_del_root fails in btrfs_drop_snapshot, we'll pick up the > error but then return 0 anyway due to mixing err and ret. > > Fixes: 79787eaab4612 ("btrfs: replace many BUG_ONs with proper error > handl

Re: [PATCH v2 0/5] define BTRFS_DEV_STATE

2017-12-04 Thread David Sterba
On Mon, Dec 04, 2017 at 12:54:51PM +0800, Anand Jain wrote: > As of now device properties and states are being represented as int > variable. So clean that up using bitwise operations. Also patches in > the ML such as device failed state needs this cleanup as well. > > V2: > Accepts all comments

Re: [PATCH] btrfs: add helper for device path or missing

2017-12-04 Thread David Sterba
On Tue, Nov 28, 2017 at 10:43:10AM +0800, Anand Jain wrote: > This patch creates a helper function to get either the rcu device path > or missing. > > Signed-off-by: Anand Jain > --- > fs/btrfs/dev-replace.c | 22 +++--- > 1 file changed, 11 insertions(+), 11 deletions(-) > > di

[PATCH] btrfs: fix missing error return in btrfs_drop_snapshot

2017-12-04 Thread jeffm
From: Jeff Mahoney If btrfs_del_root fails in btrfs_drop_snapshot, we'll pick up the error but then return 0 anyway due to mixing err and ret. Fixes: 79787eaab4612 ("btrfs: replace many BUG_ONs with proper error handling") Cc: # v3.4+ Signed-off-by: Jeff Mahoney --- fs/btrfs/extent-tree.c | 1

Re: [PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

2017-12-04 Thread Nikolay Borisov
On 4.12.2017 16:13, David Sterba wrote: > On Sun, Dec 03, 2017 at 11:43:13AM +0200, Nikolay Borisov wrote: >> On 2.12.2017 01:23, Anand Jain wrote: >>> On 12/01/2017 05:19 PM, Nikolay Borisov wrote: Currently this function executes the inner loop at most 1 due to the i = 0; i < 1

Re: [PATCH 2/5] btrfs: Remove dead code

2017-12-04 Thread Nikolay Borisov
On 4.12.2017 15:50, David Sterba wrote: > On Fri, Dec 01, 2017 at 11:19:41AM +0200, Nikolay Borisov wrote: >> 'clear' is always set to 0 (BTRFS_FEATURE_COMPAT_SAFE_CLEAR, >> BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR and BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR are >> all defined to 0). So remove the code that

Re: [PATCH 1/4] btrfs: compression: add helper for type to string conversion

2017-12-04 Thread David Sterba
On Fri, Dec 01, 2017 at 05:55:56AM +0800, Anand Jain wrote: > > > On 12/01/2017 12:09 AM, David Sterba wrote: > > There are several places opencoding this conversion, add a helper now > > that we have 3 compression algorithms. > > > > Signed-off-by: David Sterba > > --- > > fs/btrfs/compressi

Re: [PATCH 2/4] btrfs: SETFLAGS ioctl: use helper for compression type conversion

2017-12-04 Thread David Sterba
On Fri, Dec 01, 2017 at 06:08:53AM +0800, Anand Jain wrote: > > > On 12/01/2017 12:09 AM, David Sterba wrote: > > Signed-off-by: David Sterba > > --- > > fs/btrfs/ioctl.c | 10 -- > > 1 file changed, 4 insertions(+), 6 deletions(-) > > > > diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioc

Re: btrfs-transacti hammering the system

2017-12-04 Thread Austin S. Hemmelgarn
On 2017-12-04 09:10, Duncan wrote: Austin S. Hemmelgarn posted on Mon, 04 Dec 2017 07:18:11 -0500 as excerpted: On 2017-12-01 16:50, Matt McKinnon wrote: Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Meta

Re: [PATCH v4] Btrfs: compress_file_range() change page dirty status once

2017-12-04 Thread Timofey Titovets
Gentle ping 2017-10-24 1:29 GMT+03:00 Timofey Titovets : > We need to call extent_range_clear_dirty_for_io() > on compression range to prevent application from changing > page content, while pages compressing. > > extent_range_clear_dirty_for_io() run on each loop iteration, > "(end - start)" can

Re: [PATCH 5/5] btrfs: Greatly simplify btrfs_read_dev_super

2017-12-04 Thread David Sterba
On Sun, Dec 03, 2017 at 11:43:13AM +0200, Nikolay Borisov wrote: > On 2.12.2017 01:23, Anand Jain wrote: > > On 12/01/2017 05:19 PM, Nikolay Borisov wrote: > >> Currently this function executes the inner loop at most 1 due to the i > >> = 0; > >> i < 1 condition. Furthermore, the btrfs_super_gener

Re: command to get quota limit

2017-12-04 Thread Marat Khalili
AFAIK you don't need subvolume id, `btrfs qgroup show -ref /path/to/subvolume` shows necessary qgroup for me. Separating value you need is more involved: btrfs qgroup show -ref --raw /path/to/subvolume | tail -n +3 | tr -s ' ' | cut -d ' ' -f 4 Not sure how robust is this though. -- With Be

Re: btrfs-transacti hammering the system

2017-12-04 Thread Duncan
Austin S. Hemmelgarn posted on Mon, 04 Dec 2017 07:18:11 -0500 as excerpted: > On 2017-12-01 16:50, Matt McKinnon wrote: >> Well, it's at zero now... >> >> # btrfs fi df /export/ >> Data, single: total=30.45TiB, used=30.25TiB >> System, DUP: total=32.00MiB, used=3.62MiB >> Metadata, DUP: total=66

Re: command to get quota limit

2017-12-04 Thread Qu Wenruo
On 2017年12月04日 21:46, ST wrote: >>> Hello, >>> >>> what is the command to get btrfs quota limit on certain subvolume? >> >> btrfs qgroup show -re >> >> The result is for all subvolumes, then just find the subvolume you need. >> > > I need to use the quota number in a script... is there a more sp

Re: [PATCH 2/5] btrfs: Remove dead code

2017-12-04 Thread David Sterba
On Fri, Dec 01, 2017 at 11:19:41AM +0200, Nikolay Borisov wrote: > 'clear' is always set to 0 (BTRFS_FEATURE_COMPAT_SAFE_CLEAR, > BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR and BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR are > all defined to 0). So remove the code that logically can never execute. The point of the

Re: [PATCH 1/5] btrfs: Remove dead code

2017-12-04 Thread David Sterba
On Fri, Dec 01, 2017 at 11:19:40AM +0200, Nikolay Borisov wrote: > trans was statically assigned to NULL and this never changed over the course > of > btrfs_get_extent. So remove any code which checks whether trans != NULL and > just hardcode the fact trans is always NULL. This fixes CID#112806 >

Re: command to get quota limit

2017-12-04 Thread ST
> > Hello, > > > > what is the command to get btrfs quota limit on certain subvolume? > > btrfs qgroup show -re > > The result is for all subvolumes, then just find the subvolume you need. > I need to use the quota number in a script... is there a more specific command, otherwise I need to get

Re: [PATCH RESEND] btrfs: ignore return from btrfs_open_one_device()

2017-12-04 Thread David Sterba
On Fri, Dec 01, 2017 at 07:02:30AM +0800, Anand Jain wrote: > Test case like btrfs-progs test-misc/012 can recreate the same fsid with > different number of struct btrfs_fs_devices::total_devices. And the > previous device which is in the kernel device list is stale now, but as > we don't clean the

Re: [PATCH v2 06/11] writeback: add counters for metadata usage

2017-12-04 Thread Jan Kara
On Wed 22-11-17 16:16:01, Josef Bacik wrote: > From: Josef Bacik > > Btrfs has no bounds except memory on the amount of dirty memory that we have > in > use for metadata. Historically we have used a special inode so we could take > advantage of the balance_dirty_pages throttling that comes with

Re: command to get quota limit

2017-12-04 Thread Qu Wenruo
On 2017年12月04日 20:14, ST wrote: > Hello, > > what is the command to get btrfs quota limit on certain subvolume? btrfs qgroup show -re The result is for all subvolumes, then just find the subvolume you need. Thanks, Qu > > Thank you! > > -- > To unsubscribe from this list: send the line "un

Re: btrfs-transacti hammering the system

2017-12-04 Thread Austin S. Hemmelgarn
On 2017-12-01 16:50, Matt McKinnon wrote: Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Metadata, DUP: total=66.50GiB, used=65.16GiB GlobalReserve, single: total=512.00MiB, used=0.00B GlobalReserve seems to

command to get quota limit

2017-12-04 Thread ST
Hello, what is the command to get btrfs quota limit on certain subvolume? Thank you! -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs : send, truncate first to enhance many small files

2017-12-04 Thread Filipe Manana
On Mon, Dec 4, 2017 at 11:28 AM, Qu Wenruo wrote: > > > On 2017年12月04日 19:13, Filipe Manana wrote: >> On Mon, Dec 4, 2017 at 7:19 AM, Qu Wenruo wrote: >>> >>> >>> On 2017年12月04日 15:02, robbieko wrote: From: Robbie Ko The commands generated by send contain the following step:

Re: [PATCH] Btrfs : send, truncate first to enhance many small files

2017-12-04 Thread Qu Wenruo
On 2017年12月04日 19:13, Filipe Manana wrote: > On Mon, Dec 4, 2017 at 7:19 AM, Qu Wenruo wrote: >> >> >> On 2017年12月04日 15:02, robbieko wrote: >>> From: Robbie Ko >>> >>> The commands generated by send contain the following step: >>> 1. mkfile o1851-19-0 >>> 2. rename o1851-19-0 -> alsa-driver/al

Re: [PATCH] Btrfs : send, truncate first to enhance many small files

2017-12-04 Thread Filipe Manana
On Mon, Dec 4, 2017 at 7:19 AM, Qu Wenruo wrote: > > > On 2017年12月04日 15:02, robbieko wrote: >> From: Robbie Ko >> >> The commands generated by send contain the following step: >> 1. mkfile o1851-19-0 >> 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c >> 3. set_xattr alsa-driv

[PATCH v2 4/4] btrfs-progs: qgroup: allow show qgroup in order of subvol path

2017-12-04 Thread Lu Fengqi
You may want to sort qgroup with the subvol path, now you can use --sort=path. Signed-off-by: Lu Fengqi --- Changelog: v2: rebase on the qgroup cleanup Documentation/btrfs-qgroup.asciidoc | 2 +- cmds-qgroup.c | 2 +- qgroup.c| 22

[PATCH v2 2/4] btrfs-progs: qgroup: cleanup __qgroup_search

2017-12-04 Thread Lu Fengqi
Replace the if statement with the switch statement, and return the appropriate value for the future use rather than directly exit. Signed-off-by: Lu Fengqi --- Changelog: v2: 1. revoke incorrect goto pattern 2. split the error handler movement to another patch qgroup.c | 48 +++

[PATCH 0/4] qgroup cleanup and add show path feature

2017-12-04 Thread Lu Fengqi
The first two patches are the rest of the qgroup cleanup, the two remaining patches add show path feature. Lu Fengqi (4): btrfs-progs: qgroup: move btrfs_show_qgroups's error handler to __qgroup_search btrfs-progs: qgroup: cleanup __qgroup_search btrfs-progs: qgroup: show subvol path whe

[PATCH 1/4] btrfs-progs: qgroup: move btrfs_show_qgroups's error handler to __qgroup_search

2017-12-04 Thread Lu Fengqi
We have to process the return value of BTRFS_IOC_TREE_SEARCH ioctl in advance, so that we can distinguish between the two case where quota is not enabled (ioctl return -ENOENT) and either parent qgroup or child qgroup does not exist (update_qgroup_relation return -ENOENT). Besides this, any error i

[PATCH v2 3/4] btrfs-progs: qgroup: show subvol path when qgroup show

2017-12-04 Thread Lu Fengqi
Show the absolute subvol path for the associated level-0 qgroup. Signed-off-by: Lu Fengqi --- Changelog: v2: rebase on the qgroup cleanup qgroup.c | 111 +++ qgroup.h | 1 + 2 files changed, 106 insertions(+), 6 deletions(-) diff --

Re: [PATCH 0/7] retry write on error

2017-12-04 Thread Anand Jain
If the underlying protocal doesn't support retry and there are some transient errors happening somewhere in our IO stack, we'd like to give an extra chance for IO. It's true that scsi's sd layer has done SD_MAX_RETRIES(5) retries, but can we really depend on other layers for all robustness

[PATCH] btrfs: fix inconsistency during missing device rejoin

2017-12-04 Thread Anand Jain
When device is missing its not necessary that btrfs_device::name is null or the path is different when it reappears. Its possible that device can go missing after its been scanned where neither of btrfs_device::name == NULL OR btrfs_device::name != reappear_dev_path, is true. So just check for btrf