Fwd: [PATCH v2] btrfs: add mount umount logs

2017-07-13 Thread Anand Jain
Hi David, Any response on this ? Thanks, Anand Forwarded Message Subject: [PATCH v2] btrfs: add mount umount logs Date: Thu, 15 Jun 2017 17:20:42 +0800 From: Anand Jain To: linux-btrfs@vger.kernel.org CC: dste...@suse.cz From: Anand Jain By looking at the logs we should be

[PATCH 1/3] btrfs: Remove unused variables

2017-07-13 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov --- fs/btrfs/volumes.c | 9 - 1 file changed, 9 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 6466f53b1e81..9a38cbf233a1 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1873,7 +1873,6 @@ int btrfs_rm_device(struct btrf

[PATCH 2/3] btrfs: Remove unused parameters from volume.c functions

2017-07-13 Thread Nikolay Borisov
This also adjusts the respective callers in other files. Those were found with -Wunused-parameter. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 6 ++ fs/btrfs/extent_io.c | 2 +- fs/btrfs/volumes.c | 7 ++- fs/btrfs/volumes.h | 3 +-- 4 files changed, 6 insertion

[PATCH 0/3] Cleanup volume.c of unused parameters/variables

2017-07-13 Thread Nikolay Borisov
Here are 3 cleanup patches which remove unused variables, parameters and kill a dummy function. Nikolay Borisov (3): btrfs: Remove unused variables btrfs: Remove unused parameters from volume.c functions btrfs: Remove find_raid56_stripe_len fs/btrfs/extent-tree.c | 6 ++ fs/btrfs/ex

[PATCH 3/3] btrfs: Remove find_raid56_stripe_len

2017-07-13 Thread Nikolay Borisov
find_raid56_stripe_len statically returns SZ_64K which equals BTRFS_STRIPE_LEN. It's sole caller is __btrfs_alloc_chunk and it assigns the return value to ai variable which is already set to BTRFS_STRIPE_LEN. So remove the function invocation altogether and remove the function itself. Also remove t

[PATCH 3/3] btrfs: Remove find_raid56_stripe_len

2017-07-13 Thread Nikolay Borisov
find_raid56_stripe_len statically returns SZ_64K which equals BTRFS_STRIPE_LEN. It's sole caller is __btrfs_alloc_chunk and it assigns the return value to ai variable which is already set to BTRFS_STRIPE_LEN. So remove the function invocation altogether and remove the function itself. Also remove t

Re: [PATCH 3/3] btrfs: Remove find_raid56_stripe_len

2017-07-13 Thread Nikolay Borisov
On 13.07.2017 11:45, Nikolay Borisov wrote: > find_raid56_stripe_len statically returns SZ_64K which equals > BTRFS_STRIPE_LEN. > It's sole caller is __btrfs_alloc_chunk and it assigns the return value to ai > variable which is already set to BTRFS_STRIPE_LEN. So remove the function > invocation

[PATCH v2] btrfs: Remove never reached error handling code in __add_reloc_root

2017-07-13 Thread Nikolay Borisov
One of the error handling paths in __add_reloc_root contains btrfs_panic() followed by some other code. As the name implies what it does is print some error message and call BUG, naturally what follow afterwards is not invoked. So remove this extra code. Signed-off-by: Nikolay Borisov --- fs/btr

Re: [PATCH v4 0/6] Chunk level degradable check

2017-07-13 Thread Austin S. Hemmelgarn
On 2017-07-12 21:09, Adam Borowski wrote: On Thu, Jul 13, 2017 at 02:50:10AM +0200, David Sterba wrote: On Mon, Jul 10, 2017 at 09:11:50PM +0300, Dmitrii Tcvetkov wrote: Tested on top of current mainline master (commit af3c8d98508d37541d4bf57f13a984a7f73a328c). Didn't find any regressions. I'

[PATCH] btrfs: allow defrag compress to override NOCOMPRESS attribute

2017-07-13 Thread David Sterba
Currently, the BTRFS_INODE_NOCOMPRESS will prevent any compression on a given file, except when the mount is force-compress. As users have reported on IRC, this will also prevent compression when requested by defrag (btrfs fi defrag -c file). The nocompress flag is set automatically by filesystem

Re: [PATCH] btrfs-progs: add support to sort by topid

2017-07-13 Thread Jeffrey Mahoney
On 7/13/17 12:16 AM, Anand Jain wrote: > As users generally organize the subvols and snapshots based on the subvol > directory hierarchy. So providing an ability to sort them by topid would > help. Thanks. What is a topid? I needed to look at the code to discover this and it's not even documented

[PATCH] btrfs: drop newlines from strings when using btrfs_* helpers

2017-07-13 Thread David Sterba
The helpers append "\n" so we can keep the actual strings shorter. The extra newline will print an empty line. Some messages have been slightly modified to be more consistent with the rest (lowercase first letter). Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/fre

Re: [PATCH] btrfs-progs: add support to sort by topid

2017-07-13 Thread Holger Hoffstätte
On 07/13/17 15:25, Jeffrey Mahoney wrote: > On 7/13/17 12:16 AM, Anand Jain wrote: >> As users generally organize the subvols and snapshots based on the subvol >> directory hierarchy. So providing an ability to sort them by topid would >> help. Thanks. > > What is a topid? I needed to look at the

[PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread fdmanana
From: Filipe Manana The recent changes to make bio cloning faster (added in the 4.13 merge window) by using the bio_clone_fast() API introduced a regression on raid5/6 modes, because cloned bios have an invalid bi_vcnt field (therefore it can not be used) and the raid5/6 code uses the bio_for_eac

[PATCH] btrfs: test direct IO write against raid5/6 filesystems

2017-07-13 Thread fdmanana
From: Filipe Manana Test that a direct IO write works against raid5/6 filesystems and that after the write operation we are able to read back the correct data and scrub operations don't find any errors. This test is motivated by a regression introduced in the merge window for the 4.13 linux kern

BTRFS file I/O with O_DIRECT

2017-07-13 Thread Aleksandar Čekrlić
Hi all, I am currently working on a file system related code where user can opt to use direct I/O for file reads / writes. Code seems to be working and operational, and on majority of the file systems I/O is not using the page cache, but on BTRFS even though file is opened with O_DIRECT file is ca

Re: [PATCH] btrfs-progs: add support to sort by topid

2017-07-13 Thread Jeffrey Mahoney
On 7/13/17 9:58 AM, Holger Hoffstätte wrote: > On 07/13/17 15:25, Jeffrey Mahoney wrote: >> On 7/13/17 12:16 AM, Anand Jain wrote: >>> As users generally organize the subvols and snapshots based on the subvol >>> directory hierarchy. So providing an ability to sort them by topid would >>> help. Tha

Re: [PATCH] btrfs: drop newlines from strings when using btrfs_* helpers

2017-07-13 Thread Nikolay Borisov
On 13.07.2017 16:43, David Sterba wrote: > The helpers append "\n" so we can keep the actual strings shorter. The > extra newline will print an empty line. Some messages have been > slightly modified to be more consistent with the rest (lowercase first > letter). > > Signed-off-by: David Sterba

[PATCH] btrfs: btrfs_create_repair_bio never fails, skip error handling

2017-07-13 Thread David Sterba
As it uses the non-failinig bio allocation, we can remove error handling from the callers as well. Signed-off-by: David Sterba --- fs/btrfs/extent_io.c | 4 fs/btrfs/inode.c | 4 2 files changed, 8 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 2e6f699

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread David Sterba
On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > The recent changes to make bio cloning faster (added in the 4.13 merge > window) by using the bio_clone_fast() API introduced a regression on > raid5/6 modes, because cloned bios have an invalid bi_vcnt

[PATCH] btrfs: cloned bios must not be iterated by bio_for_each_segment_all

2017-07-13 Thread David Sterba
We've started using cloned bios more in 4.13, there are some specifics regarding the iteration. Filipe found [1] that the raid56 iterated a cloned bio using bio_for_each_segment_all, which is incorrect. The cloned bios have wrong bi_vcnt and this could lead to silent corruptions. This patch adds

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > The recent changes to make bio cloning faster (added in the 4.13 merge > window) by using the bio_clone_fast() API introduced a regression on > raid5/6 modes, because cloned bios have an invalid bi_vcnt

Re: [PATCH] btrfs: cloned bios must not be iterated by bio_for_each_segment_all

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 06:15:36PM +0200, David Sterba wrote: > We've started using cloned bios more in 4.13, there are some specifics > regarding the iteration. Filipe found [1] that the raid56 iterated a > cloned bio using bio_for_each_segment_all, which is incorrect. The > cloned bios have wron

Re: [PATCH] btrfs: btrfs_create_repair_bio never fails, skip error handling

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 05:42:15PM +0200, David Sterba wrote: > As it uses the non-failinig bio allocation, we can remove error handling > from the callers as well. > Looks good. Reviewed-by: Liu Bo -liubo > Signed-off-by: David Sterba > --- > fs/btrfs/extent_io.c | 4 > fs/btrfs/inode.c

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Filipe Manana
On Thu, Jul 13, 2017 at 5:36 PM, Liu Bo wrote: > On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> The recent changes to make bio cloning faster (added in the 4.13 merge >> window) by using the bio_clone_fast() API introduced a regression on >> raid

Re: BTRFS file I/O with O_DIRECT

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 04:16:44PM +0200, Aleksandar Čekrlić wrote: > Hi all, > > I am currently working on a file system related code where user can > opt to use direct I/O for file reads / writes. Code seems to be > working and operational, and on majority of the file systems I/O is > not using

Re: [PATCH] btrfs: test direct IO write against raid5/6 filesystems

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 03:10:40PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that a direct IO write works against raid5/6 filesystems and that > after the write operation we are able to read back the correct data > and scrub operations don't find any errors. > > This test

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 05:46:13PM +0100, Filipe Manana wrote: > On Thu, Jul 13, 2017 at 5:36 PM, Liu Bo wrote: > > On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: > >> From: Filipe Manana > >> [...] > > > > > >> Signed-off-by: Filipe Manana > >> --- > >> fs/btrfs/raid56.c

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 10:06:32AM -0700, Liu Bo wrote: > On Thu, Jul 13, 2017 at 05:46:13PM +0100, Filipe Manana wrote: > > On Thu, Jul 13, 2017 at 5:36 PM, Liu Bo wrote: > > > On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: > > >> From: Filipe Manana > > >> > [...] > > > >

Re: BTRFS: error (device dm-2) in btrfs_run_delayed_refs:2960: errno=-17 Object already exists

2017-07-13 Thread Chris Murphy
On Wed, Jul 12, 2017 at 7:10 PM, Marc MERLIN wrote: > On Tue, Jul 11, 2017 at 09:48:12AM -0700, Marc MERLIN wrote: >> On Tue, Jul 11, 2017 at 10:00:40AM -0600, Chris Murphy wrote: >> > > ---[ end trace feb4b95c83ac065f ]--- >> > > BTRFS: error (device dm-2) in btrfs_run_delayed_refs:2960: errno=-1

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Filipe Manana
On Thu, Jul 13, 2017 at 7:00 PM, Liu Bo wrote: > On Thu, Jul 13, 2017 at 10:06:32AM -0700, Liu Bo wrote: >> On Thu, Jul 13, 2017 at 05:46:13PM +0100, Filipe Manana wrote: >> > On Thu, Jul 13, 2017 at 5:36 PM, Liu Bo wrote: >> > > On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 07:11:24PM +0100, Filipe Manana wrote: > On Thu, Jul 13, 2017 at 7:00 PM, Liu Bo wrote: > > On Thu, Jul 13, 2017 at 10:06:32AM -0700, Liu Bo wrote: > >> On Thu, Jul 13, 2017 at 05:46:13PM +0100, Filipe Manana wrote: > >> > On Thu, Jul 13, 2017 at 5:36 PM, Liu Bo wrote: > >

Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Chris Murphy
Has anyone been working with Docker and Btrfs + overlayfs? It seems superfluous or unnecessary to use overlayfs, but the shared page cache aspect and avoiding some of the problems with large numbers of Btrfs snapshots, might make it a useful combination. But I'm not finding useful information with

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 02:49:27PM -0600, Chris Murphy wrote: > Has anyone been working with Docker and Btrfs + overlayfs? It seems > superfluous or unnecessary to use overlayfs, but the shared page cache > aspect and avoiding some of the problems with large numbers of Btrfs > snapshots, might make

Re: [PATCH] Btrfs: fix write corruption due to bio cloning on raid5/6

2017-07-13 Thread Liu Bo
On Thu, Jul 13, 2017 at 09:36:19AM -0700, Liu Bo wrote: > On Thu, Jul 13, 2017 at 03:09:54PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > The recent changes to make bio cloning faster (added in the 4.13 merge > > window) by using the bio_clone_fast() API introduced a regressi

[PATCH] Btrfs: fix unexpected return value of bio_readpage_error

2017-07-13 Thread Liu Bo
With blk_status_t, in bio_readpage_error() may return 1 as now ->submit_bio_hook() may not set %ret if it runs without problem. This fixes that unexpected return value by changing btrfs_check_repairable() to return a bool instead of updating %ret. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Chris Murphy
On Thu, Jul 13, 2017 at 4:32 PM, Liu Bo wrote: > On Thu, Jul 13, 2017 at 02:49:27PM -0600, Chris Murphy wrote: >> Has anyone been working with Docker and Btrfs + overlayfs? It seems >> superfluous or unnecessary to use overlayfs, but the shared page cache >> aspect and avoiding some of the problem

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Qu Wenruo
On 2017年07月14日 07:26, Chris Murphy wrote: On Thu, Jul 13, 2017 at 4:32 PM, Liu Bo wrote: On Thu, Jul 13, 2017 at 02:49:27PM -0600, Chris Murphy wrote: Has anyone been working with Docker and Btrfs + overlayfs? It seems superfluous or unnecessary to use overlayfs, but the shared page cache as

Re: [PATCH 3/3] btrfs: Remove find_raid56_stripe_len

2017-07-13 Thread Qu Wenruo
On 2017年07月13日 16:45, Nikolay Borisov wrote: find_raid56_stripe_len statically returns SZ_64K which equals BTRFS_STRIPE_LEN. It's sole caller is __btrfs_alloc_chunk and it assigns the return value to ai variable which is already set to BTRFS_STRIPE_LEN. So remove the function invocation altoget

Re: [PATCH 1/3] btrfs: Remove unused variables

2017-07-13 Thread Qu Wenruo
On 2017年07月13日 16:37, Nikolay Borisov wrote: Signed-off-by: Nikolay Borisov Looks good. Reviewed-by: Qu Wenruo Kernel makefile disabled unused-but-set-variable warning so we can't detect it in normal compile. Thanks, Qu --- fs/btrfs/volumes.c | 9 - 1 file changed, 9 delet

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Sargun Dhillon
On Thu, Jul 13, 2017 at 7:01 PM, Qu Wenruo wrote: > > > On 2017年07月14日 07:26, Chris Murphy wrote: >> >> On Thu, Jul 13, 2017 at 4:32 PM, Liu Bo wrote: >>> >>> On Thu, Jul 13, 2017 at 02:49:27PM -0600, Chris Murphy wrote: Has anyone been working with Docker and Btrfs + overlayfs? It seem

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Chris Murphy
On Thu, Jul 13, 2017 at 8:01 PM, Qu Wenruo wrote: > > > On 2017年07月14日 07:26, Chris Murphy wrote: >> No benchmarking comparison but it's known that deletion of snapshots >> gets more expensive when there are many snapshots due to backref >> search and metadata updates. I have no idea how it compa

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Qu Wenruo
On 2017年07月14日 10:24, Sargun Dhillon wrote: On Thu, Jul 13, 2017 at 7:01 PM, Qu Wenruo wrote: On 2017年07月14日 07:26, Chris Murphy wrote: On Thu, Jul 13, 2017 at 4:32 PM, Liu Bo wrote: On Thu, Jul 13, 2017 at 02:49:27PM -0600, Chris Murphy wrote: Has anyone been working with Docker and

Re: Containers, Btrfs vs Btrfs + overlayfs

2017-07-13 Thread Chris Murphy
On Thu, Jul 13, 2017 at 8:52 PM, Qu Wenruo wrote: > > Thanks for mentioning the underlying storage. > SSD makes FUA overhead smaller, so with SSD the metadata CoW is less > obvious. Typically there are 2 or 3 superblocks. SSD mount option causes rotation of superblock updates. I wonder the effec

Re: [PATCH] btrfs: test direct IO write against raid5/6 filesystems

2017-07-13 Thread Eryu Guan
On Thu, Jul 13, 2017 at 03:10:40PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that a direct IO write works against raid5/6 filesystems and that > after the write operation we are able to read back the correct data > and scrub operations don't find any errors. > > This test

Re: Btrfs check reports errors, filesystem seems fine

2017-07-13 Thread Qu Wenruo
Thanks for your dump. We're clear what is the direct cause of the problem. It's one corrupted DIR_ITEM causing the problem. And further more, original mode btrfs check can't detect it, and we will fix it soon. The corrupted DIR_ITEM is as the following: item 72 key (79177 DIR_ITEM 548

Re: [PATCH] btrfs-progs: add support to sort by topid

2017-07-13 Thread Anand Jain
On 07/13/2017 09:58 PM, Holger Hoffstätte wrote: On 07/13/17 15:25, Jeffrey Mahoney wrote: On 7/13/17 12:16 AM, Anand Jain wrote: As users generally organize the subvols and snapshots based on the subvol directory hierarchy. So providing an ability to sort them by topid would help. Thanks.

Re: [PATCH] btrfs-progs: add support to sort by topid

2017-07-13 Thread Anand Jain
On 07/13/2017 10:27 PM, Jeffrey Mahoney wrote: On 7/13/17 9:58 AM, Holger Hoffstätte wrote: On 07/13/17 15:25, Jeffrey Mahoney wrote: On 7/13/17 12:16 AM, Anand Jain wrote: As users generally organize the subvols and snapshots based on the subvol directory hierarchy. So providing an ability

Re: BTRFS file I/O with O_DIRECT

2017-07-13 Thread Aleksandar Čekrlić
Hi Liu, indeed setting the address alignment and read chunk to 4k did the trick, everything worked as expected. I expected in the case of the wrong alignment read() would fail with errno == EINVAL like for some other file systems, not to fallback to buffered read. Thanks a lot for the help! BR,

[PATCH v2] btrfs: Remove find_raid56_stripe_len

2017-07-13 Thread Nikolay Borisov
find_raid56_stripe_len statically returns SZ_64K which equals BTRFS_STRIPE_LEN. It's sole caller is __btrfs_alloc_chunk and it assigns the return value to ai variable which is already set to BTRFS_STRIPE_LEN. So remove the function invocation altogether and remove the function itself. Also remove t