Re: Understanding BTRFS RAID0 Performance

2018-10-05 Thread Duncan
Wilson, Ellis posted on Fri, 05 Oct 2018 15:29:52 + as excerpted: > Is there any tuning in BTRFS that limits the number of outstanding reads > at a time to a small single-digit number, or something else that could > be behind small queue depths? I can't otherwise imagine what the > difference

Re: [PATCH 02/15] xfs: refactor clonerange preparation into a separate helper

2018-10-05 Thread Dave Chinner
On Fri, Oct 05, 2018 at 10:21:59AM -0700, Darrick J. Wong wrote: > On Fri, Oct 05, 2018 at 07:02:42PM +1000, Dave Chinner wrote: > > On Fri, Oct 05, 2018 at 05:02:28PM +1000, Dave Chinner wrote: > > > On Thu, Oct 04, 2018 at 05:44:47PM -0700, Darrick J. Wong wrote: > > > > From: Darrick J. Wong >

Re: [PATCH 5/6] btrfs: introduce nparity raid_attr

2018-10-05 Thread Hans van Kranenburg
On 10/05/2018 04:42 PM, Nikolay Borisov wrote: > > > On 5.10.2018 00:24, Hans van Kranenburg wrote: >> Instead of hardcoding exceptions for RAID5 and RAID6 in the code, use an >> nparity field in raid_attr. >> >> Signed-off-by: Hans van Kranenburg >> --- >> fs/btrfs/volumes.c | 18 +++-

Re: [PATCH 08/15] vfs: change clone and dedupe range function pointers to return bytes completed

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 11:06:54AM +0300, Amir Goldstein wrote: > On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > Change the clone_file_range and dedupe_file_range functions to return > > the number of bytes they operated on. This is the precursor t

Re: BTRFS related kernel backtrace on boot on 4.18.7 after blackout due to discharged battery

2018-10-05 Thread Martin Steigerwald
Filipe Manana - 05.10.18, 17:21: > On Fri, Oct 5, 2018 at 3:23 PM Martin Steigerwald wrote: > > Hello! > > > > On ThinkPad T520 after battery was discharged and machine just > > blacked out. > > > > Is that some sign of regular consistency check / replay or something > > to investigate further?

Re: [PATCH 09/15] vfs: pass operation flags to {clone, dedupe}_file_range implementations

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 10:07:43AM +0300, Amir Goldstein wrote: > On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > Pass operational flags to the per-filesystem clone and dedupe > > implementations. This enables the vfs to signal when it can deal with

Re: [PATCH 12/15] vfs: implement opportunistic short dedupe

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 09:40:44AM +0300, Amir Goldstein wrote: > On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > For a given dedupe request, the bytes_deduped field in the control > > structure tells userspace if we managed to deduplicate some, but

Re: [PATCH 06/15] vfs: strengthen checking of file range inputs to clone/dedupe range

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 09:10:12AM +0300, Amir Goldstein wrote: > On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > Clone range is an optimization on a regular file write. File writes > > that extend the file length are subject to various constraints

Re: [PATCH 1/2] btrfs: volumes: Make sure there is no overlap dev extents at mount time

2018-10-05 Thread David Sterba
On Fri, Oct 05, 2018 at 05:22:40PM +0800, Qu Wenruo wrote: > > > On 2018/10/5 下午4:56, Nikolay Borisov wrote: > > > > > > On 5.10.2018 11:25, Qu Wenruo wrote: > >> Enhance btrfs_verify_dev_extents() to remember previous checked dev > >> extents, so it can check if one dev extent overlaps with p

Re: [PATCH 02/15] xfs: refactor clonerange preparation into a separate helper

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 07:02:42PM +1000, Dave Chinner wrote: > On Fri, Oct 05, 2018 at 05:02:28PM +1000, Dave Chinner wrote: > > On Thu, Oct 04, 2018 at 05:44:47PM -0700, Darrick J. Wong wrote: > > > From: Darrick J. Wong > > > > > > Refactor all the reflink preparation steps into a separate hel

Re: [PATCH 02/15] xfs: refactor clonerange preparation into a separate helper

2018-10-05 Thread Darrick J. Wong
On Fri, Oct 05, 2018 at 03:28:09PM +1000, Dave Chinner wrote: > On Thu, Oct 04, 2018 at 05:44:47PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Refactor all the reflink preparation steps into a separate helper that > > we'll use to land all the upcoming fixes for insufficient i

Re: Understanding BTRFS RAID0 Performance

2018-10-05 Thread Wilson, Ellis
On 10/05/2018 06:40 AM, Duncan wrote: > Wilson, Ellis posted on Thu, 04 Oct 2018 21:33:29 + as excerpted: > >> Hi all, >> >> I'm attempting to understand a roughly 30% degradation in BTRFS RAID0 >> for large read I/Os across six disks compared with ext4 atop mdadm >> RAID0. >> >> Specifically,

Re: BTRFS related kernel backtrace on boot on 4.18.7 after blackout due to discharged battery

2018-10-05 Thread Filipe Manana
On Fri, Oct 5, 2018 at 3:23 PM Martin Steigerwald wrote: > > Hello! > > On ThinkPad T520 after battery was discharged and machine just blacked > out. > > Is that some sign of regular consistency check / replay or something to > investigate further? I think it's harmless, if anything were messed u

Re: [PATCH 5/6] btrfs: introduce nparity raid_attr

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 00:24, Hans van Kranenburg wrote: > Instead of hardcoding exceptions for RAID5 and RAID6 in the code, use an > nparity field in raid_attr. > > Signed-off-by: Hans van Kranenburg > --- > fs/btrfs/volumes.c | 18 +++--- > fs/btrfs/volumes.h | 2 ++ > 2 files changed,

BTRFS related kernel backtrace on boot on 4.18.7 after blackout due to discharged battery

2018-10-05 Thread Martin Steigerwald
Hello! On ThinkPad T520 after battery was discharged and machine just blacked out. Is that some sign of regular consistency check / replay or something to investigate further? I already scrubbed all data and there are no errors. Also btrfs device stats reports no errors. SMART status appears to

Re: [PATCH] btrfs: Remove unused variable mode in btrfs_mount

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 15:26, Goldwyn Rodrigues wrote: > Code cleanup. > > Signed-off-by: Goldwyn Rodrigues Reviewed-by: Nikolay Borisov > > diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c > index e7f702761cb7..f7b8b7a6b86a 100644 > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1661,14 +16

[PATCH] btrfs: Remove unused variable mode in btrfs_mount

2018-10-05 Thread Goldwyn Rodrigues
Code cleanup. Signed-off-by: Goldwyn Rodrigues diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index e7f702761cb7..f7b8b7a6b86a 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1661,14 +1661,10 @@ static struct dentry *btrfs_mount(struct file_system_type *fs_type, int flags, {

Re: [PATCH 0/6] Chunk allocator DUP fix and cleanups

2018-10-05 Thread Hans van Kranenburg
On 10/05/2018 09:51 AM, Qu Wenruo wrote: > > > On 2018/10/5 上午5:24, Hans van Kranenburg wrote: >> This patch set contains an additional fix for a newly exposed bug after >> the previous attempt to fix a chunk allocator bug for new DUP chunks: >> >> https://lore.kernel.org/linux-btrfs/782f6000-30c

Re: Understanding BTRFS RAID0 Performance

2018-10-05 Thread Duncan
Wilson, Ellis posted on Thu, 04 Oct 2018 21:33:29 + as excerpted: > Hi all, > > I'm attempting to understand a roughly 30% degradation in BTRFS RAID0 > for large read I/Os across six disks compared with ext4 atop mdadm > RAID0. > > Specifically, I achieve performance parity with BTRFS in ter

Re: Transaction aborted error -28 clone_finish_inode_update

2018-10-05 Thread Duncan
David Goodwin posted on Thu, 04 Oct 2018 17:44:46 +0100 as excerpted: > While trying to run/use bedup ( https://github.com/g2p/bedup ) I > hit this : > > > [Thu Oct  4 15:34:51 2018] [ cut here ] > [Thu Oct  4 15:34:51 2018] BTRFS: Transaction aborted (error -28) > [

[PATCH v2 1/2] btrfs: volumes: Make sure there is no overlap dev extents at mount time

2018-10-05 Thread Qu Wenruo
Enhance btrfs_verify_dev_extents() to remember previous checked dev extents, so it can verify no dev extents can overlap. Link: https://www.spinics.net/lists/linux-btrfs/msg82370.html Reported-by: Hans van Kranenburg Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 14 ++ 1 file ch

[PATCH v2 0/2] Enhance btrfs_verify_dev_extents() to do more checks on dev extents

2018-10-05 Thread Qu Wenruo
Inspired by Hans' possible flawed DUP chunk allocator, add the following dev extents checker: 1) Dev extent overlap check Dev extents don't use extent_cache so it can't report dev extents overlap. So manually check dev extents overlap. This check is pretty simple since we're already it

[PATCH v2 2/2] btrfs: volumes: Make sure no dev extent is beyond device boundary

2018-10-05 Thread Qu Wenruo
Add extra dev extent end check against device boundary. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 17 + 1 file changed, 17 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index bf0b2c16847a..bc3ac4715694 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/v

Re: [PATCH 1/2] btrfs: volumes: Make sure there is no overlap dev extents at mount time

2018-10-05 Thread Qu Wenruo
On 2018/10/5 下午4:56, Nikolay Borisov wrote: > > > On 5.10.2018 11:25, Qu Wenruo wrote: >> Enhance btrfs_verify_dev_extents() to remember previous checked dev >> extents, so it can check if one dev extent overlaps with previous dev extent. >> >> Reported-by: Hans van Kranenburg >> Signed-off-

Re: What to do with damaged root fllesystem (opensuse leap 42.2)

2018-10-05 Thread Duncan
Beat Meier posted on Wed, 03 Oct 2018 16:20:14 -0300 as excerpted: > Hello > > I'm using btrfs on opensuse leap 42.2. > > This days I had a power loss and system does not mount anymore root > filesystem with subvolumes. > > My original problem in dmesg was skinny extents and space cache > gener

Re: [PATCH 2/2] btrfs: volumes: Make sure no dev extent is beyond device boundary

2018-10-05 Thread Qu Wenruo
On 2018/10/5 下午4:55, Nikolay Borisov wrote: > > > On 5.10.2018 11:25, Qu Wenruo wrote: >> Add extra dev extent end check against device boundary. >> >> Signed-off-by: Qu Wenruo >> --- >> fs/btrfs/volumes.c | 17 + >> 1 file changed, 17 insertions(+) >> >> diff --git a/fs/btr

Re: [PATCH 4/6] btrfs: fix ncopies raid_attr for RAID56

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 00:24, Hans van Kranenburg wrote: > RAID5 and RAID6 profile store one copy of the data, not 2 or 3. These > values are not used anywhere by the way. > > Signed-off-by: Hans van Kranenburg Reviewed-by: Nikolay Borisov > --- > fs/btrfs/volumes.c | 4 ++-- > 1 file changed, 2 in

Re: [PATCH 02/15] xfs: refactor clonerange preparation into a separate helper

2018-10-05 Thread Dave Chinner
On Fri, Oct 05, 2018 at 05:02:28PM +1000, Dave Chinner wrote: > On Thu, Oct 04, 2018 at 05:44:47PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Refactor all the reflink preparation steps into a separate helper that > > we'll use to land all the upcoming fixes for insufficient i

Re: [PATCH 2/6] btrfs: alloc_chunk: improve chunk size variable name

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 00:24, Hans van Kranenburg wrote: > num_bytes is really a way too generic name for a variable in this > function. There are a dozen other variables that hold a number of bytes > as value. > > Give it a name that actually describes what it does, which is holding > the size of the c

Re: [PATCH 1/6] btrfs: alloc_chunk: do not refurbish num_bytes

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 11:59, Nikolay Borisov wrote: > > > On 5.10.2018 00:24, Hans van Kranenburg wrote: >> num_bytes is used to store the chunk length of the chunk that we're >> allocating. Do not reuse it for something really different in the same >> function. >> >> Signed-off-by: Hans van Kranenbu

Re: [PATCH 1/6] btrfs: alloc_chunk: do not refurbish num_bytes

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 00:24, Hans van Kranenburg wrote: > num_bytes is used to store the chunk length of the chunk that we're > allocating. Do not reuse it for something really different in the same > function. > > Signed-off-by: Hans van Kranenburg nit: That's a minor cleanup and brings no functiona

Re: [PATCH 00/11] Cleanup dev-replace locking

2018-10-05 Thread Anand Jain
I do observer btrfs/011 taking longer time to complete (200sec more) and Null pointer dereference even without this patch set, tracing back lead to conclusion, that Ah. My trace back was for the circular lock dependency warning which I reported here: Subject: btrfs/011 possible circular

Re: [PATCH 1/2] btrfs: volumes: Make sure there is no overlap dev extents at mount time

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 11:25, Qu Wenruo wrote: > Enhance btrfs_verify_dev_extents() to remember previous checked dev > extents, so it can check if one dev extent overlaps with previous dev extent. > > Reported-by: Hans van Kranenburg > Signed-off-by: Qu Wenruo If this is related to the initial report

Re: [PATCH 2/2] btrfs: volumes: Make sure no dev extent is beyond device boundary

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 11:25, Qu Wenruo wrote: > Add extra dev extent end check against device boundary. > > Signed-off-by: Qu Wenruo > --- > fs/btrfs/volumes.c | 17 + > 1 file changed, 17 insertions(+) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index bf0b2c16847a..9f

Re: Understanding BTRFS RAID0 Performance

2018-10-05 Thread Nikolay Borisov
On 5.10.2018 00:33, Wilson, Ellis wrote: > Hi all, > > I'm attempting to understand a roughly 30% degradation in BTRFS RAID0 > for large read I/Os across six disks compared with ext4 atop mdadm RAID0. > > Specifically, I achieve performance parity with BTRFS in terms of > single-threaded wr

Re: [PATCH 07/15] vfs: skip zero-length dedupe requests

2018-10-05 Thread Amir Goldstein
On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong wrote: > > From: Darrick J. Wong > > Don't bother calling the filesystem for a zero-length dedupe request; > we can return zero and exit. > > Signed-off-by: Darrick J. Wong > --- > fs/read_write.c |5 + > 1 file changed, 5 insertions(+) > >

[PATCH 2/2] btrfs: volumes: Make sure no dev extent is beyond device boundary

2018-10-05 Thread Qu Wenruo
Add extra dev extent end check against device boundary. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 17 + 1 file changed, 17 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index bf0b2c16847a..9fb40e30be6a 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/v

[PATCH 1/2] btrfs: volumes: Make sure there is no overlap dev extents at mount time

2018-10-05 Thread Qu Wenruo
Enhance btrfs_verify_dev_extents() to remember previous checked dev extents, so it can check if one dev extent overlaps with previous dev extent. Reported-by: Hans van Kranenburg Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/

[PATCH 0/2] Enhance btrfs_verify_dev_extents() to do more checks on dev extents

2018-10-05 Thread Qu Wenruo
Inspired by Hans' possible flawed DUP chunk allocator, add the following dev extents checker: 1) Dev extent overlap check Dev extents don't use extent_cache so it can't report dev extents overlap. So manually check dev extents overlap. This check is pretty simple since we're already it

Re: [PATCH 08/15] vfs: change clone and dedupe range function pointers to return bytes completed

2018-10-05 Thread Amir Goldstein
On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong wrote: > > From: Darrick J. Wong > > Change the clone_file_range and dedupe_file_range functions to return > the number of bytes they operated on. This is the precursor to allowing > fs implementations to return short clone/dedupe results to the use

Re: [PATCH v6] test unaligned punch hole at ENOSPC

2018-10-05 Thread Filipe Manana
On Sun, Sep 30, 2018 at 2:40 AM Anand Jain wrote: > > Try to punch hole with unaligned size and offset when the FS is > full. Mainly holes are punched at locations which are unaligned > with the file extent boundaries when the FS is full by data. > As the punching holes at unaligned location will

Re: [PATCH 0/6] Chunk allocator DUP fix and cleanups

2018-10-05 Thread Qu Wenruo
On 2018/10/5 上午5:24, Hans van Kranenburg wrote: > This patch set contains an additional fix for a newly exposed bug after > the previous attempt to fix a chunk allocator bug for new DUP chunks: > > https://lore.kernel.org/linux-btrfs/782f6000-30c0-0085-abd2-74ec5827c...@mendix.com/T/#m609ccb5d32

Re: [PATCH 0/6] Chunk allocator DUP fix and cleanups

2018-10-05 Thread Qu Wenruo
On 2018/10/5 上午5:24, Hans van Kranenburg wrote: > This patch set contains an additional fix for a newly exposed bug after > the previous attempt to fix a chunk allocator bug for new DUP chunks: > > https://lore.kernel.org/linux-btrfs/782f6000-30c0-0085-abd2-74ec5827c...@mendix.com/T/#m609ccb5d32

Re: [PATCH 09/15] vfs: pass operation flags to {clone, dedupe}_file_range implementations

2018-10-05 Thread Amir Goldstein
On Fri, Oct 5, 2018 at 3:46 AM Darrick J. Wong wrote: > > From: Darrick J. Wong > > Pass operational flags to the per-filesystem clone and dedupe > implementations. This enables the vfs to signal when it can deal with > short clone and short dedupe operations. > > Signed-off-by: Darrick J. Wong

Re: [PATCH 02/15] xfs: refactor clonerange preparation into a separate helper

2018-10-05 Thread Dave Chinner
On Thu, Oct 04, 2018 at 05:44:47PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong > > Refactor all the reflink preparation steps into a separate helper that > we'll use to land all the upcoming fixes for insufficient input checks. > > Signed-off-by: Darrick J. Wong . > +xfs_reflink_