[GIT PULL] Btrfs

Hi Linus, I have two more in my for-linus-4.3 branch git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus-4.3 My commit fixes a bug we hit last week at FB, a combination of lots of hard links and an admin command to resolve inode numbers. Dave is adding checks to make

Re: wrong message for succesful btrfs fi resize

It turns out that the 'ERROR: unable to resize - No space left on device' IS correct for the cases I used it. It is just that in the dmesg output, the new size is reported first and then the relocation info. I misinterpreted the info. I did some balances between the resize commands (to

[PATCH] btrfs: add balance filters limits and stripes to supported mask

Enable the extended 'limit' syntax (a range) and the new 'stripes' filter in the filters mask. The patch comes separate and not within the series that introduced the new filters because the patch adding the mask was merged in a late rc. The integration branch was based on an older rc and could not

[PATCH v2] Btrfs: fix truncation of compressed and inlined extents

From: Filipe Manana When truncating a file to a smaller size which consists of an inline extent that is compressed, we did not discard (or made unusable) the data between the new file size and the old file size, wasting metadata space and allowing for the truncated data to be

[PATCH 2/2] btrfs: add balance filter for stripes

From: Gabríel Arthúr Pétursson Balance block groups which have the given number of stripes, defined by a range min..max. This is useful to selectively rebalance only chunks that do not span enough devices, applies to RAID0/10/5/6. Signed-off-by: Gabríel Arthúr Pétursson

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Fri, Oct 16, 2015 at 09:19:50AM -0400, Chris Mason wrote: > On Fri, Oct 16, 2015 at 05:25:44AM -0700, Christoph Hellwig wrote: > > On Fri, Oct 16, 2015 at 07:49:19AM -0400, Chris Mason wrote: > > > > Yes, that would be my preference. I'd also like to understand what > > > > exactly btrfs does

Re: [PATCH v6 5/4] copy_file_range.2: New page documenting copy_file_range()

It's probably worth mentioning the sparse expansion caveat in the docs, as it's important and not obvious. I.E. copy_file_range() could be called from, but would still benefit from a user space app using a SEEK_{HOLE,DATA} loop. thanks, Pádraig. -- To unsubscribe from this list: send the line

Re: [PATCH v6 5/4] copy_file_range.2: New page documenting copy_file_range()

> On Oct 16, 2015, at 3:08 PM, Anna Schumaker wrote: > > copy_file_range() is a new system call for copying ranges of data > completely in the kernel. This gives filesystems an opportunity to > implement some kind of "copy acceleration", such as reflinks or >

[PATCH v6 0/4] VFS: In-kernel copy system call

Copy system calls came up during Plumbers a while ago, mostly because several filesystems (including NFS and XFS) are currently working on copy acceleration implementations. We haven't heard from Zach Brown in a while, so I volunteered to push his patches upstream so individual filesystems don't

[PATCH v6 3/4] btrfs: add .copy_file_range file operation

From: Zach Brown This rearranges the existing COPY_RANGE ioctl implementation so that the .copy_file_range file operation can call the core loop that copies file data extent items. The extent copying loop is lifted up into its own function. It retains the core btrfs error

[PATCH v6 5/4] copy_file_range.2: New page documenting copy_file_range()

copy_file_range() is a new system call for copying ranges of data completely in the kernel. This gives filesystems an opportunity to implement some kind of "copy acceleration", such as reflinks or server-side-copy (in the case of NFS). Signed-off-by: Anna Schumaker

[PATCH v6 1/4] vfs: add copy_file_range syscall and vfs helper

From: Zach Brown Add a copy_file_range() system call for offloading copies between regular files. This gives an interface to underlying layers of the storage stack which can copy without reading and writing all the data. There are a few candidates that should support copy

[PATCH v6 4/4] vfs: Add vfs_copy_file_range() support for pagecache copies

This allows us to have an in-kernel copy mechanism that avoids frequent switches between kernel and user space. This is especially useful so NFSD can support server-side copies. The default (flags=0) means to first attempt copy acceleration, but use the pagecache if that fails. I moved the

[PATCH v6 2/4] x86: add sys_copy_file_range to syscall tables

From: Zach Brown Add sys_copy_file_range to the x86 syscall tables. Signed-off-by: Zach Brown [Anna Schumaker: Update syscall number in syscall_32.tbl] Signed-off-by: Anna Schumaker --- arch/x86/entry/syscalls/syscall_32.tbl | 1 +

Re: Recover btrfs volume which can only be mounded in read-only mode

Dmitry Katsubo posted on Thu, 15 Oct 2015 16:10:13 +0200 as excerpted: > On 15 October 2015 at 02:48, Duncan <1i5t5.dun...@cox.net> wrote: > >> [snipped] > > Thanks for this information. As far as I can see, btrfs-tools v4.1.2 in > now in experimental Debian repo (but you anyway suggest at

btrfs replace status >100%

Hi All, I am noticing some strange numbers when replacing a disk under RAID1: [root@array ~]# btrfs replace status -1 /export/archive/ 367.2% done, 0 write errs, 0 uncorr. read errs The filesystem is currently at 6x 6TB + 2x 5TB, replacing the first of each 5TB with 8TB units: [root@array ~]#

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 2015-10-16 01:38, Christoph Hellwig wrote: On Thu, Oct 15, 2015 at 08:24:51AM -0400, Austin S Hemmelgarn wrote: My only point with saying we shouldn't reflink by default is that there are many (unintelligent) people who will assume that since the syscall has copy in it's name, that's what it

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Fri, Oct 16, 2015 at 07:46:41AM -0400, Austin S Hemmelgarn wrote: > I should have been specific, what I meant was that some people will assume > that it actually creates a physical, on-disk byte-for-byte copy of the data. > There are many people out there (and sadly I have to deal with some at

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Fri, Oct 16, 2015 at 07:49:19AM -0400, Chris Mason wrote: > > Yes, that would be my preference. I'd also like to understand what > > exactly btrfs does in fallocate. > > For which part? The answer changes based on how many references there > are to a given fallocated region. Both cases.

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 16/10/15 12:46, Austin S Hemmelgarn wrote: > On 2015-10-16 01:38, Christoph Hellwig wrote: >> On Thu, Oct 15, 2015 at 08:24:51AM -0400, Austin S Hemmelgarn wrote: >>> My only point with saying we shouldn't reflink by default is that there are >>> many (unintelligent) people who will assume that

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Fri, Oct 16, 2015 at 01:02:23PM +0100, P??draig Brady wrote: > Right. reflinking is transparent to the user, though its consequences are not. > Consequences being the possible extra latency or ENOSPC on CoW. You can get all these consequences without doing the file system reflink by using a

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Wed, Oct 14, 2015 at 11:00:45PM -0700, Christoph Hellwig wrote: > On Wed, Oct 14, 2015 at 11:46:08AM -0700, Darrick J. Wong wrote: > > The documentation for fallocate ought to be updated to include that as part > > of > > guaranteeing that subsequent writes to the range won't fail due to

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 2015-10-16 08:24, Christoph Hellwig wrote: On Fri, Oct 16, 2015 at 01:02:23PM +0100, P??draig Brady wrote: Right. reflinking is transparent to the user, though its consequences are not. Consequences being the possible extra latency or ENOSPC on CoW. You can get all these consequences

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 2015-10-16 08:21, Christoph Hellwig wrote: On Fri, Oct 16, 2015 at 07:46:41AM -0400, Austin S Hemmelgarn wrote: I should have been specific, what I meant was that some people will assume that it actually creates a physical, on-disk byte-for-byte copy of the data. There are many people out

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Fri, Oct 16, 2015 at 05:25:44AM -0700, Christoph Hellwig wrote: > On Fri, Oct 16, 2015 at 07:49:19AM -0400, Chris Mason wrote: > > > Yes, that would be my preference. I'd also like to understand what > > > exactly btrfs does in fallocate. > > > > For which part? The answer changes based on

[PATCH] Btrfs: fix truncation of compressed and inlined extents

From: Filipe Manana When truncating a file to a smaller size which consists of an inline extent that is compressed, we did not discard (or made unusable) the data between the new file size and the old file size, wasting metadata space and allowing for the truncated data to be

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Fri, Oct 16, 2015 at 08:50:41AM -0400, Austin S Hemmelgarn wrote: > Certain parts of userspace do try to reflink things instead of copying (for > example, coreutils recently started doing so in mv and has had the option to > do so with cp for a while now), but a properly designed general

[PATCH] fstests: test btrfs cloning of compressed inline extent after truncation

From: Filipe Manana Test that truncating a file that consists of a compressed and inlined extent to a smaller size and then cloning it into another file is not possible and does not result in leaking stale data (data past the truncation offset) nor losing data in the clone

wrong message for succesful btrfs fi resize

While resizing a filesystem a multiple times in a row (even when unmount,mount in between), the output of btrfs tools mentions error, no space left on device, while dmesg mentions the new size and the action was in fact succesful (no errors so far). It was quite confusing the first time I found

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 2015-10-16 09:12, Christoph Hellwig wrote: On Fri, Oct 16, 2015 at 08:50:41AM -0400, Austin S Hemmelgarn wrote: Certain parts of userspace do try to reflink things instead of copying (for example, coreutils recently started doing so in mv and has had the option to do so with cp for a while

Re: btrfs replace status >100%

Hi Brett, As far as I can see, the replace will finish successfully, but the numbers are quite confusing. Maybe the percentage at the end (after more than half a day or so) gives a hint how the calculation is done, I currently can't see it. RAID1 and RAID10 have confusing output for commands

[PATCH 3/8] btrfs-progs: extend parse_range API to accept a relaxed range

In some cases we want to accept a range of type [a..a]. Add a new function to do the 'a < b' check for the caller and use it. Signed-off-by: David Sterba --- cmds-balance.c | 30 +- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git

[PATCH 1/8] btrfs-progs: cleanup and comment parse_range

Simplify a check and unindent some code. Signed-off-by: David Sterba --- cmds-balance.c | 63 ++ 1 file changed, 37 insertions(+), 26 deletions(-) diff --git a/cmds-balance.c b/cmds-balance.c index

[PATCH 4/8] btrfs-progs: add helpers for parsing 32bit ranges

Signed-off-by: David Sterba --- cmds-balance.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/cmds-balance.c b/cmds-balance.c index 1fd8d2018b7a..457d693eec42 100644 --- a/cmds-balance.c +++ b/cmds-balance.c @@ -161,6 +161,37 @@ static int

[PATCH 2/8] btrfs-progs: do not modify the string in parse_range

It's passed as const but we modify it through 'dots'. This would break parsing the string multiple times. Signed-off-by: David Sterba --- cmds-balance.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/cmds-balance.c b/cmds-balance.c index

[PATCH 6/8] btrfs-progs: extend balance args to take min/max limit filter

Add the overlapping limit and [limit_min, limit_max] members to the balance args. The min/max values are interpreted iff the corresponding flag BTRFS_BALANCE_ARGS_LIMITS is set. Note that the values are only 32bit, but this should be enough for the foreseeable future. Signed-off-by: David Sterba

[PATCH 0/8 v2] Btrfs-progs, balance filters: stripes, limits

An updated version that actually works. David Sterba (7): btrfs-progs: cleanup and comment parse_range btrfs-progs: do not modify the string in parse_range btrfs-progs: extend parse_range API to accept a relaxed range btrfs-progs: add helpers for parsing 32bit ranges btrfs-progs: add

[PATCH 8/8] btrfs-progs: balance: add stripes filter

From: Gabríel Arthúr Pétursson Add new balance filter 'stripes=' to process only chunks that are spread accross given number of chunks. The range must be specified with both values, but they can be the same to denote exact number of stripes. Signed-off-by: Gabríel Arthúr

[PATCH 7/8] btrfs-progs: balance: enhance the limit fiter with range

We can do more with the balance filer. Enhance it so we can specify also the minimum number of block groups to process. The 'limit' filter now accepts a range (a..b, can be partial) and needs kernel support. The 'limit=value' filter is equivalent to 'limit=..value' but works on older kernels as

[PATCH 5/8] btrfs-progs: add helpers to print ranges

Signed-off-by: David Sterba --- cmds-balance.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/cmds-balance.c b/cmds-balance.c index 457d693eec42..3166d7a73ba2 100644 --- a/cmds-balance.c +++ b/cmds-balance.c @@ -192,6 +192,25 @@ static int

Re: [PATCH 1/2] btrfs: extend balance filter limit to take minimum and maximum

On Fri, Oct 16, 2015 at 06:50:08PM +0200, David Sterba wrote: > The 'limit' filter is underdesigned, it should have been a range for > [min,max], with some relaxed semantics when one of the bounds is > missing. Besides that, using a full u64 for a single value is a waste of > bytes. What is min

Re: btrfs replace status >100%

Thanks Henk, That's encouraging and what I suspected - everything looks fine (even some bitrot picked up through read error corrections on checksum failures, hurray!) and yes I'll resize the new device once it completes. [root@array ~]# btrfs replace status -1 /export/archive/ 547.9% done, 0

[PULL][PATCH 0/2 v2] Balance filters: stripes, enhanced limit

Changelog (v2): * the limit_min logic was wrong and did not work, all other filters are locally applied to the processed chunk, but for the limit_min we need to count all filters first and then decide if we want to rebalance or not Update to balance filters, intended for 4.4: * new

[PATCH 1/2] btrfs: extend balance filter limit to take minimum and maximum

The 'limit' filter is underdesigned, it should have been a range for [min,max], with some relaxed semantics when one of the bounds is missing. Besides that, using a full u64 for a single value is a waste of bytes. Let's fix both by extending the use of the u64 bytes for the [min,max] range. This