[PATCH v2 6/6] btrfs-progs: add chattr support for send/receive

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Presently, btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds userspace support for inode attribute flags. Kernel support can be found under the commit: btrfs: add chattr

[PATCH v2 1/6] Btrfs-progs: send, bump stream version

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana This increases the send stream version from version 1 to version 2, adding new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in files; 3) i

[PATCH v2 4/6] Btrfs-progs: add write and clone commands debug info to receive

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana When specifying -vv print information about received write and clone commands too, as we do this for other commands already and it's very useful for debugging and troubleshooting. Signed-off-by: Filipe David Borba Manana --- cmds-receive.c | 9 + 1 file changed, 9

[RFC PATCH v2 3/6] btrfs: send, use fallocate command to punch holes

2018-05-27 Thread Howard McLauchlan
-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana --- fs/btrfs/send.c | 54 ++--- fs/btrfs/send.h | 4 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 7b184831812b

[RFC PATCH v2 4/6] btrfs: send, use fallocate command to allocate extents

2018-05-27 Thread Howard McLauchlan
s 14: [17576..21487]: 28144..32055 3912 blocks The proposed xfstest can be found at: xfstests: btrfs, test send's ability to punch holes and prealloc extents This test verifies that send-stream version 2 does space pre-allocation and hole punching. [Howar

[PATCH v2 5/6] btrfs-progs: add total data size, fallocate to dump

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Adding entries to dump for new commands (total data size, fallocate). Signed-off-by: Howard McLauchlan --- send-dump.c | 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/send-dump.c b/send-dump.c index 1591e0cc..c5a695a2 100644

[RFC PATCH v2 1/6] btrfs: send, bump stream version

2018-05-27 Thread Howard McLauchlan
(BTRFS_SEND_FLAG_CALCULATE_DATA_SIZE | BTRFS_SEND_FLAG_STREAM_V2), meaning old clients are unaffected. [Howard: rebased on 4.17-rc7] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana Reviewed-by: Omar Sandoval --- fs/btrfs/send.c| 7 ++- fs/btrfs/send.h| 22

[PATCH v2 2/6] Btrfs-progs: send, implement total data size callback and progress report

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana This is a followup to the kernel patch titled: Btrfs: send, implement total data size command to allow for progress estimation This makes the btrfs send and receive commands aware of the new send flag, named BTRFS_SEND_C_TOTAL_DATA_SIZE, which tells us the amount of fil

[PATCH v2 2/2] xfstests: btrfs, test send's ability to punch holes, prealloc extents and send data size

2018-05-27 Thread Howard McLauchlan
ated by an incremental send. Also update existing test btrfs/161 to verify total data size [Howard: rebased on kernel v4.17-rc7, btrfs progs v4.16.1] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana --- common/rc | 10 tests/btrfs/161 | 10 ++-- tests

[RFC PATCH v2 5/6] btrfs: add send_stream_version attribute to sysfs

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana So that applications can find out what's the highest send stream version supported/implemented by the running kernel: $ cat /sys/fs/btrfs/send/stream_version 2 [Howard: rebased on 4.17-rc7] Reviewed-by: Omar Sandoval Signed-off-by: Howard McLau

[RFC PATCH v2 6/6] btrfs: add chattr support for send/receive

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Presently btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds kernel support for inode attribute flags. Userspace support can be found under the commit: btrfs-progs: add

[PATCH v2 3/6] Btrfs-progs: send, implement fallocate command callback

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana The fallocate send stream command, added in stream version 2, is used to pre-allocate space for files and punch file holes. This change implements the callback for that new command, using the fallocate function from the standard C library to carry out the specified action (all

[PATCH v2 1/2] btrfs: add verify chattr support for send/receive test

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan This test aims to verify correct behaviour with chattr operations and btrfs send/receive. The intent is to check general correctness as well as special interactions with troublesome flags(immutable, append only). This test is motivated by a bug in btrfs which

[RFC PATCH v2 2/6] btrfs: send, implement total data size command to allow for progress estimation

2018-05-27 Thread Howard McLauchlan
mnt/sdc At subvol /mnt/sdd/snap_incr At subvol snap_incr About to receive 9571342213 bytes Subvolume /mnt/sdc//snap_incr, 6557345221 / 9571342213 bytes received, 68.51%, 51.04MB/s [Howard: rebased on 4.17-rc7] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Mana

[RFC PATCH v2 0/6] btrfs send stream version 2

2018-05-27 Thread Howard McLauchlan
trfs: send, implement total data size command to allow for progress estimation btrfs: send, use fallocate command to punch holes btrfs: send, use fallocate command to allocate extents btrfs: add send_stream_version attribute to sysfs Howard McLauchlan (1): btrfs: add chattr suppor

[PATCH 2/2] xfstests: btrfs, test send's ability to punch holes and prealloc extents

2018-05-08 Thread Howard McLauchlan
ated by an incremental send. [Howard: rebased on kernel v4.17-rc4, btrfs progs v4.16.1] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana --- common/rc | 10 tests/btrfs/160 | 121 tests/btrfs/160

[PATCH 1/2] btrfs: add verify chattr support for send/receive test

2018-05-08 Thread Howard McLauchlan
From: Howard McLauchlan This test aims to verify correct behaviour with chattr operations and btrfs send/receive. The intent is to check general correctness as well as special interactions with troublesome flags(immutable, append only). This test is motivated by a bug in btrfs which

[RFC PATCH 6/6] btrfs-progs: add chattr support for send/receive

2018-05-08 Thread Howard McLauchlan
From: Howard McLauchlan Presently, btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds userspace support for inode attribute flags. Kernel support can be found under the commit: btrfs: add chattr

[RFC PATCH 5/6] btrfs-progs: add total data size, fallocate to dump

2018-05-08 Thread Howard McLauchlan
From: Howard McLauchlan Adding entries to dump for new commands (total data size, fallocate). Signed-off-by: Howard McLauchlan --- send-dump.c | 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/send-dump.c b/send-dump.c index 1591e0cc..c5a695a2 100644

[RFC PATCH 4/6] Btrfs-progs: add write and clone commands debug info to receive

2018-05-08 Thread Howard McLauchlan
From: Filipe Manana When specifying -vv print information about received write and clone commands too, as we do this for other commands already and it's very useful for debugging and troubleshooting. Signed-off-by: Filipe David Borba Manana --- cmds-receive.c | 9 + 1 file changed, 9

[RFC PATCH 3/6] Btrfs-progs: send, implement fallocate command callback

2018-05-08 Thread Howard McLauchlan
From: Filipe Manana The fallocate send stream command, added in stream version 2, is used to pre-allocate space for files and punch file holes. This change implements the callback for that new command, using the fallocate function from the standard C library to carry out the specified action (all

[RFC PATCH 2/6] Btrfs-progs: send, implement total data size callback and progress report

2018-05-08 Thread Howard McLauchlan
From: Filipe Manana This is a followup to the kernel patch titled: Btrfs: send, implement total data size command to allow for progress estimation This makes the btrfs send and receive commands aware of the new send flag, named BTRFS_SEND_C_TOTAL_DATA_SIZE, which tells us the amount of fil

[RFC PATCH 1/6] Btrfs-progs: send, bump stream version

2018-05-08 Thread Howard McLauchlan
From: Filipe Manana This increases the send stream version from version 1 to version 2, adding new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in files; 3) i

[RFC PATCH 2/6] btrfs: send, implement total data size command to allow for progress estimation

2018-05-08 Thread Howard McLauchlan
mnt/sdc At subvol /mnt/sdd/snap_incr At subvol snap_incr About to receive 9571342213 bytes Subvolume /mnt/sdc//snap_incr, 6557345221 / 9571342213 bytes received, 68.51%, 51.04MB/s [Howard: rebased on 4.17-rc4] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Mana

[RFC PATCH 3/6] btrfs: send, use fallocate command to punch holes

2018-05-08 Thread Howard McLauchlan
-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana --- fs/btrfs/send.c | 54 ++--- fs/btrfs/send.h | 4 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 7b184831812b

[RFC PATCH 5/6] btrfs: add send_stream_version attribute to sysfs

2018-05-08 Thread Howard McLauchlan
From: Filipe David Borba Manana So that applications can find out what's the highest send stream version supported/implemented by the running kernel: $ cat /sys/fs/btrfs/send/stream_version 2 [Howard: rebased on 4.17-rc4] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe

[RFC PATCH 6/6] btrfs: add chattr support for send/receive

2018-05-08 Thread Howard McLauchlan
From: Howard McLauchlan Presently btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds kernel support for inode attribute flags. Userspace support can be found under the commit: btrfs-progs: add

[RFC PATCH 0/6] btrfs send stream version 2

2018-05-08 Thread Howard McLauchlan
les btrfs: send, use fallocate command to allocate extents btrfs: add send_stream_version attribute to sysfs Howard McLauchlan (1): btrfs: add chattr support for send/receive fs/btrfs/ctree.h | 2 + fs/btrfs/ioctl.c | 2 +- fs/btrfs/send.c| 496 +++

[RFC PATCH 4/6] btrfs: send, use fallocate command to allocate extents

2018-05-08 Thread Howard McLauchlan
s 14: [17576..21487]: 28144..32055 3912 blocks The proposed xfstest can be found at: xfstests: btrfs, test send's ability to punch holes and prealloc extents This test verifies that send-stream version 2 does space pre-allocation and hole punching. [Howar

[RFC PATCH 1/6] btrfs: send, bump stream version

2018-05-08 Thread Howard McLauchlan
), meaning old clients are unaffected. [Howard: rebased on 4.17-rc4] Signed-off-by: Howard McLauchlan Signed-off-by: Filipe David Borba Manana --- fs/btrfs/send.c| 7 ++- fs/btrfs/send.h| 21 - include/uapi/linux/btrfs.h | 21 - 3 files

Re: [PATCH] btrfs: add chattr support for send/receive

2018-04-20 Thread Howard McLauchlan
On 04/18/2018 01:15 AM, Filipe Manana wrote: > On Wed, Apr 18, 2018 at 12:39 AM, Howard McLauchlan > wrote: >> Presently btrfs send/receive does not propagate inode attribute flags; >> all chattr operations are effectively discarded upon transmission. >> >> This

[PATCH v2 3/3] btrfs: remove le_test_bit()

2018-04-18 Thread Howard McLauchlan
With commit b18253ec57c0 ("btrfs: optimize free space tree bitmap conversion"), there are no more callers to le_test_bit(). This patch removes le_test_bit(). Signed-off-by: Howard McLauchlan --- fs/btrfs/extent_io.h | 5 - 1 file changed, 5 deletions(-) diff --git a/fs/btrfs/ext

[PATCH v2 1/3] btrfs: clean up le_bitmap_{set, clear}()

2018-04-18 Thread Howard McLauchlan
le_bitmap_set() is only used by free-space-tree, so move it there and make it static. le_bitmap_clear() is not used, so remove it. Signed-off-by: Howard McLauchlan --- V1->V2: Also move le_bitmap_set() based on 4.17-rc1 fs/btrfs/extent_io.c | 40 --

[PATCH v2 2/3] btrfs: optimize free space tree bitmap conversion

2018-04-18 Thread Howard McLauchlan
(). Suggested-by: Omar Sandoval Signed-off-by: Howard McLauchlan --- V1->V2: Round the bitmap size accordingly when allocating bitmap. fs/btrfs/free-space-tree.c | 61 ++ 1 file changed, 23 insertions(+), 38 deletions(-) diff --git a/fs/btrfs/free-space-tree.c b

Re: [PATCH 2/2] btrfs: optimize free space tree bitmap conversion

2018-04-18 Thread Howard McLauchlan
On 04/18/2018 03:26 PM, David Sterba wrote: > On Wed, Apr 18, 2018 at 02:30:59PM -0700, Howard McLauchlan wrote: >> Presently, convert_free_space_to_extents() does a linear scan of the >> bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). >> >> This pat

[PATCH 2/2] btrfs: optimize free space tree bitmap conversion

2018-04-18 Thread Howard McLauchlan
(). Suggested-by: Omar Sandoval Signed-off-by: Howard McLauchlan --- Since we change bitmap to be unsigned long, we have to do some casting for the bitmap cursor. In le_bitmap_set() it makes sense to use u8, as we are doing bit operations. Everywhere else, we're just using it for pointer arithmeti

[PATCH 1/2] btrfs: remove le_bitmap_clear()

2018-04-18 Thread Howard McLauchlan
le_bitmap_clear() was implemented as a parallel to le_bitmap_set() but is actually not being used. This patch removes it. Signed-off-by: Howard McLauchlan --- fs/btrfs/extent_io.c | 20 fs/btrfs/extent_io.h | 1 - 2 files changed, 21 deletions(-) diff --git a/fs/btrfs

Re: [PATCH] btrfs: optimize free space tree bitmap conversion

2018-04-18 Thread Howard McLauchlan
On 04/18/2018 08:50 AM, David Sterba wrote: > On Tue, Apr 17, 2018 at 04:19:04PM -0700, Howard McLauchlan wrote: >> Presently, convert_free_space_to_extents() does a linear scan of the >> bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). >> >> This pat

Re: [PATCH] btrfs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
On 04/17/2018 04:39 PM, Howard McLauchlan wrote: > Presently btrfs send/receive does not propagate inode attribute flags; > all chattr operations are effectively discarded upon transmission. > > This patch adds kernel support for inode attribute flags. Userspace > support can be

[PATCH] btrfs-progs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
-by: Howard McLauchlan --- cmds-receive.c | 31 +++ send-dump.c| 8 +++- send-stream.c | 5 + send-stream.h | 1 + send.h | 2 ++ 5 files changed, 46 insertions(+), 1 deletion(-) diff --git a/cmds-receive.c b/cmds-receive.c index 68123a31

[PATCH] btrfs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
-off-by: Howard McLauchlan --- Based on 4.17-rc1 fs/btrfs/ctree.h | 2 + fs/btrfs/ioctl.c | 2 +- fs/btrfs/send.c | 176 +++ fs/btrfs/send.h | 2 + 4 files changed, 154 insertions(+), 28 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs

[PATCH] btrfs: optimize free space tree bitmap conversion

2018-04-17 Thread Howard McLauchlan
(). Suggested-by: Omar Sandoval Signed-off-by: Howard McLauchlan --- Based on 4.17-rc1 fs/btrfs/extent_io.c | 12 - fs/btrfs/extent_io.h | 4 +-- fs/btrfs/free-space-tree.c | 54 ++ 3 files changed, 27 insertions(+), 43 deletions(-) diff --git a

[PATCH v2] btrfs: Add nossd_spread mount option

2018-03-08 Thread Howard McLauchlan
igned-off-by: Howard McLauchlan --- V2:remove duplicate code with fallthrough Based on 4.16-rc4 fs/btrfs/super.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 4b817947e00f..5bfcbf6f7eea 100644 --- a/fs/btrfs/super.c +++

Re: [PATCH] btrfs: Add nossd_spread mount option

2018-02-23 Thread Howard McLauchlan
On 02/21/2018 05:20 PM, Hans van Kranenburg wrote: > On 02/22/2018 12:31 AM, Howard McLauchlan wrote: >> Btrfs has two mount options for SSD optimizations: ssd and ssd_spread. >> Presently there is an option to disable all SSD optimizations, but there >> isn't an option

[PATCH] btrfs: Add nossd_spread mount option

2018-02-21 Thread Howard McLauchlan
Btrfs has two mount options for SSD optimizations: ssd and ssd_spread. Presently there is an option to disable all SSD optimizations, but there isn't an option to disable just ssd_spread. This patch adds a mount option nossd_spread that disables ssd_spread only. Signed-off-by: Howard McLau

Re: [PATCH v3] btrfs: print error if primary super block write fails

2018-02-02 Thread Howard McLauchlan
On 02/02/2018 11:09 AM, Howard McLauchlan wrote: > Presently, failing a primary super block write but succeeding in at > least one super block write in general will appear to users as if > nothing important went wrong. However, upon unmounting and re-mounting, > the file system will be

[PATCH v3] btrfs: print error if primary super block write fails

2018-02-02 Thread Howard McLauchlan
. Signed-off-by: Howard McLauchlan --- V3: Rewrote boolean setting logic for clarity V2: Added devid to output, removed unnecessary fs_info parameter fs/btrfs/disk-io.c | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c

[PATCH v2] btrfs: print error if primary super block write fails

2018-01-30 Thread Howard McLauchlan
. Signed-off-by: Howard McLauchlan --- V2: Added devid to output, removed unnecessary fs_info parameter fs/btrfs/disk-io.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5da18ebc9222..6d98f2f21d5f 100644 --- a/fs

[PATCH] btrfs: print error if primary super block write fails

2018-01-26 Thread Howard McLauchlan
. Signed-off-by: Howard McLauchlan --- fs/btrfs/disk-io.c | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5da18ebc9222..8f96e1e4c613 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3293,11 +3293,13