[PATCH] btrfs: drop unused space_info parameter from create_space_info

2018-05-27 Thread Lu Fengqi
Since commit dc2d3005d27d ("btrfs: remove dead create_space_info calls"), there is only one caller btrfs_init_space_info. However, it doesn't need create_space_info to return space_info at all. Signed-off-by: Lu Fengqi --- fs/btrfs/extent-tree.c | 13 + 1 file changed, 5 insertions(+

[PATCH v2] btrfs: balance dirty metadata pages in btrfs_finish_ordered_io

2018-05-27 Thread Ethan Lien
[Problem description and how we fix it] We should balance dirty metadata pages at the end of btrfs_finish_ordered_io, since a small, unmergeable random write can potentially produce dirty metadata which is multiple times larger than the data itself. For example, a small, unmergeable 4KiB write may

Re: off-by-one uncompressed invalid ram_bytes corruptions

2018-05-27 Thread Qu Wenruo
On 2018年05月28日 11:47, Steve Leung wrote: > On 05/26/2018 06:57 PM, Qu Wenruo wrote: >> >> >> On 2018年05月26日 22:06, Steve Leung wrote: >>> On 05/20/2018 07:07 PM, Qu Wenruo wrote: On 2018年05月21日 04:43, Steve Leung wrote: > On 05/19/2018 07:02 PM, Qu Wenruo wrote: >> >> >

Re: off-by-one uncompressed invalid ram_bytes corruptions

2018-05-27 Thread Steve Leung
On 05/26/2018 06:57 PM, Qu Wenruo wrote: On 2018年05月26日 22:06, Steve Leung wrote: On 05/20/2018 07:07 PM, Qu Wenruo wrote: On 2018年05月21日 04:43, Steve Leung wrote: On 05/19/2018 07:02 PM, Qu Wenruo wrote: On 2018年05月20日 07:40, Steve Leung wrote: On 05/17/2018 11:49 PM, Qu Wenruo wrote:

[PATCH v2 6/6] btrfs-progs: add chattr support for send/receive

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Presently, btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds userspace support for inode attribute flags. Kernel support can be found under the commit: btrfs: add chattr sup

[PATCH v2 1/6] Btrfs-progs: send, bump stream version

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana This increases the send stream version from version 1 to version 2, adding new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in files; 3) i

[PATCH v2 4/6] Btrfs-progs: add write and clone commands debug info to receive

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana When specifying -vv print information about received write and clone commands too, as we do this for other commands already and it's very useful for debugging and troubleshooting. Signed-off-by: Filipe David Borba Manana --- cmds-receive.c | 9 + 1 file changed, 9

[RFC PATCH v2 3/6] btrfs: send, use fallocate command to punch holes

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana Instead of sending a write command with a data buffer filled with 0 value bytes, use the fallocate command, introduced in the send stream version 2, to tell the receiver to punch a file hole using the fallocate system call. [Howard: rebased on 4.17-rc7] Signed-off

[RFC PATCH v2 4/6] btrfs: send, use fallocate command to allocate extents

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana The send stream version 2 adds the fallocate command, which can be used to allocate extents for a file or punch holes in a file. Previously we were ignoring file prealloc extents or treating them as extents filled with 0 bytes and sending a regular write command to

[PATCH v2 5/6] btrfs-progs: add total data size, fallocate to dump

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Adding entries to dump for new commands (total data size, fallocate). Signed-off-by: Howard McLauchlan --- send-dump.c | 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/send-dump.c b/send-dump.c index 1591e0cc..c5a695a2 100644 --- a

[RFC PATCH v2 1/6] btrfs: send, bump stream version

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana This increases the send stream version from version 1 to version 2, adding new commands: 1) total data size - used to tell the receiver how much file data the stream will add or update; 2) fallocate - used to pre-allocate space for files and to punch holes in

[PATCH v2 2/6] Btrfs-progs: send, implement total data size callback and progress report

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana This is a followup to the kernel patch titled: Btrfs: send, implement total data size command to allow for progress estimation This makes the btrfs send and receive commands aware of the new send flag, named BTRFS_SEND_C_TOTAL_DATA_SIZE, which tells us the amount of fil

[PATCH v2 2/2] xfstests: btrfs, test send's ability to punch holes, prealloc extents and send data size

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana This test verifies that after an incremental btrfs send the replicated file has the same exact hole and data structure as in the origin filesystem. This didn't use to be the case before the send stream version 2 - holes were sent as write operations of 0 valued bytes instead o

[RFC PATCH v2 5/6] btrfs: add send_stream_version attribute to sysfs

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana So that applications can find out what's the highest send stream version supported/implemented by the running kernel: $ cat /sys/fs/btrfs/send/stream_version 2 [Howard: rebased on 4.17-rc7] Reviewed-by: Omar Sandoval Signed-off-by: Howard McLauchlan Sig

[RFC PATCH v2 6/6] btrfs: add chattr support for send/receive

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan Presently btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds kernel support for inode attribute flags. Userspace support can be found under the commit: btrfs-progs: add chatt

[PATCH v2 3/6] Btrfs-progs: send, implement fallocate command callback

2018-05-27 Thread Howard McLauchlan
From: Filipe Manana The fallocate send stream command, added in stream version 2, is used to pre-allocate space for files and punch file holes. This change implements the callback for that new command, using the fallocate function from the standard C library to carry out the specified action (all

[PATCH v2 1/2] btrfs: add verify chattr support for send/receive test

2018-05-27 Thread Howard McLauchlan
From: Howard McLauchlan This test aims to verify correct behaviour with chattr operations and btrfs send/receive. The intent is to check general correctness as well as special interactions with troublesome flags(immutable, append only). This test is motivated by a bug in btrfs which demonstrates

[RFC PATCH v2 2/6] btrfs: send, implement total data size command to allow for progress estimation

2018-05-27 Thread Howard McLauchlan
From: Filipe David Borba Manana This new send flag makes send calculate first the amount of new file data (in bytes) the send root has relatively to the parent root, or for the case of a non-incremental send, the total amount of file data the stream will create (including holes and prealloc ex

[RFC PATCH v2 0/6] btrfs send stream version 2

2018-05-27 Thread Howard McLauchlan
This is v2 of send stream version 2. The goal is to provide proper versioning/compatibility as new features are implemented. v1 can be found here [1]. v2 adds BTRFS_SEND_C_WRITE_COMPRESSED to patch 1 and style fixes/logic simplifications to patches 5,6. v2 also updates btrfs-progs to reflect BTRF

Re: RAID-1 refuses to balance large drive

2018-05-27 Thread Brad Templeton
BTW, I decided to follow the original double replace strategy suggested -- replace 6TB with 8TB and replace 4TB with 6TB. That should be sure to leave the 2 large drives each with 2TB free once expanded, and thus able to fully use all space. However, the first one has been going for 9 hours and i