Re: [PATCH] xfstests: make BTRFS_UTIL_PROG filesystem defragment work

2015-04-14 Thread Filipe David Manana
On Tue, Apr 14, 2015 at 10:01 AM, Liu Bo bo.li@oracle.com wrote: _require_defrag() needs to check if the command is executable, but btrfs has its subcommand filesystem defragment, which makes this checking fail. This workarounds it and now we can run case generic/324, generic/018,

directory defrag

2015-04-14 Thread Russell Coker
The current defragmentation options seem to only support defragmenting named files/directories or a recursive defragmentation of files and directories. I'd like to recursively defragment directories. One of my systems has a large number of large files, the files are write-once and read

[PATCH] Btrfs receive hardening patches

2015-04-14 Thread Lauri Võsandi
--- cmds-receive.c | 18 +++--- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index 44ef27e..6800401 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -867,15 +867,19 @@ static int do_receive(struct btrfs_receive *r, const char

Re: [PATCH v2] Btrfs: incremental send, don't rename a directory too soon

2015-04-14 Thread Robbie Ko
Hi, After applying the patch, I got WARN_ON. btrfs progs finished without any error message, but received subvolume is not the same as send subvolume. Here's the related information. thanks, robbieko uanme -a Linux ubuntu 4.0.0-rc4-custom #2 SMP Tue Apr 14 11:43:00 CST 2015 x86_64 x86_64 x86_64

Re: [PATCH v2] Btrfs: incremental send, don't rename a directory too soon

2015-04-14 Thread Filipe David Manana
On Tue, Apr 14, 2015 at 8:33 AM, Robbie Ko robbi...@synology.com wrote: Hi, After applying the patch, I got WARN_ON. btrfs progs finished without any error message, but received subvolume is not the same as send subvolume. Here's the related information. thanks, robbieko uanme -a Linux

[PATCH] xfstests: make BTRFS_UTIL_PROG filesystem defragment work

2015-04-14 Thread Liu Bo
_require_defrag() needs to check if the command is executable, but btrfs has its subcommand filesystem defragment, which makes this checking fail. This workarounds it and now we can run case generic/324, generic/018, btrfs/005. Signed-off-by: Liu Bo bo.li@oracle.com --- common/defrag | 6

Re: [PATCH] xfstests: make BTRFS_UTIL_PROG filesystem defragment work

2015-04-14 Thread Liu Bo
On Tue, Apr 14, 2015 at 10:14:59AM +0100, Filipe David Manana wrote: On Tue, Apr 14, 2015 at 10:01 AM, Liu Bo bo.li@oracle.com wrote: _require_defrag() needs to check if the command is executable, but btrfs has its subcommand filesystem defragment, which makes this checking fail. This

Re: directory defrag

2015-04-14 Thread Sander
Russell Coker wrote (ao): The current defragmentation options seem to only support defragmenting named files/directories or a recursive defragmentation of files and directories. I'd like to recursively defragment directories. find / -xdev -type d -execdir btrfs filesystem defrag -c {} +

Re: btrfs balance {,meta}data to raid5 not working?

2015-04-14 Thread Piotr Szymaniak
On Sat, Apr 11, 2015 at 02:14:44AM +, Duncan wrote: But you could try the latest 4.0-rc7+ kernel and see if it works with that, yet. Will try that. 2b) If instead your intention was to convert it to raid5 before upgrading it to three devices, just add the third device first, then do

Re: [PATCH] btrfs-progs: enforce chroot for btrfs receive

2015-04-14 Thread David Sterba
On Tue, Apr 14, 2015 at 01:44:32PM +0300, Lauri Võsandi wrote: This patch forces btrfs receive to issue chroot before parsing the btrfs stream to confine the process and minimize damage that could be done via malicious btrfs stream. Thanks. As we've discussed, there are possibly some things

Re: [PATCH v2] Btrfs: incremental send, don't rename a directory too soon

2015-04-14 Thread Robbie Ko
Hi, Sorry for not making it clear. 2015-04-14 16:16 GMT+08:00 Filipe David Manana fdman...@gmail.com: On Tue, Apr 14, 2015 at 8:33 AM, Robbie Ko robbi...@synology.com wrote: Hi, After applying the patch, I got WARN_ON. btrfs progs finished without any error message, but received subvolume

[PATCH] btrfs-progs: enforce chroot for btrfs receive

2015-04-14 Thread Lauri Võsandi
This patch forces btrfs receive to issue chroot before parsing the btrfs stream to confine the process and minimize damage that could be done via malicious btrfs stream. Signed-off-by: Lauri Võsandi lauri.vosa...@gmail.com --- cmds-receive.c | 18 ++ 1 file changed, 10

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Christoph Hellwig
On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff Layton wrote: Yuck! How the heck do you clean up the mess if that happens? I guess you're just stuck redoing the copy with normal READ/WRITE? Maybe we need to have the interface return a hard error in that case and not try to give back any sort

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Anna Schumaker
On 04/14/2015 12:53 PM, Christoph Hellwig wrote: On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff Layton wrote: Yuck! How the heck do you clean up the mess if that happens? I guess you're just stuck redoing the copy with normal READ/WRITE? Maybe we need to have the interface return a hard error

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Christoph Hellwig
On Tue, Apr 14, 2015 at 09:53:44AM -0700, Christoph Hellwig wrote: On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff Layton wrote: Yuck! How the heck do you clean up the mess if that happens? I guess you're just stuck redoing the copy with normal READ/WRITE? Maybe we need to have the

Re: [PATCH RFC 3/3] btrfs: add .copy_file_range file operation

2015-04-14 Thread Chris Mason
On 04/10/2015 06:00 PM, Zach Brown wrote: This rearranges the existing COPY_RANGE ioctl implementation so that the .copy_file_range file operation can call the core loop that copies file data extent items. The extent copying loop is lifted up into its own function. It retains the core

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread J. Bruce Fields
On Tue, Apr 14, 2015 at 11:22:41AM -0700, Zach Brown wrote: On Tue, Apr 14, 2015 at 02:19:11PM -0400, J. Bruce Fields wrote: On Tue, Apr 14, 2015 at 01:16:13PM -0400, Anna Schumaker wrote: On 04/14/2015 12:53 PM, Christoph Hellwig wrote: On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread J. Bruce Fields
On Tue, Apr 14, 2015 at 01:16:13PM -0400, Anna Schumaker wrote: On 04/14/2015 12:53 PM, Christoph Hellwig wrote: On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff Layton wrote: Yuck! How the heck do you clean up the mess if that happens? I guess you're just stuck redoing the copy with normal

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Zach Brown
On Tue, Apr 14, 2015 at 02:19:11PM -0400, J. Bruce Fields wrote: On Tue, Apr 14, 2015 at 01:16:13PM -0400, Anna Schumaker wrote: On 04/14/2015 12:53 PM, Christoph Hellwig wrote: On Sat, Apr 11, 2015 at 09:04:02AM -0400, Jeff Layton wrote: Yuck! How the heck do you clean up the mess if

Re: [PATCH] btrfs-progs: enforce chroot for btrfs receive

2015-04-14 Thread Austin S Hemmelgarn
On 2015-04-14 08:28, David Sterba wrote: On Tue, Apr 14, 2015 at 01:44:32PM +0300, Lauri Võsandi wrote: This patch forces btrfs receive to issue chroot before parsing the btrfs stream to confine the process and minimize damage that could be done via malicious btrfs stream. Thanks. As we've

Re: [PATCH v2 4/6] Btrfs: fail on mismatched subvol and subvolid mount options

2015-04-14 Thread David Sterba
On Fri, Apr 10, 2015 at 08:39:53AM +0800, Qu Wenruo wrote: There's nothing to stop a user from passing both subvol= and subvolid= to mount, but if they don't refer to the same subvolume, someone is going to be surprised at some point. Error out on this case, but allow users to pass in both

Re: [GSoC 2015] Btrfs content based storage

2015-04-14 Thread harshad shirwadkar
Thanks David and that's true: there will be a large overlap with dedup work from Liu Bo. In fact, a few days after I sent the first mail, I got in touch with Liu and he agreed upon mentoring. So, I am all set now and will start working on the project in May end.. Best, Harshad. On Mon, Apr 13,

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Zach Brown
On Tue, Apr 14, 2015 at 02:29:06PM -0400, J. Bruce Fields wrote: On Tue, Apr 14, 2015 at 11:22:41AM -0700, Zach Brown wrote: On Tue, Apr 14, 2015 at 02:19:11PM -0400, J. Bruce Fields wrote: On Tue, Apr 14, 2015 at 01:16:13PM -0400, Anna Schumaker wrote: On 04/14/2015 12:53 PM, Christoph

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Christoph Hellwig
On Tue, Apr 14, 2015 at 11:54:08AM -0700, Zach Brown wrote: Is this relying on btrfs range cloning being atomic? It certainly doesn't look atomic. It can modify items across an arbitrarily large number of leaf blocks. It can make the changes across multiple transactions which could

Re: [PATCH RFC 1/3] vfs: add copy_file_range syscall and vfs helper

2015-04-14 Thread Zach Brown
On Tue, Apr 14, 2015 at 12:23:25PM -0700, Christoph Hellwig wrote: On Tue, Apr 14, 2015 at 11:54:08AM -0700, Zach Brown wrote: Is this relying on btrfs range cloning being atomic? It certainly doesn't look atomic. It can modify items across an arbitrarily large number of leaf blocks. It

Re: [PATCH v2] Btrfs: incremental send, don't rename a directory too soon

2015-04-14 Thread Filipe David Manana
On Tue, Apr 14, 2015 at 12:09 PM, Robbie Ko robbi...@synology.com wrote: Hi, Sorry for not making it clear. 2015-04-14 16:16 GMT+08:00 Filipe David Manana fdman...@gmail.com: On Tue, Apr 14, 2015 at 8:33 AM, Robbie Ko robbi...@synology.com wrote: Hi, After applying the patch, I got

Re: directory defrag

2015-04-14 Thread David Sterba
On Tue, Apr 14, 2015 at 07:37:17AM +, Russell Coker wrote: The current defragmentation options seem to only support defragmenting named files/directories or a recursive defragmentation of files and directories. I'd like to recursively defragment directories. One of my systems has a

Re: [PATCH 1/1] btrfs-progs: improve troubleshooting avoid duplicate error strings

2015-04-14 Thread David Sterba
On Mon, Apr 13, 2015 at 08:37:01PM +0800, Anand Jain wrote: my troubleshooting experience says have unique error string per module. In the below eg, its one additional step to know error line, cat -n cmds-device.c | egrep error removing the device 185ERROR: error removing the

Re: [PATCH 1/1] Btrfs-progs: fix compile warnings

2015-04-14 Thread David Sterba
On Mon, Apr 13, 2015 at 10:48:54PM +0800, Anand Jain wrote: simple compile time warning fixes. cmds-check.c: In function ‘del_file_extent_hole’: cmds-check.c:289: warning: ‘prev.len’ may be used uninitialized in this function cmds-check.c:289: warning: ‘prev.start’ may be used