Re: [Ocfs2-devel] [PATCH 6/6] ocfs2: implement the VFS clone_range, copy_range, and dedupe_range features

2016-11-11 Thread Darrick J. Wong 
On Fri, Nov 11, 2016 at 02:45:54PM +0800, Eric Ren wrote:
> On 11/11/2016 02:20 PM, Darrick J. Wong wrote:
> >On Fri, Nov 11, 2016 at 01:49:48PM +0800, Eric Ren wrote:
> >>Hi,
> >>
> >>A few issues obvious to me:
> >>
> >>On 11/10/2016 06:51 AM, Darrick J. Wong wrote:
> >>>Connect the new VFS clone_range, copy_range, and dedupe_range features
> >>>to the existing reflink capability of ocfs2.  Compared to the existing
> >>>ocfs2 reflink ioctl We have to do things a little differently to support
> >>>the VFS semantics (we can clone subranges of a file but we don't clone
> >>>xattrs), but the VFS ioctls are more broadly supported.
> >>How can I test the new ocfs2 reflink (with this patch) manually? What
> >>commands should I use to do xxx_range things?
> >See the 'reflink', 'dedupe', and 'copy_range' commands in xfs_io.
> >
> >The first two were added in xfsprogs 4.3, and copy_range in 4.7.
> 
> OK, thanks. I think you are missing the following two inline comments:
> 
> >>>+  spin_lock(_I(dest)->ip_lock);
> >>>+  if (newlen > i_size_read(dest)) {
> >>>+  i_size_write(dest, newlen);
> >>>+  di->i_size = newlen;
> >>di->i_size = cpu_to_le64(newlen);

Good catch!

> >>>+  }
> >>>+  spin_unlock(_I(dest)->ip_lock);
> >>>+
> >>Add ocfs2_update_inode_fsync_trans() here? Looks this function was
> >>introduced by you to improve efficiency.
> >>Just want to awake your memory about this, though I don't know about the
> >>details why it should be.

D'oh!  Yes, I did miss that.

The function updates the destination inode's information.  Specifically,
it updates i_size if we reflinked blocks into the file past EOF.
Looking at it some more, I also need to update i_blocks or the stat(2) info
will be wrong, and I also need to convert inline data to extents prior
to reflinking.

--D

> >>
> >>Eric
> Thanks,
> Eric

___
Ocfs2-devel mailing list
Ocfs2-devel@oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

Re: [Ocfs2-devel] [PATCH 6/6] ocfs2: implement the VFS clone_range, copy_range, and dedupe_range features

2016-11-10 Thread Eric Ren 
On 11/11/2016 02:20 PM, Darrick J. Wong wrote:
> On Fri, Nov 11, 2016 at 01:49:48PM +0800, Eric Ren wrote:
>> Hi,
>>
>> A few issues obvious to me:
>>
>> On 11/10/2016 06:51 AM, Darrick J. Wong wrote:
>>> Connect the new VFS clone_range, copy_range, and dedupe_range features
>>> to the existing reflink capability of ocfs2.  Compared to the existing
>>> ocfs2 reflink ioctl We have to do things a little differently to support
>>> the VFS semantics (we can clone subranges of a file but we don't clone
>>> xattrs), but the VFS ioctls are more broadly supported.
>> How can I test the new ocfs2 reflink (with this patch) manually? What
>> commands should I use to do xxx_range things?
> See the 'reflink', 'dedupe', and 'copy_range' commands in xfs_io.
>
> The first two were added in xfsprogs 4.3, and copy_range in 4.7.

OK, thanks. I think you are missing the following two inline comments:

>>> +   spin_lock(_I(dest)->ip_lock);
>>> +   if (newlen > i_size_read(dest)) {
>>> +   i_size_write(dest, newlen);
>>> +   di->i_size = newlen;
>> di->i_size = cpu_to_le64(newlen);
>>
>>> +   }
>>> +   spin_unlock(_I(dest)->ip_lock);
>>> +
>> Add ocfs2_update_inode_fsync_trans() here? Looks this function was
>> introduced by you to improve efficiency.
>> Just want to awake your memory about this, though I don't know about the
>> details why it should be.
>>
>> Eric
Thanks,
Eric

___
Ocfs2-devel mailing list
Ocfs2-devel@oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

Re: [Ocfs2-devel] [PATCH 6/6] ocfs2: implement the VFS clone_range, copy_range, and dedupe_range features

2016-11-10 Thread Darrick J. Wong 
On Fri, Nov 11, 2016 at 01:49:48PM +0800, Eric Ren wrote:
> Hi,
> 
> A few issues obvious to me:
> 
> On 11/10/2016 06:51 AM, Darrick J. Wong wrote:
> >Connect the new VFS clone_range, copy_range, and dedupe_range features
> >to the existing reflink capability of ocfs2.  Compared to the existing
> >ocfs2 reflink ioctl We have to do things a little differently to support
> >the VFS semantics (we can clone subranges of a file but we don't clone
> >xattrs), but the VFS ioctls are more broadly supported.
> 
> How can I test the new ocfs2 reflink (with this patch) manually? What
> commands should I use to do xxx_range things?

See the 'reflink', 'dedupe', and 'copy_range' commands in xfs_io.

The first two were added in xfsprogs 4.3, and copy_range in 4.7.

--D

> 
> >
> >Signed-off-by: Darrick J. Wong 
> >---
> >  fs/ocfs2/file.c |   62 -
> >  fs/ocfs2/file.h |3
> >  fs/ocfs2/refcounttree.c |  619 
> > +++
> >  fs/ocfs2/refcounttree.h |7 +
> >  4 files changed, 688 insertions(+), 3 deletions(-)
> >
> >
> >diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
> >index 000c234..d5a022d 100644
> >--- a/fs/ocfs2/file.c
> >+++ b/fs/ocfs2/file.c
> >@@ -1667,9 +1667,9 @@ static void ocfs2_calc_trunc_pos(struct inode *inode,
> > *done = ret;
> >  }
> >-static int ocfs2_remove_inode_range(struct inode *inode,
> >-struct buffer_head *di_bh, u64 byte_start,
> >-u64 byte_len)
> >+int ocfs2_remove_inode_range(struct inode *inode,
> >+ struct buffer_head *di_bh, u64 byte_start,
> >+ u64 byte_len)
> >  {
> > int ret = 0, flags = 0, done = 0, i;
> > u32 trunc_start, trunc_len, trunc_end, trunc_cpos, phys_cpos;
> >@@ -2440,6 +2440,56 @@ static loff_t ocfs2_file_llseek(struct file *file, 
> >loff_t offset, int whence)
> > return offset;
> >  }
> >+static ssize_t ocfs2_file_copy_range(struct file *file_in,
> >+ loff_t pos_in,
> >+ struct file *file_out,
> >+ loff_t pos_out,
> >+ size_t len,
> >+ unsigned int flags)
> >+{
> >+int error;
> >+
> >+error = ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
> >+  len, false);
> >+if (error)
> >+return error;
> >+return len;
> >+}
> >+
> >+static int ocfs2_file_clone_range(struct file *file_in,
> >+  loff_t pos_in,
> >+  struct file *file_out,
> >+  loff_t pos_out,
> >+  u64 len)
> >+{
> >+return ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
> >+ len, false);
> >+}
> >+
> >+#define OCFS2_MAX_DEDUPE_LEN(16 * 1024 * 1024)
> >+static ssize_t ocfs2_file_dedupe_range(struct file *src_file,
> >+   u64 loff,
> >+   u64 len,
> >+   struct file *dst_file,
> >+   u64 dst_loff)
> >+{
> >+int error;
> >+
> >+/*
> >+ * Limit the total length we will dedupe for each operation.
> >+ * This is intended to bound the total time spent in this
> >+ * ioctl to something sane.
> >+ */
> >+if (len > OCFS2_MAX_DEDUPE_LEN)
> >+len = OCFS2_MAX_DEDUPE_LEN;
> >+
> >+error = ocfs2_reflink_remap_range(src_file, loff, dst_file, dst_loff,
> >+  len, true);
> >+if (error)
> >+return error;
> >+return len;
> >+}
> >+
> >  const struct inode_operations ocfs2_file_iops = {
> > .setattr= ocfs2_setattr,
> > .getattr= ocfs2_getattr,
> >@@ -2479,6 +2529,9 @@ const struct file_operations ocfs2_fops = {
> > .splice_read= generic_file_splice_read,
> > .splice_write   = iter_file_splice_write,
> > .fallocate  = ocfs2_fallocate,
> >+.copy_file_range = ocfs2_file_copy_range,
> >+.clone_file_range = ocfs2_file_clone_range,
> >+.dedupe_file_range = ocfs2_file_dedupe_range,
> >  };
> >  const struct file_operations ocfs2_dops = {
> >@@ -2524,6 +2577,9 @@ const struct file_operations ocfs2_fops_no_plocks = {
> > .splice_read= generic_file_splice_read,
> > .splice_write   = iter_file_splice_write,
> > .fallocate  = ocfs2_fallocate,
> >+.copy_file_range = ocfs2_file_copy_range,
> >+.clone_file_range = ocfs2_file_clone_range,
> >+.dedupe_file_range = ocfs2_file_dedupe_range,
> >  };
> >  const struct file_operations ocfs2_dops_no_plocks = {
> >diff --git a/fs/ocfs2/file.h b/fs/ocfs2/file.h
> >index e8c62f2..897fd9a 100644
> >--- a/fs/ocfs2/file.h
> >+++ b/fs/ocfs2/file.h
> >@@ -82,4 +82,7 @@ int

Re: [Ocfs2-devel] [PATCH 6/6] ocfs2: implement the VFS clone_range, copy_range, and dedupe_range features

2016-11-10 Thread Eric Ren 
Hi,

A few issues obvious to me:

On 11/10/2016 06:51 AM, Darrick J. Wong wrote:
> Connect the new VFS clone_range, copy_range, and dedupe_range features
> to the existing reflink capability of ocfs2.  Compared to the existing
> ocfs2 reflink ioctl We have to do things a little differently to support
> the VFS semantics (we can clone subranges of a file but we don't clone
> xattrs), but the VFS ioctls are more broadly supported.

How can I test the new ocfs2 reflink (with this patch) manually? What commands 
should I
use to do xxx_range things?

>
> Signed-off-by: Darrick J. Wong 
> ---
>   fs/ocfs2/file.c |   62 -
>   fs/ocfs2/file.h |3
>   fs/ocfs2/refcounttree.c |  619 
> +++
>   fs/ocfs2/refcounttree.h |7 +
>   4 files changed, 688 insertions(+), 3 deletions(-)
>
>
> diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
> index 000c234..d5a022d 100644
> --- a/fs/ocfs2/file.c
> +++ b/fs/ocfs2/file.c
> @@ -1667,9 +1667,9 @@ static void ocfs2_calc_trunc_pos(struct inode *inode,
>   *done = ret;
>   }
>   
> -static int ocfs2_remove_inode_range(struct inode *inode,
> - struct buffer_head *di_bh, u64 byte_start,
> - u64 byte_len)
> +int ocfs2_remove_inode_range(struct inode *inode,
> +  struct buffer_head *di_bh, u64 byte_start,
> +  u64 byte_len)
>   {
>   int ret = 0, flags = 0, done = 0, i;
>   u32 trunc_start, trunc_len, trunc_end, trunc_cpos, phys_cpos;
> @@ -2440,6 +2440,56 @@ static loff_t ocfs2_file_llseek(struct file *file, 
> loff_t offset, int whence)
>   return offset;
>   }
>   
> +static ssize_t ocfs2_file_copy_range(struct file *file_in,
> +  loff_t pos_in,
> +  struct file *file_out,
> +  loff_t pos_out,
> +  size_t len,
> +  unsigned int flags)
> +{
> + int error;
> +
> + error = ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
> +   len, false);
> + if (error)
> + return error;
> + return len;
> +}
> +
> +static int ocfs2_file_clone_range(struct file *file_in,
> +   loff_t pos_in,
> +   struct file *file_out,
> +   loff_t pos_out,
> +   u64 len)
> +{
> + return ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
> +  len, false);
> +}
> +
> +#define OCFS2_MAX_DEDUPE_LEN (16 * 1024 * 1024)
> +static ssize_t ocfs2_file_dedupe_range(struct file *src_file,
> +u64 loff,
> +u64 len,
> +struct file *dst_file,
> +u64 dst_loff)
> +{
> + int error;
> +
> + /*
> +  * Limit the total length we will dedupe for each operation.
> +  * This is intended to bound the total time spent in this
> +  * ioctl to something sane.
> +  */
> + if (len > OCFS2_MAX_DEDUPE_LEN)
> + len = OCFS2_MAX_DEDUPE_LEN;
> +
> + error = ocfs2_reflink_remap_range(src_file, loff, dst_file, dst_loff,
> +   len, true);
> + if (error)
> + return error;
> + return len;
> +}
> +
>   const struct inode_operations ocfs2_file_iops = {
>   .setattr= ocfs2_setattr,
>   .getattr= ocfs2_getattr,
> @@ -2479,6 +2529,9 @@ const struct file_operations ocfs2_fops = {
>   .splice_read= generic_file_splice_read,
>   .splice_write   = iter_file_splice_write,
>   .fallocate  = ocfs2_fallocate,
> + .copy_file_range = ocfs2_file_copy_range,
> + .clone_file_range = ocfs2_file_clone_range,
> + .dedupe_file_range = ocfs2_file_dedupe_range,
>   };
>   
>   const struct file_operations ocfs2_dops = {
> @@ -2524,6 +2577,9 @@ const struct file_operations ocfs2_fops_no_plocks = {
>   .splice_read= generic_file_splice_read,
>   .splice_write   = iter_file_splice_write,
>   .fallocate  = ocfs2_fallocate,
> + .copy_file_range = ocfs2_file_copy_range,
> + .clone_file_range = ocfs2_file_clone_range,
> + .dedupe_file_range = ocfs2_file_dedupe_range,
>   };
>   
>   const struct file_operations ocfs2_dops_no_plocks = {
> diff --git a/fs/ocfs2/file.h b/fs/ocfs2/file.h
> index e8c62f2..897fd9a 100644
> --- a/fs/ocfs2/file.h
> +++ b/fs/ocfs2/file.h
> @@ -82,4 +82,7 @@ int ocfs2_change_file_space(struct file *file, unsigned int 
> cmd,
>   
>   int ocfs2_check_range_for_refcount(struct inode *inode, loff_t pos,
>  size_t count);
> +int ocfs2_remove_inode_range(struct inode *inode,
> +

[Ocfs2-devel] [PATCH 6/6] ocfs2: implement the VFS clone_range, copy_range, and dedupe_range features

2016-11-09 Thread Darrick J. Wong 
Connect the new VFS clone_range, copy_range, and dedupe_range features
to the existing reflink capability of ocfs2.  Compared to the existing
ocfs2 reflink ioctl We have to do things a little differently to support
the VFS semantics (we can clone subranges of a file but we don't clone
xattrs), but the VFS ioctls are more broadly supported.

Signed-off-by: Darrick J. Wong 
---
 fs/ocfs2/file.c |   62 -
 fs/ocfs2/file.h |3 
 fs/ocfs2/refcounttree.c |  619 +++
 fs/ocfs2/refcounttree.h |7 +
 4 files changed, 688 insertions(+), 3 deletions(-)


diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
index 000c234..d5a022d 100644
--- a/fs/ocfs2/file.c
+++ b/fs/ocfs2/file.c
@@ -1667,9 +1667,9 @@ static void ocfs2_calc_trunc_pos(struct inode *inode,
*done = ret;
 }
 
-static int ocfs2_remove_inode_range(struct inode *inode,
-   struct buffer_head *di_bh, u64 byte_start,
-   u64 byte_len)
+int ocfs2_remove_inode_range(struct inode *inode,
+struct buffer_head *di_bh, u64 byte_start,
+u64 byte_len)
 {
int ret = 0, flags = 0, done = 0, i;
u32 trunc_start, trunc_len, trunc_end, trunc_cpos, phys_cpos;
@@ -2440,6 +2440,56 @@ static loff_t ocfs2_file_llseek(struct file *file, 
loff_t offset, int whence)
return offset;
 }
 
+static ssize_t ocfs2_file_copy_range(struct file *file_in,
+loff_t pos_in,
+struct file *file_out,
+loff_t pos_out,
+size_t len,
+unsigned int flags)
+{
+   int error;
+
+   error = ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
+ len, false);
+   if (error)
+   return error;
+   return len;
+}
+
+static int ocfs2_file_clone_range(struct file *file_in,
+ loff_t pos_in,
+ struct file *file_out,
+ loff_t pos_out,
+ u64 len)
+{
+   return ocfs2_reflink_remap_range(file_in, pos_in, file_out, pos_out,
+len, false);
+}
+
+#define OCFS2_MAX_DEDUPE_LEN   (16 * 1024 * 1024)
+static ssize_t ocfs2_file_dedupe_range(struct file *src_file,
+  u64 loff,
+  u64 len,
+  struct file *dst_file,
+  u64 dst_loff)
+{
+   int error;
+
+   /*
+* Limit the total length we will dedupe for each operation.
+* This is intended to bound the total time spent in this
+* ioctl to something sane.
+*/
+   if (len > OCFS2_MAX_DEDUPE_LEN)
+   len = OCFS2_MAX_DEDUPE_LEN;
+
+   error = ocfs2_reflink_remap_range(src_file, loff, dst_file, dst_loff,
+ len, true);
+   if (error)
+   return error;
+   return len;
+}
+
 const struct inode_operations ocfs2_file_iops = {
.setattr= ocfs2_setattr,
.getattr= ocfs2_getattr,
@@ -2479,6 +2529,9 @@ const struct file_operations ocfs2_fops = {
.splice_read= generic_file_splice_read,
.splice_write   = iter_file_splice_write,
.fallocate  = ocfs2_fallocate,
+   .copy_file_range = ocfs2_file_copy_range,
+   .clone_file_range = ocfs2_file_clone_range,
+   .dedupe_file_range = ocfs2_file_dedupe_range,
 };
 
 const struct file_operations ocfs2_dops = {
@@ -2524,6 +2577,9 @@ const struct file_operations ocfs2_fops_no_plocks = {
.splice_read= generic_file_splice_read,
.splice_write   = iter_file_splice_write,
.fallocate  = ocfs2_fallocate,
+   .copy_file_range = ocfs2_file_copy_range,
+   .clone_file_range = ocfs2_file_clone_range,
+   .dedupe_file_range = ocfs2_file_dedupe_range,
 };
 
 const struct file_operations ocfs2_dops_no_plocks = {
diff --git a/fs/ocfs2/file.h b/fs/ocfs2/file.h
index e8c62f2..897fd9a 100644
--- a/fs/ocfs2/file.h
+++ b/fs/ocfs2/file.h
@@ -82,4 +82,7 @@ int ocfs2_change_file_space(struct file *file, unsigned int 
cmd,
 
 int ocfs2_check_range_for_refcount(struct inode *inode, loff_t pos,
   size_t count);
+int ocfs2_remove_inode_range(struct inode *inode,
+struct buffer_head *di_bh, u64 byte_start,
+u64 byte_len);
 #endif /* OCFS2_FILE_H */
diff --git a/fs/ocfs2/refcounttree.c b/fs/ocfs2/refcounttree.c
index d92b6c6..3e2198c 100644
--- a/fs/ocfs2/refcounttree.c
+++ b/fs/ocfs2/refcounttree.c
@@ -34,6 +34,7 @@
 #include "xattr.h"
 #include "namei.h"
 #include