Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-10-01 Thread Andreas Dilger
On Sep 20, 2018, at 10:40 PM, Zygo Blaxell wrote: > > On Fri, Sep 21, 2018 at 12:59:31PM +1000, Dave Chinner wrote: >> On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: > [...] >> With no DMAPI in the future, people with custom HSM-like interfaces >> based on dmapi are starting to tu

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-20 Thread Zygo Blaxell
On Fri, Sep 21, 2018 at 12:59:31PM +1000, Dave Chinner wrote: > On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: [...] > With no DMAPI in the future, people with custom HSM-like interfaces > based on dmapi are starting to turn to fanotify and friends to > provide them with the change n

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-20 Thread Dave Chinner
On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: > On Mon, Sep 10, 2018 at 07:06:46PM +1000, Dave Chinner wrote: > > On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > > > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > > > On Fri, Aug 31, 2018 at 01:10:4

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-18 Thread Zygo Blaxell
On Mon, Sep 10, 2018 at 07:06:46PM +1000, Dave Chinner wrote: > On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > > > On Thu, Aug 30, 2018 at 04:27:4

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-10 Thread Dave Chinner
On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > > > On Thu, Aug 23, 2018 at 08:58:4

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-06 Thread Zygo Blaxell
On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > > On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > > > > On Mon, Aug 20, 2018 at 08:33:4

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-06 Thread Dave Chinner
On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > > > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > > > On Mon, Aug 20, 2018 at 11:0

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-30 Thread Zygo Blaxell
On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > > > - is documenting rej

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-29 Thread Dave Chinner
On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > > - is documenting rejection on request alignment grounds > > > (i.e. EINVAL) in the man page

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-23 Thread Zygo Blaxell
On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > > - should we just round down the EOF dedupe request to the > > > block before EOF so dedupe s

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-23 Thread Zygo Blaxell
On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > - is documenting rejection on request alignment grounds > > (i.e. EINVAL) in the man page sufficient for app > > developers to understand what is goin

Re: [PATCH] generic: test for deduplication between different files

2018-08-21 Thread Filipe Manana
On Mon, Aug 20, 2018 at 12:11 AM, Dave Chinner wrote: > [cc linux-...@vger.kernel.org] > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Test that deduplication of an entire file that has a size that is not >> aligned to the filesystem's block s

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-21 Thread Filipe Manana
On Mon, Aug 20, 2018 at 2:09 AM, Dave Chinner wrote: > [cc linux-fsdevel now, too] > > On Mon, Aug 20, 2018 at 09:11:26AM +1000, Dave Chinner wrote: >> [cc linux-...@vger.kernel.org] >> >> On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: >> > From: Filipe Manana >> > >> > Test

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:17:18PM -0500, Eric Sandeen wrote: > > > On 8/20/18 7:49 PM, Dave Chinner wrote: > > Upon successful completion of this ioctl, the number of > > bytes successfully deduplicated is returned in bytes_deduped > > and a status code for the deduplication operatio

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Eric Sandeen
On 8/20/18 7:49 PM, Dave Chinner wrote: > Upon successful completion of this ioctl, the number of > bytes successfully deduplicated is returned in bytes_deduped > and a status code for the deduplication operation is > returned in status. If even a single byte in the rang

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > So why was this dedupe request even accepted by the kernel? Well, > > I think there's a bug in the check just above this: > > > > /* If we're linking to

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Darrick J. Wong
On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > [cc linux-fsdevel now, too] > > On Mon, Aug 20, 2018 at 09:11:26AM +1000, Dave Chinner wrote: > > [cc linux-...@vger.kernel.org] > > > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana

[patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-19 Thread Dave Chinner
[cc linux-fsdevel now, too] On Mon, Aug 20, 2018 at 09:11:26AM +1000, Dave Chinner wrote: > [cc linux-...@vger.kernel.org] > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Test that deduplication of an entire file that has a size that is n

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Dave Chinner
[cc linux-...@vger.kernel.org] On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that deduplication of an entire file that has a size that is not > aligned to the filesystem's block size into a different file does not > corrupt the destination's

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Filipe Manana
On Sun, Aug 19, 2018 at 5:19 PM, Eryu Guan wrote: > On Sun, Aug 19, 2018 at 04:41:31PM +0100, Filipe Manana wrote: >> On Sun, Aug 19, 2018 at 3:07 PM, Eryu Guan wrote: >> > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: >> >> From: Filipe Manana >> >> >> >> Test that dedupl

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Eryu Guan
On Sun, Aug 19, 2018 at 04:41:31PM +0100, Filipe Manana wrote: > On Sun, Aug 19, 2018 at 3:07 PM, Eryu Guan wrote: > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > >> From: Filipe Manana > >> > >> Test that deduplication of an entire file that has a size that is not > >>

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Filipe Manana
On Sun, Aug 19, 2018 at 3:07 PM, Eryu Guan wrote: > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Test that deduplication of an entire file that has a size that is not >> aligned to the filesystem's block size into a different file does not >> c

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Eryu Guan
On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that deduplication of an entire file that has a size that is not > aligned to the filesystem's block size into a different file does not > corrupt the destination's file data. > > This test is mot