On Mon, Oct 22, 2018 at 08:42:29AM +0300, Amir Goldstein wrote:
> On Mon, Oct 22, 2018 at 8:09 AM Dave Chinner wrote:
> >
> > On Mon, Oct 22, 2018 at 05:52:49AM +0100, Al Viro wrote:
> > > On Mon, Oct 22, 2018 at 03:37:41PM +1100, Dave Chinner wrote:
> > >
> > > > Ok, this is a bit of a mess. the
On 2018/10/22 下午2:29, Otto Kekäläinen wrote:
> I never got a reply to this thread,
I replied to you but got no rely:
https://lore.kernel.org/linux-btrfs/eba5de6f-535a-0f5d-e415-9cd622d71...@gmx.com/
And your steps are just what I suggested.
Thanks,
Qu
> but I am not replying to myself in
>
I never got a reply to this thread, but I am not replying to myself in
case somebody has the same issue and is reading the archive:
The problem went away after:
- deleted all snapshots as they seemed to slow down btrfs I/O so much
that simple commands like rm and rsync were unusable
- replaced the
On Mon, Oct 22, 2018 at 8:09 AM Dave Chinner wrote:
>
> On Mon, Oct 22, 2018 at 05:52:49AM +0100, Al Viro wrote:
> > On Mon, Oct 22, 2018 at 03:37:41PM +1100, Dave Chinner wrote:
> >
> > > Ok, this is a bit of a mess. the patches do not merge cleanly to a
> > > 4.19-rc1 base kernel because of all
On Mon, Oct 22, 2018 at 05:52:49AM +0100, Al Viro wrote:
> On Mon, Oct 22, 2018 at 03:37:41PM +1100, Dave Chinner wrote:
>
> > Ok, this is a bit of a mess. the patches do not merge cleanly to a
> > 4.19-rc1 base kernel because of all the changes to
> > include/linux/fs.h that have hit the tree aft
On Mon, Oct 22, 2018 at 03:37:41PM +1100, Dave Chinner wrote:
> Ok, this is a bit of a mess. the patches do not merge cleanly to a
> 4.19-rc1 base kernel because of all the changes to
> include/linux/fs.h that have hit the tree after this. There's also
> failures against Documentation/filesystems/
On Mon, Oct 22, 2018 at 01:21:12PM +1100, Dave Chinner wrote:
> On Sun, Oct 21, 2018 at 09:15:03AM -0700, Darrick J. Wong wrote:
> > Hi all,
> >
> > Dave, Eric, and I have been chasing a stale data exposure bug in the XFS
> > reflink implementation, and tracked it down to reflink forgetting to do
On Sun, Oct 21, 2018 at 09:15:03AM -0700, Darrick J. Wong wrote:
> Hi all,
>
> Dave, Eric, and I have been chasing a stale data exposure bug in the XFS
> reflink implementation, and tracked it down to reflink forgetting to do
> some of the file-extending activities that must happen for regular
> w
On Sun, Oct 21, 2018 at 09:18:17AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong
>
> Now that the vfs remap helper dirties the inode [cm]time for us, xfs no
> longer needs to do that on its own.
>
> Signed-off-by: Darrick J. Wong
looks good.
Reviewed-by: Dave Chinner
--
Dave Chinner
On Sun, Oct 21, 2018 at 09:18:11AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong
>
> Since xfs_file_remap_range is a thin wrapper, move the contents of
> xfs_reflink_remap_range into the shell. This cuts down on the vfs
> calls being made from internal xfs code.
>
> Signed-off-by: Darri
On Sun, Oct 21, 2018 at 09:17:57AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong
>
> Back when the XFS reflink code only supported clone_file_range, we were
> only able to return zero or negative error codes to userspace. However,
> now that copy_file_range (which returns bytes copied) c
On Sun, Oct 21, 2018 at 09:17:50AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong
>
> Move the offset <-> blocks unit conversions into
> xfs_reflink_remap_blocks to make the call site less ugly.
>
> Signed-off-by: Darrick J. Wong
Looks fine.
Reviewed-by: Dave Chinner
--
Dave Chinner
Le 20/10/2018 à 10:39, Cerem Cem ASLAN a écrit :
> You don't even need dd_rescue or something for RaspberryPi creation,
> you may use
> https://github.com/aktos-io/dcs-tools#cloning-a-target-into-a-new-target
That's interesting, but a bit overkill for my needs : I'll stay with my
simple script fo
From: Darrick J. Wong
Prior to remapping blocks, it is necessary to remove pages from the
destination file's page cache. Unfortunately, the truncation is not
aggressive enough -- if page size > block size, we'll end up zeroing
subpage blocks instead of removing them. So, round the start offset
From: Darrick J. Wong
Since the remap prep function can update the length of the remap
request, we can change this function to return the usual return status
instead of the odd behavior it has now.
Signed-off-by: Darrick J. Wong
Reviewed-by: Christoph Hellwig
---
fs/ocfs2/refcounttree.c |
From: Darrick J. Wong
There are no callers of vfs_dedupe_file_range_compare, so we might as
well make it a static helper and remove the export.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
Reviewed-by: Christoph Hellwig
---
fs/read_write.c| 187 +---
From: Darrick J. Wong
Since xfs_file_remap_range is a thin wrapper, move the contents of
xfs_reflink_remap_range into the shell. This cuts down on the vfs
calls being made from internal xfs code.
Signed-off-by: Darrick J. Wong
---
fs/xfs/xfs_file.c| 65 ++
From: Darrick J. Wong
Pass the same remap flags to generic_remap_checks for consistency.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
Reviewed-by: Christoph Hellwig
---
fs/read_write.c|2 +-
include/linux/fs.h |2 +-
mm/filemap.c |4 ++--
3 files changed,
From: Darrick J. Wong
Now that the vfs remap helper dirties the inode [cm]time for us, xfs no
longer needs to do that on its own.
Signed-off-by: Darrick J. Wong
---
fs/xfs/xfs_reflink.c |7 +--
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/
From: Darrick J. Wong
Back when the XFS reflink code only supported clone_file_range, we were
only able to return zero or negative error codes to userspace. However,
now that copy_file_range (which returns bytes copied) can use XFS'
clone_file_range, we have the opportunity to return partial res
From: Darrick J. Wong
Move the offset <-> blocks unit conversions into
xfs_reflink_remap_blocks to make the call site less ugly.
Signed-off-by: Darrick J. Wong
---
fs/xfs/xfs_reflink.c | 37 ++---
1 file changed, 18 insertions(+), 19 deletions(-)
diff --git
From: Darrick J. Wong
Now that we've moved the partial EOF block checks to the VFS helpers, we
can remove the redundant functionality from XFS.
Signed-off-by: Darrick J. Wong
Reviewed-by: Dave Chinner
---
fs/xfs/xfs_reflink.c | 19 ---
1 file changed, 19 deletions(-)
diff
From: Darrick J. Wong
Change the ocfs2 remap code to allow for returning partial results.
Signed-off-by: Darrick J. Wong
---
fs/ocfs2/file.c |7 +
fs/ocfs2/refcounttree.c | 72 +--
fs/ocfs2/refcounttree.h | 12
3 files ch
From: Darrick J. Wong
Since ocfs2_remap_file_range is a thin shell around
ocfs2_remap_remap_range, move everything from the latter into the
former.
Signed-off-by: Darrick J. Wong
---
fs/ocfs2/file.c | 68 +++-
fs/ocfs2/refcounttree.c | 113 +++
From: Darrick J. Wong
Prior to remapping blocks, it is necessary to remove pages from the
destination file's page cache. Unfortunately, the truncation is not
aggressive enough -- if page size > block size, we'll end up zeroing
subpage blocks instead of removing them. So, round the start offset
From: Darrick J. Wong
When cloning blocks into another file, truncate the page cache before we
start remapping blocks so that concurrent reads wait for us to finish.
Signed-off-by: Darrick J. Wong
---
fs/ocfs2/refcounttree.c | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
From: Darrick J. Wong
A deduplication data corruption is exposed in XFS and btrfs. It is
caused by extending the block match range to include the partial EOF
block, but then allowing unknown data beyond EOF to be considered a
"match" to data in the destination file because the comparison is only
From: Darrick J. Wong
Plumb in a remap flag that enables the filesystem remap handler to
shorten remapping requests for callers that can handle it. Now
copy_file_range can report partial success (in case we run up against
alignment problems, resource limits, etc.).
We also enable CAN_SHORTEN fo
From: Darrick J. Wong
Combine the clone_file_range and dedupe_file_range operations into a
single remap_file_range file operation dispatch since they're
fundamentally the same operation. The differences between the two can
be made in the prep functions.
Signed-off-by: Darrick J. Wong
Reviewed-
From: Darrick J. Wong
Plumb the remap flags through the filesystem from the vfs function
dispatcher all the way to the prep function to prepare for behavior
changes in subsequent patches.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
Reviewed-by: Christoph Hellwig
---
fs/ocfs2/f
From: Darrick J. Wong
Change the remap_file_range functions to take a number of bytes to
operate upon and return the number of bytes they operated on. This is a
requirement for allowing fs implementations to return short clone/dedupe
results to the user, which will enable us to obey resource lim
From: Darrick J. Wong
Plumb a remap_flags argument through the {do,vfs}_clone_file_range
functions so that clone can take advantage of it.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
---
fs/ioctl.c |2 +-
fs/nfsd/vfs.c |2 +-
fs/overlayfs/copy_up.c
From: Darrick J. Wong
Plumb a remap_flags argument through the vfs_dedupe_file_range_one
functions so that dedupe can take advantage of it.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
---
fs/overlayfs/file.c |3 ++-
fs/read_write.c |9 ++---
include/linux/fs.h
From: Darrick J. Wong
Extend generic_remap_file_range_prep to handle inode metadata updates
when remapping into a file. If the operation can possibly alter the
file contents, we must update the ctime and mtime and remove security
privileges, just like we do for regular file writes.
Signed-off-b
From: Darrick J. Wong
File range remapping, if allowed to run past the destination file's EOF,
is an optimization on a regular file write. Regular file writes that
extend the file length are subject to various constraints which are not
checked by range cloning.
This is a correctness problem bec
From: Darrick J. Wong
Since we use clone_verify_area for both clone and dedupe range checks,
rename the function to make it clear that it's for both.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
---
fs/read_write.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(
From: Darrick J. Wong
The vfs_clone_file_prep is a generic function to be called by filesystem
implementations only. Rename the prefix to generic_ and make it more
clear that it applies to remap operations, not just clones.
Signed-off-by: Darrick J. Wong
Reviewed-by: Amir Goldstein
---
fs/oc
From: Darrick J. Wong
Don't bother calling the filesystem for a zero-length dedupe request;
we can return zero and exit.
Signed-off-by: Darrick J. Wong
Reviewed-by: Christoph Hellwig
Reviewed-by: Amir Goldstein
---
fs/read_write.c |5 +
1 file changed, 5 insertions(+)
diff --git a/
From: Darrick J. Wong
Move the file range checks from vfs_clone_file_prep into a separate
generic_remap_checks function so that all the checks are collected in a
central location. This forms the basis for adding more checks from
generic_write_checks that will make cloning's input checking more
c
From: Darrick J. Wong
If a remap caller asks us to remap to the source file's EOF and the
source file length leaves us with a zero byte request, exit early.
Signed-off-by: Darrick J. Wong
Reviewed-by: Christoph Hellwig
---
fs/read_write.c |2 ++
1 file changed, 2 insertions(+)
diff --gi
Hi all,
Dave, Eric, and I have been chasing a stale data exposure bug in the XFS
reflink implementation, and tracked it down to reflink forgetting to do
some of the file-extending activities that must happen for regular
writes.
We then started auditing the clone, dedupe, and copyfile code and
rea
From: Darrick J. Wong
vfs_clone_file_prep_inodes cannot return 0 if it is asked to remap from
a zero byte file because that's what btrfs does.
Signed-off-by: Darrick J. Wong
Reviewed-by: Christoph Hellwig
---
fs/read_write.c |3 ---
1 file changed, 3 deletions(-)
diff --git a/fs/read_wr
On 10/21/2018 05:31 PM, Filipe Manana wrote:
On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov wrote:
On 21.10.2018 10:16, Filipe Manana wrote:
On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote:
Make sure missing device is included in the alloc list when it is
scanned on a mounted FS.
On Tue, Oct 09, 2018 at 02:28:21AM +0800, Anand Jain wrote:
> We have a known bug in btrfs, that we let the device path be changed
> after the device has been mounted. So using this loop hole the new
> copied device would appears as if its mounted immediately after its
> been copied. So this test c
On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov wrote:
>
>
>
> On 21.10.2018 10:16, Filipe Manana wrote:
> > On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote:
> >>
> >> Make sure missing device is included in the alloc list when it is
> >> scanned on a mounted FS.
> >>
> >> This test case needs
On 21.10.2018 10:16, Filipe Manana wrote:
> On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote:
>>
>> Make sure missing device is included in the alloc list when it is
>> scanned on a mounted FS.
>>
>> This test case needs btrfs kernel patch which is in the ML
>> [PATCH] btrfs: handle dynamical
On Mon, Nov 13, 2017 at 2:26 AM Anand Jain wrote:
>
> Make sure missing device is included in the alloc list when it is
> scanned on a mounted FS.
>
> This test case needs btrfs kernel patch which is in the ML
> [PATCH] btrfs: handle dynamically reappearing missing device
> Without the kernel pa
On Sun, Oct 21, 2018 at 6:05 AM Andrew Nelson wrote:
>
> Also, is the drive in a safe state to use? Is there anything I should
> run on the drive to check consistency?
It should be in a safe state. You can verify it running "btrfs check
/dev/" (it's a readonly operation).
If you are able to patc
48 matches
Mail list logo
