Re: Btrfs send to send out metadata and data separately

2016-08-01 Thread Filipe Manana
On Fri, Jul 29, 2016 at 1:40 PM, Qu Wenruo wrote: > Hi Filipe, and maintainers, > > I'm recently working on the root fix to free send from calling backref walk. > > My current idea is to send data and metadata separately, and only do clone > detection inside the send subvolume. > > This method nee

Re: Btrfs send to send out metadata and data separately

2016-08-03 Thread Filipe Manana
On Tue, Aug 2, 2016 at 2:20 AM, Qu Wenruo wrote: > > > At 08/02/2016 02:00 AM, Filipe Manana wrote: >> >> On Fri, Jul 29, 2016 at 1:40 PM, Qu Wenruo wrote: >>> >>> Hi Filipe, and maintainers, >>> >>> I'm recently working on the r

Re: [PATCH v3 2/3] btrfs: relocation: Fix leaking qgroups numbers on data extents

2016-08-12 Thread Filipe Manana
l account these data extents correctly. Hi Qu, This changelog should mention this fixes a regression introduced in the 4.2 kernel. It's specially important for people responsible to backport fixes to earlier kernel releases. > > Cc: Mark Fasheh > Reported-by: Mark Fashe

Re: Btrfs send to send out metadata and data separately

2016-08-24 Thread Filipe Manana
On Wed, Aug 24, 2016 at 3:36 AM, Qu Wenruo wrote: > > > At 08/04/2016 09:52 AM, Qu Wenruo wrote: >> >> >> >> At 08/03/2016 05:05 PM, Filipe Manana wrote: >>> >>> On Tue, Aug 2, 2016 at 2:20 AM, Qu Wenruo >>> wrote: >

Re: [PATCH] Btrfs: handle pending renames with recycled inodes properly

2016-08-24 Thread Filipe Manana
On Tue, Aug 23, 2016 at 6:22 PM, Josef Bacik wrote: > Suppose you have the following tree in snap1 on a file system mounted with -o > inode_cache so that inode numbers are recycled > > └── [258] a > └── [257] b > > and then you remove b, rename a to c, and then re-create b in c so yo

Re: [PATCH] btrfs: Fix extent map leak in find_first_block_group

2016-08-24 Thread Filipe Manana
On Wed, Aug 24, 2016 at 5:13 AM, Qu Wenruo wrote: > The following commit introduced the extent map leak: > commit 6fb37b756acce6d6e045f79c3764206033f617b4 > Author: Liu Bo > Date: Wed Jun 22 18:31:27 2016 -0700 > > Btrfs: check inconsistence between chunk and block group > > This leads to s

Re: [PATCH] Btrfs: fix BUG_ON in btrfs_mark_buffer_dirty

2016-09-05 Thread Filipe Manana
On Fri, Sep 2, 2016 at 8:35 PM, Liu Bo wrote: > This can only happen with CONFIG_BTRFS_FS_CHECK_INTEGRITY=y. > > Commit 1ba98d0 ("Btrfs: detect corruption when non-root leaf has zero item") > assumes that a leaf is its root when leaf->bytenr == btrfs_root_bytenr(root), > however, we should not use

Re: Is stability a joke? (wiki updated)

2016-09-12 Thread Filipe Manana
On Mon, Sep 12, 2016 at 5:56 PM, Austin S. Hemmelgarn wrote: > On 2016-09-12 12:27, David Sterba wrote: >> >> On Mon, Sep 12, 2016 at 04:27:14PM +0200, David Sterba wrote: I therefore would like to propose that some sort of feature / stability matrix for the latest kernel is added t

Re: send -p stuck with kernel bug message

2016-09-26 Thread Filipe Manana
On Mon, Sep 26, 2016 at 3:16 PM, Romulo wrote: > Hi, > > I faced a problem with send -p command, that occur with an specif > snapshot and any parent before it, I isolated the problem being in > send as it occur even in a "send -p> /dev/null". I've tried "-vvv" > with no extra information succe

Re: [PATCH] Btrfs: fix enospc in punch hole

2016-10-07 Thread Filipe Manana
On Fri, Oct 7, 2016 at 7:09 AM, robbieko wrote: > From: Robbie Ko > > when extent-tree level > BTRFS_MAX_LEVEL / 2, > __btrfs_drop_extents -> btrfs_duplicate_item -> > setup_leaf_for_split -> split_leaf > maybe enospc, because min_size is too small, > need use btrfs_calc_trans_metadata_size. Thi

Re: [PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

2016-10-07 Thread Filipe Manana
On Fri, Oct 7, 2016 at 11:05 AM, robbieko wrote: > From: Robbie Ko > > We found a fsync deadlock ie. 32021->32020->32028->14431->14436->32021, > in log_new_dir_dentries, because btrfs_search_forward get path lock, then > call btrfs_iget will get another extent_buffer lock, maybe occur deadlock.

Re: [PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

2016-10-07 Thread Filipe Manana
replies with top posting, it just breaks the thread. thanks > > Thanks. > robbieko > > Filipe Manana 於 2016-10-07 18:23 寫道: > > On Fri, Oct 7, 2016 at 11:05 AM, robbieko wrote: >> From: Robbie Ko >> >> We found a fsync deadlock ie. 32021->32020->32028-

Re: [PATCH 5/5] Btrfs: incremental send, add gen check if has waiting_dir_move in the will_overwrite_ref

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 9:12 AM, robbieko wrote: > From: Robbie Ko > > There a some case similar as before. As before what? Each change log should be complete and the reader is not supposed to guess what's the previous patch or commit this is referring to. Imagine yourself or someone else readin

Re: [PATCH] Btrfs: fix fsync deadlock in log_new_dir_dentries

2016-10-12 Thread Filipe Manana
13120, len:16384 > locker pid: 14431 write lock > wait pid: 32028 read lock > extent_buffer: start:446503845888, len: 16384 > locker pid: 14436 write lock > wait pid: 14431 write lock > extent_buffer: start: 446504386560, len: 16384 > locker pid: 32021 write l

Re: [PATCH 1/5] Btrfs: incremental send, fix don't skip root inode in overwrite_ref

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 9:12 AM, robbieko wrote: > From: Robbie Ko > > When root dir item change, don't skip will_overwrite_ref, > because root inode always exist. What do you mean by root dir item change? You mean indoe 256 changed, how did it change (and how can it change)? > > Example: > Par

Re: [PATCH 2/5] Btrfs: incremental send, add gen for is_waiting_for_rm when some corner case

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 9:12 AM, robbieko wrote: > From: Robbie Ko > > There a one case for old_gen waiting_for_rm, > but new_gen use get_cur_path with the same inode. > > Example: > Parent snapshot: > | dir258/ (ino 258, dir) > |--- dir257/ (ino 257, dir) > | dir259/

Re: [PATCH] Btrfs: fix enospc in punch hole

2016-10-12 Thread Filipe Manana
node might be added to each level of the tree (since there's a new key and every parent node was full). Having detailed and well written change logs is important... Thanks > > Thanks. > robbieko > > Filipe Manana 於 2016-10-07 18:18 寫到: > >> On Fri, Oct 7, 2016 at

Re: [PATCH] btrfs: fix silent data corruption while reading compressed inline extents

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 3:28 AM, Zygo Blaxell wrote: > rsync -S causes a large number of small writes separated by small seeks > to form sparse holes in files that contain runs of zero bytes. Rarely, > this can lead btrfs to write a file with a compressed inline extent > followed by other data, l

Re: [PATCH 4/5] Btrfs: incremental send, add gen check in did_overwrite_ref

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 9:12 AM, robbieko wrote: > From: Robbie Ko > > There a some case similar as before. As before what? Each change log should be complete and the reader is not supposed to guess what's the previous patch or commit this is referring to. Imagine yourself or someone else readin

Re: [PATCH 3/5] Btrfs: incremental send, add gen in waiting_dir_move for some corner case

2016-10-12 Thread Filipe Manana
On Wed, Oct 12, 2016 at 9:12 AM, robbieko wrote: > From: Robbie Ko > > There a some case similar as before. As before what? Each change log should be complete and the reader is not supposed to guess what's the previous patch or commit this is referring to. Imagine yourself or someone else readin

Re: [PATCH] Btrfs: fix BUG_ON in btrfs_mark_buffer_dirty

2016-10-12 Thread Filipe Manana
On Tue, Sep 6, 2016 at 10:51 PM, Liu Bo wrote: > Hi Filipe, > > On Mon, Sep 05, 2016 at 04:28:09PM +0100, Filipe Manana wrote: >> On Fri, Sep 2, 2016 at 8:35 PM, Liu Bo wrote: >> > This can only happen with CONFIG_BTRFS_FS_CHECK_INTEGRITY=y. >> > >> >

Re: [PATCH] Btrfs: fix BUG_ON in btrfs_mark_buffer_dirty

2016-10-13 Thread Filipe Manana
On Thu, Oct 13, 2016 at 1:37 AM, Liu Bo wrote: > On Wed, Oct 12, 2016 at 10:23:52PM +0100, Filipe Manana wrote: >> On Tue, Sep 6, 2016 at 10:51 PM, Liu Bo wrote: >> > Hi Filipe, >> > >> > On Mon, Sep 05, 2016 at 04:28:09PM +0100, Filipe Manana wrote: >>

Re: [PATCH 1/4] btrfs: add test for an incremental send don't skip overwrite ref for inode 256

2016-10-13 Thread Filipe Manana
at an incremental send operation does not skip inode 256 check > +# overwrite ref, because inode 256 always exist. > +# > +#------- > +# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved. > +# Author: Filipe Manana No, I'm

Re: [PATCH 0/4] Btrfs: add serval test case for incremental send

2016-10-13 Thread Filipe Manana
On Thu, Oct 13, 2016 at 12:04 PM, robbieko wrote: > From: Robbie Ko > > Patch for test btrfs incremental send. > > Robbie Ko (4): > btrfs: add test for an incremental send don't skip overwrite ref for > inode 256 > btrfs: add test for an incremental send add gen for is_waiting_for_rm >

Re: [PATCH] btrfs-progs: Initialize btrfs_path before use

2016-10-24 Thread Filipe Manana
On Mon, Oct 24, 2016 at 3:57 PM, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > While performing an fsck, an assertion failure occurs because of reusing path > in a loop. > ctree.c:1112: btrfs_search_slot: Warning: assertion `p->nodes[0] != NULL` > failed, value 0 > > Signed-off-by: Gol

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Filipe Manana
On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When creating a block group we don't need to set the log for full commit > > if the new block group is not used for

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Filipe Manana
On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > > > > > On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When creating a block group we d

Re: [PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

2018-11-09 Thread Filipe Manana
> > has a path loop with 267, > > and then we add 259, 265 to the stack, but we don't remove from > > pending_dir_moves rb_tree. > > > > 7. In round 15, we processing 266, we delayed the rename because 266 > > has a path loop with 270, > > So we look f

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-09 Thread Filipe Manana
On Fri, Nov 9, 2018 at 12:27 AM Qu Wenruo wrote: > > > > On 2018/11/8 下午10:48, Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > >> > >> On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > >>> > >>>

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-13 Thread Filipe Manana
On Tue, Nov 13, 2018 at 2:31 PM David Sterba wrote: > > On Thu, Nov 08, 2018 at 02:48:29PM +, Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > > > > > > On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > > >

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-13 Thread Filipe Manana
On Tue, Nov 13, 2018 at 5:08 PM David Sterba wrote: > > On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > > This patchset can be fetched from github: > > > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Filipe Manana
On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: > > Support for a new command 'btrfs dev forget [dev]' is proposed here > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > this patch proposes to use ioctl #5 as it was empty. > IOW(BTRFS_IOCTL_MAGIC, 5, ..) > This patch ad

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Filipe Manana
On Wed, Nov 14, 2018 at 11:15 AM Filipe Manana wrote: > > On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: > > > > Support for a new command 'btrfs dev forget [dev]' is proposed here > > to undo the effects of 'btrfs dev scan [dev]'. For this purpos

Re: [PATCH] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Filipe Manana
On Mon, Nov 19, 2018 at 11:09 AM Qu Wenruo wrote: > > > > On 2018/11/19 下午5:48, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If the quota enable and snapshot creation ioctls are called concurrently > > we can get into a deadlock where the task

Re: [PATCH] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Filipe Manana
On Mon, Nov 19, 2018 at 11:35 AM Qu Wenruo wrote: > > > > On 2018/11/19 下午7:13, Filipe Manana wrote: > > On Mon, Nov 19, 2018 at 11:09 AM Qu Wenruo wrote: > >> > >> > >> > >> On 2018/11/19 下午5:48, fdman...@kernel.org wrote: > >>>

Re: [PATCH] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Filipe Manana
On Mon, Nov 19, 2018 at 11:52 AM Filipe Manana wrote: > > On Mon, Nov 19, 2018 at 11:35 AM Qu Wenruo wrote: > > > > > > > > On 2018/11/19 下午7:13, Filipe Manana wrote: > > > On Mon, Nov 19, 2018 at 11:09 AM Qu Wenruo wrote: > > >> > > &

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Filipe Manana
On Mon, Nov 19, 2018 at 2:48 PM Qu Wenruo wrote: > > > > On 2018/11/19 下午10:15, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If the quota enable and snapshot creation ioctls are called concurrently > > we can get into a deadlock where the task

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-23 Thread Filipe Manana
On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > I noticed in a giant dbench run that we spent a lot of time on lock > contention while running transaction commit. This is because dbench > results in a lot of fsync()'s that do a btrfs_transaction_commit(), and > they all run the delayed re

Re: [PATCH] Btrfs: bring back key search optimization to btrfs_search_old_slot()

2018-11-26 Thread Filipe Manana
On Fri, Nov 16, 2018 at 11:09 AM wrote: > > From: Filipe Manana > > Commit d7396f07358a ("Btrfs: optimize key searches in btrfs_search_slot"), > dated from August 2013, introduced an optimization to search for keys in a > node/leaf to both btrfs_search_slot() and btrf

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-26 Thread Filipe Manana
On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote: > > On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When a transaction commit starts, it attempts to pause scrub and it blocks > > until the scrub is pause

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-27 Thread Filipe Manana
On Tue, Nov 27, 2018 at 7:22 PM Josef Bacik wrote: > > On Fri, Nov 23, 2018 at 04:59:32PM +, Filipe Manana wrote: > > On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > > > > > I noticed in a giant dbench run that we spent a lot of time on lock > > &g

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-28 Thread Filipe Manana
On Wed, Nov 28, 2018 at 9:26 AM Lu Fengqi wrote: > > On Wed, Nov 28, 2018 at 09:48:07AM +0200, Nikolay Borisov wrote: > > > > > >On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: > >> On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: > >>> > >>> > >>> On 28.11.18 г. 5:07 ч., Lu Fengq

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-28 Thread Filipe Manana
On Wed, Nov 28, 2018 at 2:22 PM David Sterba wrote: > > On Mon, Nov 26, 2018 at 08:10:30PM +, Filipe Manana wrote: > > On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote: > > > > > > On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrot

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-28 Thread Filipe Manana
On Wed, Nov 28, 2018 at 7:09 PM David Sterba wrote: > > On Tue, Nov 27, 2018 at 03:08:08PM -0500, Josef Bacik wrote: > > On Tue, Nov 27, 2018 at 07:59:42PM +, Chris Mason wrote: > > > On 27 Nov 2018, at 14:54, Josef Bacik wrote: > > > > > > > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay B

Re: [PATCH] btrfs: skip file_extent generation check for free_space_inode in run_delalloc_nocow

2018-11-29 Thread Filipe Manana
n directly write the inode to the disk. > > Fixes: 78d4295b1eee ("btrfs: lift some btrfs_cross_ref_exist checks in nocow > path") > Signed-off-by: Lu Fengqi The code changes look good to me. Reviewed-by: Filipe Manana Thanks. > --- > fs/btrfs/inode.c | 3 ++- >

Re: [PATCH v2 1/3] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-29 Thread Filipe Manana
On Thu, Nov 29, 2018 at 9:27 AM Anand Jain wrote: > > The device_list_mutex and scrub_lock creates a nested locks in > btrfs_scrub_dev(). > > During lock the order is device_list_mutex and then scrub_lock, and during > unlock, the order is device_list_mutex and then scrub_lock. > Fix this to the l

Re: [PATCH 2/2] btrfs: run delayed items before dropping the snapshot

2018-11-30 Thread Filipe Manana
s smarter in the future by making the delayed > items > per-root, and then simply drop any delayed items for roots that we are going > to > delete. But for now just a quick and easy solution is the safest. > > Cc: sta...@vger.kernel.org > Signed-off-by: Josef Bacik Reviewed-by: Fi

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-11-30 Thread Filipe Manana
\n"); Please use btrfs_warn(), it makes sure we use a consistent message style, identifies the fs, etc. Also, "thats" should be "that is" or "that's". With that fixed, Reviewed-by: Filipe Manana > + > if (trans->transaction != fs_info->

Re: [PATCH 2/2] btrfs: run delayed items before dropping the snapshot

2018-11-30 Thread Filipe Manana
On Fri, Nov 30, 2018 at 5:12 PM Filipe Manana wrote: > > On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > > > From: Josef Bacik > > > > With my delayed refs patches in place we started seeing a large amount > > of aborts in __btrfs_free_extent > >

Re: [PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-05 Thread Filipe Manana
s smarter in the future by making the delayed > items > per-root, and then simply drop any delayed items for roots that we are going > to > delete. But for now just a quick and easy solution is the safest. > > Cc: sta...@vger.kernel.org > Signed-off-by: Josef Bacik Reviewed-by: Filipe Man

Re: [PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-10-12 Thread Filipe Manana
On Thu, Oct 11, 2018 at 5:13 AM Darrick J. Wong wrote: > > From: Darrick J. Wong > > A deduplication data corruption is exposed by fstests generic/505 on > XFS. (and btrfs) Btw, the generic test I wrote was indeed numbered 505, however it was never committed and there's now a generic/505 which

Re: [PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-11-02 Thread Filipe Manana
On Mon, Oct 15, 2018 at 1:31 AM Dave Chinner wrote: > > On Fri, Oct 12, 2018 at 09:22:18PM +0100, Filipe Manana wrote: > > On Thu, Oct 11, 2018 at 5:13 AM Darrick J. Wong > > wrote: > > > > > > From: Darrick J. Wong > > > > > > A deduplica

Re: [PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-11-02 Thread Filipe Manana
On Fri, Nov 2, 2018 at 5:42 PM Darrick J. Wong wrote: > > On Fri, Nov 02, 2018 at 12:04:39PM +, Filipe Manana wrote: > > On Mon, Oct 15, 2018 at 1:31 AM Dave Chinner wrote: > > > > > > On Fri, Oct 12, 2018 at 09:22:18PM +0100, Filipe Manana wrote: > >

Re: [PATCH 05/25] vfs: avoid problematic remapping requests into partial EOF block

2018-11-02 Thread Filipe Manana
On Fri, Nov 2, 2018 at 6:18 PM Filipe Manana wrote: > > On Fri, Nov 2, 2018 at 5:42 PM Darrick J. Wong > wrote: > > > > On Fri, Nov 02, 2018 at 12:04:39PM +, Filipe Manana wrote: > > > On Mon, Oct 15, 2018 at 1:31 AM Dave Chinner wrote: > > > >

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-10 Thread Filipe Manana
On Sat, Dec 8, 2018 at 12:51 AM Qu Wenruo wrote: > > > > On 2018/12/8 上午8:47, David Sterba wrote: > > On Fri, Dec 07, 2018 at 06:51:21AM +0800, Qu Wenruo wrote: > >> > >> > >> On 2018/12/7 上午3:35, David Sterba wrote: > >>> On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > On Thu

Re: [PATCH] Btrfs: do not overwrite error return value in the balance ioctl

2018-12-17 Thread Filipe Manana
On Mon, Dec 17, 2018 at 8:25 AM Anand Jain wrote: > > > > On 12/15/2018 03:45 AM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If the call to btrfs_balance() failed we would overwrite the error > > returned to user space with -EFAULT if the call

Re: [PATCH v2 2/2] btrfs: volumes: Make sure no dev extent is beyond device boundary

2019-01-07 Thread Filipe Manana
On Fri, Oct 5, 2018 at 10:46 AM Qu Wenruo wrote: > > Add extra dev extent end check against device boundary. > > Signed-off-by: Qu Wenruo > --- > fs/btrfs/volumes.c | 17 + > 1 file changed, 17 insertions(+) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index bf0b2c1

Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.

2019-01-07 Thread Filipe Manana
On Tue, Jan 1, 2019 at 7:07 PM Giuseppe Della Bianca wrote: > > In data giovedì 9 agosto 2018 20:48:03 CEST, Jeff Mahoney ha scritto: > > On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote: > > > Hi. > > > > > > My system: > > > - Fedora 28 x86_64 > > > - kernel-4.17.7-200 > > > - btrfs-progs-4.15.1-

Re: [For 5.0-rc PATCH] btrfs: Use real device structure to verify dev extent

2019-01-08 Thread Filipe Manana
ss the device boundary check. > > [FIX] > This patch will try to verify the device returned by btrfs_find_device() > and if it's a dummy then re-search in seed devices. > > Reported-by: Filipe Manana > Fixes: cf90d884b347 ("btrfs: Introduce mount time chunk <

Re: [PATCH] Btrfs: avoid deadlock with memory reclaim due to allocation of devices

2019-01-08 Thread Filipe Manana
On Thu, Dec 13, 2018 at 9:18 PM wrote: > > From: Filipe Manana > > Several places allocate a device while holding the device list mutex. This > can result in a deadlock if reclaim happens because the device, and its > flush bio, are allocated using GFP_KERNEL mode (by __alloc_d

Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.

2019-01-08 Thread Filipe Manana
On Tue, Jan 8, 2019 at 12:14 PM gius db wrote: > > Il giorno mar 8 gen 2019 alle ore 00:11 Filipe Manana > ha scritto: > > ]zac[ > > > [ 1558.056931] Call Trace: > > > [ 1558.057011] __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs] > > > [ 1558.057011]

Re: [PATCH] Btrfs: fix deadlock when using free space tree due to block group creation

2019-01-08 Thread Filipe Manana
On Tue, Jan 8, 2019 at 4:14 PM David Sterba wrote: > > On Tue, Jan 08, 2019 at 11:44:41AM +, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When modifying the free space tree we can end up COWing one of its extent > > buffers which in turn might re

Re: [PATCH] Btrfs: avoid deadlock with memory reclaim due to allocation of devices

2019-01-09 Thread Filipe Manana
On Wed, Jan 9, 2019 at 6:27 PM David Sterba wrote: > > On Thu, Dec 13, 2018 at 09:17:25PM +, fdman...@kernel.org wrote: > > - if (list_empty(&fs_devices->resized_devices)) > > - return; > > - > > - mutex_lock(&fs_devices->device_list_mutex); > > mutex_lock(&fs_info->c

Re: [PATCH 2/2] fstests: btrfs: Introduce stress test for deadlock between snapshot delete and other read-write operations

2019-01-10 Thread Filipe Manana
On Thu, Jan 10, 2019 at 6:15 AM Qu Wenruo wrote: > > Commit fb235dc06fac ("btrfs: qgroup: Move half of the qgroup accounting > time out of commit trans") could cause ABBA deadlock between backref > lookup with write lock hold (subvolume deletion) and other read/write > operations. > > It's going t

Re: [PATCH 2/2] fstests: btrfs: Introduce stress test for deadlock between snapshot delete and other read-write operations

2019-01-10 Thread Filipe Manana
On Thu, Jan 10, 2019 at 1:49 PM Qu Wenruo wrote: > > > > On 2019/1/10 下午8:08, Filipe Manana wrote: > > On Thu, Jan 10, 2019 at 6:15 AM Qu Wenruo wrote: > >> > >> Commit fb235dc06fac ("btrfs: qgroup: Move half of the qgroup accounting > >> ti

Re: [PATCH v2 2/2] fstests: btrfs: Introduce stress test for deadlock between snapshot delete and other read-write operations

2019-01-11 Thread Filipe Manana
t needs some time to generate enough files to bump the tree height to > trigger the bug. > In my test environment, with 'unsafe' cache mode for the VM, it triggers > the bug at around 70~90 seconds. So I leave the default runtime to 120s > to make sure the bug will be triggered. &

Re: [PATCH v2 4/6] btrfs: Introduce mount time chunk <-> dev extent mapping check

2019-01-14 Thread Filipe Manana
On Wed, Aug 1, 2018 at 3:39 AM Qu Wenruo wrote: > > This patch will introduce chunk <-> dev extent mapping check, to protect > us against invalid dev extents or chunks. > > Since chunk mapping is the fundamental infrastructure of btrfs, extra > check at mount time could prevent a lot of unexpected

Re: [PATCH 3/3] Btrfs-progs: add test for receive

2019-01-14 Thread Filipe Manana
On Mon, Jan 14, 2019 at 2:08 PM David Disseldorp wrote: > > On Mon, 14 Jan 2019 13:31:46 +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > Add a test for a scenario that used to fail due to find_mount_root() > > incorrectly determining the mou

Re: [PATCH 1/3] Btrfs-progs: fix mount point detection due to partial prefix match

2019-01-14 Thread Filipe Manana
On Mon, Jan 14, 2019 at 1:59 PM David Disseldorp wrote: > > On Mon, 14 Jan 2019 13:30:24 +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > When attempting to find the mount point of a path we can end up returning > > an incorrect mount point. T

Re: [PATCH 2/2] Btrfs: fix unprotected deletion from pending_chunks list

2019-01-21 Thread Filipe Manana
On Mon, Jan 21, 2019 at 7:07 PM Alex Lyakas wrote: > > Hi Filipe, > > On Tue, Dec 2, 2014 at 8:08 PM Filipe Manana wrote: > > > > On block group remove if the corresponding extent map was on the > > transaction->pending_chunks list, we were deleting the exten

Re: btrfs receive deadlock and questions

2019-01-22 Thread Filipe Manana
On Tue, Jan 22, 2019 at 3:26 PM Eli V wrote: > > I seem to have it a deadlock trying out btrfs send & receive. Now I > haven't used btrfs send & receive much, so don't have much experience > with them. Anyways, bug report and stack traces: > https://bugzilla.kernel.org/show_bug.cgi?id=202383 This

Re: btrfs receive deadlock and questions

2019-01-22 Thread Filipe Manana
On Tue, Jan 22, 2019 at 5:20 PM Eli V wrote: > > On Tue, Jan 22, 2019 at 11:03 AM Eli V wrote: > > > > On Tue, Jan 22, 2019 at 10:42 AM Filipe Manana wrote: > > > > > > On Tue, Jan 22, 2019 at 3:26 PM Eli V wrote: > > > > > > > >

Re: [PATCH] btrfs: Handle ENOMEM gracefully in cow_file_range_async

2019-01-25 Thread Filipe Manana
On Fri, Jan 25, 2019 at 3:08 PM David Sterba wrote: > > On Wed, Jan 09, 2019 at 04:43:03PM +0200, Nikolay Borisov wrote: > > If we run out of memory during delalloc filling in compress case btrfs > > is going to BUG_ON. This is unnecessary since the higher levels code > > (btrfs_run_delalloc_range

Re: [PATCH] btrfs: Test if btrfs will report false ENOSPC error balnacing small metadata chunk

2019-01-29 Thread Filipe Manana
standard environment, filters and checks > +. ./common/rc > +. ./common/filter > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_fs btrfs > +_supported_os Linux > +_

Re: [PATCH v2] btrfs: Test if btrfs will commit too many transaction for balance

2019-01-29 Thread Filipe Manana
((after_gen - before_gen))" > + echo "theoretic generation difference: ${theoretic_gen}" > + echo "threshold generation difference: ${threshold_gen}" > +fi > + > +echo "super generation before balance: ${before_gen}" >> $seqres.full

Re: [PATCH] btrfs: Test if btrfs hits EDQUOT without trying to reclaim some space

2019-01-29 Thread Filipe Manana
t; > /dev/null > +$BTRFS_UTIL_PROG qgroup limit -e 1G "$SCRATCH_MNT" > + > +$XFS_IO_PROG -f -c "falloc 0 900M" "$SCRATCH_MNT/padding" | _filter_xfs_io No need to use the _filter_xfs_io for falloc (it doesn't output throughput values such as

Re: [PATCH 4/4] Btrfs: remove no longer needed range length checks for deduplication

2019-01-31 Thread Filipe Manana
On Wed, Dec 12, 2018 at 6:07 PM wrote: > > From: Filipe Manana > > Comparing the content of the pages in the range to deduplicate is now done > by the generic helper generic_remap_file_range_prep(), which takes care of > ensuring we do not compare/deduplicate undefined data b

Re: [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication

2019-01-31 Thread Filipe Manana
On Thu, Dec 13, 2018 at 4:08 PM David Sterba wrote: > > On Wed, Dec 12, 2018 at 06:05:58PM +, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Checking if the destination root is read-only was being performed only for > > clone operations. Make dedupli

Re: [PATCH 1/2] btrfs: check for refs on snapshot delete resume

2019-02-07 Thread Filipe Manana
ee > should be in a normal state from then on, and any problems we run into > from there are different issues. I tested this with an existing broken > fs and my reproducer that creates a broken fs and it fixed both file > systems. > > Signed-off-by: Josef Bacik Reviewed-by: Fili

Re: [PATCH 2/2] btrfs: save drop_progress if we drop refs at all

2019-02-07 Thread Filipe Manana
t place we dropped a reference for > our block in do_walk_down. Then if wc->stage == UPDATE_BACKREF we know > we'll start over from a place we meant to, and otherwise things continue > to work as they did before. > > I have a complicated reproducer for this problem, without t

Re: [PATCH v2 2/2] fstests: btrfs: Introduce stress test for deadlock between snapshot delete and other read-write operations

2019-02-12 Thread Filipe Manana
On Tue, Feb 12, 2019 at 5:14 AM Qu Wenruo wrote: > > > > On 2019/1/11 下午1:01, Qu Wenruo wrote: > [snip] > > +# FS QA Test 179 > > +# > > +# Test if btrfs will lockup at subvolume deletion when qgroups are enabled. > > +# > > +# This bug is going to be fixed by a patch for the kernel titled > > +#

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-12 Thread Filipe Manana
On Tue, Feb 12, 2019 at 3:11 AM Zygo Blaxell wrote: > > Still reproducible on 4.20.7. I tried your reproducer when you first reported it, on different machines with different kernel versions. Never managed to reproduce it, nor see anything obviously wrong in relevant code paths. > > The behavior

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-12 Thread Filipe Manana
On Tue, Feb 12, 2019 at 5:01 PM Zygo Blaxell wrote: > > On Tue, Feb 12, 2019 at 03:35:37PM +, Filipe Manana wrote: > > On Tue, Feb 12, 2019 at 3:11 AM Zygo Blaxell > > wrote: > > > > > > Still reproducible on 4.20.7. > > > > I tried your rep

Re: [PATCH 4/4] Btrfs: remove no longer needed range length checks for deduplication

2019-02-12 Thread Filipe Manana
On Thu, Jan 31, 2019 at 4:31 PM Filipe Manana wrote: > > On Wed, Dec 12, 2018 at 6:07 PM wrote: > > > > From: Filipe Manana > > > > Comparing the content of the pages in the range to deduplicate is now done > > by the generic helper generic_remap_fi

Re: [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication

2019-02-12 Thread Filipe Manana
On Thu, Jan 31, 2019 at 4:39 PM Filipe Manana wrote: > > On Thu, Dec 13, 2018 at 4:08 PM David Sterba wrote: > > > > On Wed, Dec 12, 2018 at 06:05:58PM +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > Checking if the destinati

Re: [PATCH] fstests: fix fssum to actually ignore file holes when supposed to

2019-02-13 Thread Filipe Manana
On Wed, Feb 13, 2019 at 5:44 AM Qu Wenruo wrote: > > > > On 2018/10/29 下午5:43, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Unless the '-s' option is passed to fssum, it should not detect file holes > > and have their existence in

Re: [PATCH v2] btrfs: honor path->skip_locking in backref code

2019-02-13 Thread Filipe Manana
ck A to unlock. > While thread B holds write lock A, while needs lock D to unlock. > > This will cause a deadlock. > > This is not only limited to snapshot dropping case. > As the backref walk, even only happens on commit trees, is breaking the > normal top-down locking ord

Re: [PATCH 1/2] btrfs: reserve space for inheriting properties

2019-02-13 Thread Filipe Manana
time. NO_FLUSH can transiently fail, but we'll still complain. It's > just an extra credit, so simply add that to the places that call > btrfs_new_inode and call it good enough. > > Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana Loogs good, thanks. Just one minor comment

Re: [PATCH 2/2] btrfs: use the existing credit for our first prop

2019-02-13 Thread Filipe Manana
ra space reservation. If we do add more properties in the future > we should re-visit how we calculate the space reservation needs by the > callers. > > Signed-off-by: Josef Bacik Reviewed-by: Filipe Manana Looks good, thanks. > --- > fs/btrfs/props.c | 32

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-13 Thread Filipe Manana
On Tue, Feb 12, 2019 at 6:14 PM Zygo Blaxell wrote: > > On Tue, Feb 12, 2019 at 05:56:24PM +, Filipe Manana wrote: > > On Tue, Feb 12, 2019 at 5:01 PM Zygo Blaxell > > wrote: > > > > > > On Tue, Feb 12, 2019 at 03:35:37PM +, Filipe Manana wrote: >

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-13 Thread Filipe Manana
On Wed, Feb 13, 2019 at 5:36 PM Filipe Manana wrote: > > On Tue, Feb 12, 2019 at 6:14 PM Zygo Blaxell > wrote: > > > > On Tue, Feb 12, 2019 at 05:56:24PM +, Filipe Manana wrote: > > > On Tue, Feb 12, 2019 at 5:01 PM Zygo Blaxell > > > wrote: > &

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-13 Thread Filipe Manana
On Wed, Feb 13, 2019 at 6:14 PM Filipe Manana wrote: > > On Wed, Feb 13, 2019 at 5:36 PM Filipe Manana wrote: > > > > On Tue, Feb 12, 2019 at 6:14 PM Zygo Blaxell > > wrote: > > > > > > On Tue, Feb 12, 2019 at 05:56:24PM +, Filipe Manana wrote: &g

Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7

2019-02-15 Thread Filipe Manana
On Thu, Feb 14, 2019 at 11:10 PM Christoph Anton Mitterer wrote: > > On Thu, 2019-02-14 at 01:22 +, Filipe Manana wrote: > > The following one liner fixes it: > > https://friendpaste.com/22t4OdktHQTl0aMGxcWLj3 > > Great to see that fixed... is there any advise that

Re: [PATCH 2/2] generic/059: also test that the file's mtime and ctime are updated

2019-06-21 Thread Filipe Manana
On Fri, Jun 21, 2019 at 11:36 AM Eryu Guan wrote: > > On Wed, Jun 19, 2019 at 01:06:24PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Test as well that hole punch operations that affect a single file block > > also update the file's mtime a

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Filipe Manana
On Tue, Jun 25, 2019 at 8:58 PM Goldwyn Rodrigues wrote: > > On 9:05 24/06, Christoph Hellwig wrote: > > On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > > > From: Goldwyn Rodrigues > > > > > > btrfs uses page->private as well to store extent_buffer. Make > > > the check stri

Re: [PATCH] generic: test cloning large exents to a file with many small extents

2019-06-27 Thread Filipe Manana
On Thu, Jun 27, 2019 at 9:28 PM Darrick J. Wong wrote: > > On Thu, Jun 27, 2019 at 06:00:30PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Test that if we clone a file with some large extents into a file that has > > many small extents, when th

Re: [PATCH v2] generic: test cloning large exents to a file with many small extents

2019-07-05 Thread Filipe Manana
On Fri, Jul 5, 2019 at 8:43 AM Eryu Guan wrote: > > On Fri, Jun 28, 2019 at 11:08:36PM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Test that if we clone a file with some large extents into a file that has > > many small extents, when th

Re: [PATCH 2/5] Btrfs: fix inode cache block reserve leak on failure to allocate data space

2019-07-05 Thread Filipe Manana
On Fri, Jul 5, 2019 at 11:01 AM Nikolay Borisov wrote: > > > > On 4.07.19 г. 18:24 ч., fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If we failed to allocate the data extent(s) for the inode space cache, we > > were bailing out without releas

Re: [PATCH 1/5] Btrfs: fix hang when loading existing inode cache off disk

2019-07-05 Thread Filipe Manana
On Fri, Jul 5, 2019 at 10:09 AM Nikolay Borisov wrote: > > > > On 4.07.19 г. 18:24 ч., fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If we are able to load an existing inode cache off disk, we set the state > > of the cache to BTRFS_CACHE_FI

Re: [PATCH 2/5] Btrfs: fix inode cache block reserve leak on failure to allocate data space

2019-07-05 Thread Filipe Manana
On Fri, Jul 5, 2019 at 3:09 PM Nikolay Borisov wrote: > > > > On 5.07.19 г. 13:42 ч., Filipe Manana wrote: > > On Fri, Jul 5, 2019 at 11:01 AM Nikolay Borisov wrote: > >> > >> > >> > >> On 4.07.19 г. 18:24 ч., fdman...@kernel.org wrote:

  1   2   3   4   5   6   7   8   9   10   >