[PATCH v2] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Qu Wenruo
[BUG] For certains crafted image, whose csum root leaf has missing backref, if we try to trigger write with data csum, it could cause deadlock with the following kernel WARN_ON(): -- WARNING: CPU: 1 PID: 41 at fs/btrfs/locking.c:230 btrfs_tree_lock+0x3e2/0x400 CPU: 1 PID: 41 Comm: kworker/u4:1

Re: [PATCH 1/1] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Qu Wenruo
On 2018年07月31日 14:44, Qu Wenruo wrote: > > > On 2018年07月31日 14:48, Su Yue wrote: >> >> >> On 07/30/2018 02:17 PM, Qu Wenruo wrote: >>> [BUG] >>> For certains crafted image, whose csum root leaf has missing backref, if >>> we try to trigger write with data csum, it could cause deadlock with the

Re: [PATCH 1/1] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Qu Wenruo
On 2018年07月31日 14:48, Su Yue wrote: > > > On 07/30/2018 02:17 PM, Qu Wenruo wrote: >> [BUG] >> For certains crafted image, whose csum root leaf has missing backref, if >> we try to trigger write with data csum, it could cause deadlock with the >> following kernel WARN_ON(): >> -- >> WARNIN

Re: [PATCH 1/1] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Su Yue
On 07/30/2018 02:17 PM, Qu Wenruo wrote: [BUG] For certains crafted image, whose csum root leaf has missing backref, if we try to trigger write with data csum, it could cause deadlock with the following kernel WARN_ON(): -- WARNING: CPU: 1 PID: 41 at fs/btrfs/locking.c:230 btrfs_tree_lock+

Re: [PATCH 0/2] btrfs-progs: completion: Small fixes to make debug simpler

2018-07-30 Thread Misono Tomohiro
This is useful. Reviewed-by: Misono Tomohiro On 2018/07/17 15:15, Qu Wenruo wrote: > For developer, it's pretty common to use "btrfs check" or "btrfs ins > dump-tree" on raw dumps. > > However "btrfs check" can only complete real block devices, and > "btrfs inspect dump-tree" can only complete

Re: csum failed on raid1 even after clean scrub?

2018-07-30 Thread Sterling Windmill
Thanks for the speedy reply! Here's my kernel version: 4.17.9-200.fc28.x86_64 dmesg doesn't show any USB related info at all, no signs of errors / warnings. Both drives are identical, Seagate 8TB external drives connected to the following PCIe controller: 03:00.0 USB controller: NEC Corporati

Re: csum failed on raid1 even after clean scrub?

2018-07-30 Thread Qu Wenruo
On 2018年07月31日 08:43, Sterling Windmill wrote: > I am using a two disk raid1 btrfs filesystem spanning two external hard > drives connected via USB 3.0. Is there any speed difference between the two device? And are these 2 devices under the same USB3.0 root hub or different root hubs? lsusb out

csum failed on raid1 even after clean scrub?

2018-07-30 Thread Sterling Windmill
I am using a two disk raid1 btrfs filesystem spanning two external hard drives connected via USB 3.0. While copying ~6TB of data from this filesystem to local disk via rsync I am seeing messages like the following in dmesg output: [ 2213.406267] BTRFS warning (device sdj1): csum failed root 5 ino

Re: [PATCH] btrfs: btrfs_shrink_device should call commit transaction

2018-07-30 Thread Nikolay Borisov
On 25.07.2018 16:27, Anand Jain wrote: > test case btrfs/164 reported UAF.. > > [ 6712.084324] general protection fault: [#1] PREEMPT SMP > :: > [ 6712.195423] btrfs_update_commit_device_size+0x75/0xf0 [btrfs] > [ 6712.201424] btrfs_commit_transaction+0x57d/0xa90 [btrfs] > [ 6712.206999]

Re: [PATCH] btrfs: backref.c: Use ERR_CAST() to return error code

2018-07-30 Thread Nikolay Borisov
On 26.07.2018 04:22, Misono Tomohiro wrote: > Use ERR_CAST() instead of void * to make meaning clear. > > Signed-off-by: Misono Tomohiro Reviewed-by: Nikolay Borisov > --- > fs/btrfs/backref.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/btrfs/backref.c b/fs

Re: [PATCH] btrfs: extent-tree.c: Remove unused __btrfs_free_block_rsv()

2018-07-30 Thread Nikolay Borisov
On 26.07.2018 05:40, Misono Tomohiro wrote: > There is no user of this function. > > This is forgotten to get removed in commit a575ceeb1338 > ("Btrfs: get rid of unused orphan infrastructure"). > > Signed-off-by: Misono Tomohiro Reviewed-by: Nikolay Borisov > --- > fs/btrfs/ctree.h

Re: [PATCH] btrfs: revert fs_devices state on error of btrfs_init_new_device()

2018-07-30 Thread Filipe Manana
On Fri, Jul 27, 2018 at 1:04 AM, Naohiro Aota wrote: > When btrfs hits error after modifying fs_devices in > btrfs_init_new_device() (such as btrfs_add_dev_item() returns error), it > leaves everything as is, but frees allocated btrfs_device. As a result, > fs_devices->devices and fs_devices->allo

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-30 Thread Filipe Manana
On Mon, Jul 30, 2018 at 12:28 PM, Filipe Manana wrote: > On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana wrote: >> On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: >>> From: Robbie Ko >>> >>> Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") >>> modified the nocow writeback mechani

[PATCH] btrfs: test send with prealloc extent beyond EOF and hole punching

2018-07-30 Thread fdmanana
From: Filipe Manana Test that an incremental send operation produces correct results if a file that has a prealloc (unwritten) extent beyond its EOF gets a hole punched in a section of that prealloc extent. This test is motivated by a bug found in btrfs which is fixed by a patch for the linux ke

[PATCH] Btrfs: send, fix incorrect file layout after hole punching beyond eof

2018-07-30 Thread fdmanana
From: Filipe Manana When doing an incremental send, if we have a file in the parent snapshot that has prealloc extents beyond EOF and in the send snapshot it got a hole punch that partially covers the prealloc extents, the send stream, when replayed by a receiver, can result in a file that has a

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-30 Thread Filipe Manana
On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana wrote: > On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: >> From: Robbie Ko >> >> Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") >> modified the nocow writeback mechanism, if you create a snapshot, >> it will always switch to cow w

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-30 Thread Filipe Manana
On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: > From: Robbie Ko > > Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") > modified the nocow writeback mechanism, if you create a snapshot, > it will always switch to cow writeback. > > This will cause data loss when there is no space,

[PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-30 Thread robbieko
From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") modified the nocow writeback mechanism, if you create a snapshot, it will always switch to cow writeback. This will cause data loss when there is no space, because when the space is full, the write will not reserve any

Re: [PATCH 14/15] btrfs-progs: Wire up delayed refs

2018-07-30 Thread Nikolay Borisov
On 30.07.2018 11:33, Misono Tomohiro wrote: > On 2018/06/08 21:47, Nikolay Borisov wrote: >> This commit enables the delayed refs infrastructures. This entails doing >> the following: >> >> 1. Replacing existing calls of btrfs_extent_post_op (which is the >> equivalent of delayed refs) with the

Re: [PATCH 11/15] btrfs-progs: Add delayed refs infrastructure

2018-07-30 Thread Nikolay Borisov
On 30.07.2018 11:34, Misono Tomohiro wrote: > On 2018/06/08 21:47, Nikolay Borisov wrote: >> This commit pulls those portions of the kernel implementation of >> delayed refs which are necessary to have them working in user-space. >> I've done the following modifications: >> >> 1. Replaced all km

Re: [PATCH 11/15] btrfs-progs: Add delayed refs infrastructure

2018-07-30 Thread Misono Tomohiro
On 2018/06/08 21:47, Nikolay Borisov wrote: > This commit pulls those portions of the kernel implementation of > delayed refs which are necessary to have them working in user-space. > I've done the following modifications: > > 1. Replaced all kmem_cache_alloc calls to kmalloc. > > 2. Removed all

Re: [PATCH 14/15] btrfs-progs: Wire up delayed refs

2018-07-30 Thread Misono Tomohiro
On 2018/06/08 21:47, Nikolay Borisov wrote: > This commit enables the delayed refs infrastructures. This entails doing > the following: > > 1. Replacing existing calls of btrfs_extent_post_op (which is the > equivalent of delayed refs) with the proper btrfs_run_delayed_refs. > As well as eliminati

Re: [PATCH 1/1] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Qu Wenruo
On 2018年07月30日 15:18, Nikolay Borisov wrote: > > > On 30.07.2018 09:17, Qu Wenruo wrote: >> -void btrfs_tree_lock(struct extent_buffer *eb) >> +int btrfs_tree_lock(struct extent_buffer *eb) >> { >> -WARN_ON(eb->lock_owner == current->pid); >> +if (unlikely(eb->lock_owner == current->p

Re: [PATCH 1/1] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-30 Thread Nikolay Borisov
On 30.07.2018 09:17, Qu Wenruo wrote: > -void btrfs_tree_lock(struct extent_buffer *eb) > +int btrfs_tree_lock(struct extent_buffer *eb) > { > - WARN_ON(eb->lock_owner == current->pid); > + if (unlikely(eb->lock_owner == current->pid)) { > + WARN(1, > +"tree block %llu alrea