[PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread fdmanana
From: Filipe Manana When a transaction commit starts, it attempts to pause scrub and it blocks until the scrub is paused. So while the transaction is blocked waiting for scrub to pause, we can not do memory allocation with GFP_KERNEL from scrub, otherwise we risk getting into a deadlock with

Re: Btrfs and fanotify filesystem watches

2018-11-23 Thread Amir Goldstein
On Fri, Nov 23, 2018 at 3:34 PM Amir Goldstein wrote: > > On Fri, Nov 23, 2018 at 2:52 PM Jan Kara wrote: > > > > Changed subject to better match what we discuss and added btrfs list to CC. > > > > On Thu 22-11-18 17:18:25, Amir Goldstein wrote: > > > On Thu, Nov 22, 2018 at 3:26 PM Jan Kara

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-23 Thread Filipe Manana
On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > I noticed in a giant dbench run that we spent a lot of time on lock > contention while running transaction commit. This is because dbench > results in a lot of fsync()'s that do a btrfs_transaction_commit(), and > they all run the delayed

[PATCH v3] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread fdmanana
From: Filipe Manana When a transaction commit starts, it attempts to pause scrub and it blocks until the scrub is paused. So while the transaction is blocked waiting for scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub is running, we must use GFP_NOS to avoid deadlock

Re: [PATCH v2] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread Nikolay Borisov
On 23.11.18 г. 18:05 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > When a transaction commit starts, it attempts to pause scrub and it blocks > until the scrub is paused. So while the transaction is blocked waiting for > scrub to pause, we can not do memory allocation with

[PATCH v2] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread fdmanana
From: Filipe Manana When a transaction commit starts, it attempts to pause scrub and it blocks until the scrub is paused. So while the transaction is blocked waiting for scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub is running, we must use GFP_NOS to avoid deadlock

Re: [PATCH 0/6] Delayed refs rsv

2018-11-23 Thread David Sterba
On Wed, Nov 21, 2018 at 01:59:06PM -0500, Josef Bacik wrote: > This patchset changes how we do space reservations for delayed refs. We were > hitting probably 20-40 enospc abort's per day in production while running > delayed refs at transaction commit time. This means we ran out of space in >

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-23 Thread David Sterba
On Wed, Nov 21, 2018 at 01:59:09PM -0500, Josef Bacik wrote: > From: Josef Bacik > > The cleanup_extent_op function actually would run the extent_op if it > needed running, which made the name sort of a misnomer. Change it to > run_and_cleanup_extent_op, and move the actual cleanup work to >

Re: [PATCH 2/6] btrfs: add cleanup_ref_head_accounting helper

2018-11-23 Thread David Sterba
On Thu, Nov 22, 2018 at 09:06:30AM +0800, Qu Wenruo wrote: > > > On 2018/11/22 上午2:59, Josef Bacik wrote: > > From: Josef Bacik > > > > We were missing some quota cleanups in check_ref_cleanup, so break the > > ref head accounting cleanup into a helper and call that from both > >

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-23 Thread Nikolay Borisov
On 23.11.18 г. 15:45 ч., David Sterba wrote: > On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote: >> >> >> On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: >>> >>> >>> On 21.11.18 г. 20:59 ч., Josef Bacik wrote: From: Josef Bacik We do this dance in cleanup_ref_head

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-23 Thread David Sterba
On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote: > > > On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: > > > > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > >> From: Josef Bacik > >> > >> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > >> into a

[PATCH] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread fdmanana
From: Filipe Manana When a transaction commit starts, it attempts to pause scrub and it blocks until the scrub is paused. So while the transaction is blocked waiting for scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub is running, we must use GFP_NOS to avoid deadlock

Re: Btrfs and fanotify filesystem watches

2018-11-23 Thread Amir Goldstein
On Fri, Nov 23, 2018 at 2:52 PM Jan Kara wrote: > > Changed subject to better match what we discuss and added btrfs list to CC. > > On Thu 22-11-18 17:18:25, Amir Goldstein wrote: > > On Thu, Nov 22, 2018 at 3:26 PM Jan Kara wrote: > > > > > > On Thu 22-11-18 14:36:35, Amir Goldstein wrote: > >

Btrfs and fanotify filesystem watches

2018-11-23 Thread Jan Kara
Changed subject to better match what we discuss and added btrfs list to CC. On Thu 22-11-18 17:18:25, Amir Goldstein wrote: > On Thu, Nov 22, 2018 at 3:26 PM Jan Kara wrote: > > > > On Thu 22-11-18 14:36:35, Amir Goldstein wrote: > > > > > Regardless, IIUC, btrfs_statfs() returns an fsid which

Re: [PATCH] btrfs: remove btrfs_bio_end_io_t

2018-11-23 Thread David Sterba
On Fri, Nov 23, 2018 at 09:42:27AM +0100, Johannes Thumshirn wrote: > The btrfs_bio_end_io_t typedef was introduced with commit > a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio") > but never used anywhere. This commit also introduced a forward declaration > of 'struct btrfs_bio'

Re: [[Missing subject]]

2018-11-23 Thread Qu Wenruo
On 2018/11/23 下午2:41, Andy Leadbetter wrote: > I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I > have added a new 2TB disk to the computer, and started a BTRFS replace > for the old and new disk. The process starts correctly however some > hours into the job, there is an

Re:

2018-11-23 Thread Andy Leadbetter
Will capture all of that this evening, and try it with the latest kernel and tools. Thanks for the input on what info is relevant, with gather it asap. On Fri, 23 Nov 2018 at 07:53, Chris Murphy wrote: > > On Thu, Nov 22, 2018 at 11:41 PM Andy Leadbetter > wrote: > > > > I have a failing 2TB

Re: [PATCH] btrfs: remove btrfs_bio_end_io_t

2018-11-23 Thread Nikolay Borisov
On 23.11.18 г. 10:42 ч., Johannes Thumshirn wrote: > The btrfs_bio_end_io_t typedef was introduced with commit > a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio") > but never used anywhere. This commit also introduced a forward declaration > of 'struct btrfs_bio' which is only

[PATCH v2] btrfs: improve error handling of btrfs_add_link()

2018-11-23 Thread Johannes Thumshirn
err holds the return value of either btrfs_del_root_ref() or btrfs_del_inode_ref() but it hasn't been checked since it's introduction with commit fe66a05a0679 (Btrfs: improve error handling for btrfs_insert_dir_item callers) in 2012. Instead of silently ignoring the return values, print a message

[PATCH] btrfs: remove btrfs_bio_end_io_t

2018-11-23 Thread Johannes Thumshirn
The btrfs_bio_end_io_t typedef was introduced with commit a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio") but never used anywhere. This commit also introduced a forward declaration of 'struct btrfs_bio' which is only needed for btrfs_bio_end_io_t. Remove both as they're not needed

Re: [PATCH 1/4] btrfs: merge btrfs_submit_bio_done to its caller

2018-11-23 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH 2/4] btrfs: replace async_cow::root with fs_info

2018-11-23 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH 3/4] btrfs: remove redundant csum buffer in btrfs_io_bio

2018-11-23 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH 4/4] btrfs: replace btrfs_io_bio::end_io with a simple helper

2018-11-23 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG