Re: [f2fs-dev] [PATCH] f2fs: avoid deadlock in gc thread under low memory

2022-04-13 Thread Michal Hocko via Linux-f2fs-devel
On Wed 13-04-22 19:20:06, Wu Yan wrote: > On 4/13/22 17:46, Michal Hocko wrote: > > On Wed 13-04-22 16:44:32, Rokudo Yan wrote: > > > There is a potential deadlock in gc thread may happen > > > under low memory as below: > > > > > > gc_thread_func

Re: [f2fs-dev] [PATCH] f2fs: avoid deadlock in gc thread under low memory

2022-04-13 Thread Michal Hocko via Linux-f2fs-devel
writeback for cgroup v1. Also are you sure that the mapping's gfp mask has __GFP_FS set for this allocation? f2fs_iget uses GFP_NOFS like mask for some inode types. All that being said, you will need to change the above call chain but it would be worth double checking the dead lock is real.

Re: [f2fs-dev] [PATCH] f2fs: initialize page->private when using for our internal use

2021-07-11 Thread Michal Hocko via Linux-f2fs-devel
>private)) > + pr_info("2order:%x, origpage.private:%x", order, > orig_page->private); > + } Why is this expected? Buddy allocator uses page private to store order. Whether we are merging to the freed page or coalesce it to a different page is not

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-18 Thread Michal Hocko
> > should not be exposed to other MM parts. > > I agree that the current semantics are mentally taxing and we should > generally avoid exposing the implementation details outside of memcg where > possible. Do you have a suggested rework? :-) I would really prefer to do that wo

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-17 Thread Michal Hocko
On Wed 17-06-20 21:23:05, Naresh Kamboju wrote: > On Wed, 17 Jun 2020 at 19:41, Michal Hocko wrote: > > > > [Our emails have crossed] > > > > On Wed 17-06-20 14:57:58, Chris Down wrote: > > > Naresh Kamboju writes: > > > > mkfs -t ext4 /dev

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-17 Thread Michal Hocko
in > and mem_cgroup_below_low changed to `>`? If that fixes it, then that gives a > strong hint about what's going on here. This would work but I believe an explicit check for the root memcg would be easier to spot the reasoning. -- Michal Hocko SUSE Labs ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-17 Thread Michal Hocko
On Wed 17-06-20 19:07:20, Naresh Kamboju wrote: > On Thu, 21 May 2020 at 22:04, Michal Hocko wrote: > > > > On Thu 21-05-20 11:55:16, Michal Hocko wrote: > > > On Wed 20-05-20 20:09:06, Chris Down wrote: > > > > Hi Naresh, > > > > >

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-12 Thread Michal Hocko
On Fri 12-06-20 15:13:22, Naresh Kamboju wrote: > On Thu, 11 Jun 2020 at 15:25, Michal Hocko wrote: > > > > On Fri 29-05-20 11:49:20, Michal Hocko wrote: > > > On Fri 29-05-20 02:56:44, Chris Down wrote: > > > > Yafang Shao writes: > > > Agreed. E

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-06-11 Thread Michal Hocko
On Fri 29-05-20 11:49:20, Michal Hocko wrote: > On Fri 29-05-20 02:56:44, Chris Down wrote: > > Yafang Shao writes: > > > Look at this patch[1] carefully you will find that it introduces the > > > same issue that I tried to fix in another patch [2]. Even more sad is >

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-29 Thread Michal Hocko
ld confirm. Maybe my debugging patch is incomplete or used incorrectly (maybe it would be esier to use printk rather than trace_printk?). -- Michal Hocko SUSE Labs ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-28 Thread Michal Hocko
s this still hold? Because I still have a hard time to understand how those three patches could have the observed effects. -- Michal Hocko SUSE Labs ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-28 Thread Michal Hocko
0 Not tainted > > > > > 5.7.0-rc6-next-20200519+ #1 > > > > > [ 35.532121] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS > > > > > 2.2 05/23/2018 > > > > > [ 35.539507] EIP: mem_cgroup_get_nr_swap_pages+0x28/0x60 > > Swap

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-21 Thread Michal Hocko
On Thu 21-05-20 11:55:16, Michal Hocko wrote: > On Wed 20-05-20 20:09:06, Chris Down wrote: > > Hi Naresh, > > > > Naresh Kamboju writes: > > > As a part of investigation on this issue LKFT teammate Anders Roxell > > > git bisected the problem an

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-21 Thread Michal Hocko
On Thu 21-05-20 05:24:27, Hugh Dickins wrote: > On Thu, 21 May 2020, Michal Hocko wrote: > > On Thu 21-05-20 16:11:11, Naresh Kamboju wrote: > > > On Thu, 21 May 2020 at 15:25, Michal Hocko wrote: > > > > > > > > On Wed 20-05-20 20:09

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-21 Thread Michal Hocko
On Thu 21-05-20 16:11:11, Naresh Kamboju wrote: > On Thu, 21 May 2020 at 15:25, Michal Hocko wrote: > > > > On Wed 20-05-20 20:09:06, Chris Down wrote: > > > Hi Naresh, > > > > > > Naresh Kamboju writes: > > > > As a part of investigation

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-21 Thread Michal Hocko
ed. If memory.{low,min} is not used then the patch should be effectively a nop. Btw. do you see the problem when booting with cgroup_disable=memory kernel command line parameter? I suspect that something might be initialized for memcg incorrectly and the patch just makes it more visible for some reason

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-19 Thread Michal Hocko
On Tue 19-05-20 10:11:25, Arnd Bergmann wrote: > On Tue, May 19, 2020 at 9:52 AM Michal Hocko wrote: > > > > On Mon 18-05-20 19:40:55, Naresh Kamboju wrote: > > > Thanks for looking into this problem. > > > > > > On Sat, 2 May 2020 at 02:28, Andrew Morto

Re: [f2fs-dev] mm: mkfs.ext4 invoked oom-killer on i386 - pagecache_get_page

2020-05-19 Thread Michal Hocko
of the low mem is in the Normal zone which is completely missing here. How have you got to that configuration? I have to say I haven't seen anything like that on i386. The failing request is GFP_USER so highmem is not really allowed but free pages are way above watermarks so the allocation should hav

Re: [f2fs-dev] deadlock during writeback when using f2fs filesystem

2018-06-01 Thread Michal Hocko
On Fri 01-06-18 16:50:50, Sahitya Tummala wrote: > On Fri, Jun 01, 2018 at 12:26:09PM +0200, Michal Hocko wrote: > > On Fri 01-06-18 15:02:35, Sahitya Tummala wrote: > > > Hi, > > > > > > We are observing a deadlock scenario during FS writeback under

Re: [f2fs-dev] deadlock during writeback when using f2fs filesystem

2018-06-01 Thread Michal Hocko
(GFP_NOFS | __GFP_ZERO) > #define GFP_F2FS_HIGH_ZERO (GFP_NOFS | __GFP_ZERO | __GFP_HIGHMEM) > +#define GFP_F2FS_NODE_MAPPING (GFP_NOWAIT | __GFP_IO | __GFP_ZERO) > > Thanks, > Sahitya. > -- > -- > Sent by a consultant of the Qualcomm Innovation Center, Inc. &g

Re: [f2fs-dev] [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-10 Thread Michal Hocko
CLAIM_MASK filtering and a warning in slab for __GFP_ZERO looks like a reasonable step forward. -- Michal Hocko SUSE Labs -- Check out the vibrant tech community on one of the world's most engaging tech sites,

Re: [f2fs-dev] [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-10 Thread Michal Hocko
On Tue 10-04-18 05:05:28, Matthew Wilcox wrote: > On Tue, Apr 10, 2018 at 10:26:43AM +0200, Michal Hocko wrote: > > On Mon 09-04-18 12:40:44, Matthew Wilcox wrote: > > > The problem is that the mapping gfp flags are used not only for allocating > > > pages, but also fo

Re: [f2fs-dev] [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-10 Thread Michal Hocko
644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -785,7 +785,7 @@ int replace_page_cache_page(struct page *old, struct page > *new, gfp_t gfp_mask) > VM_BUG_ON_PAGE(!PageLocked(new), new); > VM_BUG_ON_PAGE(new->mapping, new); > >

Re: [f2fs-dev] [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-09 Thread Michal Hocko
On Mon 09-04-18 06:41:14, Matthew Wilcox wrote: > On Mon, Apr 09, 2018 at 02:48:52PM +0200, Michal Hocko wrote: > > On Mon 09-04-18 20:25:06, Chao Yu wrote: > > [...] > > > diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c > > > index c852e800..cc63f8c448f0 1

Re: [f2fs-dev] [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-09 Thread Michal Hocko
question. Why do you make all allocations for the mapping NOFS automatically? What kind of reclaim recursion problems are you trying to prevent? -- Michal Hocko SUSE Labs -- Check out the vibrant tech community on one of t

Re: [f2fs-dev] [PATCH 1/3] f2fs: avoid using __GFP_NOFAIL

2017-11-06 Thread Michal Hocko
onditions. -- Michal Hocko SUSE Labs -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Linux-f2fs

Re: [f2fs-dev] [PATCH 1/3] f2fs: avoid using __GFP_NOFAIL

2017-11-06 Thread Michal Hocko
ck(&im->ino_lock); > - radix_tree_preload_end(); > + > + if (preloaded) > + radix_tree_preload_end(); > > if (e != tmp) > kmem_cache_free(ino_entry_slab, tmp); > -- > 2.14.1.145.gb3622a4ee -- Michal Hocko

Re: [f2fs-dev] [PATCH 4/7] mm: introduce memalloc_nofs_{save, restore} API

2017-03-07 Thread Michal Hocko
On Mon 06-03-17 13:22:14, Andrew Morton wrote: > On Mon, 6 Mar 2017 14:14:05 +0100 Michal Hocko wrote: [...] > > --- a/include/linux/gfp.h > > +++ b/include/linux/gfp.h > > @@ -210,8 +210,16 @@ struct vm_area_struct; > > * > > * GFP_NOIO will use direct recla

[f2fs-dev] [PATCH 1/7] lockdep: teach lockdep about memalloc_noio_save

2017-03-06 Thread Michal Hocko
path_setxattr+0x8f/0xc0 [ 644.174012] SyS_lsetxattr+0x11/0x20 [ 644.174012] entry_SYSCALL_64_fastpath+0x23/0xc6 Let's fix this by making lockdep explicitly do the shaving of respective GFP flags. Fixes: 934f3072c17c ("mm: clear __GFP_FS when PF_MEMALLOC_NOIO is set") Acked-by: M

[f2fs-dev] [PATCH 6/7] jbd2: mark the transaction context with the scope GFP_NOFS context

2017-03-06 Thread Michal Hocko
From: Michal Hocko now that we have memalloc_nofs_{save,restore} api we can mark the whole transaction context as implicitly GFP_NOFS. All allocations will automatically inherit GFP_NOFS this way. This means that we do not have to mark any of those requests with GFP_NOFS and moreover all the

[f2fs-dev] [PATCH 3/7] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS

2017-03-06 Thread Michal Hocko
From: Michal Hocko xfs has defined PF_FSTRANS to declare a scope GFP_NOFS semantic quite some time ago. We would like to make this concept more generic and use it for other filesystems as well. Let's start by giving the flag a more generic name PF_MEMALLOC_NOFS which is in line with an ex

[f2fs-dev] [PATCH 0/7 v5] scope GFP_NOFS api

2017-03-06 Thread Michal Hocko
| 6 +++--- 16 files changed, 106 insertions(+), 37 deletions(-) Shortlog: Michal Hocko (6): lockdep: allow to disable reclaim lockup detection xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS mm: introduce memalloc_nofs_{save,restore} API xfs: use memalloc_nofs_{save,restore} in

[f2fs-dev] [PATCH 7/7] jbd2: make the whole kjournald2 kthread NOFS safe

2017-03-06 Thread Michal Hocko
From: Michal Hocko kjournald2 is central to the transaction commit processing. As such any potential allocation from this kernel thread has to be GFP_NOFS. Make sure to mark the whole kernel thread GFP_NOFS by the memalloc_nofs_save. Suggested-by: Jan Kara Reviewed-by: Jan Kara Signed-off-by

[f2fs-dev] [PATCH 5/7] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-03-06 Thread Michal Hocko
From: Michal Hocko kmem_zalloc_large and _xfs_buf_map_pages use memalloc_noio_{save,restore} API to prevent from reclaim recursion into the fs because vmalloc can invoke unconditional GFP_KERNEL allocations and these functions might be called from the NOFS contexts. The memalloc_noio_save will

[f2fs-dev] [PATCH 4/7] mm: introduce memalloc_nofs_{save, restore} API

2017-03-06 Thread Michal Hocko
From: Michal Hocko GFP_NOFS context is used for the following 5 reasons currently - to prevent from deadlocks when the lock held by the allocation context would be needed during the memory reclaim - to prevent from stack overflows during the reclaim because

[f2fs-dev] [PATCH 2/7] lockdep: allow to disable reclaim lockup detection

2017-03-06 Thread Michal Hocko
From: Michal Hocko The current implementation of the reclaim lockup detection can lead to false positives and those even happen and usually lead to tweak the code to silence the lockdep by using GFP_NOFS even though the context can use __GFP_FS just fine. See http://lkml.kernel.org/r

Re: [f2fs-dev] [PATCH 7/8] Revert "ext4: avoid deadlocks in the writeback path by using sb_getblk_gfp"

2017-03-06 Thread Michal Hocko
On Tue 17-01-17 08:54:50, Michal Hocko wrote: > On Mon 16-01-17 22:01:18, Theodore Ts'o wrote: > > On Fri, Jan 06, 2017 at 03:11:06PM +0100, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > This reverts commit c45653c341f5c8a0ce19c8f0ad46786408

Re: [f2fs-dev] [PATCH 4/6] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-02-07 Thread Michal Hocko
On Tue 07-02-17 09:51:50, Dave Chinner wrote: > On Mon, Feb 06, 2017 at 07:47:43PM +0100, Michal Hocko wrote: > > On Mon 06-02-17 10:32:37, Darrick J. Wong wrote: [...] > > > I prefer to keep the "...yet we are likely to be under GFP_NOFS..." > > > wording o

Re: [f2fs-dev] [PATCH 4/6] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-02-07 Thread Michal Hocko
On Mon 06-02-17 11:51:11, Darrick J. Wong wrote: > On Mon, Feb 06, 2017 at 07:47:43PM +0100, Michal Hocko wrote: > > On Mon 06-02-17 10:32:37, Darrick J. Wong wrote: > > > On Mon, Feb 06, 2017 at 06:44:15PM +0100, Michal Hocko wrote: > > > > On Mon 06-02-17

Re: [f2fs-dev] [PATCH 4/6] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-02-07 Thread Michal Hocko
On Mon 06-02-17 10:32:37, Darrick J. Wong wrote: > On Mon, Feb 06, 2017 at 06:44:15PM +0100, Michal Hocko wrote: > > On Mon 06-02-17 07:39:23, Matthew Wilcox wrote: > > > On Mon, Feb 06, 2017 at 03:07:16PM +0100, Michal Hocko wrote: > > > > +++ b/fs/xfs/xfs_bu

Re: [f2fs-dev] [PATCH 4/6] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-02-07 Thread Michal Hocko
On Mon 06-02-17 07:39:23, Matthew Wilcox wrote: > On Mon, Feb 06, 2017 at 03:07:16PM +0100, Michal Hocko wrote: > > +++ b/fs/xfs/xfs_buf.c > > @@ -442,17 +442,17 @@ _xfs_buf_map_pages( > > bp->b_addr = NULL; > > } else { > > int r

[f2fs-dev] [PATCH 5/6] jbd2: mark the transaction context with the scope GFP_NOFS context

2017-02-06 Thread Michal Hocko
From: Michal Hocko now that we have memalloc_nofs_{save,restore} api we can mark the whole transaction context as implicitly GFP_NOFS. All allocations will automatically inherit GFP_NOFS this way. This means that we do not have to mark any of those requests with GFP_NOFS and moreover all the

[f2fs-dev] [PATCH 4/6] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-02-06 Thread Michal Hocko
From: Michal Hocko kmem_zalloc_large and _xfs_buf_map_pages use memalloc_noio_{save,restore} API to prevent from reclaim recursion into the fs because vmalloc can invoke unconditional GFP_KERNEL allocations and these functions might be called from the NOFS contexts. The memalloc_noio_save will

Re: [f2fs-dev] [PATCH 1/6] lockdep: allow to disable reclaim lockup detection

2017-02-06 Thread Michal Hocko
On Mon 06-02-17 06:26:41, Matthew Wilcox wrote: > On Mon, Feb 06, 2017 at 03:07:13PM +0100, Michal Hocko wrote: > > While we are at it also make sure that the radix tree doesn't > > accidentaly override tags stored in the upper part of the gfp_mask. > > > diff --git a

Re: [f2fs-dev] [PATCH 1/6] lockdep: allow to disable reclaim lockup detection

2017-02-06 Thread Michal Hocko
On Mon 06-02-17 07:24:00, Matthew Wilcox wrote: > On Mon, Feb 06, 2017 at 03:34:50PM +0100, Michal Hocko wrote: > > This part is not needed for the patch, strictly speaking but I wanted to > > make the code more future proof. > > Understood. I took an extra bit myself for m

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-02-06 Thread Michal Hocko
On Mon 30-01-17 09:12:10, Michal Hocko wrote: > On Fri 27-01-17 11:40:42, Theodore Ts'o wrote: > > On Fri, Jan 27, 2017 at 10:37:35AM +0100, Michal Hocko wrote: > > > If this ever turn out to be a problem and with the vmapped stacks we > > > have good chances t

[f2fs-dev] [PATCH 0/6 v4] scope GFP_NOFS api

2017-02-06 Thread Michal Hocko
ge_alloc.c | 10 ++ mm/vmscan.c | 6 +++--- 15 files changed, 100 insertions(+), 36 deletions(-) Shortlog: Michal Hocko (6): lockdep: allow to disable reclaim lockup detection xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS mm: introduce memalloc_nofs_{sa

[f2fs-dev] [PATCH 3/6] mm: introduce memalloc_nofs_{save, restore} API

2017-02-06 Thread Michal Hocko
From: Michal Hocko GFP_NOFS context is used for the following 5 reasons currently - to prevent from deadlocks when the lock held by the allocation context would be needed during the memory reclaim - to prevent from stack overflows during the reclaim because

[f2fs-dev] [PATCH 6/6] jbd2: make the whole kjournald2 kthread NOFS safe

2017-02-06 Thread Michal Hocko
From: Michal Hocko kjournald2 is central to the transaction commit processing. As such any potential allocation from this kernel thread has to be GFP_NOFS. Make sure to mark the whole kernel thread GFP_NOFS by the memalloc_nofs_save. Suggested-by: Jan Kara Reviewed-by: Jan Kara Signed-off-by

[f2fs-dev] [PATCH 2/6] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS

2017-02-06 Thread Michal Hocko
From: Michal Hocko xfs has defined PF_FSTRANS to declare a scope GFP_NOFS semantic quite some time ago. We would like to make this concept more generic and use it for other filesystems as well. Let's start by giving the flag a more generic name PF_MEMALLOC_NOFS which is in line with an ex

[f2fs-dev] [PATCH 1/6] lockdep: allow to disable reclaim lockup detection

2017-02-06 Thread Michal Hocko
From: Michal Hocko The current implementation of the reclaim lockup detection can lead to false positives and those even happen and usually lead to tweak the code to silence the lockdep by using GFP_NOFS even though the context can use __GFP_FS just fine. See http://lkml.kernel.org/r

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-30 Thread Michal Hocko
On Fri 27-01-17 11:40:42, Theodore Ts'o wrote: > On Fri, Jan 27, 2017 at 10:37:35AM +0100, Michal Hocko wrote: > > If this ever turn out to be a problem and with the vmapped stacks we > > have good chances to get a proper stack traces on a potential overflow > > we can ad

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-27 Thread Michal Hocko
On Thu 19-01-17 10:44:05, Michal Hocko wrote: > On Thu 19-01-17 10:22:36, Jan Kara wrote: > > On Thu 19-01-17 09:39:56, Michal Hocko wrote: > > > On Tue 17-01-17 18:29:25, Jan Kara wrote: > > > > On Tue 17-01-17 17:16:19, Michal Hocko wrote: > > > > >

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-27 Thread Michal Hocko
On Fri 27-01-17 01:13:18, Theodore Ts'o wrote: > On Thu, Jan 26, 2017 at 08:44:55AM +0100, Michal Hocko wrote: > > > > I'm convinced the current series is OK, only real life will tell us > > > > whether > > > > we missed something or not ;) >

Re: [f2fs-dev] [PATCH 7/8] Revert "ext4: avoid deadlocks in the writeback path by using sb_getblk_gfp"

2017-01-19 Thread Michal Hocko
On Mon 16-01-17 22:01:18, Theodore Ts'o wrote: > On Fri, Jan 06, 2017 at 03:11:06PM +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > This reverts commit c45653c341f5c8a0ce19c8f0ad4678640849cb86 because > > sb_getblk_gfp is not really needed as

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Mon 16-01-17 21:56:07, Theodore Ts'o wrote: > On Fri, Jan 06, 2017 at 03:11:07PM +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > This reverts commit 216553c4b7f3e3e2beb4981cddca9b2027523928. Now that > > the transaction context uses memalloc_nofs_sav

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Tue 17-01-17 09:24:25, Michal Hocko wrote: > On Mon 16-01-17 21:56:07, Theodore Ts'o wrote: > > On Fri, Jan 06, 2017 at 03:11:07PM +0100, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > This reverts commit 216553c4b7f3e3e2beb4981cddca9b2

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Tue 17-01-17 18:29:25, Jan Kara wrote: > On Tue 17-01-17 17:16:19, Michal Hocko wrote: > > > > But before going to play with that I am really wondering whether we need > > > > all this with no journal at all. AFAIU what Jack told me it is the > > > > journ

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Thu 19-01-17 10:22:36, Jan Kara wrote: > On Thu 19-01-17 09:39:56, Michal Hocko wrote: > > On Tue 17-01-17 18:29:25, Jan Kara wrote: > > > On Tue 17-01-17 17:16:19, Michal Hocko wrote: > > > > > > But before going to play with that I am really wo

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Tue 17-01-17 14:04:03, Andreas Dilger wrote: > On Jan 17, 2017, at 8:59 AM, Theodore Ts'o wrote: > > > > On Tue, Jan 17, 2017 at 04:18:17PM +0100, Michal Hocko wrote: > >> > >> OK, so I've been staring into the code and AFAIU current->journal_

Re: [f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-19 Thread Michal Hocko
On Tue 17-01-17 10:59:16, Theodore Ts'o wrote: > On Tue, Jan 17, 2017 at 04:18:17PM +0100, Michal Hocko wrote: > > > > OK, so I've been staring into the code and AFAIU current->journal_info > > can contain my stored information. I could either hijack part of t

Re: [f2fs-dev] [PATCH] mm: support __GFP_REPEAT in kvmalloc_node

2017-01-09 Thread Michal Hocko
On Fri 06-01-17 13:09:36, Vlastimil Babka wrote: > On 01/04/2017 07:12 PM, Michal Hocko wrote: > > While checking opencoded users I've encountered that vhost code would > > really like to use kvmalloc with __GFP_REPEAT [1] so the following patch > > adds support for __GF

Re: [f2fs-dev] [PATCH 4/8] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-01-09 Thread Michal Hocko
On Mon 09-01-17 15:08:27, Vlastimil Babka wrote: > On 01/06/2017 03:11 PM, Michal Hocko wrote: > > From: Michal Hocko > > > > kmem_zalloc_large and _xfs_buf_map_pages use memalloc_noio_{save,restore} > > API to prevent from reclaim recursion into the fs bec

Re: [f2fs-dev] [PATCH 3/8] mm: introduce memalloc_nofs_{save, restore} API

2017-01-09 Thread Michal Hocko
On Mon 09-01-17 14:42:10, Michal Hocko wrote: > On Mon 09-01-17 14:04:21, Vlastimil Babka wrote: [...] > Now that you have opened this I have noticed that the code is wrong > here because GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK would overwrite > the removed GFP_FS. Blee, it

Re: [f2fs-dev] [PATCH 2/8] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS

2017-01-09 Thread Michal Hocko
On Mon 09-01-17 13:59:05, Vlastimil Babka wrote: > On 01/06/2017 03:11 PM, Michal Hocko wrote: > > From: Michal Hocko > > > > xfs has defined PF_FSTRANS to declare a scope GFP_NOFS semantic quite > > some time ago. We would like to make this concept more gener

Re: [f2fs-dev] [PATCH 3/8] mm: introduce memalloc_nofs_{save, restore} API

2017-01-09 Thread Michal Hocko
p_zone(gfp_mask); struct scan_control sc = { .nr_to_reclaim = max(nr_pages, SWAP_CLUSTER_MAX), - .gfp_mask = (gfp_mask = current_gfp_context(gfp_mask)), + .gfp_mask = gfp_mask, .order =

Re: [f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-06 Thread Michal Hocko
On Fri 06-01-17 15:36:04, Vlastimil Babka wrote: > On 01/04/2017 03:20 PM, Michal Hocko wrote: > > diff --git a/net/netfilter/x_tables.c b/net/netfilter/x_tables.c > > index 2ff499680cc6..0a5cc1237afe 100644 > > --- a/net/netfilter/x_tables.c > > +++ b/net/netfilte

[f2fs-dev] [DEBUG PATCH 0/2] debug explicit GFP_NO{FS, IO} usage from the scope context

2017-01-06 Thread Michal Hocko
These two patches should help to identify explicit GFP_NO{FS,IO} usage from withing a scope context and reduce such a usage as a result. Such a usage can be changed to the full GFP_KERNEL because all the calls from within the NO{FS,IO} scope will drop the __GFP_FS resp. __GFP_IO automatically and i

[f2fs-dev] [PATCH 1/8] lockdep: allow to disable reclaim lockup detection

2017-01-06 Thread Michal Hocko
From: Michal Hocko The current implementation of the reclaim lockup detection can lead to false positives and those even happen and usually lead to tweak the code to silence the lockdep by using GFP_NOFS even though the context can use __GFP_FS just fine. See http://lkml.kernel.org/r

[f2fs-dev] [PATCH 8/8] Revert "ext4: fix wrong gfp type under transaction"

2017-01-06 Thread Michal Hocko
From: Michal Hocko This reverts commit 216553c4b7f3e3e2beb4981cddca9b2027523928. Now that the transaction context uses memalloc_nofs_save and all allocations within the this context inherit GFP_NOFS automatically, there is no reason to mark specific allocations explicitly. This patch should not

[f2fs-dev] [PATCH 5/8] jbd2: mark the transaction context with the scope GFP_NOFS context

2017-01-06 Thread Michal Hocko
From: Michal Hocko now that we have memalloc_nofs_{save,restore} api we can mark the whole transaction context as implicitly GFP_NOFS. All allocations will automatically inherit GFP_NOFS this way. This means that we do not have to mark any of those requests with GFP_NOFS and moreover all the

[f2fs-dev] [PATCH 2/8] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS

2017-01-06 Thread Michal Hocko
From: Michal Hocko xfs has defined PF_FSTRANS to declare a scope GFP_NOFS semantic quite some time ago. We would like to make this concept more generic and use it for other filesystems as well. Let's start by giving the flag a more generic name PF_MEMALLOC_NOFS which is in line with an ex

[f2fs-dev] [PATCH 0/8 v3] scope GFP_NOFS api

2017-01-06 Thread Michal Hocko
de/linux/sched.h | 32 ++-- kernel/locking/lockdep.c | 6 +- lib/radix-tree.c | 2 ++ mm/page_alloc.c | 8 +--- mm/vmscan.c | 6 +++--- 19 files changed, 109 insertions(+), 45 deletions(-) Shortlog: Michal Hocko (8): lockdep: allow to disable reclaim l

[f2fs-dev] [PATCH 6/8] jbd2: make the whole kjournald2 kthread NOFS safe

2017-01-06 Thread Michal Hocko
From: Michal Hocko kjournald2 is central to the transaction commit processing. As such any potential allocation from this kernel thread has to be GFP_NOFS. Make sure to mark the whole kernel thread GFP_NOFS by the memalloc_nofs_save. Suggested-by: Jan Kara Signed-off-by: Michal Hocko Reviewed

[f2fs-dev] [DEBUG PATCH 2/2] silent warnings which we cannot do anything about

2017-01-06 Thread Michal Hocko
From: Michal Hocko THIS PATCH IS FOR TESTING ONLY AND NOT MEANT TO HIT LINUS TREE There are some code paths used by all the filesystems which we cannot change to drop the GFP_NOFS, yet they generate a lot of warnings. Provide {disable,enable}_scope_gfp_check to silence those. alloc_page_buffers

[f2fs-dev] [DEBUG PATCH 1/2] mm, debug: report when GFP_NO{FS, IO} is used explicitly from memalloc_no{fs, io}_{save, restore} context

2017-01-06 Thread Michal Hocko
From: Michal Hocko THIS PATCH IS FOR TESTING ONLY AND NOT MEANT TO HIT LINUS TREE It is desirable to reduce the direct GFP_NO{FS,IO} usage at minimum and prefer scope usage defined by memalloc_no{fs,io}_{save,restore} API. Let's help this process and add a debugging tool to catch wh

[f2fs-dev] [PATCH 4/8] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2017-01-06 Thread Michal Hocko
From: Michal Hocko kmem_zalloc_large and _xfs_buf_map_pages use memalloc_noio_{save,restore} API to prevent from reclaim recursion into the fs because vmalloc can invoke unconditional GFP_KERNEL allocations and these functions might be called from the NOFS contexts. The memalloc_noio_save will

Re: [f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-06 Thread Michal Hocko
On Fri 06-01-17 14:29:33, Vlastimil Babka wrote: > On 01/02/2017 02:37 PM, Michal Hocko wrote: > > --- a/drivers/vhost/vhost.c > > +++ b/drivers/vhost/vhost.c > > @@ -514,18 +514,9 @@ long vhost_dev_set_owner(struct vhost_dev *dev) > > } > > EXPO

[f2fs-dev] [PATCH 3/8] mm: introduce memalloc_nofs_{save, restore} API

2017-01-06 Thread Michal Hocko
From: Michal Hocko GFP_NOFS context is used for the following 5 reasons currently - to prevent from deadlocks when the lock held by the allocation context would be needed during the memory reclaim - to prevent from stack overflows during the reclaim because

Re: [f2fs-dev] [PATCH] mm: support __GFP_REPEAT in kvmalloc_node

2017-01-06 Thread Michal Hocko
On Fri 06-01-17 13:09:36, Vlastimil Babka wrote: > On 01/04/2017 07:12 PM, Michal Hocko wrote: [...] > > diff --git a/mm/util.c b/mm/util.c > > index 8e4ea6cbe379..a2bfb85e60e5 100644 > > --- a/mm/util.c > > +++ b/mm/util.c > > @@ -348,8 +348,13 @@ void *kvmalloc_n

[f2fs-dev] [PATCH 7/8] Revert "ext4: avoid deadlocks in the writeback path by using sb_getblk_gfp"

2017-01-06 Thread Michal Hocko
From: Michal Hocko This reverts commit c45653c341f5c8a0ce19c8f0ad4678640849cb86 because sb_getblk_gfp is not really needed as sb_getblk __getblk_gfp __getblk_slow grow_buffers grow_dev_page gfp_mask = mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS) | gfp

Re: [f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-05 Thread Michal Hocko
>From 72a8493c4f692fca8980b59e0a79d09041164203 Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Wed, 4 Jan 2017 13:30:32 +0100 Subject: [PATCH] treewide: use kv[mz]alloc* rather than opencoded variants There are many code paths opencoding kvmalloc. Let's use the helper instead. This shouldn&#x

[f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-05 Thread Michal Hocko
From: Michal Hocko Using kmalloc with the vmalloc fallback for larger allocations is a common pattern in the kernel code. Yet we do not have any common helper for that and so users have invented their own helpers. Some of them are really creative when doing so. Let's just add kv[mz]allo

[f2fs-dev] [PATCH] mm: support __GFP_REPEAT in kvmalloc_node

2017-01-05 Thread Michal Hocko
about potential split ups or cleanups few more days and then repost the whole series. [1] http://lkml.kernel.org/r/20170104150800.go25...@dhcp22.suse.cz --- >From 0b92e4d2e040524b878d4e7b9ee88fbad5284b33 Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Wed, 4 Jan 2017 18:01:39 +0100 Subject: [P

Re: [f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-05 Thread Michal Hocko
On Mon 02-01-17 07:55:22, Joe Perches wrote: > On Mon, 2017-01-02 at 14:37 +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > Using kmalloc with the vmalloc fallback for larger allocations is a > > common pattern in the kernel code. Yet we do not have any common

Re: [f2fs-dev] [PATCH] mm: introduce kv[mz]alloc helpers

2017-01-05 Thread Michal Hocko
On Tue 03-01-17 11:23:04, Vlastimil Babka wrote: > On 01/02/2017 02:37 PM, Michal Hocko wrote: > > From: Michal Hocko > > > > Using kmalloc with the vmalloc fallback for larger allocations is a > > common pattern in the kernel code. Yet we do not have any common helpe

Re: [f2fs-dev] [PATCH 0/9 v2] scope GFP_NOFS api

2016-12-30 Thread Michal Hocko
this API. Can we find a way to use it? -- Michal Hocko SUSE Labs -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/sla

Re: [f2fs-dev] [RFC PATCH] mm: introduce kv[mz]alloc helpers

2016-12-30 Thread Michal Hocko
It seems that this email didn't get delivered due to some stupid gmail spam policy. Let me try to repost via a different relay. Sorry to those who have seen the original message and get a duplicate now. On Wed 21-12-16 08:03:53, Michal Hocko wrote: > On Tue 20-12-16 14:13:41, Andrew Mort

Re: [f2fs-dev] [PATCH 2/9] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives

2016-12-20 Thread Michal Hocko
On Tue 20-12-16 08:24:13, Dave Chinner wrote: > On Thu, Dec 15, 2016 at 03:07:08PM +0100, Michal Hocko wrote: > > From: Michal Hocko > > > > Now that the page allocator offers __GFP_NOLOCKDEP let's introduce > > KM_NOLOCKDEP alias for the xfs allocation APIs. Wh

Re: [f2fs-dev] [RFC PATCH] mm: introduce kv[mz]alloc helpers

2016-12-20 Thread Michal Hocko
On Wed 14-12-16 09:59:16, Michal Hocko wrote: > On Tue 13-12-16 14:07:33, Joe Perches wrote: > > On Tue, 2016-12-13 at 11:14 +0100, Michal Hocko wrote: > > > Are there any more comments or objections to this patch? Is this a good > > > start or kv[mz]alloc has to prov

[f2fs-dev] [PATCH 9/9] Revert "ext4: fix wrong gfp type under transaction"

2016-12-16 Thread Michal Hocko
From: Michal Hocko This reverts commit 216553c4b7f3e3e2beb4981cddca9b2027523928. Now that the transaction context uses memalloc_nofs_save and all allocations within the this context inherit GFP_NOFS automatically, there is no reason to mark specific allocations explicitly. This patch should not

[f2fs-dev] [PATCH 5/9 v2] xfs: use memalloc_nofs_{save, restore} instead of memalloc_noio*

2016-12-16 Thread Michal Hocko
On Fri 16-12-16 11:38:11, Brian Foster wrote: > On Thu, Dec 15, 2016 at 03:07:11PM +0100, Michal Hocko wrote: [...] > > @@ -459,7 +459,7 @@ _xfs_buf_map_pages( > > break; > > vm_unmap_aliases(); > >

Re: [f2fs-dev] [PATCH 2/9 v2] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives

2016-12-16 Thread Michal Hocko
On Fri 16-12-16 11:37:50, Brian Foster wrote: > On Fri, Dec 16, 2016 at 04:40:41PM +0100, Michal Hocko wrote: > > Updated patch after Mike noticed a BUG_ON when KM_NOLOCKDEP is used. > > --- > > From 1497e713e11639157aef21cae29052cb3dc7ab44 Mon Sep 17 00:00:00 2001 > > F

[f2fs-dev] [DEBUG PATCH 1/2] mm, debug: report when GFP_NO{FS, IO} is used explicitly from memalloc_no{fs, io}_{save, restore} context

2016-12-16 Thread Michal Hocko
From: Michal Hocko THIS PATCH IS FOR TESTING ONLY AND NOT MEANT TO HIT LINUS TREE It is desirable to reduce the direct GFP_NO{FS,IO} usage at minimum and prefer scope usage defined by memalloc_no{fs,io}_{save,restore} API. Let's help this process and add a debugging tool to catch wh

[f2fs-dev] [PATCH 0/9 v2] scope GFP_NOFS api

2016-12-16 Thread Michal Hocko
deletions(-) Shortlog: Michal Hocko (9): lockdep: allow to disable reclaim lockup detection xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS mm: introduce memalloc_nofs_{save,restore} API

[f2fs-dev] [DEBUG PATCH 0/2] debug explicit GFP_NO{FS, IO} usage from the scope context

2016-12-16 Thread Michal Hocko
Hi, I've forgot to add the following two patches which should help to identify explicit GFP_NO{FS,IO} usage from withing a scope context. Such a usage can be changed to the full GFP_KERNEL because all the calls from within the NO{FS,IO} scope will drop the __GFP_FS resp. __GFP_IO automatically and

[f2fs-dev] [PATCH 7/9] jbd2: make the whole kjournald2 kthread NOFS safe

2016-12-16 Thread Michal Hocko
From: Michal Hocko kjournald2 is central to the transaction commit processing. As such any potential allocation from this kernel thread has to be GFP_NOFS. Make sure to mark the whole kernel thread GFP_NOFS by the memalloc_nofs_save. Suggested-by: Jan Kara Signed-off-by: Michal Hocko --- fs

[f2fs-dev] [PATCH 2/9 v2] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives

2016-12-16 Thread Michal Hocko
Updated patch after Mike noticed a BUG_ON when KM_NOLOCKDEP is used. --- >From 1497e713e11639157aef21cae29052cb3dc7ab44 Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Thu, 15 Dec 2016 13:06:43 +0100 Subject: [PATCH] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep fa

[f2fs-dev] [PATCH 1/9] lockdep: allow to disable reclaim lockup detection

2016-12-16 Thread Michal Hocko
From: Michal Hocko The current implementation of the reclaim lockup detection can lead to false positives and those even happen and usually lead to tweak the code to silence the lockdep by using GFP_NOFS even though the context can use __GFP_FS just fine. See http://lkml.kernel.org/r

[f2fs-dev] [PATCH 4/9] mm: introduce memalloc_nofs_{save, restore} API

2016-12-16 Thread Michal Hocko
From: Michal Hocko GFP_NOFS context is used for the following 5 reasons currently - to prevent from deadlocks when the lock held by the allocation context would be needed during the memory reclaim - to prevent from stack overflows during the reclaim because

  1   2   >