Re: [Intel-gfx] [PATCH v2 3/6] mm: replace vma->vm_flags direct modifications with modifier calls

2023-02-02 Thread Michal Hocko
sors which would also prevent any future direct setting of those flags in uncontrolled way as well. Anyway Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [Intel-gfx] [PATCH v2 6/6] mm: export dump_mm()

2023-02-02 Thread Michal Hocko
gt; > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > mm/debug.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mm/debug.c b/mm/debug.c > index 9d3d893dc7f4..96d594e16292 100644 > --- a/mm/debug.c > +++ b/mm/debug.c > @@ -215,6

Re: [Intel-gfx] [PATCH v2 2/6] mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK

2023-02-02 Thread Michal Hocko
On Wed 25-01-23 00:38:47, Suren Baghdasaryan wrote: > To simplify the usage of VM_LOCKED_CLEAR_MASK in clear_vm_flags(), > replace it with VM_LOCKED_MASK bitmask and convert all users. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > include/linux/mm.h

Re: [Intel-gfx] [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-02-02 Thread Michal Hocko
On Wed 25-01-23 08:57:48, Suren Baghdasaryan wrote: > On Wed, Jan 25, 2023 at 1:38 AM 'Michal Hocko' via kernel-team > wrote: > > > > On Wed 25-01-23 00:38:49, Suren Baghdasaryan wrote: > > > Replace indirect modifications to vma->vm_flags with calls t

Re: [Intel-gfx] [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-02-02 Thread Michal Hocko
cation attempts. Those BUG_ONs scream to much IMHO. KSM is an MM internal code so I gueess we should be willing to trust it. > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [Intel-gfx] [PATCH v2 1/6] mm: introduce vma->vm_flags modifier functions

2023-02-02 Thread Michal Hocko
; operations. Introduce modifier functions for vm_flags to be used whenever > flags are updated. This way we can better check and control correct > locking behavior during these updates. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- >

Re: [Intel-gfx] [PATCH v2 5/6] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

2023-02-02 Thread Michal Hocko
e isolated before we downgraded mmap_lock. > + */ > + unmap_region(mm, &mt_detach, vma, prev, next, start, end, !downgrade); > /* Statistics and freeing VMAs */ > mas_set(&mas_detach, start); > remove_mt(mm, &mas_detach); > @@ -2704,7 +2708,7

Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional

2020-09-29 Thread Michal Hocko
On Tue 29-09-20 11:00:03, Daniel Vetter wrote: > On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote: > > On Wed 16-09-20 23:43:02, Daniel Vetter wrote: > > > I can > > > then figure out whether it's better to risk not spotting issues with > > >

Re: [Intel-gfx] [patch 00/13] preempt: Make preempt count unconditional

2020-09-29 Thread Michal Hocko
o carefuly consider failure. This is not a random allocation mode. -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] mm: Skip opportunistic reclaim for dma pinned pages

2020-06-25 Thread Michal Hocko
On Thu 25-06-20 12:00:47, Chris Wilson wrote: > Quoting Michal Hocko (2020-06-25 08:57:25) > > On Wed 24-06-20 20:14:17, Chris Wilson wrote: > > > A general rule of thumb is that shrinkers should be fast and effective. > > > They are called from direct reclaim at the mos

Re: [Intel-gfx] [PATCH] mm: Skip opportunistic reclaim for dma pinned pages

2020-06-25 Thread Michal Hocko
be done. The page is going to be pinned no matter how many page tables are mapping it right? > + > if (flags & TTU_SPLIT_HUGE_PMD) { > split_huge_pmd_address(vma, address, > flags & TTU_SPLIT_FREEZE, page); > -- > 2.20.1 -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Paging out when free memory is low but not exhausted (and available memory remains high)

2020-02-10 Thread Michal Hocko
lags=17 > kswapd0-90[000] 251817.911052: i915_gem_shrink: dev=0, > target=605, flags=3 > kswapd0-90[000] 251817.911080: i915_gem_shrink: dev=0, > target=474, flags=17 > kswapd0-90[000] 251817.964954: i915_gem_shrink: dev=0, &

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-20 Thread Michal Hocko
On Fri 16-08-19 11:31:45, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 02:26:25PM +0200, Michal Hocko wrote: [...] > > I believe I have given some examples when introducing __GFP_NOLOCKDEP. > > Okay, I think that is 7e7844226f10 ("lockdep: allow to disable reclaim >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Fri 16-08-19 09:19:06, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 10:10:29AM +0200, Michal Hocko wrote: > > On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > > > > > The l

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 22:16:43, Daniel Vetter wrote: > On Thu, Aug 15, 2019 at 9:35 PM Michal Hocko wrote: [...] > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > > I

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 15:15:09, Andrew Morton wrote: > On Thu, 15 Aug 2019 10:44:29 +0200 Michal Hocko wrote: > > > > I continue to struggle with this. It introduces a new kernel state > > > "running preemptibly but must not call schedule()". How does this make >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > >

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 16:18:10, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:05:25PM +0200, Michal Hocko wrote: > > > This is what you claim and I am saying that fs_reclaim is about a > > restricted reclaim context and it is an ugly hack. It has proven to > > report false

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 15:24:48, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 07:42:07PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > > > > > AFAIK

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack of __GFP_FS and > > > __GFP_DIRECT_RECLAIM.. > > > > > > This matches the

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 11:12:19, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 03:21:27PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 09:23:44, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 08:58:29AM +0200, Daniel Vetter wrote: > > > > On Wed, Aug 14, 2

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 10:04:15, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 10:44:29AM +0200, Michal Hocko wrote: > > > As the oom reaper is the primary guarantee of the oom handling forward > > progress it cannot be blocked on anything that might depend on blockable > > m

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
ou posted for > blockable had to do with not recursing into reclaim and deadlocking, > and didn't seem to have much to do with blocking. > > I'm asking if *non-blocking* is really the requirement or if this is > just the usual 'do not deadlock on the allocator' thing

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
ng state happens that we can end up in a silent hang with an unusable machine. Now we hope for reasonable implementations of mmu notifiers (strong words I know ;) and this should be relatively simple and effective catch all tool to detect something suspicious is going on. Does that make the situ

Re: [Intel-gfx] [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-07 Thread Michal Hocko
On Wed 07-08-19 10:37:26, Jan Kara wrote: > On Fri 02-08-19 12:14:09, John Hubbard wrote: > > On 8/2/19 7:52 AM, Jan Kara wrote: > > > On Fri 02-08-19 07:24:43, Matthew Wilcox wrote: > > > > On Fri, Aug 02, 2019 at 02:41:46PM +0200, Jan Kara wrote: > > > >

Re: [Intel-gfx] [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-02 Thread Michal Hocko
e changes that would break the balance though. -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-22 Thread Michal Hocko
On Wed 22-05-19 08:13:57, Chris Wilson wrote: > Quoting Michal Hocko (2019-05-22 07:34:42) > > On Wed 22-05-19 06:06:31, Tetsuo Handa wrote: > > [...] > > > Since OOM notifier will be called after shrinkers are attempted, > > > can i915 move from OOM notifier to

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
On Wed 22-05-19 06:06:31, Tetsuo Handa wrote: [...] > Since OOM notifier will be called after shrinkers are attempted, > can i915 move from OOM notifier to shrinker? That would be indeed preferable. OOM notifier is an API from hell. -- Michal Hocko SUS

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
t_sleep() debug checks. Add a non_block_start/end() > > pair to annotate these. > > Just putting preempt on/off around these is not sufficient? It is not a critical section. It is a _debugging_ facility to help discover blocking contexts. -- Michal Hocko SUSE Labs _

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
On Tue 21-05-19 20:04:34, Tetsuo Handa wrote: > On 2019/05/21 19:51, Michal Hocko wrote: > > On Tue 21-05-19 19:44:01, Tetsuo Handa wrote: > >> On 2019/05/21 19:06, Daniel Vetter wrote: > >>> In some special cases we must not block, but there's not a >

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
ut then we should be using the non-blocking API if this is a real problem. The above code just doesn't make any sense. We have a blocking API called and wrapped by non-blocking one. -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
catch spinlocks on top, but Michal > said those are less of a problem because spinlocks can't have an > indirect dependency upon the page allocator and hence close the loop > with the oom reaper. > > Suggested by Michal Hocko. > > v2: > - Improve commit message (Michal

Re: [Intel-gfx] [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
On Mon 10-12-18 16:22:53, Peter Zijlstra wrote: > On Mon, Dec 10, 2018 at 04:01:59PM +0100, Michal Hocko wrote: > > On Mon 10-12-18 15:47:11, Peter Zijlstra wrote: > > > On Mon, Dec 10, 2018 at 03:13:37PM +0100, Michal Hocko wrote: > > > > I do not see any sched

Re: [Intel-gfx] [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
On Mon 10-12-18 15:47:11, Peter Zijlstra wrote: > On Mon, Dec 10, 2018 at 03:13:37PM +0100, Michal Hocko wrote: > > I do not see any scheduler guys Cced and it would be really great to get > > their opinion here. > > > > On Mon 10-12-18 11:36:39, Daniel Vetter wrote:

Re: [Intel-gfx] [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
,enable}, and that really has a different semantic, I think this makes some sense. The cotext is preemptible but we do not want notifier to sleep on any locks, WQ etc. > Suggested by Michal Hocko. > > Cc: Andrew Morton > Cc: Michal Hocko > Cc: David Rientjes > Cc: "Chris

Re: [Intel-gfx] [PATCH 1/4] mm: Check if mmu notifier callbacks are allowed to fail

2018-12-10 Thread Michal Hocko
ir enough. If this is going to help with testing then I do not have any objections of course. > v2: Drop the full WARN_ON backtrace in favour of just a pr_warn for > the problematic case (Michal Hocko). Thanks! > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Christian König"

Re: [Intel-gfx] [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 14:15:11, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 1:43 PM Michal Hocko wrote: > > On Fri 23-11-18 13:30:57, Daniel Vetter wrote: > > > On Fri, Nov 23, 2018 at 12:15:57PM +0100, Michal Hocko wrote: > > > > On Thu 22-11-18 17:51:04, Daniel Vett

Re: [Intel-gfx] [PATCH 2/3] mm, notifier: Catch sleeping/blocking for !blockable

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 13:38:38, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 12:12:37PM +0100, Michal Hocko wrote: > > On Thu 22-11-18 17:51:05, Daniel Vetter wrote: > > > We need to make sure implementations don't cheat and don't have a > > > possible schedul

Re: [Intel-gfx] [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 13:30:57, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 12:15:57PM +0100, Michal Hocko wrote: > > On Thu 22-11-18 17:51:04, Daniel Vetter wrote: > > > Just a bit of paranoia, since if we start pushing this deep into > > > callchains it's hard

Re: [Intel-gfx] [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
pr_info? Is really backtrace that interesting? > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Christian König" > Cc: David Rientjes > Cc: Daniel Vetter > Cc: "Jérôme Glisse" > Cc: linux...@kvack.org > Cc: Paolo Bonzini > Signed-off-by: Dan

Re: [Intel-gfx] [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
ng to free any kind of memory. And where the process is > gone already, so semantics of what exactly happens don't matter that much > anymore. Yes this was indeed the case. There is still the exit path which would do the rest of the work so we are not leaving anything behind. --

Re: [Intel-gfx] [PATCH 2/3] mm, notifier: Catch sleeping/blocking for !blockable

2018-11-23 Thread Michal Hocko
e > might_sleep() callsites trigger, and it's a bit ugly in the code flow. > But it gets the job done. Yeah, it is quite ugly. Especially because it makes DEBUG config bahavior much different. So is this really worth it? Has this already discovered any existing bug? > Cc: Andrew Morto

Re: [Intel-gfx] [PATCH v6] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-06 Thread Michal Hocko
tree needs that. Up to Andrew but this doesn't seem to be conflicting with anything that is going on in MM. -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH v5] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-06 Thread Michal Hocko
> > Export pagevec API check_move_unevictable_pages(). > > This patch was inspired by Chris Wilson's change [1]. > > [1]: https://patchwork.kernel.org/patch/9768741/ > > Cc: Chris Wilson > Cc: Joonas Lahtinen > Cc: Peter Zijlstra > Cc: Andrew Morton > Cc: Dave

Re: [Intel-gfx] [PATCH v4] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-05 Thread Michal Hocko
On Mon 05-11-18 22:33:13, Kuo-Hsin Yang wrote: > On Mon, Nov 5, 2018 at 9:02 PM Michal Hocko wrote: > > > > On Mon 05-11-18 19:13:48, Kuo-Hsin Yang wrote: [...] > > > + * @pvec: pagevec with pages to check > > > * > > > - * Checks pages for evictabi

Re: [Intel-gfx] [PATCH v4] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-05 Thread Michal Hocko
On Mon 05-11-18 14:02:09, Michal Hocko wrote: > On Mon 05-11-18 19:13:48, Kuo-Hsin Yang wrote: > > The i915 driver uses shmemfs to allocate backing storage for gem > > objects. These shmemfs pages can be pinned (increased ref count) by > > shmem_read_mapping_page_gfp(). Wh

Re: [Intel-gfx] [PATCH v4] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-05 Thread Michal Hocko
work.kernel.org/patch/9768741/ I would recommend using msg-id based url. > Cc: Chris Wilson > Cc: Michal Hocko > Cc: Joonas Lahtinen > Cc: Peter Zijlstra > Cc: Andrew Morton > Cc: Dave Hansen > Signed-off-by: Kuo-Hsin Yang other than that Acked-by: Michal Hocko

Re: [Intel-gfx] [PATCH v3] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-02 Thread Michal Hocko
On Fri 02-11-18 20:35:11, Vovo Yang wrote: > On Thu, Nov 1, 2018 at 9:10 PM Michal Hocko wrote: > > OK, so that explain my question about the test case. Even though you > > generate a lot of page cache, the amount is still too small to trigger > > pagecache mostly recl

Re: [Intel-gfx] [PATCH v3] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-11-01 Thread Michal Hocko
On Thu 01-11-18 19:28:46, Vovo Yang wrote: > On Thu, Nov 1, 2018 at 12:42 AM Michal Hocko wrote: > > On Wed 31-10-18 07:40:14, Dave Hansen wrote: > > > Didn't we create the unevictable lists in the first place because > > > scanning alone was observed to b

Re: [Intel-gfx] [PATCH v3] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-10-31 Thread Michal Hocko
On Wed 31-10-18 07:40:14, Dave Hansen wrote: > On 10/31/18 7:24 AM, Michal Hocko wrote: > > I am also wondering whether unevictable pages culling can be > > really visible when we do the anon LRU reclaim because the swap path is > > quite expensinve on its own. >

Re: [Intel-gfx] [PATCH v3] mm, drm/i915: mark pinned shmemfs pages as unevictable

2018-10-31 Thread Michal Hocko
do not reclaim anon LRUs at all. Maybe I have misunderstood the test though. I am also wondering whether unevictable pages culling can be really visible when we do the anon LRU reclaim because the swap path is quite expensinve on its own. -- Michal Hocko SUSE Labs _

Re: [Intel-gfx] [PATCH 2/2] drm/i915: Mark pinned shmemfs pages as unevictable

2018-10-18 Thread Michal Hocko
n scan anonymous LRU lists. You would have to generate a swapout workload to test this properly. On the other hand if mapping_set_unevictable has really a measurably bad performance impact then this is probably not worth much because most workloads are swap modest. -- Michal Hocko SUSE Labs _

Re: [Intel-gfx] [PATCH 2/2] drm/i915: Mark pinned shmemfs pages as unevictable

2018-10-16 Thread Michal Hocko
On Tue 16-10-18 19:31:06, Chris Wilson wrote: > Quoting Michal Hocko (2018-10-16 19:21:55) > > On Wed 17-10-18 01:43:00, Kuo-Hsin Yang wrote: > > > The i915 driver use shmemfs to allocate backing storage for gem objects. > > > These shmemfs pages can be pinn

Re: [Intel-gfx] [PATCH 2/2] drm/i915: Mark pinned shmemfs pages as unevictable

2018-10-16 Thread Michal Hocko
a movable zone. Does mapping_gfp_constraint contains __GFP_MOVABLE? If yes, we want to drop it as well. Other than that the patch makes sense with my very limited knowlege of the i915 code of course. -- Michal Hocko SUSE Labs ___ Intel-gfx ma

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
about? From f7ac75277d526dccd011f343818dc6af627af2af Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Fri, 24 Aug 2018 15:32:24 +0200 Subject: [PATCH] mm, mmu_notifier: be explicit about range invalition non-blocking mode If invalidate_range_start is called for !blocking mode then all callbacks have to guarantee

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 23:52:25, Tetsuo Handa wrote: > On 2018/08/24 22:32, Michal Hocko wrote: > > On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > >> I worry that (currently > >> out-of-tree) users of this API are involving work / recursion. > > > > I do not giv

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:44:03, Christian König wrote: > Am 24.08.2018 um 15:40 schrieb Michal Hocko: > > On Fri 24-08-18 15:28:33, Christian König wrote: > > > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > > > On Fri 24-08-18 15:10:08, Christian König wrote: > >

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:28:33, Christian König wrote: > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > On Fri 24-08-18 15:10:08, Christian König wrote: > > > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > > > On Fri 24-08-18 14:52:26, Christian König wrote: > >

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > On 2018/08/24 20:36, Michal Hocko wrote: > >> That is, this API seems to be currently used by only out-of-tree users. > >> Since > >> we can't check that nobody has memory allocation dependency, I think that >

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:10:08, Christian König wrote: > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > On Fri 24-08-18 14:52:26, Christian König wrote: > > > Am 24.08.2018 um 14:33 schrieb Michal Hocko: > > [...] > > > > Thiking about it some more, I can im

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:52:26, Christian König wrote: > Am 24.08.2018 um 14:33 schrieb Michal Hocko: [...] > > Thiking about it some more, I can imagine that a notifier callback which > > performs an allocation might trigger a memory reclaim and that in turn > > might trigger a n

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:18:44, Christian König wrote: > Am 24.08.2018 um 14:03 schrieb Michal Hocko: > > On Fri 24-08-18 13:57:52, Christian König wrote: > > > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > > > On Fri 24-08-18 13:43:16, Christian König wrote: > >

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:57:52, Christian König wrote: > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > On Fri 24-08-18 13:43:16, Christian König wrote: [...] > > > That won't work like this there might be multiple > > > invalidate_range_start()/invalidate_ra

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:43:16, Christian König wrote: > Am 24.08.2018 um 13:32 schrieb Michal Hocko: > > On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: > > > Two more worries for this patch. > > > > > > > > > > > > > --- a/drivers/gpu/drm

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
pagetables(mirror, > action, > > That is, this API seems to be currently used by only out-of-tree users. Since > we can't check that nobody has memory allocation dependency, I think that > hmm_invalidate_range_start() should return -EAGAIN if blockable == false for > now

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable) */ static void amdgpu_mn_read_unlock(struct amdgpu_mn *amn) { - if (atomic_dec_return(&amn->recursion) == 0) - up_read_non_owner(&amn->lock); + up_read(&amn->lock); } /** -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
unction a bit to have unified function exit paths. Suggested-by: Andrew Morton Signed-off-by: Michal Hocko " -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
oom_reaper: reaped process %d (%s), now anon-rss:%lukB, > file-rss:%lukB, shmem-rss:%lukB\n", > task_pid_nr(tsk), tsk->comm, -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
K(get_mm_counter(mm, MM_SHMEMPAGES))); +out_finish: + trace_finish_task_reaping(tsk->pid); +out_unlock: up_read(&mm->mmap_sem); - trace_finish_task_reaping(tsk->pid); - return true; + return ret; } #define MAX_OOM_REAP_RETRIES 10 -- Mich

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 16:01:25, Andrew Morton wrote: > On Tue, 17 Jul 2018 10:12:01 +0200 Michal Hocko wrote: > > > > Any suggestions regarding how the driver developers can test this code > > > path? I don't think we presently have a way to fake an oom-killing > &

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:11:54, Michal Hocko wrote: > On Mon 23-07-18 09:03:06, Michal Hocko wrote: > > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > > [...] > > > Please take a look? > > > > Are you OK to have these in a separate patch? > > Btw. I will

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:03:06, Michal Hocko wrote: > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > [...] > > Please take a look? > > Are you OK to have these in a separate patch? Btw. I will rebase this patch once oom stuff in linux-next settles. At least oom_lock removal fr

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 17:09:02, Andrew Morton wrote: [...] > Please take a look? Are you OK to have these in a separate patch? -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mail

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-19 Thread Michal Hocko
Does anybody see any reasons why this should get into mmotm tree? I do not want to rush this in but if general feeling is to push it for the upcoming merge window then I will not object. -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx

Re: [Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-17 Thread Michal Hocko
On Mon 16-07-18 16:12:49, Andrew Morton wrote: > On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > > > From: Michal Hocko > > > > There are several blockable mmu notifiers which might sleep in > > mmu_notifier_invalidate_range_start and that is a problem

[Intel-gfx] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-16 Thread Michal Hocko
From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-11 Thread Michal Hocko
On Wed 11-07-18 13:14:47, Leon Romanovsky wrote: > On Wed, Jul 11, 2018 at 11:03:53AM +0200, Michal Hocko wrote: > > On Tue 10-07-18 19:20:20, Leon Romanovsky wrote: > > > On Tue, Jul 10, 2018 at 04:14:10PM +0200, Michal Hocko wrote: > > > > On Tue 10-07-18 1

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-11 Thread Michal Hocko
On Tue 10-07-18 19:20:20, Leon Romanovsky wrote: > On Tue, Jul 10, 2018 at 04:14:10PM +0200, Michal Hocko wrote: > > On Tue 10-07-18 16:40:40, Leon Romanovsky wrote: > > > On Mon, Jul 09, 2018 at 02:29:08PM +0200, Michal Hocko wrote: > > > > On Wed 27-06-1

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-10 Thread Michal Hocko
On Tue 10-07-18 16:40:40, Leon Romanovsky wrote: > On Mon, Jul 09, 2018 at 02:29:08PM +0200, Michal Hocko wrote: > > On Wed 27-06-18 09:44:21, Michal Hocko wrote: > > > This is the v2 of RFC based on the feedback I've received so far. The > > > code even compiles

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-09 Thread Michal Hocko
On Wed 27-06-18 09:44:21, Michal Hocko wrote: > This is the v2 of RFC based on the feedback I've received so far. The > code even compiles as a bonus ;) I haven't runtime tested it yet, mostly > because I have no idea how. > > Any further feedback is highly apprecia

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
de would simply back of without releasing any memory. The patch should help to reclaim some memory. > But do you know a way to let the OOM killer kill a specific process? Yes, you can set its oom_score_adj to 1000 which means always select that task.

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 14:24:29, Christian König wrote: > Am 02.07.2018 um 14:20 schrieb Michal Hocko: > > On Mon 02-07-18 14:13:42, Christian König wrote: > > > Am 02.07.2018 um 13:54 schrieb Michal Hocko: > > > > On Mon 02-07-18 11:14:58, Christian König wrote: > >

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 14:13:42, Christian König wrote: > Am 02.07.2018 um 13:54 schrieb Michal Hocko: > > On Mon 02-07-18 11:14:58, Christian König wrote: > > > Am 27.06.2018 um 09:44 schrieb Michal Hocko: > > > > This is the v2 of RFC based on the feedback I've rec

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 11:14:58, Christian König wrote: > Am 27.06.2018 um 09:44 schrieb Michal Hocko: > > This is the v2 of RFC based on the feedback I've received so far. The > > code even compiles as a bonus ;) I haven't runtime tested it yet, mostly > > because I

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-27 Thread Michal Hocko
7 00:00:00 2001 From: Michal Hocko Date: Wed, 20 Jun 2018 15:03:20 +0200 Subject: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There are several blockable mmu notifiers which migh

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-25 Thread Michal Hocko
On Mon 25-06-18 10:01:03, Michal Hocko wrote: > On Fri 22-06-18 16:09:06, Felix Kuehling wrote: > > On 2018-06-22 11:24 AM, Michal Hocko wrote: > > > On Fri 22-06-18 17:13:02, Christian König wrote: > > >> Hi Michal, > > >> > > >> [Adding Fel

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-25 Thread Michal Hocko
On Fri 22-06-18 16:09:06, Felix Kuehling wrote: > On 2018-06-22 11:24 AM, Michal Hocko wrote: > > On Fri 22-06-18 17:13:02, Christian König wrote: > >> Hi Michal, > >> > >> [Adding Felix as well] > >> > >> Well first of all you have a misconcep

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-22 Thread Michal Hocko
[Resnding with the CC list fixed] On Fri 22-06-18 18:40:26, Michal Hocko wrote: > On Fri 22-06-18 12:18:46, Jerome Glisse wrote: > > On Fri, Jun 22, 2018 at 05:57:16PM +0200, Michal Hocko wrote: > > > On Fri 22-06-18 16:36:49, Chris Wilson wrote: > > > > Quoting Mic

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-22 Thread Michal Hocko
[Hmm, the cc list got mangled somehow - you have just made many people to work for suse ;) and to kvack.org in the preious one - fixed up hopefully] On Fri 22-06-18 17:07:21, Chris Wilson wrote: > Quoting Michal Hocko (2018-06-22 16:57:16) > > On Fri 22-06-18 16:36:49, Chris Wil

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-22 Thread Michal Hocko
On Fri 22-06-18 16:36:49, Chris Wilson wrote: > Quoting Michal Hocko (2018-06-22 16:02:42) > > Hi, > > this is an RFC and not tested at all. I am not very familiar with the > > mmu notifiers semantics very much so this is a crude attempt to achieve > > what I need basica

Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-22 Thread Michal Hocko
> > @@ -244,7 +258,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct > > mmu_notifier *mn, > > /* notification is exclusive, but interval is inclusive */ > > end -= 1; > > - amdgpu_mn_read_lock(rmn); > > + if (amdgpu_mn_read_lock(rmn, blockable)) > > + return -EAGAIN; > > it = interval_tree_iter_first(&rmn->objects, start, end); > > while (it) { > > @@ -262,6 +277,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct > > mmu_notifier *mn, > > amdgpu_amdkfd_evict_userptr(mem, mm); > > } > > } > > + > > + return 0; > > } > > /** -- Michal Hocko SUSE Labs ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-22 Thread Michal Hocko
From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an oom

Re: [Intel-gfx] [RFC] [PATCH] mm, oom: Offload OOM notify callback to a kernel thread.

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 17:29:47, Michael S. Tsirkin wrote: > On Mon, Oct 02, 2017 at 04:19:00PM +0200, Michal Hocko wrote: > > On Mon 02-10-17 17:11:55, Michael S. Tsirkin wrote: > > > On Mon, Oct 02, 2017 at 01:50:35PM +0200, Michal Hocko wrote: > > [...] > > > >

Re: [Intel-gfx] [RFC] [PATCH] mm, oom: Offload OOM notify callback to a kernel thread.

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 17:11:55, Michael S. Tsirkin wrote: > On Mon, Oct 02, 2017 at 01:50:35PM +0200, Michal Hocko wrote: [...] > > and some > > other call path is allocating while holding the lock. But you seem to be > > right and > > leak_balloon > > tell_

Re: [Intel-gfx] [RFC] [PATCH] mm, oom: Offload OOM notify callback to a kernel thread.

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 20:33:52, Tetsuo Handa wrote: > Michal Hocko wrote: > > [Hmm, I do not see the original patch which this has been a reply to] > > urbl.hostedemail.com and b.barracudacentral.org blocked my IP address, > and the rest are "Recipient address rejected: Greylist

Re: [Intel-gfx] [RFC] [PATCH] mm, oom: Offload OOM notify callback to a kernel thread.

2017-10-02 Thread Michal Hocko
lloon(). This is really nasty! And I would argue that this is an abuse of the oom notifier interface from the virtio code. OOM notifiers are an ugly hack on its own but all its users have to be really careful to not depend on any allocation request because that is a straight deadlock situatio

Re: [Intel-gfx] [RFC] mm, drm/i915: Mark pinned shmemfs pages as unevictable

2017-08-21 Thread Michal Hocko
On Sat 19-08-17 14:15:35, Chris Wilson wrote: > Quoting Michal Hocko (2017-06-06 13:14:18) > > On Tue 06-06-17 13:04:36, Chris Wilson wrote: > > > Similar in principle to the treatment of get_user_pages, pages that > > > i915.ko acquires from shmemfs are not imm

Re: [Intel-gfx] [PATCH v2 5/5] drm/i915: Start writeback from the shrinker

2017-06-09 Thread Michal Hocko
On Fri 09-06-17 12:51:57, Chris Wilson wrote: > Quoting Michal Hocko (2017-06-09 12:17:26) > > [Add Hugh] > > > > On Fri 09-06-17 12:03:50, Chris Wilson wrote: > > > When we are called to relieve mempressue via the shrinker, the only way > > > we can make pr

Re: [Intel-gfx] [PATCH v2 5/5] drm/i915: Start writeback from the shrinker

2017-06-09 Thread Michal Hocko
re just extending the > impact of swap thrashing to them). I am not sure you can start writeback on shmem while it is not in the swapcache. Hugh? > Signed-off-by: Chris Wilson > Cc: Mika Kuoppala > Cc: Joonas Lahtinen > Cc: Tvrtko Ursulin > Cc: Matthew Auld > Cc: Dan

Re: [Intel-gfx] [PATCH v2 2/5] drm/i915: Remove __GFP_NORETRY from our buffer allocator

2017-06-09 Thread Michal Hocko
d_swapping > Signed-off-by: Chris Wilson > Cc: Mika Kuoppala > Cc: Joonas Lahtinen > Cc: Daniel Vetter > Cc: Michal Hocko OK, this looks good to me. I will follow with my __GFP_RETRY_MAYFAIL and will convert this caller to use it sometimes next week. Thanks! > --- > dri

  1   2   >