Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-11 Thread Michal Hocko
On Wed 11-09-19 19:41:44, Yunsheng Lin wrote: > On 2019/9/11 19:03, Yunsheng Lin wrote: > > On 2019/9/11 15:34, Michal Hocko wrote: > >> On Wed 11-09-19 15:22:30, Yunsheng Lin wrote: > >> [...] > >>> It seems that there is no protection that prevent setting

Re: [PATCH] memcg, kmem: do not fail __GFP_NOFAIL charges

2019-09-11 Thread Michal Hocko
On Mon 09-09-19 13:22:45, Michal Hocko wrote: > On Fri 06-09-19 11:24:55, Shakeel Butt wrote: [...] > > I wonder what has changed since > > <http://lkml.kernel.org/r/20180525185501.82098-1-shake...@google.com/>. > > I have completely forgot about that one. It seems t

Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread Michal Hocko
it should provide a similar functionality. Did you see that approach? How does this compares to it? Or am I completely off when comparing them? [1] mostly likely not the latest version of the patchset http://lkml.kernel.org/r/1502940416-42944-5-git-send-email-wei.w.w...@intel.com -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-11 Thread Michal Hocko
suspect they are a result of random attempts to fortify the code in many cases. Consistency is certainly good but spreading more checks all over the place just adds more cargo cult. Each check should be reasonably justified. -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-11 Thread Michal Hocko
On Wed 11-09-19 14:15:51, Yunsheng Lin wrote: > On 2019/9/11 13:33, Michal Hocko wrote: > > On Tue 10-09-19 14:53:39, Michal Hocko wrote: > >> On Tue 10-09-19 20:47:40, Yunsheng Lin wrote: > >>> On 2019/9/10 19:12, Greg KH wrote: > >>>> On Tue, Sep 10,

Re: [PATCH] mm: Add callback for defining compaction completion

2019-09-11 Thread Michal Hocko
trol > over how to compaction work to do, I have added a compaction callback > which controls how much work is done in one compaction cycle. Why is a more fine grained control really needed? Sure compacting everything is heavy weight but how often do you have to do that. Your changelog starts with a usecase when there is a high demand for large pages at the startup. What prevents you do compaction at that time. If the workload is longterm then the initial price should just pay back, no? -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 14:53:39, Michal Hocko wrote: > On Tue 10-09-19 20:47:40, Yunsheng Lin wrote: > > On 2019/9/10 19:12, Greg KH wrote: > > > On Tue, Sep 10, 2019 at 01:04:51PM +0200, Michal Hocko wrote: > > >> On Tue 10-09-19 18:58:05, Yunsheng Lin wrote: > > &

Re: [PATCH] mm: Add callback for defining compaction completion

2019-09-10 Thread Michal Hocko
de > > /* > * The set of flags that only affect watermark checking and reclaim > @@ -203,6 +204,7 @@ struct compact_control { > bool whole_zone;/* Whole zone should/has been scanned */ > bool contended; /* Signal lock or sched contention */ > bool rescan;/* Rescanning the same pageblock */ > + compact_finished_cb compact_finished_cb; > }; > > /* > -- > 2.21.0 -- Michal Hocko SUSE Labs

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-10 Thread Michal Hocko
migrated out successfully, driver should clear > the movable flag in the page. Look at reset_page in zs_page_migrate. > So, other thread couldn't isolate the page during the window. > > If I miss something, let me know it. Please have a look at http://lkml.kernel.org/r/157fc541501a9c4c

Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 19:52:13, Michal Hocko wrote: > On Tue 10-09-19 09:05:43, Alexander Duyck wrote: [...] > > All this is providing is just a report and it is optional if the > > hypervisor will act on it or not. If the hypervisor takes some sort of > > action on the page,

Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 09:05:43, Alexander Duyck wrote: > On Tue, Sep 10, 2019 at 7:47 AM Michal Hocko wrote: > > > > On Tue 10-09-19 07:42:43, Alexander Duyck wrote: > > > On Tue, Sep 10, 2019 at 5:42 AM Michal Hocko wrote: > > > > > > > > I wanted to re

Re: [PATCH v9 3/8] mm: Move set/get_pcppage_migratetype to mmzone.h

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 07:46:50, Alexander Duyck wrote: > On Tue, Sep 10, 2019 at 5:23 AM Michal Hocko wrote: > > > > On Sat 07-09-19 10:25:28, Alexander Duyck wrote: > > > From: Alexander Duyck > > > > > > In order to support page reporting it wil

Re: [RFC PATCH v2] mm: initialize struct pages reserved by ZONE_DEVICE driver.

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 07:53:17, Dan Williams wrote: > On Tue, Sep 10, 2019 at 7:01 AM Michal Hocko wrote: > > > > On Fri 06-09-19 08:09:52, Toshiki Fukasawa wrote: > > [...] > > > @@ -5856,8 +5855,6 @@ void __meminit memmap_init_zone(unsigned long size, >

Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 07:42:43, Alexander Duyck wrote: > On Tue, Sep 10, 2019 at 5:42 AM Michal Hocko wrote: > > > > I wanted to review "mm: Introduce Reported pages" just realize that I > > have no clue on what is going on so returned to the cover and it didn't > &

Re: [RFC PATCH v2] mm: initialize struct pages reserved by ZONE_DEVICE driver.

2019-09-10 Thread Michal Hocko
anything AFAICS. Btw. irrespective to this issue all three callers should be using pfn_to_online_page rather than pfn_to_page AFAICS. It doesn't really make sense to collect data for offline pfn ranges. They might be uninitialized even without zone device. -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 20:47:40, Yunsheng Lin wrote: > On 2019/9/10 19:12, Greg KH wrote: > > On Tue, Sep 10, 2019 at 01:04:51PM +0200, Michal Hocko wrote: > >> On Tue 10-09-19 18:58:05, Yunsheng Lin wrote: > >>> On 2019/9/10 17:31, Greg KH wrote: > >>>> On Tu

Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-10 Thread Michal Hocko
s with additional non-reported > pages being pulled until the free areas all consist of reported pages. -- Michal Hocko SUSE Labs

Re: [PATCH v9 4/8] mm: Use zone and order instead of free area in free_list manipulators

2019-09-10 Thread Michal Hocko
functions down so that we have the zone defined before we define the > list manipulation functions. > > Reviewed-by: Dan Williams > Reviewed-by: David Hildenbrand > Reviewed-by: Pankaj Gupta > Signed-off-by: Alexander Duyck Acked-by: Micha

Re: [PATCH v9 3/8] mm: Move set/get_pcppage_migratetype to mmzone.h

2019-09-10 Thread Michal Hocko
- * other index - this ensures that it will be put on the correct CMA > freelist. > - */ > -static inline int get_pcppage_migratetype(struct page *page) > -{ > - return page->index; > -} > - > -static inline void set_pcppage_migratetype(struct page *page, int > migratetype) > -{ > - page->index = migratetype; > -} > - > #ifdef CONFIG_PM_SLEEP > /* > * The following functions are used by the suspend/hibernate code to > temporarily -- Michal Hocko SUSE Labs

Re: [PATCH v9 2/8] mm: Adjust shuffle code to allow for future coalescing

2019-09-10 Thread Michal Hocko
rea_random(page, >free_area[order], > - migratetype); > + area = >free_area[order]; > + if (is_shuffle_order(order) ? shuffle_pick_tail() : > + buddy_merge_likely(pfn, buddy_pfn, page, order)) Ouch this is just awful don't you think? -- Michal Hocko SUSE Labs

Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling

2019-09-10 Thread Michal Hocko
expensive as a result since the memory was both not local to the > CPU and was being updated by multiple sockets. What was the pattern of page freeing in your testing? I am wondering because order 0 pages should be prevailing and those usually go via pcp lists so they do not get shuffled unless the batch is full IIRC. -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
KASAN as below: OK, I seem to remember this being brought up already. And now when I think about it, we really want to make cpumask_of_node NUMA_NO_NODE aware. That means using the same trick the allocator does for this special case. -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 18:40:12, Yunsheng Lin wrote: > On 2019/9/10 15:24, Michal Hocko wrote: > > Our emails crossed, sorry about that. > > > > On Tue 10-09-19 15:08:20, Yunsheng Lin wrote: > >> On 2019/9/10 2:50, Michal Hocko wrote: > >>> On

Re: [LKP] [mm, memcg] 1e577f970f: will-it-scale.per_process_ops -7.2% regression

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 15:08:41, Huang, Ying wrote: > Michal Hocko writes: > > > On Mon 09-09-19 10:15:44, kernel test robot wrote: > >> Greeting, > >> > >> FYI, we noticed a -7.2% regression of will-it-scale.per_process_ops due to > >> commi

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
Our emails crossed, sorry about that. On Tue 10-09-19 15:08:20, Yunsheng Lin wrote: > On 2019/9/10 2:50, Michal Hocko wrote: > > On Mon 09-09-19 14:04:23, Yunsheng Lin wrote: [...] > >> Even if a device's numa node is not specified, the device really > >> does belong to

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-10 Thread Michal Hocko
eally have to know about the node apart the allocator? -- Michal Hocko SUSE Labs

Re: [PATCH] mm: add dummy can_do_mlock() helper

2019-09-10 Thread Michal Hocko
n bool can_do_mlock(void); > +#else > +static inline bool can_do_mlock(void) { return false; } > +#endif > extern int user_shm_lock(size_t, struct user_struct *); > extern void user_shm_unlock(size_t, struct user_struct *); > > -- > 2.20.0 > -- Michal Hocko SUSE Labs

Re: [patch for-5.3 0/4] revert immediate fallback to remote hugepages

2019-09-09 Thread Michal Hocko
ch series, that it fixes the swap > > storms. But upstream 5.3 fixes the swap storms too and what you sent > > is not nearly equivalent to the mempolicy that Michal was willing > > to provide you and that we thought you needed to get bigger guarantees > > of getting only local 2m or local 4k pages. > > > > I haven't seen such a patch series, is there a link? not yet unfortunatelly. So far I haven't heard that you are even interested in that policy. You have never commented on that IIRC. -- Michal Hocko SUSE Labs

Re: [PATCH] driver core: ensure a device has valid node id in device_add()

2019-09-09 Thread Michal Hocko
r with generic layer. */ > /* we require the name to be set before, and pass NULL */ > diff --git a/include/linux/numa.h b/include/linux/numa.h > index 110b0e5..eccc757 100644 > --- a/include/linux/numa.h > +++ b/include/linux/numa.h > @@ -13,4 +13,6 @@ > > #define NUMA_NO_NODE(-1) > > +#define numa_node_valid(node)((unsigned int)(node) < nr_node_ids) > + > #endif /* _LINUX_NUMA_H */ > -- > 2.8.1 -- Michal Hocko SUSE Labs

Re: [PATCH (resend)] mm,oom: Defer dump_tasks() output.

2019-09-09 Thread Michal Hocko
On Mon 09-09-19 21:40:24, Tetsuo Handa wrote: > On 2019/09/09 20:36, Michal Hocko wrote: > > On Sat 07-09-19 19:54:32, Tetsuo Handa wrote: > >> (Resending to LKML as linux-mm ML dropped my posts.) > >> > >> If /proc/sys/vm/oom_dump_tasks != 0, dump_header

Re: [PATCH (resend)] mm,oom: Defer dump_tasks() output.

2019-09-09 Thread Michal Hocko
mp for an arbitrary amount of time because the worder context might be stalled for an arbitrary time. Even long after the oom is resolved. Not to mention that 1:1 (oom to tasks) information dumping is fundamentally broken. Any task might be on an oom list of different OOM contexts in different

Re: [PATCH] memcg, kmem: do not fail __GFP_NOFAIL charges

2019-09-09 Thread Michal Hocko
On Fri 06-09-19 11:24:55, Shakeel Butt wrote: > On Fri, Sep 6, 2019 at 5:56 AM Michal Hocko wrote: > > > > From: Michal Hocko > > > > Thomas has noticed the following NULL ptr dereference when using cgroup > > v1 kmem limit: > > BUG: unable to

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-09 Thread Michal Hocko
On Thu 05-09-19 01:44:12, sunqiuyang wrote: > > > > ____ > > From: Michal Hocko [mho...@kernel.org] > > Sent: Wednesday, September 04, 2019 20:52 > > To: sunqiuyang > > Cc: linux-kernel@vger.kernel.org; linux...@kvack.org >

Re: [patch for-5.3 0/4] revert immediate fallback to remote hugepages

2019-09-09 Thread Michal Hocko
umentation loop. We are more likely to make a forward progress. 5.3 managed to fix the worst case behavior, now let's talk about more clever tuning. You cannot expect such a tuning is an overnight work. This area is full of subtle side effects and few liners might have hard to predict consequence

Re: [PATCH 0/3] Remove __online_page_set_limits()

2019-09-09 Thread Michal Hocko
- > include/linux/memory_hotplug.h | 1 - > mm/memory_hotplug.c| 5 - > 4 files changed, 8 deletions(-) To the whole series Acked-by: Michal Hocko Thanks! -- Michal Hocko SUSE Labs

[PATCH] memcg, kmem: do not fail __GFP_NOFAIL charges

2019-09-06 Thread Michal Hocko
From: Michal Hocko Thomas has noticed the following NULL ptr dereference when using cgroup v1 kmem limit: BUG: unable to handle kernel NULL pointer dereference at 0008 PGD 0 P4D 0 Oops: [#1] PREEMPT SMP PTI CPU: 3 PID: 16923 Comm: gtk-update-icon Not tainted 4.19.51 #42 Hardware

Re: [RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-06 Thread Michal Hocko
On Fri 06-09-19 19:46:10, Tetsuo Handa wrote: > On 2019/09/05 23:08, Michal Hocko wrote: > > On Thu 05-09-19 22:39:47, Tetsuo Handa wrote: > > [...] > >> There is nothing that prevents users from enabling oom_dump_tasks by > >> sysctl. > >> But that re

Re: [PATCH v2] mm: emit tracepoint when RSS changes by threshold

2019-09-05 Thread Michal Hocko
[Add Steven] On Wed 04-09-19 12:28:08, Joel Fernandes wrote: > On Wed, Sep 4, 2019 at 11:38 AM Michal Hocko wrote: > > > > On Wed 04-09-19 11:32:58, Joel Fernandes wrote: [...] > > > but also for reducing > > > tracing noise. Flooding the traces mak

Re: [PATCH v2] mm: emit tracepoint when RSS changes by threshold

2019-09-05 Thread Michal Hocko
On Thu 05-09-19 10:14:52, Joel Fernandes wrote: > On Thu, Sep 05, 2019 at 12:54:24PM +0200, Michal Hocko wrote: > > On Wed 04-09-19 12:28:08, Joel Fernandes wrote: > > > On Wed, Sep 4, 2019 at 11:38 AM Michal Hocko wrote: > > > > > > > > On W

Re: [RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-05 Thread Michal Hocko
insisting is not a way to cooperate. -- Michal Hocko SUSE Labs

Re: [PATCH v2] mm: emit tracepoint when RSS changes by threshold

2019-09-05 Thread Michal Hocko
On Wed 04-09-19 12:28:08, Joel Fernandes wrote: > On Wed, Sep 4, 2019 at 11:38 AM Michal Hocko wrote: > > > > On Wed 04-09-19 11:32:58, Joel Fernandes wrote: > > > On Wed, Sep 04, 2019 at 10:45:08AM +0200, Michal Hocko wrote: > > > > On Tue 03-09-19 16:

Re: [rfc 3/4] mm, page_alloc: avoid expensive reclaim when compaction may not succeed

2019-09-05 Thread Michal Hocko
compact_result == COMPACT_DEFERRED) > + goto nopage; > + } > + > /* >* Checks for costly allocations with __GFP_NORETRY, which >* includes THP page fault allocations -- Michal Hocko SUSE Labs

Re: [PATCH v2] mm: emit tracepoint when RSS changes by threshold

2019-09-04 Thread Michal Hocko
On Wed 04-09-19 11:32:58, Joel Fernandes wrote: > On Wed, Sep 04, 2019 at 10:45:08AM +0200, Michal Hocko wrote: > > On Tue 03-09-19 16:09:05, Joel Fernandes (Google) wrote: > > > Useful to track how RSS is changing per TGID to detect spikes in RSS and > > > memory hogs.

Re: [PATCH v1 0/7] mm/memcontrol: recharge mlocked pages

2019-09-04 Thread Michal Hocko
| 14 +++ > mm/rmap.c |5 + > mm/vmscan.c | 17 ++-- > 12 files changed, 121 insertions(+), 52 deletions(-) > > -- > Signature -- Michal Hocko SUSE Labs

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-04 Thread Michal Hocko
Or did I miss something? Thanks, This talks about mapping rather than flags stored in the mapping. I can see that in tree migration handlers (z3fold_page_migrate, vmballoon_migratepage via balloon_page_delete, zs_page_migrate via reset_page) all reset the movable flag. I am not sure whether that is a documented requirement or just a coincidence. Maybe it should be documented. I would like to hear from Minchan. -- Michal Hocko SUSE Labs

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-04 Thread Michal Hocko
On Wed 04-09-19 07:59:17, Qian Cai wrote: > On Wed, 2019-09-04 at 10:25 +0200, Michal Hocko wrote: > > On Wed 04-09-19 16:00:42, Sergey Senozhatsky wrote: > > > On (09/04/19 15:41), Sergey Senozhatsky wrote: > > > > But the thing is different in case of dump_stack

Re: [PATCH v2] mm: emit tracepoint when RSS changes by threshold

2019-09-04 Thread Michal Hocko
er has to > + * advance before we trace it. Should be a power of 2. It is to reduce > unwanted > + * trace overhead. The counter is in units of number of pages. > + */ > +#define TRACE_MM_COUNTER_THRESHOLD 128 > + > +void mm_trace_rss_stat(int member, long count, long value) > +{ > + long thresh_mask = ~(TRACE_MM_COUNTER_THRESHOLD - 1); > + > + if (!trace_rss_stat_enabled()) > + return; > + > + /* Threshold roll-over, trace it */ > + if ((count & thresh_mask) != ((count - value) & thresh_mask)) > + trace_rss_stat(member, count); > +} > > #if defined(SPLIT_RSS_COUNTING) > > -- > 2.23.0.187.g17f5b7556c-goog -- Michal Hocko SUSE Labs

Re: [PATCH v5 1/2] mm: Allow the page cache to allocate large pages

2019-09-04 Thread Michal Hocko
On Tue 03-09-19 21:30:30, William Kucharski wrote: > > > > On Sep 3, 2019, at 5:57 AM, Michal Hocko wrote: > > > > On Mon 02-09-19 03:23:40, William Kucharski wrote: > >> Add an 'order' argument to __page_cache_alloc() and > >> do_read_cache_page(

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-04 Thread Michal Hocko
start_throttle(rate, number); /* here goes your operation */ end_throttle(); one operation is not considered done until the whole section ends. Or something along those lines. In this particular case we can increase the rate limit parameters of course but I think that longterm we need a better api. -- Michal Hocko SUSE Labs

Re: Re: Re: Re: [PATCH] mm: Add nr_free_highatomimic to fix incorrect watermatk routine

2019-09-04 Thread Michal Hocko
... > if (likely(!alloc_harder)) { > free_pages -= z->nr_reserved_highatomic; > ... > } > > However, free_page excludes the size already allocated by hiahtomic. > If highatomic block is small(Under 4GB RAM), it could be no problem. > But, the larger the memory size, the greater the chance of problems. > (Becasue highatomic size can be increased up to 1% of memory) I still do not understand. NR_FREE_PAGES should include the amount of hhighatomic reserves, right. So reducing the free_pages for normal allocations just makes sense. Or what do I miss? I am sorry but I find your reasoning really hard to follow. -- Michal Hocko SUSE Labs

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-04 Thread Michal Hocko
s); > > And this page will be added to another list. > Or, do you see any reason that the page cannot go through this path? The page shouldn't be __PageMovable after the migration is done. All the state should have been transfered to the new page IIUC. -- Michal Hocko SUSE Labs

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-04 Thread Michal Hocko
On Wed 04-09-19 15:41:44, Sergey Senozhatsky wrote: > On (09/04/19 08:15), Michal Hocko wrote: > > > If you look at the original report, the failed allocation dump_stack() is, > > > > > >   > > >  warn_alloc.cold.43+0x8a/0x148 > > >  __alloc_pages_n

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-04 Thread Michal Hocko
list_del(>lru) But the page has been migrated already and not freed yet because there is still a pin on it. So nobody should be touching it at this stage. Or do I still miss something? -- Michal Hocko SUSE Labs

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-04 Thread Michal Hocko
> On Tue, 2019-09-03 at 20:53 +0200, Michal Hocko wrote: > > On Tue 03-09-19 11:42:22, Qian Cai wrote: > > > On Tue, 2019-09-03 at 15:22 +0200, Michal Hocko wrote: > > > > On Fri 30-08-19 18:15:22, Eric Dumazet wrote: > > > > > If there is

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-04 Thread Michal Hocko
logd > irq_work_queue > __irq_work_queue_local > arch_irq_work_raise > apic->send_IPI_self(IRQ_WORK_VECTOR) > smp_irq_work_interrupt > exiting_irq > irq_exit > > and end up processing pending net_rx_action softirqs again which are plenty > due > to connected via ssh etc. -- Michal Hocko SUSE Labs

Re: [RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-03 Thread Michal Hocko
On Wed 04-09-19 05:52:43, Tetsuo Handa wrote: > On 2019/09/03 23:45, Michal Hocko wrote: > > It's primary purpose is > > to help analyse oom victim selection decision. > > I disagree, for I use the process list for understanding what / how many > processes are consu

Re: [PATCH v5 1/2] mm: Allow the page cache to allocate large pages

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 09:28:31, Matthew Wilcox wrote: > On Tue, Sep 03, 2019 at 02:19:52PM +0200, Michal Hocko wrote: > > On Tue 03-09-19 05:11:55, Matthew Wilcox wrote: > > > On Tue, Sep 03, 2019 at 01:57:48PM +0200, Michal Hocko wrote: > > > > On Mon 02-09-19 03:

Re: [PATCH v5 2/2] mm,thp: Add experimental config option RO_EXEC_FILEMAP_HUGE_FAULT_THP

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 08:10:15, Matthew Wilcox wrote: > On Tue, Sep 03, 2019 at 02:51:50PM +0200, Michal Hocko wrote: > > On Tue 03-09-19 05:22:08, Matthew Wilcox wrote: > > > On Tue, Sep 03, 2019 at 02:14:24PM +0200, Michal Hocko wrote: > > > > On Mon 02-09-19 03:

Re: [RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 11:32:58, Qian Cai wrote: > On Tue, 2019-09-03 at 17:13 +0200, Michal Hocko wrote: > > On Tue 03-09-19 11:02:46, Qian Cai wrote: > > > On Tue, 2019-09-03 at 16:45 +0200, Michal Hocko wrote: > > > > From: Michal Hocko > > > > > >

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 11:42:22, Qian Cai wrote: > On Tue, 2019-09-03 at 15:22 +0200, Michal Hocko wrote: > > On Fri 30-08-19 18:15:22, Eric Dumazet wrote: > > > If there is a risk of flooding the syslog, we should fix this generically > > > in mm layer, not adding hundr

Re: [RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 11:02:46, Qian Cai wrote: > On Tue, 2019-09-03 at 16:45 +0200, Michal Hocko wrote: > > From: Michal Hocko > > > > dump_tasks has been introduced by quite some time ago fef1bdd68c81 > > ("oom: add sysctl to enable task memory dump"). It's prima

[RFC PATCH] mm, oom: disable dump_tasks by default

2019-09-03 Thread Michal Hocko
From: Michal Hocko dump_tasks has been introduced by quite some time ago fef1bdd68c81 ("oom: add sysctl to enable task memory dump"). It's primary purpose is to help analyse oom victim selection decision. This has been certainly useful at times when the heuristic to chose a victim was

Re: [PATCH] net/skbuff: silence warnings under memory pressure

2019-09-03 Thread Michal Hocko
ferent parameters. Or maybe it is the ratelimiting which doesn't work here. Hard to tell and something to explore. -- Michal Hocko SUSE Labs

Re: [PATCH 1/1] mm/migrate: fix list corruption in migration of non-LRU movable pages

2019-09-03 Thread Michal Hocko
*/ > - list_del(>lru); > - > - /* >* Compaction can migrate also non-LRU pages which are >* not accounted to NR_ISOLATED_*. They can be recognized >* as __PageMovable > -- > 1.8.3.1 -- Michal Hocko SUSE Labs

Re: [PATCH v5 2/2] mm,thp: Add experimental config option RO_EXEC_FILEMAP_HUGE_FAULT_THP

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 05:22:08, Matthew Wilcox wrote: > On Tue, Sep 03, 2019 at 02:14:24PM +0200, Michal Hocko wrote: > > On Mon 02-09-19 03:23:41, William Kucharski wrote: > > > Add filemap_huge_fault() to attempt to satisfy page > > > faults on memory-mapped read-onl

Re: [PATCH v5 1/2] mm: Allow the page cache to allocate large pages

2019-09-03 Thread Michal Hocko
On Tue 03-09-19 05:11:55, Matthew Wilcox wrote: > On Tue, Sep 03, 2019 at 01:57:48PM +0200, Michal Hocko wrote: > > On Mon 02-09-19 03:23:40, William Kucharski wrote: > > > Add an 'order' argument to __page_cache_alloc() and > > > do_read_cache_page(). Ensure the allocat

Re: [PATCH v5 2/2] mm,thp: Add experimental config option RO_EXEC_FILEMAP_HUGE_FAULT_THP

2019-09-03 Thread Michal Hocko
addr = get_unmapped_area(file, addr, len, pgoff, flags); > + } else { > +#endif > + addr = get_unmapped_area(file, addr, len, pgoff, flags); > +#ifdef CONFIG_RO_EXEC_FILEMAP_HUGE_FAULT_THP > + } > +#endif > + > if (offset_in_pag

Re: [PATCH v5 1/2] mm: Allow the page cache to allocate large pages

2019-09-03 Thread Michal Hocko
gelist_reserve(struct ceph_pagelist *pl, > size_t space) > space = (space + PAGE_SIZE - 1) >> PAGE_SHIFT; /* conv to num pages */ > > while (space > pl->num_pages_free) { > - struct page *page = __page_cache_alloc(GFP_NOFS); > + struct

Re: Re: Re: [PATCH] mm: Add nr_free_highatomimic to fix incorrect watermatk routine

2019-09-03 Thread Michal Hocko
eservation. [...] > In my test, most case are using camera. So, memory usage is increased > momentarily, > it cause free page go to under low value of watermark. > If free page is under low and 0-order fail is occured, its normal operation. > But, although free page is higher than min, fail is occurred. > After fix routin for checking highatomic size, it's not reproduced. But you are stealing from the atomic reserves and thus defeating the purpose of it. -- Michal Hocko SUSE Labs

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-09-02 Thread Michal Hocko
ry why an early and more aggressive reclaim pays off. -- Michal Hocko SUSE Labs

Re: Re: [PATCH] mm: Add nr_free_highatomimic to fix incorrect watermatk routine

2019-09-02 Thread Michal Hocko
tack+0xc4/0x100 > warn_alloc+0x100/0x198 > __alloc_pages_nodemask+0x116c/0x1188 > do_swap_page+0x10c/0x6f0 > handle_pte_fault+0x12c/0xfe0 > handle_mm_fault+0x1d0/0x328 > do_page_fault+0x2a0/0x3e0 > do_translation_fault+0x44/0xa8 > do_mem_abort+0x4c/0xd0 > el1_da+0x24/0x84 > __arch_copy_to_user+0x5c/0x220 > binder_ioctl+0x20c/0x740 > compat_SyS_ioctl+0x128/0x248 > __sys_trace_return+0x0/0x4 -- Michal Hocko SUSE Labs

Re: poisoned pages do not play well in the buddy allocator

2019-08-30 Thread Michal Hocko
gly think we do really need to re-work that a bit, to make it more > simple. > > Since it will take a bit to have everything ready, I just wanted to > let you know. Thanks for the heads up and the work! The plan sounds great to me. -- Michal Hocko SUSE Labs

Re: [PATCH] arm64: numa: check the node id before accessing node_to_cpumask_map

2019-08-30 Thread Michal Hocko
On Fri 30-08-19 17:49:46, Yunsheng Lin wrote: > On 2019/8/30 16:39, Michal Hocko wrote: > > On Fri 30-08-19 16:08:14, Yunsheng Lin wrote: [...] > >> It seems the cpumask_of_node with CONFIG_DEBUG_PER_CPU_MAPS is used > >> to catch the erorr case and give a w

Re: [PATCH] mm: Add nr_free_highatomimic to fix incorrect watermatk routine

2019-08-30 Thread Michal Hocko
process context is killed by the oom killer. Also please note that atomic reserves are released when the memory pressure is high and we cannot reclaim any other memory. Have a look at unreserve_highatomic_pageblock called from should_reclaim_retry. -- Michal Hocko SUSE Labs

Re: [PATCH] arm64: numa: check the node id before accessing node_to_cpumask_map

2019-08-30 Thread Michal Hocko
On Fri 30-08-19 16:08:14, Yunsheng Lin wrote: > On 2019/8/30 14:44, Michal Hocko wrote: > > On Fri 30-08-19 14:35:26, Yunsheng Lin wrote: > >> On 2019/8/30 13:55, Michal Hocko wrote: > >>> On Fri 30-08-19 10:26:31, Yunsheng Lin wrote: > >>>> So

Re: [PATCH v2 3/6] mm/memory_hotplug: Process all zones when removing memory

2019-08-30 Thread Michal Hocko
On Fri 30-08-19 09:07:59, David Hildenbrand wrote: > On 30.08.19 08:47, Michal Hocko wrote: > > On Fri 30-08-19 08:20:32, David Hildenbrand wrote: > > [...] > >> Regarding shrink_zone_span(), I suspect it was introduced by > >> > >> d0dc12e86b31 (&

Re: [PATCH v2 3/6] mm/memory_hotplug: Process all zones when removing memory

2019-08-30 Thread Michal Hocko
eeded for Fixes tag. -- Michal Hocko SUSE Labs

Re: [PATCH] arm64: numa: check the node id before accessing node_to_cpumask_map

2019-08-30 Thread Michal Hocko
On Fri 30-08-19 14:35:26, Yunsheng Lin wrote: > On 2019/8/30 13:55, Michal Hocko wrote: > > On Fri 30-08-19 10:26:31, Yunsheng Lin wrote: > >> Some buggy bios may not set the device' numa id, and dev_to_node > >> will return -1, which may cause global-out-of-bounds e

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-30 Thread Michal Hocko
On Thu 29-08-19 10:03:21, Yang Shi wrote: > On Wed, Aug 28, 2019 at 9:02 AM Michal Hocko wrote: > > > > On Wed 28-08-19 17:46:59, Kirill A. Shutemov wrote: > > > On Wed, Aug 28, 2019 at 02:12:53PM +, Michal Hocko wrote: > > > > On Wed 28-08-1

Re: [PATCH v2 3/6] mm/memory_hotplug: Process all zones when removing memory

2019-08-30 Thread Michal Hocko
On Thu 29-08-19 18:59:31, David Hildenbrand wrote: > On 29.08.19 18:27, Michal Hocko wrote: [...] > > No rush, really... It seems this is quite unlikely event as most hotplug > > usecases simply online memory before removing it later on. > > > > I can trigger it relia

Re: [PATCH] arm64: numa: check the node id before accessing node_to_cpumask_map

2019-08-29 Thread Michal Hocko
100644 > --- a/arch/arm64/mm/numa.c > +++ b/arch/arm64/mm/numa.c > @@ -46,7 +46,7 @@ EXPORT_SYMBOL(node_to_cpumask_map); > */ > const struct cpumask *cpumask_of_node(int node) > { > - if (WARN_ON(node >= nr_node_ids)) > + if (WARN_ON(node >= nr_node_ids || node

Re: [PATCH] mm: memcontrol: fix percpu vmstats and vmevents flush

2019-08-29 Thread Michal Hocko
ontrol: flush percpu vmevents before releasing > memcg") > Fixes: c350a99ea2b1 ("mm: memcontrol: flush percpu vmstats before releasing > memcg") > Signed-off-by: Shakeel Butt > Cc: Roman Gushchin > Cc: Michal Hocko > Cc: Johannes Weiner > Cc: Vladimir Davydov &

Re: [PATCH] mm/vmalloc: move 'area->pages' after if statement

2019-08-29 Thread Michal Hocko
On Fri 30-08-19 12:57:16, Austin Kim wrote: > If !area->pages statement is true where memory allocation fails, > area is freed. > > In this case 'area->pages = pages' should not executed. > So move 'area->pages = pages' after if statement. > > Signed-off-by: Austi

[PATCH] mm, oom: consider present pages for the node size

2019-08-29 Thread Michal Hocko
From: Michal Hocko constrained_alloc calculates the size of the oom domain by using node_spanned_pages which is incorrect because this is the full range of the physical memory range that the numa node occupies rather than the memory that backs that range which is represented

Re: [PATCH v2 3/6] mm/memory_hotplug: Process all zones when removing memory

2019-08-29 Thread Michal Hocko
On Thu 29-08-19 17:54:35, David Hildenbrand wrote: > On 29.08.19 17:39, Michal Hocko wrote: > > On Mon 26-08-19 12:10:09, David Hildenbrand wrote: > >> It is easier than I though to trigger a kernel bug by removing memory that > >> was never onlined. With CONFIG_DEBUG_V

Re: [PATCH 00/10] OOM Debug print selection and additional information

2019-08-29 Thread Michal Hocko
On Thu 29-08-19 08:03:19, Edward Chron wrote: > On Thu, Aug 29, 2019 at 4:56 AM Michal Hocko wrote: [...] > > Or simply provide a hook with the oom_control to be called to report > > without replacing the whole oom killer behavior. That is not necessary. > > For very si

Re: [PATCH v2 3/6] mm/memory_hotplug: Process all zones when removing memory

2019-08-29 Thread Michal Hocko
proper fix is to remove it from that level down. Moreover the zone is only needed for the shrinking code and zone continuous thingy. The later belongs to offlining code unless I am missing something. I can see that you are removing zone parameter in a later patch but wouldn't it be just better to remove the whole zone thing in a single patch and have this as a bug fix for a rare bug with a fixes tag? -- Michal Hocko SUSE Labs

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-29 Thread Michal Hocko
as anyone got any ideas > with regards to what we can do about that? This is what we have the page cache for. -- Michal Hocko SUSE Labs

Re: [PATCH 00/10] OOM Debug print selection and additional information

2019-08-29 Thread Michal Hocko
On Thu 29-08-19 19:14:46, Tetsuo Handa wrote: > On 2019/08/29 16:11, Michal Hocko wrote: > > On Wed 28-08-19 12:46:20, Edward Chron wrote: > >> Our belief is if you really think eBPF is the preferred mechanism > >> then move OOM reporting to an eBPF. > > >

Re: [PATCH] mm: Remove NULL check in clear_hwpoisoned_pages()

2019-08-29 Thread Michal Hocko
hether section_mem_map could have been NULL before then but the important part is that NULL is not possible anymore as pfn_to_page shouldn't ever return NULL. > Signed-off-by: Alastair D'Silva Acked-by: Michal Hocko Thanks! > --- > mm/sparse.c | 3 --- > 1 file changed, 3 de

Re: [PATCH v3 0/6] hugetlb_cgroup: Add hugetlb_cgroup reservation limits

2019-08-29 Thread Michal Hocko
[Cc cgroups maintainers] On Wed 28-08-19 10:58:00, Mina Almasry wrote: > On Wed, Aug 28, 2019 at 4:23 AM Michal Hocko wrote: > > > > On Mon 26-08-19 16:32:34, Mina Almasry wrote: > > > mm/hugetlb.c | 493 -- &g

Re: [PATCH 00/10] OOM Debug print selection and additional information

2019-08-29 Thread Michal Hocko
ready responded to this kind of argumentation elsewhere. This is not a relevant argument for any kernel implementation. This is a data process management process. -- Michal Hocko SUSE Labs

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 17:46:59, Kirill A. Shutemov wrote: > On Wed, Aug 28, 2019 at 02:12:53PM +0000, Michal Hocko wrote: > > On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > > > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > > > > On Tue 27-

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > > On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > > > > > > > On 8/27/19 5:59 AM, Kirill A. Shutemov wrote: > > > > On Tue, Aug

Re: [PATCH v2] fs/proc/page: Skip uninitialized page when iterating page structures

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 09:46:21, Waiman Long wrote: > On 8/28/19 4:00 AM, Michal Hocko wrote: > > On Tue 27-08-19 16:22:38, Michal Hocko wrote: > >> Dan, isn't this something we have discussed recently? > > This was > > http://lkml.kernel.org/r/20190725023100.31

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-28 Thread Michal Hocko
ose watermarks that your monitoring > > tool can use to alter the reclaim behavior. > Just to confirm here, I am aware of one way which is to alter > min_kfree_bytes values. What other ways are there to alter watermarks > from user space? /proc/sys/vm/watermark_*factor -- Michal Hocko SUSE Labs

Re: [PATCH v3 0/6] hugetlb_cgroup: Add hugetlb_cgroup reservation limits

2019-08-28 Thread Michal Hocko
roupv1 is feature frozen and I am not aware of any plans to port the controller to v2. That all doesn't sound in favor of this change. Mike is the maintainer of the hugetlb code so I will defer to him to make a decision but I wouldn't recommend that. -- Michal Hocko SUSE Labs

Re: [PATCH 00/10] OOM Debug print selection and additional information

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 19:56:58, Tetsuo Handa wrote: > On 2019/08/28 19:32, Michal Hocko wrote: > >> Speak of my cases, those who take care of their systems are not developers. > >> And they afraid changing code that runs in kernel mode. They unlikely give > >> permis

Re: [PATCH 00/10] OOM Debug print selection and additional information

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 19:12:41, Tetsuo Handa wrote: > On 2019/08/28 16:08, Michal Hocko wrote: > > On Tue 27-08-19 19:47:22, Edward Chron wrote: > >> For production systems installing and updating EBPF scripts may someday > >> be very common, but I wonder how data center

Re: [PATCH v2] fs/proc/page: Skip uninitialized page when iterating page structures

2019-08-28 Thread Michal Hocko
On Tue 27-08-19 16:22:38, Michal Hocko wrote: > Dan, isn't this something we have discussed recently? This was http://lkml.kernel.org/r/20190725023100.31141-3-t-fukas...@vx.jp.nec.com and talked about /proc/kpageflags but this is essentially the same thing AFAIU. I hope we get a consist

<    12   13   14   15   16   17   18   19   20   21   >