Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2018-02-05 Thread Pavel Tatashin
2017 5:50 AM >>>>> To: Pavel Tatashin >>>>> Cc: Michal Hocko ; YASUAKI ISHIMATSU >>>>> ; Andrew Morton ; >>>>> Linux Memory Management List ; linux- >>>>> ker...@vger.kernel.org; Sanagi, Koki ; Steve >>>>> Si

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2018-02-05 Thread Masayoshi Mizuma
, 2017 5:50 AM >>>> To: Pavel Tatashin >>>> Cc: Michal Hocko ; YASUAKI ISHIMATSU >>>> ; Andrew Morton ; >>>> Linux Memory Management List ; linux- >>>> ker...@vger.kernel.org; Sanagi, Koki ; Steve >>>> Sistare >

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2018-01-31 Thread Pavel Tatashin
List ; linux- >>>ker...@vger.kernel.org; Sanagi, Koki ; Steve >>>Sistare >>>Subject: Re: [PATCH] mm, meminit: Serially initialise deferred memory if >>>trace_buf_size is specified >>> >>>On Wed, Nov 29, 2017 at 10:41:59PM -0500, Pavel Tatash

RE: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2018-01-31 Thread koki.san...@us.fujitsu.com
SU >>; Andrew Morton ; >>Linux Memory Management List ; linux- >>ker...@vger.kernel.org; Sanagi, Koki ; Steve >>Sistare >>Subject: Re: [PATCH] mm, meminit: Serially initialise deferred memory if >>trace_buf_size is specified >> >>On Wed, Nov 29, 2017

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-12-06 Thread Mel Gorman
On Wed, Nov 29, 2017 at 10:41:59PM -0500, Pavel Tatashin wrote: > Hi Mel, > > Thank you very much for your feedback, my replies below: > > > A lack of involvement from admins is indeed desirable. For example, > > while I might concede on using a disable-everything-switch, I would not > > be happy

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-29 Thread Pavel Tatashin
>> Hi Mel, >> >> The forth approach is the best as it is seamless for admins and >> engineers, it will also work on any system configuration with any >> parameters without any special involvement. > > Apart from what-mel-said, I'd be concerned that this failsafe would > almost never get tested. We

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-29 Thread Pavel Tatashin
Hi Mel, Thank you very much for your feedback, my replies below: > A lack of involvement from admins is indeed desirable. For example, > while I might concede on using a disable-everything-switch, I would not > be happy to introduce a switch that specified how much memory per node > to initialise

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-20 Thread Andrew Morton
On Fri, 17 Nov 2017 13:19:56 -0500 Pavel Tatashin wrote: > On Thu, Nov 16, 2017 at 5:06 AM, Mel Gorman > wrote: > > 4. Put a check into the page allocator slowpath that triggers serialised > >init if the system is booting and an allocation is about to fail. It > >would be such a cold p

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-17 Thread Mel Gorman
On Fri, Nov 17, 2017 at 01:19:56PM -0500, Pavel Tatashin wrote: > On Thu, Nov 16, 2017 at 5:06 AM, Mel Gorman > wrote: > > 4. Put a check into the page allocator slowpath that triggers serialised > >init if the system is booting and an allocation is about to fail. It > >would be such a co

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-17 Thread Pavel Tatashin
On Thu, Nov 16, 2017 at 5:06 AM, Mel Gorman wrote: > 4. Put a check into the page allocator slowpath that triggers serialised >init if the system is booting and an allocation is about to fail. It >would be such a cold path that it would never be noticable although it >would leave dead

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-16 Thread Mel Gorman
On Thu, Nov 16, 2017 at 09:54:33AM +0100, Michal Hocko wrote: > On Wed 15-11-17 14:17:52, YASUAKI ISHIMATSU wrote: > > Hi Michal and Mel, > > > > To reproduce the issue, I specified the large trace buffer. The issue also > > occurs with > > trace_buf_size=12M and movable_node on 4.14.0. > > This

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-16 Thread Michal Hocko
On Wed 15-11-17 14:17:52, YASUAKI ISHIMATSU wrote: > Hi Michal and Mel, > > To reproduce the issue, I specified the large trace buffer. The issue also > occurs with > trace_buf_size=12M and movable_node on 4.14.0. This is still 10x more than the default. Why do you need it in the first place? Yo

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-16 Thread Mel Gorman
On Wed, Nov 15, 2017 at 11:49:19AM -0800, Andrew Morton wrote: > On Wed, 15 Nov 2017 08:55:56 + Mel Gorman > wrote: > > > Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was > > specified on a machine with many CPUs. The kernel tried to allocate 38.4GB > > but only 16GB w

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Andrew Morton
On Wed, 15 Nov 2017 08:55:56 + Mel Gorman wrote: > Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was > specified on a machine with many CPUs. The kernel tried to allocate 38.4GB > but only 16GB was available due to deferred memory initialisation. > > The allocation con

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread YASUAKI ISHIMATSU
Hi Michal and Mel, To reproduce the issue, I specified the large trace buffer. The issue also occurs with trace_buf_size=12M and movable_node on 4.14.0. In my system, there are 384 CPUs and 8 nodes. So when not using movable_node boot option, kernel can use about 16GB memory for trace buffer. S

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Michal Hocko
On Wed 15-11-17 14:43:14, Mel Gorman wrote: > On Wed, Nov 15, 2017 at 03:28:16PM +0100, Michal Hocko wrote: > > On Wed 15-11-17 14:13:29, Mel Gorman wrote: > > [...] > > > I doubt anyone well. Even the original reporter appeared to pick that > > > particular value just to trigger the OOM. > > > >

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Mel Gorman
On Wed, Nov 15, 2017 at 03:28:16PM +0100, Michal Hocko wrote: > On Wed 15-11-17 14:13:29, Mel Gorman wrote: > [...] > > I doubt anyone well. Even the original reporter appeared to pick that > > particular value just to trigger the OOM. > > Then why do we care at all? The trace buffer size can be c

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Michal Hocko
On Wed 15-11-17 14:13:29, Mel Gorman wrote: [...] > I doubt anyone well. Even the original reporter appeared to pick that > particular value just to trigger the OOM. Then why do we care at all? The trace buffer size can be configured from the userspace if it is not sufficiently large IIRC. -- Mi

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Mel Gorman
On Wed, Nov 15, 2017 at 12:55:59PM +0100, Michal Hocko wrote: > On Wed 15-11-17 08:55:56, Mel Gorman wrote: > > Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was > > specified on a machine with many CPUs. The kernel tried to allocate 38.4GB > > but only 16GB was available due

Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Michal Hocko
On Wed 15-11-17 08:55:56, Mel Gorman wrote: > Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was > specified on a machine with many CPUs. The kernel tried to allocate 38.4GB > but only 16GB was available due to deferred memory initialisation. > > The allocation context is with

[PATCH] mm, meminit: Serially initialise deferred memory if trace_buf_size is specified

2017-11-15 Thread Mel Gorman
Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was specified on a machine with many CPUs. The kernel tried to allocate 38.4GB but only 16GB was available due to deferred memory initialisation. The allocation context is within smp_init() so there are no opportunities to do the