Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-30 Thread Kirill A. Shutemov
On Wed, Aug 28, 2019 at 06:02:24PM +0200, Michal Hocko wrote: > > > > > > Any idea about a bad case? > > > > Not really. > > > > How bad you want it to get? How many processes share the page? Access > > pattern? Locking situation? > > Let's say how hard a regular user can make this? It bumped

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-30 Thread Michal Hocko
On Thu 29-08-19 10:03:21, Yang Shi wrote: > On Wed, Aug 28, 2019 at 9:02 AM Michal Hocko wrote: > > > > On Wed 28-08-19 17:46:59, Kirill A. Shutemov wrote: > > > On Wed, Aug 28, 2019 at 02:12:53PM +, Michal Hocko wrote: > > > > On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > > > > > On

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-29 Thread Yang Shi
On Wed, Aug 28, 2019 at 9:02 AM Michal Hocko wrote: > > On Wed 28-08-19 17:46:59, Kirill A. Shutemov wrote: > > On Wed, Aug 28, 2019 at 02:12:53PM +, Michal Hocko wrote: > > > On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > > > > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 17:46:59, Kirill A. Shutemov wrote: > On Wed, Aug 28, 2019 at 02:12:53PM +, Michal Hocko wrote: > > On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > > > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > > > > On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > >

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Kirill A. Shutemov
On Wed, Aug 28, 2019 at 02:12:53PM +, Michal Hocko wrote: > On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > > > On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > > > > > > > > > > On 8/27/19 5:59 AM, Kirill A. Shutemov

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 17:03:29, Kirill A. Shutemov wrote: > On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > > On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > > > > > > > On 8/27/19 5:59 AM, Kirill A. Shutemov wrote: > > > > On Tue, Aug 27, 2019 at 03:17:39PM +0300, Kirill A. Shutemov

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Kirill A. Shutemov
On Wed, Aug 28, 2019 at 09:57:08AM +0200, Michal Hocko wrote: > On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > > > > On 8/27/19 5:59 AM, Kirill A. Shutemov wrote: > > > On Tue, Aug 27, 2019 at 03:17:39PM +0300, Kirill A. Shutemov wrote: > > > > On Tue, Aug 27, 2019 at 02:09:23PM +0200, Michal

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-28 Thread Michal Hocko
On Tue 27-08-19 10:06:20, Yang Shi wrote: > > > On 8/27/19 5:59 AM, Kirill A. Shutemov wrote: > > On Tue, Aug 27, 2019 at 03:17:39PM +0300, Kirill A. Shutemov wrote: > > > On Tue, Aug 27, 2019 at 02:09:23PM +0200, Michal Hocko wrote: > > > > On Tue 27-08-19 14:01:56, Vlastimil Babka wrote: > > >

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Yang Shi
On 8/27/19 5:59 AM, Kirill A. Shutemov wrote: On Tue, Aug 27, 2019 at 03:17:39PM +0300, Kirill A. Shutemov wrote: On Tue, Aug 27, 2019 at 02:09:23PM +0200, Michal Hocko wrote: On Tue 27-08-19 14:01:56, Vlastimil Babka wrote: On 8/27/19 1:02 PM, Kirill A. Shutemov wrote: On Tue, Aug 27,

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Kirill A. Shutemov
On Tue, Aug 27, 2019 at 03:17:39PM +0300, Kirill A. Shutemov wrote: > On Tue, Aug 27, 2019 at 02:09:23PM +0200, Michal Hocko wrote: > > On Tue 27-08-19 14:01:56, Vlastimil Babka wrote: > > > On 8/27/19 1:02 PM, Kirill A. Shutemov wrote: > > > > On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Kirill A. Shutemov
On Tue, Aug 27, 2019 at 02:09:23PM +0200, Michal Hocko wrote: > On Tue 27-08-19 14:01:56, Vlastimil Babka wrote: > > On 8/27/19 1:02 PM, Kirill A. Shutemov wrote: > > > On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal Hocko wrote: > > >> On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: > > >>>

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Michal Hocko
On Tue 27-08-19 14:01:56, Vlastimil Babka wrote: > On 8/27/19 1:02 PM, Kirill A. Shutemov wrote: > > On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal Hocko wrote: > >> On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: > >>> > >>> Unmapped completely pages will be freed with current code.

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Vlastimil Babka
On 8/27/19 1:02 PM, Kirill A. Shutemov wrote: > On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal Hocko wrote: >> On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: >>> >>> Unmapped completely pages will be freed with current code. Deferred split >>> only applies to partly mapped THPs: at least on

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Michal Hocko
On Tue 27-08-19 14:02:10, Kirill A. Shutemov wrote: > On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal Hocko wrote: > > On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: > > > On Mon, Aug 26, 2019 at 09:40:35AM +0200, Michal Hocko wrote: > > > > On Thu 22-08-19 18:29:34, Kirill A. Shutemov

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Kirill A. Shutemov
On Tue, Aug 27, 2019 at 08:01:39AM +0200, Michal Hocko wrote: > On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: > > On Mon, Aug 26, 2019 at 09:40:35AM +0200, Michal Hocko wrote: > > > On Thu 22-08-19 18:29:34, Kirill A. Shutemov wrote: > > > > On Thu, Aug 22, 2019 at 02:56:56PM +0200,

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Michal Hocko
On Tue 27-08-19 11:32:15, Kirill A. Shutemov wrote: > On Tue, Aug 27, 2019 at 07:59:41AM +0200, Michal Hocko wrote: > > > > > > IIUC deferred splitting is mostly a workaround for nasty locking > > > > > > issues > > > > > > during splitting, right? This is not really an optimization to cache > >

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Kirill A. Shutemov
On Tue, Aug 27, 2019 at 07:59:41AM +0200, Michal Hocko wrote: > > > > > IIUC deferred splitting is mostly a workaround for nasty locking > > > > > issues > > > > > during splitting, right? This is not really an optimization to cache > > > > > THPs for reuse or something like that. What is the

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Michal Hocko
On Mon 26-08-19 16:15:38, Kirill A. Shutemov wrote: > On Mon, Aug 26, 2019 at 09:40:35AM +0200, Michal Hocko wrote: > > On Thu 22-08-19 18:29:34, Kirill A. Shutemov wrote: > > > On Thu, Aug 22, 2019 at 02:56:56PM +0200, Vlastimil Babka wrote: > > > > On 8/22/19 10:04 AM, Michal Hocko wrote: > > >

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-27 Thread Michal Hocko
On Mon 26-08-19 21:27:38, Yang Shi wrote: > > > On 8/26/19 12:43 AM, Michal Hocko wrote: > > On Thu 22-08-19 08:33:40, Yang Shi wrote: > > > > > > On 8/22/19 1:04 AM, Michal Hocko wrote: > > > > On Thu 22-08-19 01:55:25, Yang Shi wrote: > > [...] > > > > > And, they seems very common with the

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-26 Thread Yang Shi
On 8/26/19 12:43 AM, Michal Hocko wrote: On Thu 22-08-19 08:33:40, Yang Shi wrote: On 8/22/19 1:04 AM, Michal Hocko wrote: On Thu 22-08-19 01:55:25, Yang Shi wrote: [...] And, they seems very common with the common workloads when THP is enabled. A simple run with MariaDB test of mmtest

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-26 Thread Kirill A. Shutemov
On Mon, Aug 26, 2019 at 09:40:35AM +0200, Michal Hocko wrote: > On Thu 22-08-19 18:29:34, Kirill A. Shutemov wrote: > > On Thu, Aug 22, 2019 at 02:56:56PM +0200, Vlastimil Babka wrote: > > > On 8/22/19 10:04 AM, Michal Hocko wrote: > > > > On Thu 22-08-19 01:55:25, Yang Shi wrote: > > > >>

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-26 Thread Michal Hocko
On Thu 22-08-19 08:33:40, Yang Shi wrote: > > > On 8/22/19 1:04 AM, Michal Hocko wrote: > > On Thu 22-08-19 01:55:25, Yang Shi wrote: [...] > > > And, they seems very common with the common workloads when THP is > > > enabled. A simple run with MariaDB test of mmtest with THP enabled as > > >

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-26 Thread Michal Hocko
On Thu 22-08-19 18:29:34, Kirill A. Shutemov wrote: > On Thu, Aug 22, 2019 at 02:56:56PM +0200, Vlastimil Babka wrote: > > On 8/22/19 10:04 AM, Michal Hocko wrote: > > > On Thu 22-08-19 01:55:25, Yang Shi wrote: > > >> Available memory is one of the most important metrics for memory > > >>

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Yang Shi
On 8/22/19 5:56 AM, Vlastimil Babka wrote: On 8/22/19 10:04 AM, Michal Hocko wrote: On Thu 22-08-19 01:55:25, Yang Shi wrote: Available memory is one of the most important metrics for memory pressure. I would disagree with this statement. It is a rough estimate that tells how much memory

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Kirill A. Shutemov
On Thu, Aug 22, 2019 at 02:56:56PM +0200, Vlastimil Babka wrote: > >> @@ -2816,11 +2821,14 @@ void free_transhuge_page(struct page *page) > >>ds_queue->split_queue_len--; > >>list_del(page_deferred_list(page)); > >>} > >> + __mod_node_page_state(page_pgdat(page),

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Yang Shi
On 8/22/19 1:04 AM, Michal Hocko wrote: On Thu 22-08-19 01:55:25, Yang Shi wrote: Available memory is one of the most important metrics for memory pressure. I would disagree with this statement. It is a rough estimate that tells how much memory you can allocate before going into a more

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Kirill A. Shutemov
On Thu, Aug 22, 2019 at 02:56:56PM +0200, Vlastimil Babka wrote: > On 8/22/19 10:04 AM, Michal Hocko wrote: > > On Thu 22-08-19 01:55:25, Yang Shi wrote: > >> Available memory is one of the most important metrics for memory > >> pressure. > > > > I would disagree with this statement. It is a

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Vlastimil Babka
On 8/22/19 10:04 AM, Michal Hocko wrote: > On Thu 22-08-19 01:55:25, Yang Shi wrote: >> Available memory is one of the most important metrics for memory >> pressure. > > I would disagree with this statement. It is a rough estimate that tells > how much memory you can allocate before going into a

Re: [v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-22 Thread Michal Hocko
On Thu 22-08-19 01:55:25, Yang Shi wrote: > Available memory is one of the most important metrics for memory > pressure. I would disagree with this statement. It is a rough estimate that tells how much memory you can allocate before going into a more expensive reclaim (mostly swapping).

[v2 PATCH -mm] mm: account deferred split THPs into MemAvailable

2019-08-21 Thread Yang Shi
Available memory is one of the most important metrics for memory pressure. Currently, the deferred split THPs are not accounted into available memory, but they are reclaimable actually, like reclaimable slabs. And, they seems very common with the common workloads when THP is enabled. A simple