Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-28 Thread Fengguang Wu
Hi Jonathan, Thanks for showing the gap on tracking hot accesses from devices. On Mon, Jan 28, 2019 at 05:42:39PM +, Jonathan Cameron wrote: On Wed, 2 Jan 2019 12:21:10 + Jonathan Cameron wrote: On Fri, 28 Dec 2018 20:52:24 +0100 Michal Hocko wrote: > [Ccing Mel and Andrea] > Hi,

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-28 Thread Jonathan Cameron
On Wed, 2 Jan 2019 12:21:10 + Jonathan Cameron wrote: > On Fri, 28 Dec 2018 20:52:24 +0100 > Michal Hocko wrote: > > > [Ccing Mel and Andrea] > > Hi, I just wanted to highlight this section as I didn't feel we really addressed this in the earlier conversation. > * Hot pages may not be

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Jonathan Cameron
On Tue, 8 Jan 2019 15:52:56 +0100 Michal Hocko wrote: > On Wed 02-01-19 12:21:10, Jonathan Cameron wrote: > [...] > > So ideally I'd love this set to head in a direction that helps me tick off > > at least some of the above usecases and hopefully have some visibility on > > how to address the oth

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Jerome Glisse
On Thu, Jan 10, 2019 at 05:50:01PM +0100, Michal Hocko wrote: > On Thu 10-01-19 11:25:56, Jerome Glisse wrote: > > On Fri, Dec 28, 2018 at 08:52:24PM +0100, Michal Hocko wrote: > > > [Ccing Mel and Andrea] > > > > > > On Fri 28-12-18 21:31:11, Wu Fengguang wrote: > > > > > > > I haven't looked at

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Jerome Glisse
On Thu, Jan 10, 2019 at 05:42:48PM +0100, Michal Hocko wrote: > On Thu 10-01-19 10:53:17, Jerome Glisse wrote: > > On Tue, Jan 08, 2019 at 03:52:56PM +0100, Michal Hocko wrote: > > > On Wed 02-01-19 12:21:10, Jonathan Cameron wrote: > > > [...] > > > > So ideally I'd love this set to head in a dire

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Michal Hocko
On Thu 10-01-19 11:25:56, Jerome Glisse wrote: > On Fri, Dec 28, 2018 at 08:52:24PM +0100, Michal Hocko wrote: > > [Ccing Mel and Andrea] > > > > On Fri 28-12-18 21:31:11, Wu Fengguang wrote: > > > > > > I haven't looked at the implementation yet but if you are proposing > > > > > > a > > > > > >

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Michal Hocko
On Thu 10-01-19 10:53:17, Jerome Glisse wrote: > On Tue, Jan 08, 2019 at 03:52:56PM +0100, Michal Hocko wrote: > > On Wed 02-01-19 12:21:10, Jonathan Cameron wrote: > > [...] > > > So ideally I'd love this set to head in a direction that helps me tick off > > > at least some of the above usecases a

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Jerome Glisse
On Fri, Dec 28, 2018 at 08:52:24PM +0100, Michal Hocko wrote: > [Ccing Mel and Andrea] > > On Fri 28-12-18 21:31:11, Wu Fengguang wrote: > > > > > I haven't looked at the implementation yet but if you are proposing a > > > > > special cased zone lists then this is something CDM (Coherent Device >

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-10 Thread Jerome Glisse
On Tue, Jan 08, 2019 at 03:52:56PM +0100, Michal Hocko wrote: > On Wed 02-01-19 12:21:10, Jonathan Cameron wrote: > [...] > > So ideally I'd love this set to head in a direction that helps me tick off > > at least some of the above usecases and hopefully have some visibility on > > how to address t

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-08 Thread Michal Hocko
On Wed 02-01-19 10:12:04, Dave Hansen wrote: > On 12/28/18 12:41 AM, Michal Hocko wrote: > >> > >> It can be done in kernel page reclaim path, near the anonymous page > >> swap out point. Instead of swapping out, we now have the option to > >> migrate cold pages to PMEM NUMA nodes. > > OK, this mak

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-08 Thread Michal Hocko
On Wed 02-01-19 12:21:10, Jonathan Cameron wrote: [...] > So ideally I'd love this set to head in a direction that helps me tick off > at least some of the above usecases and hopefully have some visibility on > how to address the others moving forwards, Is it sufficient to have such a memory marke

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-03 Thread Mel Gorman
On Fri, Dec 28, 2018 at 08:52:24PM +0100, Michal Hocko wrote: > [Ccing Mel and Andrea] > > On Fri 28-12-18 21:31:11, Wu Fengguang wrote: > > > > > I haven't looked at the implementation yet but if you are proposing a > > > > > special cased zone lists then this is something CDM (Coherent Device >

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2019-01-02 Thread Dave Hansen
On 12/28/18 12:41 AM, Michal Hocko wrote: >> >> It can be done in kernel page reclaim path, near the anonymous page >> swap out point. Instead of swapping out, we now have the option to >> migrate cold pages to PMEM NUMA nodes. > OK, this makes sense to me except I am not sure this is something tha

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Michal Hocko
[Ccing Mel and Andrea] On Fri 28-12-18 21:31:11, Wu Fengguang wrote: > > > > I haven't looked at the implementation yet but if you are proposing a > > > > special cased zone lists then this is something CDM (Coherent Device > > > > Memory) was trying to do two years ago and there was quite some >

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Michal Hocko
[Cc Mel and Andrea - the thread started http://lkml.kernel.org/r/20181226131446.330864...@intel.com] On Fri 28-12-18 21:15:42, Wu Fengguang wrote: > On Fri, Dec 28, 2018 at 01:15:15PM +0100, Michal Hocko wrote: > > On Fri 28-12-18 17:42:08, Wu Fengguang wrote: > > [...] > > > Those look unnecessar

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Yang Shi
On Fri, Dec 28, 2018 at 5:31 AM Fengguang Wu wrote: > > >> > I haven't looked at the implementation yet but if you are proposing a > >> > special cased zone lists then this is something CDM (Coherent Device > >> > Memory) was trying to do two years ago and there was quite some > >> > skepticism in

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Fengguang Wu
> I haven't looked at the implementation yet but if you are proposing a > special cased zone lists then this is something CDM (Coherent Device > Memory) was trying to do two years ago and there was quite some > skepticism in the approach. It looks we are pretty different than CDM. :) We creating

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Fengguang Wu
On Fri, Dec 28, 2018 at 01:15:15PM +0100, Michal Hocko wrote: On Fri 28-12-18 17:42:08, Wu Fengguang wrote: [...] Those look unnecessary complexities for this post. This v2 patchset mainly fulfills our first milestone goal: a minimal viable solution that's relatively clean to backport. Even when

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Michal Hocko
On Fri 28-12-18 17:42:08, Wu Fengguang wrote: [...] > Those look unnecessary complexities for this post. This v2 patchset > mainly fulfills our first milestone goal: a minimal viable solution > that's relatively clean to backport. Even when preparing for new > upstreamable versions, it may be good

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Fengguang Wu
On Fri, Dec 28, 2018 at 09:41:05AM +0100, Michal Hocko wrote: On Fri 28-12-18 13:08:06, Wu Fengguang wrote: [...] Optimization: do hot/cold page tracking and migration = Since PMEM is slower than DRAM, we need to make sure hot pages go to DRAM

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-28 Thread Michal Hocko
On Fri 28-12-18 13:08:06, Wu Fengguang wrote: [...] > Optimization: do hot/cold page tracking and migration > = > > Since PMEM is slower than DRAM, we need to make sure hot pages go to > DRAM and cold pages stay in PMEM, to get the best out of PM

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-27 Thread Fengguang Wu
On Thu, Dec 27, 2018 at 09:31:58PM +0100, Michal Hocko wrote: On Wed 26-12-18 21:14:46, Wu Fengguang wrote: This is an attempt to use NVDIMM/PMEM as volatile NUMA memory that's transparent to normal applications and virtual machines. The code is still in active development. It's provided for ea

Re: [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-27 Thread Michal Hocko
On Wed 26-12-18 21:14:46, Wu Fengguang wrote: > This is an attempt to use NVDIMM/PMEM as volatile NUMA memory that's > transparent to normal applications and virtual machines. > > The code is still in active development. It's provided for early design > review. So can we get a high level descrip

[RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration

2018-12-26 Thread Fengguang Wu
This is an attempt to use NVDIMM/PMEM as volatile NUMA memory that's transparent to normal applications and virtual machines. The code is still in active development. It's provided for early design review. Key functionalities: 1) create and describe PMEM NUMA node for NVDIMM memory 2) dumb /proc