Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Michal Hocko
On Thu 28-03-19 12:40:14, Yang Shi wrote: > > > On 3/28/19 12:12 PM, Michal Hocko wrote: > > On Thu 28-03-19 11:58:57, Yang Shi wrote: > > > > > > On 3/27/19 11:58 PM, Michal Hocko wrote: > > > > On Wed 27-03-19 19:09:10, Yang Shi wrote: > > > > > One question, when doing demote and promote we

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Yang Shi
On 3/28/19 12:12 PM, Michal Hocko wrote: On Thu 28-03-19 11:58:57, Yang Shi wrote: On 3/27/19 11:58 PM, Michal Hocko wrote: On Wed 27-03-19 19:09:10, Yang Shi wrote: One question, when doing demote and promote we need define a path, for example, DRAM <-> PMEM (assume two tier memory).

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Michal Hocko
On Thu 28-03-19 11:58:57, Yang Shi wrote: > > > On 3/27/19 11:58 PM, Michal Hocko wrote: > > On Wed 27-03-19 19:09:10, Yang Shi wrote: > > > One question, when doing demote and promote we need define a path, for > > > example, DRAM <-> PMEM (assume two tier memory). When determining what > > >

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Yang Shi
On 3/27/19 11:58 PM, Michal Hocko wrote: On Wed 27-03-19 19:09:10, Yang Shi wrote: One question, when doing demote and promote we need define a path, for example, DRAM <-> PMEM (assume two tier memory). When determining what nodes are "DRAM" nodes, does it make sense to assume the nodes with

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Dan Williams
On Wed, Mar 27, 2019 at 7:09 PM Yang Shi wrote: > On 3/27/19 1:09 PM, Michal Hocko wrote: > > On Wed 27-03-19 11:59:28, Yang Shi wrote: > >> > >> On 3/27/19 10:34 AM, Dan Williams wrote: > >>> On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: > On Tue 26-03-19 19:58:56, Yang Shi wrote: >

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-28 Thread Michal Hocko
On Wed 27-03-19 19:09:10, Yang Shi wrote: > One question, when doing demote and promote we need define a path, for > example, DRAM <-> PMEM (assume two tier memory). When determining what nodes > are "DRAM" nodes, does it make sense to assume the nodes with both cpu and > memory are DRAM nodes

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Yang Shi
On 3/27/19 1:09 PM, Michal Hocko wrote: On Wed 27-03-19 11:59:28, Yang Shi wrote: On 3/27/19 10:34 AM, Dan Williams wrote: On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: On Tue 26-03-19 19:58:56, Yang Shi wrote: [...] It is still NUMA, users still can see all the NUMA nodes. No,

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Dave Hansen
On 3/27/19 1:35 PM, Matthew Wilcox wrote: > > pmem1 --- node1 --- node2 --- pmem2 > | \ / | > |X| > | / \ | > pmem3 --- node3 --- node4 --- pmem4 > > which I could actually see someone building with normal DRAM, and we > should probably handle

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Matthew Wilcox
On Wed, Mar 27, 2019 at 10:34:11AM -0700, Dan Williams wrote: > On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: > > No, Linux NUMA implementation makes all numa nodes available by default > > and provides an API to opt-in for more fine tuning. What you are > > suggesting goes against that

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Dave Hansen
On 3/27/19 11:59 AM, Yang Shi wrote: > In real production environment we don't know what kind of applications > would end up on PMEM (DRAM may be full, allocation fall back to PMEM) > then have unexpected performance degradation. I understand to have > mempolicy to choose to avoid it. But, there

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Michal Hocko
On Wed 27-03-19 11:59:28, Yang Shi wrote: > > > On 3/27/19 10:34 AM, Dan Williams wrote: > > On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: > > > On Tue 26-03-19 19:58:56, Yang Shi wrote: [...] > > > > It is still NUMA, users still can see all the NUMA nodes. > > > No, Linux NUMA

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Yang Shi
On 3/27/19 10:34 AM, Dan Williams wrote: On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: On Tue 26-03-19 19:58:56, Yang Shi wrote: On 3/26/19 11:37 AM, Michal Hocko wrote: On Tue 26-03-19 11:33:17, Yang Shi wrote: On 3/26/19 6:58 AM, Michal Hocko wrote: On Sat 23-03-19 12:44:25,

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Dan Williams
On Wed, Mar 27, 2019 at 2:01 AM Michal Hocko wrote: > > On Tue 26-03-19 19:58:56, Yang Shi wrote: > > > > > > On 3/26/19 11:37 AM, Michal Hocko wrote: > > > On Tue 26-03-19 11:33:17, Yang Shi wrote: > > > > > > > > On 3/26/19 6:58 AM, Michal Hocko wrote: > > > > > On Sat 23-03-19 12:44:25, Yang

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-27 Thread Michal Hocko
On Tue 26-03-19 19:58:56, Yang Shi wrote: > > > On 3/26/19 11:37 AM, Michal Hocko wrote: > > On Tue 26-03-19 11:33:17, Yang Shi wrote: > > > > > > On 3/26/19 6:58 AM, Michal Hocko wrote: > > > > On Sat 23-03-19 12:44:25, Yang Shi wrote: > > > > > With Dave Hansen's patches merged into Linus's

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-26 Thread Yang Shi
On 3/26/19 11:37 AM, Michal Hocko wrote: On Tue 26-03-19 11:33:17, Yang Shi wrote: On 3/26/19 6:58 AM, Michal Hocko wrote: On Sat 23-03-19 12:44:25, Yang Shi wrote: With Dave Hansen's patches merged into Linus's tree

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-26 Thread Michal Hocko
On Tue 26-03-19 11:33:17, Yang Shi wrote: > > > On 3/26/19 6:58 AM, Michal Hocko wrote: > > On Sat 23-03-19 12:44:25, Yang Shi wrote: > > > With Dave Hansen's patches merged into Linus's tree > > > > > >

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-26 Thread Yang Shi
On 3/26/19 6:58 AM, Michal Hocko wrote: On Sat 23-03-19 12:44:25, Yang Shi wrote: With Dave Hansen's patches merged into Linus's tree https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 PMEM could be hot plugged as NUMA

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-26 Thread Michal Hocko
On Sat 23-03-19 12:44:25, Yang Shi wrote: > > With Dave Hansen's patches merged into Linus's tree > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 > > PMEM could be hot plugged as NUMA node now. But, how to use PMEM as

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-26 Thread Jonathan Cameron
On Mon, 25 Mar 2019 16:37:07 -0700 Dan Williams wrote: > On Mon, Mar 25, 2019 at 4:09 PM Brice Goglin wrote: > > > > > > Le 25/03/2019 à 20:29, Dan Williams a écrit : > > > Perhaps "path" might be a suitable replacement identifier rather than > > > type. I.e. memory that originates from an

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Dan Williams
On Mon, Mar 25, 2019 at 4:09 PM Brice Goglin wrote: > > > Le 25/03/2019 à 20:29, Dan Williams a écrit : > > Perhaps "path" might be a suitable replacement identifier rather than > > type. I.e. memory that originates from an ACPI.NFIT root device is > > likely "pmem". > > > Could work. > > What

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Brice Goglin
Le 25/03/2019 à 20:29, Dan Williams a écrit : > Perhaps "path" might be a suitable replacement identifier rather than > type. I.e. memory that originates from an ACPI.NFIT root device is > likely "pmem". Could work. What kind of "path" would we get for other types of memory? (DDR,

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Yang Shi
On 3/25/19 9:15 AM, Brice Goglin wrote: Le 23/03/2019 à 05:44, Yang Shi a écrit : With Dave Hansen's patches merged into Linus's tree https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 PMEM could be hot plugged as NUMA

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Dan Williams
On Mon, Mar 25, 2019 at 10:45 AM Brice Goglin wrote: > > Le 25/03/2019 à 17:56, Dan Williams a écrit : > > > > I'm generally against the concept that a "pmem" or "type" flag should > > indicate anything about the expected performance of the address range. > > The kernel should explicitly look to

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Brice Goglin
Le 25/03/2019 à 17:56, Dan Williams a écrit : > > I'm generally against the concept that a "pmem" or "type" flag should > indicate anything about the expected performance of the address range. > The kernel should explicitly look to the HMAT for performance data and > not otherwise make type-based

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Dan Williams
On Mon, Mar 25, 2019 at 9:15 AM Brice Goglin wrote: > > > Le 23/03/2019 à 05:44, Yang Shi a écrit : > > With Dave Hansen's patches merged into Linus's tree > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 > > > > PMEM

Re: [RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-25 Thread Brice Goglin
Le 23/03/2019 à 05:44, Yang Shi a écrit : > With Dave Hansen's patches merged into Linus's tree > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 > > PMEM could be hot plugged as NUMA node now. But, how to use PMEM as NUMA

[RFC PATCH 0/10] Another Approach to Use PMEM as NUMA Node

2019-03-22 Thread Yang Shi
With Dave Hansen's patches merged into Linus's tree https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308fd01d9fb33a16f64d2fd95f8830a4 PMEM could be hot plugged as NUMA node now. But, how to use PMEM as NUMA node effectively and efficiently is still a