Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Wed, Mar 28, 2018 at 05:00:24PM -0700, Alison Schofield wrote: > From: Alison Schofield > > Intel's Skylake Server CPUs have a different LLC topology than previous > generations. When in Sub-NUMA-Clustering (SNC) mode, the package is > divided into two "slices",

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Wed, Mar 28, 2018 at 05:00:24PM -0700, Alison Schofield wrote: > From: Alison Schofield > > Intel's Skylake Server CPUs have a different LLC topology than previous > generations. When in Sub-NUMA-Clustering (SNC) mode, the package is > divided into two "slices", each containing half the

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

Hi Alison, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on v4.16-rc7] [also build test WARNING on next-20180329] [cannot apply to tip/x86/core] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

Hi Alison, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on v4.16-rc7] [also build test WARNING on next-20180329] [cannot apply to tip/x86/core] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

RE: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

> Hmm, if we shrink llc-size by splitting it, do we also need to create a > unique "id" for each slice? RDT uses the cache id ... but it doesn't play well with cluster on die mode ... so our recommendation is to not use RDT if COD mode is enabled. If the result of these changes happens to be

RE: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

> Hmm, if we shrink llc-size by splitting it, do we also need to create a > unique "id" for each slice? RDT uses the cache id ... but it doesn't play well with cluster on die mode ... so our recommendation is to not use RDT if COD mode is enabled. If the result of these changes happens to be

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 09:09 AM, Peter Zijlstra wrote: > On Thu, Mar 29, 2018 at 07:34:58AM -0700, Dave Hansen wrote: >> What should we say, though? >> >> /* >>* false means 'c' does not share the LLC of 'o'. >>* Note: this decision gets reflected all the way >>

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 09:09 AM, Peter Zijlstra wrote: > On Thu, Mar 29, 2018 at 07:34:58AM -0700, Dave Hansen wrote: >> What should we say, though? >> >> /* >>* false means 'c' does not share the LLC of 'o'. >>* Note: this decision gets reflected all the way >>

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 07:34:58AM -0700, Dave Hansen wrote: > What should we say, though? > > /* > * false means 'c' does not share the LLC of 'o'. > * Note: this decision gets reflected all the way > * out to userspace > */ >

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 07:34:58AM -0700, Dave Hansen wrote: > What should we say, though? > > /* > * false means 'c' does not share the LLC of 'o'. > * Note: this decision gets reflected all the way > * out to userspace > */ >

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 07:37:29AM -0700, Dave Hansen wrote: > On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > > Further I think Dave argued that we should not change the llc-size, > > because while SNC presents a subset of the cache to local CPUs, for > > remote data the whole cache is still

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 07:37:29AM -0700, Dave Hansen wrote: > On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > > Further I think Dave argued that we should not change the llc-size, > > because while SNC presents a subset of the cache to local CPUs, for > > remote data the whole cache is still

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > Further I think Dave argued that we should not change the llc-size, > because while SNC presents a subset of the cache to local CPUs, for > remote data the whole cache is still available, again something some > applications might rely on. BTW, I may

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > Further I think Dave argued that we should not change the llc-size, > because while SNC presents a subset of the cache to local CPUs, for > remote data the whole cache is still available, again something some > applications might rely on. BTW, I may

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > The issue is that HPC workloads care about cache-size-per-cpu measure, > and the way they go about obtaining that is reading the cache-size and > dividing it by the h-weight of the cache-mask. That works, but only if the memory being accessed is

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:47 AM, Peter Zijlstra wrote: > The issue is that HPC workloads care about cache-size-per-cpu measure, > and the way they go about obtaining that is reading the cache-size and > dividing it by the h-weight of the cache-mask. That works, but only if the memory being accessed is

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 06:45:12AM -0700, Dave Hansen wrote: > On 03/29/2018 06:16 AM, Thomas Gleixner wrote: > >> This is OK at least on the hardware we are immediately concerned about > >> because the LLC sharing happens at both the slice and at the package > >> level, which are also NUMA

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 06:45:12AM -0700, Dave Hansen wrote: > On 03/29/2018 06:16 AM, Thomas Gleixner wrote: > >> This is OK at least on the hardware we are immediately concerned about > >> because the LLC sharing happens at both the slice and at the package > >> level, which are also NUMA

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 03:16:12PM +0200, Thomas Gleixner wrote: > On Wed, 28 Mar 2018, Alison Schofield wrote: > > From: Alison Schofield > > > > Intel's Skylake Server CPUs have a different LLC topology than previous > > generations. When in Sub-NUMA-Clustering

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Thu, Mar 29, 2018 at 03:16:12PM +0200, Thomas Gleixner wrote: > On Wed, 28 Mar 2018, Alison Schofield wrote: > > From: Alison Schofield > > > > Intel's Skylake Server CPUs have a different LLC topology than previous > > generations. When in Sub-NUMA-Clustering (SNC) mode, the package is > >

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:16 AM, Thomas Gleixner wrote: >> This is OK at least on the hardware we are immediately concerned about >> because the LLC sharing happens at both the slice and at the package >> level, which are also NUMA boundaries. > So that addresses the scheduler interaction, but it still

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On 03/29/2018 06:16 AM, Thomas Gleixner wrote: >> This is OK at least on the hardware we are immediately concerned about >> because the LLC sharing happens at both the slice and at the package >> level, which are also NUMA boundaries. > So that addresses the scheduler interaction, but it still

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Wed, 28 Mar 2018, Alison Schofield wrote: > From: Alison Schofield > > Intel's Skylake Server CPUs have a different LLC topology than previous > generations. When in Sub-NUMA-Clustering (SNC) mode, the package is > divided into two "slices", each containing half

Re: [PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

On Wed, 28 Mar 2018, Alison Schofield wrote: > From: Alison Schofield > > Intel's Skylake Server CPUs have a different LLC topology than previous > generations. When in Sub-NUMA-Clustering (SNC) mode, the package is > divided into two "slices", each containing half the cores, half the LLC, > and

[PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

From: Alison Schofield Intel's Skylake Server CPUs have a different LLC topology than previous generations. When in Sub-NUMA-Clustering (SNC) mode, the package is divided into two "slices", each containing half the cores, half the LLC, and one memory controller and

[PATCH v3] x86,sched: allow topologies where NUMA nodes share an LLC

From: Alison Schofield Intel's Skylake Server CPUs have a different LLC topology than previous generations. When in Sub-NUMA-Clustering (SNC) mode, the package is divided into two "slices", each containing half the cores, half the LLC, and one memory controller and each slice is enumerated to