Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-09 Thread Mel Gorman
On Wed, Apr 09, 2014 at 11:18:27AM -0400, Konrad Rzeszutek Wilk wrote: > On Tue, Apr 08, 2014 at 05:51:23PM +0100, Mel Gorman wrote: > > On Tue, Apr 08, 2014 at 12:02:50PM -0400, Konrad Rzeszutek Wilk wrote: > > > .snip.. > > > > >>> David Vrabel has a patchset which I presumed would be pulled thro

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-09 Thread Konrad Rzeszutek Wilk
On Tue, Apr 08, 2014 at 05:51:23PM +0100, Mel Gorman wrote: > On Tue, Apr 08, 2014 at 12:02:50PM -0400, Konrad Rzeszutek Wilk wrote: > > .snip.. > > > >>> David Vrabel has a patchset which I presumed would be pulled through > > > >the > > > >>> Xen tree this merge window: > > > >>> > > > >>> [PATCH

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-09 Thread Peter Zijlstra
On Wed, Apr 09, 2014 at 11:04:48AM -0400, Konrad Rzeszutek Wilk wrote: > On Tue, Apr 08, 2014 at 01:59:09PM -0700, H. Peter Anvin wrote: > > On 04/08/2014 01:51 PM, Steven Noonan wrote: > > > On Tue, Apr 8, 2014 at 8:16 AM, H. Peter Anvin wrote: > > >> > > >> > > >> Of course, it would also be pr

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-09 Thread Konrad Rzeszutek Wilk
On Tue, Apr 08, 2014 at 01:59:09PM -0700, H. Peter Anvin wrote: > On 04/08/2014 01:51 PM, Steven Noonan wrote: > > On Tue, Apr 8, 2014 at 8:16 AM, H. Peter Anvin wrote: > >> > >> > >> Of course, it would also be preferable if Amazon (or anything else) didn't > >> need Xen PV :( > > > > Well Ama

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread H. Peter Anvin
On 04/08/2014 01:51 PM, Steven Noonan wrote: > On Tue, Apr 8, 2014 at 8:16 AM, H. Peter Anvin wrote: >> >> >> Of course, it would also be preferable if Amazon (or anything else) didn't >> need Xen PV :( > > Well Amazon doesn't expose NUMA on PV, only on HVM guests. > Yes, but Amazon is one of

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread Steven Noonan
On Tue, Apr 8, 2014 at 8:16 AM, H. Peter Anvin wrote: > > > Of course, it would also be preferable if Amazon (or anything else) didn't > need Xen PV :( Well Amazon doesn't expose NUMA on PV, only on HVM guests. > On April 7, 2014 9:04:53 PM PDT, Steven Noonan wrote: >>On Mon, Apr 7, 2014 at 2

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread David Vrabel
On 08/04/14 17:16, H. Peter Anvin wrote: > On 04/08/2014 09:02 AM, Konrad Rzeszutek Wilk wrote: Amazon EC2 does have large memory instance types with NUMA exposed to the guest (e.g. c3.8xlarge, i2.8xlarge, etc), so it'd be preferable (to me anyway) if we didn't require !XEN. >>

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread Mel Gorman
On Tue, Apr 08, 2014 at 12:02:50PM -0400, Konrad Rzeszutek Wilk wrote: > .snip.. > > >>> David Vrabel has a patchset which I presumed would be pulled through > > >the > > >>> Xen tree this merge window: > > >>> > > >>> [PATCHv5 0/8] x86/xen: fixes for mapping high MMIO regions (and > > >remove > >

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread Mel Gorman
On Tue, Apr 08, 2014 at 09:16:49AM -0700, H. Peter Anvin wrote: > On 04/08/2014 09:02 AM, Konrad Rzeszutek Wilk wrote: > >>> > >>> Amazon EC2 does have large memory instance types with NUMA exposed to > >>> the guest (e.g. c3.8xlarge, i2.8xlarge, etc), so it'd be preferable > >>> (to me anyway) if

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread H. Peter Anvin
On 04/08/2014 09:02 AM, Konrad Rzeszutek Wilk wrote: >>> >>> Amazon EC2 does have large memory instance types with NUMA exposed to >>> the guest (e.g. c3.8xlarge, i2.8xlarge, etc), so it'd be preferable >>> (to me anyway) if we didn't require !XEN. > > What about the patch that David Vrabel posted

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread Konrad Rzeszutek Wilk
.snip.. > >>> David Vrabel has a patchset which I presumed would be pulled through > >the > >>> Xen tree this merge window: > >>> > >>> [PATCHv5 0/8] x86/xen: fixes for mapping high MMIO regions (and > >remove > >>> _PAGE_IOMAP) > >>> > >>> That frees up this bit. > >>> > >> > >> Thanks, I was not

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread H. Peter Anvin
Of course, it would also be preferable if Amazon (or anything else) didn't need Xen PV :( On April 7, 2014 9:04:53 PM PDT, Steven Noonan wrote: >On Mon, Apr 7, 2014 at 2:25 PM, Mel Gorman wrote: >> On Mon, Apr 07, 2014 at 12:42:40PM -0700, H. Peter Anvin wrote: >>> On 04/07/2014 12:36 PM, Cyr

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-08 Thread David Vrabel
On 07/04/14 20:36, Cyrill Gorcunov wrote: > On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: >> On 04/07/2014 11:28 AM, Mel Gorman wrote: >>> >>> I had considered the soft-dirty tracking usage of the same bit. I thought >>> I'd >>> be able to swizzle around it or a further worst cas

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Steven Noonan
On Mon, Apr 7, 2014 at 2:25 PM, Mel Gorman wrote: > On Mon, Apr 07, 2014 at 12:42:40PM -0700, H. Peter Anvin wrote: >> On 04/07/2014 12:36 PM, Cyrill Gorcunov wrote: >> > On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: >> >> On 04/07/2014 11:28 AM, Mel Gorman wrote: >> >>> >> >>> I

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Mel Gorman
On Mon, Apr 07, 2014 at 12:42:40PM -0700, H. Peter Anvin wrote: > On 04/07/2014 12:36 PM, Cyrill Gorcunov wrote: > > On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: > >> On 04/07/2014 11:28 AM, Mel Gorman wrote: > >>> > >>> I had considered the soft-dirty tracking usage of the same

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Mel Gorman
On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: > On 04/07/2014 11:28 AM, Mel Gorman wrote: > > > > I had considered the soft-dirty tracking usage of the same bit. I thought > > I'd > > be able to swizzle around it or a further worst case of having soft-dirty > > and > > automati

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Dave Hansen
On 04/07/2014 08:10 AM, Mel Gorman wrote: > +/* > + * Software bits ignored by the page table walker > + * At the time of writing, different levels have bits that are ignored. Due > + * to physical address limitations, bits 52:62 should be ignored for the PMD > + * and PTE levels and are available

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread H. Peter Anvin
On 04/07/2014 12:36 PM, Cyrill Gorcunov wrote: > On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: >> On 04/07/2014 11:28 AM, Mel Gorman wrote: >>> >>> I had considered the soft-dirty tracking usage of the same bit. I thought >>> I'd >>> be able to swizzle around it or a further wors

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Cyrill Gorcunov
On Mon, Apr 07, 2014 at 12:27:10PM -0700, H. Peter Anvin wrote: > On 04/07/2014 11:28 AM, Mel Gorman wrote: > > > > I had considered the soft-dirty tracking usage of the same bit. I thought > > I'd > > be able to swizzle around it or a further worst case of having soft-dirty > > and > > automati

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread H. Peter Anvin
On 04/07/2014 11:28 AM, Mel Gorman wrote: > > I had considered the soft-dirty tracking usage of the same bit. I thought I'd > be able to swizzle around it or a further worst case of having soft-dirty and > automatic NUMA balancing mutually exclusive. Unfortunately upon examination > it's not obvio

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Cyrill Gorcunov
On Mon, Apr 07, 2014 at 07:28:54PM +0100, Mel Gorman wrote: > > > I didn't bother spelling it out in case I gave the impression that I was > > > blaming Xen for the problem. As the bit is now changes, does it help > > > the Xen problem or cause another collision of some sort? There is no > > > gua

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Mel Gorman
On Mon, Apr 07, 2014 at 08:19:10PM +0400, Cyrill Gorcunov wrote: > On Mon, Apr 07, 2014 at 04:49:35PM +0100, Mel Gorman wrote: > > On Mon, Apr 07, 2014 at 04:32:39PM +0100, David Vrabel wrote: > > > On 07/04/14 16:10, Mel Gorman wrote: > > > > _PAGE_NUMA is currently an alias of _PROT_PROTNONE to t

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Cyrill Gorcunov
On Mon, Apr 07, 2014 at 04:49:35PM +0100, Mel Gorman wrote: > On Mon, Apr 07, 2014 at 04:32:39PM +0100, David Vrabel wrote: > > On 07/04/14 16:10, Mel Gorman wrote: > > > _PAGE_NUMA is currently an alias of _PROT_PROTNONE to trap NUMA hinting > > > faults. As the bit is shared care is taken that _P

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Mel Gorman
On Mon, Apr 07, 2014 at 04:32:39PM +0100, David Vrabel wrote: > On 07/04/14 16:10, Mel Gorman wrote: > > _PAGE_NUMA is currently an alias of _PROT_PROTNONE to trap NUMA hinting > > faults. As the bit is shared care is taken that _PAGE_NUMA is only used in > > places where _PAGE_PROTNONE could not r

Re: [PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread David Vrabel
On 07/04/14 16:10, Mel Gorman wrote: > _PAGE_NUMA is currently an alias of _PROT_PROTNONE to trap NUMA hinting > faults. As the bit is shared care is taken that _PAGE_NUMA is only used in > places where _PAGE_PROTNONE could not reach but this still causes problems > on Xen and conceptually difficul

[PATCH 2/3] x86: Define _PAGE_NUMA with unused physical address bits PMD and PTE levels

2014-04-07 Thread Mel Gorman
_PAGE_NUMA is currently an alias of _PROT_PROTNONE to trap NUMA hinting faults. As the bit is shared care is taken that _PAGE_NUMA is only used in places where _PAGE_PROTNONE could not reach but this still causes problems on Xen and conceptually difficult. Fundamentally, we only need the _PAGE_NUM