Re: [PATCH] mm: Speed up mremap on large regions

2018-10-12 Thread Juergen Gross
On 12/10/2018 07:34, Jann Horn wrote: > On Fri, Oct 12, 2018 at 7:29 AM Juergen Gross wrote: >> On 12/10/2018 05:21, Jann Horn wrote: >>> +cc xen maintainers and kvm folks >>> >>> On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) >>> wrote: Android needs to mremap large regions of

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-12 Thread Juergen Gross
On 12/10/2018 07:34, Jann Horn wrote: > On Fri, Oct 12, 2018 at 7:29 AM Juergen Gross wrote: >> On 12/10/2018 05:21, Jann Horn wrote: >>> +cc xen maintainers and kvm folks >>> >>> On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) >>> wrote: Android needs to mremap large regions of

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-12 Thread Paolo Bonzini
On 12/10/2018 05:21, Jann Horn wrote: > I don't know how this interacts with shadow paging implementations. Shadow paging simply uses MMU notifiers and that does not assume that PTE invalidation is atomic. The invalidate_range_start and invalidate_range_end calls are not affected by Joel's

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-12 Thread Paolo Bonzini
On 12/10/2018 05:21, Jann Horn wrote: > I don't know how this interacts with shadow paging implementations. Shadow paging simply uses MMU notifiers and that does not assume that PTE invalidation is atomic. The invalidate_range_start and invalidate_range_end calls are not affected by Joel's

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Jann Horn
On Fri, Oct 12, 2018 at 7:29 AM Juergen Gross wrote: > On 12/10/2018 05:21, Jann Horn wrote: > > +cc xen maintainers and kvm folks > > > > On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) > > wrote: > >> Android needs to mremap large regions of memory during memory management > >> related

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Jann Horn
On Fri, Oct 12, 2018 at 7:29 AM Juergen Gross wrote: > On 12/10/2018 05:21, Jann Horn wrote: > > +cc xen maintainers and kvm folks > > > > On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) > > wrote: > >> Android needs to mremap large regions of memory during memory management > >> related

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Juergen Gross
On 12/10/2018 05:21, Jann Horn wrote: > +cc xen maintainers and kvm folks > > On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) > wrote: >> Android needs to mremap large regions of memory during memory management >> related operations. The mremap system call can be really slow if THP is >>

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Juergen Gross
On 12/10/2018 05:21, Jann Horn wrote: > +cc xen maintainers and kvm folks > > On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) > wrote: >> Android needs to mremap large regions of memory during memory management >> related operations. The mremap system call can be really slow if THP is >>

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Jann Horn
+cc xen maintainers and kvm folks On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables,

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Jann Horn
+cc xen maintainers and kvm folks On Fri, Oct 12, 2018 at 4:40 AM Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables,

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Joel Fernandes
On Thu, Oct 11, 2018 at 11:11:11AM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 08:14:19AM +0300, Kirill A. Shutemov wrote: > > On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > > > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Joel Fernandes
On Thu, Oct 11, 2018 at 11:11:11AM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 08:14:19AM +0300, Kirill A. Shutemov wrote: > > On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > > > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Michal Hocko
On Thu 11-10-18 11:17:19, Kirill A. Shutemov wrote: [...] > > The thing is its quite a lot of change, I wrote a coccinelle script to do it > > tree wide, following is the diffstat: > > 48 files changed, 91 insertions(+), 124 deletions(-) > > > > Imagine then having to add the address argument

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Michal Hocko
On Thu 11-10-18 11:17:19, Kirill A. Shutemov wrote: [...] > > The thing is its quite a lot of change, I wrote a coccinelle script to do it > > tree wide, following is the diffstat: > > 48 files changed, 91 insertions(+), 124 deletions(-) > > > > Imagine then having to add the address argument

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Kirill A. Shutemov
On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: > > On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > > > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > > > On Tue, Oct

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Kirill A. Shutemov
On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: > > On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > > > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > > > On Tue, Oct

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Kirill A. Shutemov
On Thu, Oct 11, 2018 at 08:14:19AM +0300, Kirill A. Shutemov wrote: > On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > > b/arch/powerpc/include/asm/book3s/64/pgalloc.h > > index 391ed2c3b697..8a33f2044923 100644 > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-11 Thread Kirill A. Shutemov
On Thu, Oct 11, 2018 at 08:14:19AM +0300, Kirill A. Shutemov wrote: > On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > > b/arch/powerpc/include/asm/book3s/64/pgalloc.h > > index 391ed2c3b697..8a33f2044923 100644 > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Kirill A. Shutemov
On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > b/arch/powerpc/include/asm/book3s/64/pgalloc.h > index 391ed2c3b697..8a33f2044923 100644 > --- a/arch/powerpc/include/asm/book3s/64/pgalloc.h > +++

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Kirill A. Shutemov
On Wed, Oct 10, 2018 at 05:46:18PM -0700, Joel Fernandes wrote: > diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h > b/arch/powerpc/include/asm/book3s/64/pgalloc.h > index 391ed2c3b697..8a33f2044923 100644 > --- a/arch/powerpc/include/asm/book3s/64/pgalloc.h > +++

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 5:46 PM, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: [...] >> >> My worry is that some architecture has to allocate page table differently >> depending on virtual address (due to aliasing or something). Original page >> table

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 5:46 PM, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: [...] >> >> My worry is that some architecture has to allocate page table differently >> depending on virtual address (due to aliasing or something). Original page >> table

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: > On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 01:00:11PM +0300, Kirill A. Shutemov wrote: > On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Kirill A. Shutemov
On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-10 Thread Kirill A. Shutemov
On Tue, Oct 09, 2018 at 04:04:47PM -0700, Joel Fernandes wrote: > On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes
On Tue, Oct 09, 2018 at 02:38:59PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 13:14:00 -0700 "Joel Fernandes (Google)" > wrote: > > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes
On Tue, Oct 09, 2018 at 02:38:59PM -0700, Andrew Morton wrote: > On Tue, 9 Oct 2018 13:14:00 -0700 "Joel Fernandes (Google)" > wrote: > > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes
On Wed, Oct 10, 2018 at 01:02:22AM +0300, Kirill A. Shutemov wrote: > On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Kirill A. Shutemov
On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Kirill A. Shutemov
On Tue, Oct 09, 2018 at 01:14:00PM -0700, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each >

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Andrew Morton
On Tue, 9 Oct 2018 13:14:00 -0700 "Joel Fernandes (Google)" wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each > pte

Re: [PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Andrew Morton
On Tue, 9 Oct 2018 13:14:00 -0700 "Joel Fernandes (Google)" wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each > pte

[PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes (Google)
Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow if THP is not enabled. The bottleneck is move_page_tables, which is copying each pte at a time, and can be really slow across a large map. Turning on THP may not

[PATCH] mm: Speed up mremap on large regions

2018-10-09 Thread Joel Fernandes (Google)
Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow if THP is not enabled. The bottleneck is move_page_tables, which is copying each pte at a time, and can be really slow across a large map. Turning on THP may not