Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Wed, 1 Aug 2012, Nikunj A Dadhania wrote: > Hi Stefano, > > On Wed, 1 Aug 2012 12:23:37 +0100, Stefano Stabellini > wrote: > > On Tue, 5 Jun 2012, Stefano Stabellini wrote: > > > On Tue, 5 Jun 2012, Peter Zijlstra wrote: > > > > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > > > > PeterZ, is 7/7 alright to be picked? > > > > > > > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > > > > though. But these two patches together should make it work for x86. > > > > > > > > > > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for > > > 3.6? > > > > > > > Hello Nikunj, > > what happened to this patch series? > > In particular I am interested in the following two patches: > > > > kvm,x86: RCU based table free > > Flush page-table pages before freeing them > > > > do you still intend to carry on with the development? Is there anything > > missing that is preventing them from going upstream? > > > I have posted a v3 on the kvm-list: > http://www.spinics.net/lists/kvm/msg76955.html > > I am carrying the above two patches(with one fix) in my series as well > for completeness. > > I have picked up the patches from PeterZ's "Unify TLB gather > implementations -v3" > http://article.gmane.org/gmane.linux.kernel.mm/81278 It is good to see that you are following up on this; since I didn't see any updates I got worried :) Cheers, Stefano -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
Hi Stefano, On Wed, 1 Aug 2012 12:23:37 +0100, Stefano Stabellini wrote: > On Tue, 5 Jun 2012, Stefano Stabellini wrote: > > On Tue, 5 Jun 2012, Peter Zijlstra wrote: > > > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > > > PeterZ, is 7/7 alright to be picked? > > > > > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > > > though. But these two patches together should make it work for x86. > > > > > > > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for > > 3.6? > > > > Hello Nikunj, > what happened to this patch series? > In particular I am interested in the following two patches: > > kvm,x86: RCU based table free > Flush page-table pages before freeing them > > do you still intend to carry on with the development? Is there anything > missing that is preventing them from going upstream? > I have posted a v3 on the kvm-list: http://www.spinics.net/lists/kvm/msg76955.html I am carrying the above two patches(with one fix) in my series as well for completeness. I have picked up the patches from PeterZ's "Unify TLB gather implementations -v3" http://article.gmane.org/gmane.linux.kernel.mm/81278 Regards Nikunj -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012, Stefano Stabellini wrote: > On Tue, 5 Jun 2012, Peter Zijlstra wrote: > > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > > PeterZ, is 7/7 alright to be picked? > > > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > > though. But these two patches together should make it work for x86. > > > > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for > 3.6? > Hello Nikunj, what happened to this patch series? In particular I am interested in the following two patches: kvm,x86: RCU based table free Flush page-table pages before freeing them do you still intend to carry on with the development? Is there anything missing that is preventing them from going upstream? Cheers, Stefano -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 05 Jun 2012 15:08:08 +0200, Peter Zijlstra wrote: > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > PeterZ, is 7/7 alright to be picked? > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > though. But these two patches together should make it work for x86. > I haven't added your SOB yet to this, though I have your name in mentioned "From". Should I add your SOB to this, I have added a minor fix for !CONFIG_HAVE_RCU_TABLE_FREE case? Regards Nikunj -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012, Peter Zijlstra wrote: > On Tue, 2012-06-05 at 14:26 +0100, Stefano Stabellini wrote: > > On Tue, 5 Jun 2012, Peter Zijlstra wrote: > > > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > > > PeterZ, is 7/7 alright to be picked? > > > > > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > > > though. But these two patches together should make it work for x86. > > > > > > > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for > > 3.6? > > I wouldn't rush this stuff, but if you're up for it and can convince > Linus that its all peachy you can try ;-) > Fair enough, I think I'll let Konrad make this call :-) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 2012-06-05 at 14:26 +0100, Stefano Stabellini wrote: > On Tue, 5 Jun 2012, Peter Zijlstra wrote: > > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > > PeterZ, is 7/7 alright to be picked? > > > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > > though. But these two patches together should make it work for x86. > > > > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for > 3.6? I wouldn't rush this stuff, but if you're up for it and can convince Linus that its all peachy you can try ;-) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012, Peter Zijlstra wrote: > On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > > PeterZ, is 7/7 alright to be picked? > > Yeah, I guess it is.. I haven't had time to rework my tlb series yet > though. But these two patches together should make it work for x86. > Good. Do you think they are OK for 3.5-rc2? Or is it better to wait for 3.6? -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012, Nikunj A Dadhania wrote: > On Tue, 5 Jun 2012 12:58:32 +0100, Stefano Stabellini > wrote: > > On Tue, 5 Jun 2012, Nikunj A Dadhania wrote: > > > On Tue, 5 Jun 2012 11:48:02 +0100, Stefano Stabellini > > > wrote: > > > > > > > > I am also interested in introducing HAVE_RCU_TABLE_FREE on x86 for Xen. > > > > Maybe we can pull our efforts together :-) > > > > > > > > Giving a look at this patch, it doesn't look like it is introducing > > > > CONFIG_HAVE_RCU_TABLE_FREE anywhere under arch/x86. > > > > How is the user supposed to set it? > > > > > > > I am doing that in the next patch only for KVM-ParavirtTLB flush, as > > > there is a bug in this implementation that patch [7/7] fixes. > > > > > > Refer following thread for details: > > > http://mid.gmane.org/1337254086.4281.26.camel@twins > > > http://mid.gmane.org/1337273959.4281.62.camel@twins > > > > Thanks, somehow I missed the 7/7 patch. > > > > From the Xen POV, your patch is fine because we'll just select > > PARAVIRT_TLB_FLUSH on CONFIG_XEN (see appended patch for completeness). > > > Selecting ARCH_HW_WALKS_PAGE_TABLE in place of PARAVIRT_TLB_FLUSH should > suffice. We would also need to select HAVE_RCU_TABLE_FREE, but it could be a good idea, not go through PARAVIRT_TLB_FLUSH. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012 12:58:32 +0100, Stefano Stabellini wrote: > On Tue, 5 Jun 2012, Nikunj A Dadhania wrote: > > On Tue, 5 Jun 2012 11:48:02 +0100, Stefano Stabellini > > wrote: > > > > > > I am also interested in introducing HAVE_RCU_TABLE_FREE on x86 for Xen. > > > Maybe we can pull our efforts together :-) > > > > > > Giving a look at this patch, it doesn't look like it is introducing > > > CONFIG_HAVE_RCU_TABLE_FREE anywhere under arch/x86. > > > How is the user supposed to set it? > > > > > I am doing that in the next patch only for KVM-ParavirtTLB flush, as > > there is a bug in this implementation that patch [7/7] fixes. > > > > Refer following thread for details: > > http://mid.gmane.org/1337254086.4281.26.camel@twins > > http://mid.gmane.org/1337273959.4281.62.camel@twins > > Thanks, somehow I missed the 7/7 patch. > > From the Xen POV, your patch is fine because we'll just select > PARAVIRT_TLB_FLUSH on CONFIG_XEN (see appended patch for completeness). > Selecting ARCH_HW_WALKS_PAGE_TABLE in place of PARAVIRT_TLB_FLUSH should suffice. > The main difference between the two approaches is that a kernel with > PARAVIRT_TLB_FLUSH and/or CONFIG_XEN enabled is going to have > HAVE_RCU_TABLE_FREE even when running on native. > > Are you proposing this series for 3.5? > If not (because it depends on ticketlocks and KVM Paravirt Spinlock > patches), > 3.6 I suppose as the merge window is already closed and we are having some discussions on PLE results. > could you extract patch 6/7 and 7/7 and send them out > separately? > > I am saying this because Xen needs the HAVE_RCU_TABLE_FREE fix even if > pv ticketlock are not accepted. This is an outstanding bug for us > unfortunately. > PeterZ has a patch in his tlb-unify: mm, x86: Add HAVE_RCU_TABLE_FREE support Implements optional HAVE_RCU_TABLE_FREE support for x86. This is useful for things like Xen and KVM where paravirt tlb flush means the software page table walkers like GUP-fast cannot rely on IRQs disabling like regular x86 can. Cc: Nikunj A Dadhania Cc: Jeremy Fitzhardinge Cc: Avi Kivity Signed-off-by: Peter Zijlstra http://git.kernel.org/?p=linux/kernel/git/peterz/mmu.git;a=commit;h=8a7e6fa5be9d2645c3394892c870113e6e5d9309 PeterZ, is 7/7 alright to be picked? Regards Nikunj -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 2012-06-05 at 18:34 +0530, Nikunj A Dadhania wrote: > PeterZ, is 7/7 alright to be picked? Yeah, I guess it is.. I haven't had time to rework my tlb series yet though. But these two patches together should make it work for x86. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012, Nikunj A Dadhania wrote: > On Tue, 5 Jun 2012 11:48:02 +0100, Stefano Stabellini > wrote: > > > > I am also interested in introducing HAVE_RCU_TABLE_FREE on x86 for Xen. > > Maybe we can pull our efforts together :-) > > > > Giving a look at this patch, it doesn't look like it is introducing > > CONFIG_HAVE_RCU_TABLE_FREE anywhere under arch/x86. > > How is the user supposed to set it? > > > I am doing that in the next patch only for KVM-ParavirtTLB flush, as > there is a bug in this implementation that patch [7/7] fixes. > > Refer following thread for details: > http://mid.gmane.org/1337254086.4281.26.camel@twins > http://mid.gmane.org/1337273959.4281.62.camel@twins Thanks, somehow I missed the 7/7 patch. >From the Xen POV, your patch is fine because we'll just select PARAVIRT_TLB_FLUSH on CONFIG_XEN (see appended patch for completeness). The main difference between the two approaches is that a kernel with PARAVIRT_TLB_FLUSH and/or CONFIG_XEN enabled is going to have HAVE_RCU_TABLE_FREE even when running on native. Are you proposing this series for 3.5? If not (because it depends on ticketlocks and KVM Paravirt Spinlock patches), could you extract patch 6/7 and 7/7 and send them out separately? I am saying this because Xen needs the HAVE_RCU_TABLE_FREE fix even if pv ticketlock are not accepted. This is an outstanding bug for us unfortunately. --- xen: select PARAVIRT_TLB_FLUSH if SMP At the moment get_user_pages_fast is unsafe on Xen, because it relies on the fact that flush_tlb_others sends an IPI to flush the tlb but xen_flush_tlb_others doesn't send any IPIs and always returns succesfully and immediately. Select PARAVIRT_TLB_FLUSH, that enables an RCU lock to protect this kind of software pagetable walks (also see HAVE_RCU_TABLE_FREE and include/asm-generic/tlb.h). Signed-off-by: Stefano Stabellini diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig index fdce49c..18c9876 100644 --- a/arch/x86/xen/Kconfig +++ b/arch/x86/xen/Kconfig @@ -6,6 +6,7 @@ config XEN bool "Xen guest support" select PARAVIRT select PARAVIRT_CLOCK + select PARAVIRT_TLB_FLUSH if SMP depends on X86_64 || (X86_32 && X86_PAE && !X86_VISWS) depends on X86_CMPXCHG && X86_TSC help -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Tue, 5 Jun 2012 11:48:02 +0100, Stefano Stabellini wrote: > > I am also interested in introducing HAVE_RCU_TABLE_FREE on x86 for Xen. > Maybe we can pull our efforts together :-) > > Giving a look at this patch, it doesn't look like it is introducing > CONFIG_HAVE_RCU_TABLE_FREE anywhere under arch/x86. > How is the user supposed to set it? > I am doing that in the next patch only for KVM-ParavirtTLB flush, as there is a bug in this implementation that patch [7/7] fixes. Refer following thread for details: http://mid.gmane.org/1337254086.4281.26.camel@twins http://mid.gmane.org/1337273959.4281.62.camel@twins Regards Nikunj -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 6/7] kvm,x86: RCU based table free
On Mon, 4 Jun 2012, Nikunj A. Dadhania wrote:
> From: Peter Zijlstra
>
> get_user_pages_fast() depends on the IPI to hold off page table
> teardown while they are locklessly walked with interrupts disabled.
> If a vcpu were to be preempted while in this critical section, another
> vcpu tearing down page tables would go ahead and destroy them. when
> the preempted vcpu resumes it then touches the freed pages.
>
> Using HAVE_RCU_TABLE_FREE:
>
> By using call_rcu_sched() to free the page-tables you'd need to
> receive and process at least one tick on the woken up cpu after the
> freeing, but since the in-progress gup_fast() will have IRQs disabled
> this will be delayed.
>
> http://article.gmane.org/gmane.linux.kernel/1290539
>
> Tested-by: Nikunj A. Dadhania
> Signed-off-by: Nikunj A. Dadhania
>
> arch/powerpc/include/asm/pgalloc.h |1 +
> arch/s390/mm/pgtable.c |1 +
> arch/sparc/include/asm/pgalloc_64.h |1 +
> arch/x86/mm/pgtable.c |6 +++---
> include/asm-generic/tlb.h |9 +
> mm/memory.c |7 +++
> 6 files changed, 22 insertions(+), 3 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/pgalloc.h
> b/arch/powerpc/include/asm/pgalloc.h
> index bf301ac..c33ae79 100644
> --- a/arch/powerpc/include/asm/pgalloc.h
> +++ b/arch/powerpc/include/asm/pgalloc.h
> @@ -49,6 +49,7 @@ static inline void __tlb_remove_table(void *_table)
>
> pgtable_free(table, shift);
> }
> +#define __tlb_remove_table __tlb_remove_table
> #else /* CONFIG_SMP */
> static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table,
> unsigned shift)
> {
> diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c
> index 6e765bf..7029ed7 100644
> --- a/arch/s390/mm/pgtable.c
> +++ b/arch/s390/mm/pgtable.c
> @@ -730,6 +730,7 @@ void __tlb_remove_table(void *_table)
> else
> free_pages((unsigned long) table, ALLOC_ORDER);
> }
> +#define __tlb_remove_table __tlb_remove_table
>
> static void tlb_remove_table_smp_sync(void *arg)
> {
> diff --git a/arch/sparc/include/asm/pgalloc_64.h
> b/arch/sparc/include/asm/pgalloc_64.h
> index 40b2d7a..d10913a 100644
> --- a/arch/sparc/include/asm/pgalloc_64.h
> +++ b/arch/sparc/include/asm/pgalloc_64.h
> @@ -106,6 +106,7 @@ static inline void __tlb_remove_table(void *_table)
> is_page = true;
> pgtable_free(table, is_page);
> }
> +#define __tlb_remove_table __tlb_remove_table
> #else /* CONFIG_SMP */
> static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table,
> bool is_page)
> {
> diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
> index 8573b83..34fa168 100644
> --- a/arch/x86/mm/pgtable.c
> +++ b/arch/x86/mm/pgtable.c
> @@ -51,21 +51,21 @@ void ___pte_free_tlb(struct mmu_gather *tlb, struct page
> *pte)
> {
> pgtable_page_dtor(pte);
> paravirt_release_pte(page_to_pfn(pte));
> - tlb_remove_page(tlb, pte);
> + tlb_remove_table(tlb, pte);
> }
>
> #if PAGETABLE_LEVELS > 2
> void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd)
> {
> paravirt_release_pmd(__pa(pmd) >> PAGE_SHIFT);
> - tlb_remove_page(tlb, virt_to_page(pmd));
> + tlb_remove_table(tlb, virt_to_page(pmd));
> }
>
> #if PAGETABLE_LEVELS > 3
> void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud)
> {
> paravirt_release_pud(__pa(pud) >> PAGE_SHIFT);
> - tlb_remove_page(tlb, virt_to_page(pud));
> + tlb_remove_table(tlb, virt_to_page(pud));
> }
> #endif /* PAGETABLE_LEVELS > 3 */
> #endif /* PAGETABLE_LEVELS > 2 */
> diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
> index f96a5b5..9ac30f7 100644
> --- a/include/asm-generic/tlb.h
> +++ b/include/asm-generic/tlb.h
> @@ -19,6 +19,8 @@
> #include
> #include
>
> +static inline void tlb_remove_page(struct mmu_gather *tlb, struct page
> *page);
> +
> #ifdef CONFIG_HAVE_RCU_TABLE_FREE
> /*
> * Semi RCU freeing of the page directories.
> @@ -60,6 +62,13 @@ struct mmu_table_batch {
> extern void tlb_table_flush(struct mmu_gather *tlb);
> extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
>
> +#else
> +
> +static inline void tlb_remove_table(struct mmu_gather *tlb, void *page)
> +{
> + tlb_remove_page(tlb, page);
> +}
> +
> #endif
>
> /*
> diff --git a/mm/memory.c b/mm/memory.c
> index 6105f47..c12685d 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -297,6 +297,13 @@ int __tlb_remove_page(struct mmu_gather *tlb, struct
> page *page)
> * See the comment near struct mmu_table_batch.
> */
>
> +#ifndef __tlb_remove_table
> +static void __tlb_remove_table(void *table)
> +{
> + free_page_and_swap_cache(table);
> +}
> +#endif
> +
> static void tlb_remove_table_smp_sync(void *arg)
> {
> /* Simply deliver the interrupt */
I am also interested in introducing HAVE_RCU_TABLE_FREE on x86 for Xen.
Maybe we can pull our efforts together :-)
Giving a look at th
[PATCH v2 6/7] kvm,x86: RCU based table free
From: Peter Zijlstra
get_user_pages_fast() depends on the IPI to hold off page table
teardown while they are locklessly walked with interrupts disabled.
If a vcpu were to be preempted while in this critical section, another
vcpu tearing down page tables would go ahead and destroy them. when
the preempted vcpu resumes it then touches the freed pages.
Using HAVE_RCU_TABLE_FREE:
By using call_rcu_sched() to free the page-tables you'd need to
receive and process at least one tick on the woken up cpu after the
freeing, but since the in-progress gup_fast() will have IRQs disabled
this will be delayed.
http://article.gmane.org/gmane.linux.kernel/1290539
Tested-by: Nikunj A. Dadhania
Signed-off-by: Nikunj A. Dadhania
---
arch/powerpc/include/asm/pgalloc.h |1 +
arch/s390/mm/pgtable.c |1 +
arch/sparc/include/asm/pgalloc_64.h |1 +
arch/x86/mm/pgtable.c |6 +++---
include/asm-generic/tlb.h |9 +
mm/memory.c |7 +++
6 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/pgalloc.h
b/arch/powerpc/include/asm/pgalloc.h
index bf301ac..c33ae79 100644
--- a/arch/powerpc/include/asm/pgalloc.h
+++ b/arch/powerpc/include/asm/pgalloc.h
@@ -49,6 +49,7 @@ static inline void __tlb_remove_table(void *_table)
pgtable_free(table, shift);
}
+#define __tlb_remove_table __tlb_remove_table
#else /* CONFIG_SMP */
static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table,
unsigned shift)
{
diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c
index 6e765bf..7029ed7 100644
--- a/arch/s390/mm/pgtable.c
+++ b/arch/s390/mm/pgtable.c
@@ -730,6 +730,7 @@ void __tlb_remove_table(void *_table)
else
free_pages((unsigned long) table, ALLOC_ORDER);
}
+#define __tlb_remove_table __tlb_remove_table
static void tlb_remove_table_smp_sync(void *arg)
{
diff --git a/arch/sparc/include/asm/pgalloc_64.h
b/arch/sparc/include/asm/pgalloc_64.h
index 40b2d7a..d10913a 100644
--- a/arch/sparc/include/asm/pgalloc_64.h
+++ b/arch/sparc/include/asm/pgalloc_64.h
@@ -106,6 +106,7 @@ static inline void __tlb_remove_table(void *_table)
is_page = true;
pgtable_free(table, is_page);
}
+#define __tlb_remove_table __tlb_remove_table
#else /* CONFIG_SMP */
static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table, bool
is_page)
{
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 8573b83..34fa168 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -51,21 +51,21 @@ void ___pte_free_tlb(struct mmu_gather *tlb, struct page
*pte)
{
pgtable_page_dtor(pte);
paravirt_release_pte(page_to_pfn(pte));
- tlb_remove_page(tlb, pte);
+ tlb_remove_table(tlb, pte);
}
#if PAGETABLE_LEVELS > 2
void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd)
{
paravirt_release_pmd(__pa(pmd) >> PAGE_SHIFT);
- tlb_remove_page(tlb, virt_to_page(pmd));
+ tlb_remove_table(tlb, virt_to_page(pmd));
}
#if PAGETABLE_LEVELS > 3
void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud)
{
paravirt_release_pud(__pa(pud) >> PAGE_SHIFT);
- tlb_remove_page(tlb, virt_to_page(pud));
+ tlb_remove_table(tlb, virt_to_page(pud));
}
#endif /* PAGETABLE_LEVELS > 3 */
#endif /* PAGETABLE_LEVELS > 2 */
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index f96a5b5..9ac30f7 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -19,6 +19,8 @@
#include
#include
+static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page);
+
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
/*
* Semi RCU freeing of the page directories.
@@ -60,6 +62,13 @@ struct mmu_table_batch {
extern void tlb_table_flush(struct mmu_gather *tlb);
extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
+#else
+
+static inline void tlb_remove_table(struct mmu_gather *tlb, void *page)
+{
+ tlb_remove_page(tlb, page);
+}
+
#endif
/*
diff --git a/mm/memory.c b/mm/memory.c
index 6105f47..c12685d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -297,6 +297,13 @@ int __tlb_remove_page(struct mmu_gather *tlb, struct page
*page)
* See the comment near struct mmu_table_batch.
*/
+#ifndef __tlb_remove_table
+static void __tlb_remove_table(void *table)
+{
+ free_page_and_swap_cache(table);
+}
+#endif
+
static void tlb_remove_table_smp_sync(void *arg)
{
/* Simply deliver the interrupt */
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
