Rather than unconditionally reloading cr3, only do so if the pud we're
updating is within the active pgd.
This eliminates TLB flushes most of the time. The
performance-critical uses of pud_clear are during execve and exit, but
in those cases cr3 is referring to some other pagetable. The only
other use of pud_clear is during a large (1Gbyte+) munmap, and those
are sufficiently rare that a couple of cr3 reloads won't hurt.
Signed-off-by: Jeremy Fitzhardinge <[EMAIL PROTECTED]>
---
include/asm-x86/pgtable-3level.h | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/include/asm-x86/pgtable-3level.h b/include/asm-x86/pgtable-3level.h
--- a/include/asm-x86/pgtable-3level.h
+++ b/include/asm-x86/pgtable-3level.h
@@ -93,17 +93,20 @@ static inline void native_pmd_clear(pmd_
static inline void pud_clear(pud_t *pudp)
{
+ unsigned long pgd;
+
set_pud(pudp, __pud(0));
/*
* Pentium-II erratum A13: in PAE mode we explicitly have to flush
* the TLB via cr3 if the top-level pgd is changed...
*
- * XXX I don't think we need to worry about this here, since
- * when clearing the pud, the calling code needs to flush the
- * tlb anyway. But do it now for safety's sake. - jsgf
+ * Make sure the pud entry we're updating is within the
+ * current pgd to avoid unnecessary TLB flushes.
*/
- write_cr3(read_cr3());
+ pgd = read_cr3();
+ if (__pa(pudp) >= pgd && __pa(pudp) < (pgd +
sizeof(pgd_t)*PTRS_PER_PGD))
+ write_cr3(pgd);
}
#define pud_page(pud) \
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
