Re: [PATCH] mm: cache largest vma

2013-11-13 Thread Peter Zijlstra
On Mon, Nov 11, 2013 at 12:47:28PM -0800, Davidlohr Bueso wrote: > On Mon, 2013-11-11 at 13:04 +0100, Ingo Molnar wrote: > in find_vma() but the cost of maintaining it comes free. I just ran into > a similar idea from 2 years ago: > http://lkml.indiana.edu/hypermail/linux/kernel/1112.1/01352.html

Re: [PATCH] mm: cache largest vma

2013-11-13 Thread Ingo Molnar
* Davidlohr Bueso wrote: > On Mon, 2013-11-11 at 12:47 -0800, Davidlohr Bueso wrote: > > On Mon, 2013-11-11 at 13:04 +0100, Ingo Molnar wrote: > > > * Michel Lespinasse wrote: > > > > > > > On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso > > > > wrote: > > > > > 2) Oracle Data mining (4K pa

Re: [PATCH] mm: cache largest vma

2013-11-13 Thread Davidlohr Bueso
On Mon, 2013-11-11 at 12:47 -0800, Davidlohr Bueso wrote: > On Mon, 2013-11-11 at 13:04 +0100, Ingo Molnar wrote: > > * Michel Lespinasse wrote: > > > > > On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso wrote: > > > > 2) Oracle Data mining (4K pages) > > > > ++--

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Ingo Molnar
* Davidlohr Bueso wrote: > > Or is access to varied in the Oracle case that it's missing the cache > > all the time, because the rbtree causes many cachemisses as the > > separate nodes are accessed during an rb-walk? > > Similar to get_cycles(), is there anyway to quickly measure the amount

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Davidlohr Bueso
On Mon, 2013-11-11 at 21:47 +0100, Ingo Molnar wrote: > * Davidlohr Bueso wrote: > > > On Mon, 2013-11-11 at 13:01 +0100, Ingo Molnar wrote: > > > * Davidlohr Bueso wrote: > > > > > > > Hi Ingo, > > > > > > > > On Mon, 2013-11-04 at 08:36 +0100, Ingo Molnar wrote: > > > > > * Davidlohr Bueso

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Davidlohr Bueso
On Mon, 2013-11-11 at 13:04 +0100, Ingo Molnar wrote: > * Michel Lespinasse wrote: > > > On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso wrote: > > > 2) Oracle Data mining (4K pages) > > > ++--+--+-+ > > > |mmap_cache type | hit-ra

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Ingo Molnar
* Davidlohr Bueso wrote: > On Mon, 2013-11-11 at 13:01 +0100, Ingo Molnar wrote: > > * Davidlohr Bueso wrote: > > > > > Hi Ingo, > > > > > > On Mon, 2013-11-04 at 08:36 +0100, Ingo Molnar wrote: > > > > * Davidlohr Bueso wrote: > > > > > > > > > I will look into doing the vma cache per thre

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Davidlohr Bueso
On Mon, 2013-11-11 at 13:01 +0100, Ingo Molnar wrote: > * Davidlohr Bueso wrote: > > > Hi Ingo, > > > > On Mon, 2013-11-04 at 08:36 +0100, Ingo Molnar wrote: > > > * Davidlohr Bueso wrote: > > > > > > > I will look into doing the vma cache per thread instead of mm (I hadn't > > > > really loo

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Ingo Molnar
* Michel Lespinasse wrote: > On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso wrote: > > 2) Oracle Data mining (4K pages) > > ++--+--+-+ > > |mmap_cache type | hit-rate | cycles (billion) | stddev | > > ++-

Re: [PATCH] mm: cache largest vma

2013-11-11 Thread Ingo Molnar
* Davidlohr Bueso wrote: > Hi Ingo, > > On Mon, 2013-11-04 at 08:36 +0100, Ingo Molnar wrote: > > * Davidlohr Bueso wrote: > > > > > I will look into doing the vma cache per thread instead of mm (I hadn't > > > really looked at the problem like this) as well as Ingo's suggestion on > > > th

Re: [PATCH] mm: cache largest vma

2013-11-10 Thread Michel Lespinasse
On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso wrote: > 2) Oracle Data mining (4K pages) > ++--+--+-+ > |mmap_cache type | hit-rate | cycles (billion) | stddev | > ++--+--+-+

Re: converting unicore32 to gate_vma as done for arm (was Re:?? [PATCH] mm: cache largest vma)

2013-11-10 Thread Al Viro
On Tue, Nov 05, 2013 at 10:49:15AM +0800, ? wrote: > The patch is ok for unicore32. Thanks Al. > > While testing this patch, a bug is found in > arch/unicore32/include/asm/pgtable.h: > > @@ -96,7 +96,7 @@ extern pgprot_t pgprot_kernel; >

Re: [PATCH] mm: cache largest vma

2013-11-10 Thread Davidlohr Bueso
Hi Ingo, On Mon, 2013-11-04 at 08:36 +0100, Ingo Molnar wrote: > * Davidlohr Bueso wrote: > > > I will look into doing the vma cache per thread instead of mm (I hadn't > > really looked at the problem like this) as well as Ingo's suggestion on > > the weighted LRU approach. However, having see

Re: [PATCH] mm: cache largest vma

2013-11-06 Thread Konstantin Khlebnikov
Some time ago I've thought about caching vma on PTE's struct page. This will work for all huge vmas not only for largest ones. Of course this requires some reordering in do_page_fault because currently it lookups vma before pte for obvious reason. On Wed, Nov 6, 2013 at 10:01 AM, Ingo Molnar wr

Re: [PATCH] mm: cache largest vma

2013-11-05 Thread Ingo Molnar
* Jiri Olsa wrote: > > But success primarily depends on how useful the tooling UI turns out > > to be: create a nice Slang or GTK UI for kprobes and triggers, and/or > > turn it into a really intuitive command line UI, and people will use > > it. > > > > I think annotated assembly/source out

Re: [PATCH] mm: cache largest vma

2013-11-05 Thread Jiri Olsa
On Tue, Nov 05, 2013 at 09:24:51AM +0100, Ingo Molnar wrote: SNIP > > > > Yeah that would be a nice interface. Speaking about that, it would be nice > > to get your input > > on the proposed interface for toggle events. > > > > It's still in an RFC state, although it's getting quite elaborated

Re: [PATCH] mm: cache largest vma

2013-11-05 Thread Ingo Molnar
* Frederic Weisbecker wrote: > On Mon, Nov 04, 2013 at 06:52:45PM +0100, Ingo Molnar wrote: > > > > * Frederic Weisbecker wrote: > > > > > On Mon, Nov 04, 2013 at 08:05:00AM +0100, Ingo Molnar wrote: > > > > > > > > * Davidlohr Bueso wrote: > > > > > > > > > Btw, do you suggest using a hig

Re: converting unicore32 to gate_vma as done for arm (was Re: [PATCH] mm: cache largest vma)

2013-11-04 Thread 管雪涛
The patch is ok for unicore32. Thanks Al. While testing this patch, a bug is found in arch/unicore32/include/asm/pgtable.h: @@ -96,7 +96,7 @@ extern pgprot_t pgprot_kernel; | PTE_EXEC) #define PAGE_READONLY __pgprot(pgprot

Re: [PATCH] mm: cache largest vma

2013-11-04 Thread Frederic Weisbecker
On Mon, Nov 04, 2013 at 06:52:45PM +0100, Ingo Molnar wrote: > > * Frederic Weisbecker wrote: > > > On Mon, Nov 04, 2013 at 08:05:00AM +0100, Ingo Molnar wrote: > > > > > > * Davidlohr Bueso wrote: > > > > > > > Btw, do you suggest using a high level tool such as perf for getting > > > > thi

Re: [PATCH] mm: cache largest vma

2013-11-04 Thread Ingo Molnar
* Frederic Weisbecker wrote: > On Mon, Nov 04, 2013 at 08:05:00AM +0100, Ingo Molnar wrote: > > > > * Davidlohr Bueso wrote: > > > > > Btw, do you suggest using a high level tool such as perf for getting > > > this data or sprinkling get_cycles() in find_vma() -- I'd think that the > > > fi

Re: [PATCH] mm: cache largest vma

2013-11-04 Thread Michel Lespinasse
On Sun, Nov 3, 2013 at 11:36 PM, Ingo Molnar wrote: > So I think it all really depends on the hit/miss cost difference. It makes > little sense to add a more complex scheme if it washes out most of the > benefits! > > Also note the historic context: the _original_ mmap_cache, that I > implemented

Re: [PATCH] mm: cache largest vma

2013-11-04 Thread Frederic Weisbecker
On Mon, Nov 04, 2013 at 08:05:00AM +0100, Ingo Molnar wrote: > > * Davidlohr Bueso wrote: > > > Btw, do you suggest using a high level tool such as perf for getting > > this data or sprinkling get_cycles() in find_vma() -- I'd think that the > > first isn't fine grained enough, while the later

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Ingo Molnar
* Davidlohr Bueso wrote: > I will look into doing the vma cache per thread instead of mm (I hadn't > really looked at the problem like this) as well as Ingo's suggestion on > the weighted LRU approach. However, having seen that we can cheaply and > easily reach around ~70% hit rate in a lot o

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Ingo Molnar
* Davidlohr Bueso wrote: > Btw, do you suggest using a high level tool such as perf for getting > this data or sprinkling get_cycles() in find_vma() -- I'd think that the > first isn't fine grained enough, while the later will probably variate a > lot from run to run but the ratio should be r

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Christoph Hellwig
On Sun, Nov 03, 2013 at 10:51:27AM -0800, Linus Torvalds wrote: > Ugh. This patch makes me angry. It looks way too ad-hoc. > > I can well imagine that our current one-entry cache is crap and could > be improved, but this looks too random. Different code for the > CONFIG_MMU case? Same name, but fo

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Ingo Molnar
* Davidlohr Bueso wrote: > On Sun, 2013-11-03 at 11:12 +0100, Ingo Molnar wrote: > > * Davidlohr Bueso wrote: > > > > > While caching the last used vma already does a nice job avoiding > > > having to iterate the rbtree in find_vma, we can improve. After > > > studying the hit rate on a load o

converting unicore32 to gate_vma as done for arm (was Re: [PATCH] mm: cache largest vma)

2013-11-03 Thread Al Viro
On Sun, Nov 03, 2013 at 08:20:10PM -0800, Davidlohr Bueso wrote: > > > diff --git a/arch/unicore32/include/asm/mmu_context.h > > > b/arch/unicore32/include/asm/mmu_context.h > > > index fb5e4c6..38cc7fc 100644 > > > --- a/arch/unicore32/include/asm/mmu_context.h > > > +++ b/arch/unicore32/include/

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Davidlohr Bueso
On Sun, 2013-11-03 at 18:57 -0500, KOSAKI Motohiro wrote: > >> I'm slightly surprised this cache makes 15% hit. Which application > >> get a benefit? You listed a lot of applications, but I'm not sure > >> which is highly depending on largest vma. > > > > Well I chose the largest vma because it giv

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Davidlohr Bueso
On Sun, 2013-11-03 at 11:12 +0100, Ingo Molnar wrote: > * Davidlohr Bueso wrote: > > > While caching the last used vma already does a nice job avoiding > > having to iterate the rbtree in find_vma, we can improve. After > > studying the hit rate on a load of workloads and environments, > > it was

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Davidlohr Bueso
On Sun, 2013-11-03 at 10:51 -0800, Linus Torvalds wrote: > Ugh. This patch makes me angry. It looks way too ad-hoc. > > I can well imagine that our current one-entry cache is crap and could > be improved, but this looks too random. Indeed, my approach is random *because* I wanted to keep things

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread KOSAKI Motohiro
>> I'm slightly surprised this cache makes 15% hit. Which application >> get a benefit? You listed a lot of applications, but I'm not sure >> which is highly depending on largest vma. > > Well I chose the largest vma because it gives us a greater chance of > being already cached when we do the look

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Linus Torvalds
Ugh. This patch makes me angry. It looks way too ad-hoc. I can well imagine that our current one-entry cache is crap and could be improved, but this looks too random. Different code for the CONFIG_MMU case? Same name, but for non-MMU it's a single entry, for MMU it's an array? And the whole "large

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Ingo Molnar
* Davidlohr Bueso wrote: > While caching the last used vma already does a nice job avoiding > having to iterate the rbtree in find_vma, we can improve. After > studying the hit rate on a load of workloads and environments, > it was seen that it was around 45-50% - constant for a standard > deskt

Re: [PATCH] mm: cache largest vma

2013-11-03 Thread Ingo Molnar
* Davidlohr Bueso wrote: > On Fri, 2013-11-01 at 16:38 -0400, KOSAKI Motohiro wrote: > > (11/1/13 4:17 PM), Davidlohr Bueso wrote: > > > > > While caching the last used vma already does a nice job avoiding > > > having to iterate the rbtree in find_vma, we can improve. After > > > studying the

Re: [PATCH] mm: cache largest vma

2013-11-01 Thread Rik van Riel
On 11/01/2013 04:17 PM, Davidlohr Bueso wrote: > While caching the last used vma already does a nice job avoiding > having to iterate the rbtree in find_vma, we can improve. After > studying the hit rate on a load of workloads and environments, > it was seen that it was around 45-50% - constant for

Re: [PATCH] mm: cache largest vma

2013-11-01 Thread Davidlohr Bueso
On Fri, 2013-11-01 at 16:38 -0400, KOSAKI Motohiro wrote: > (11/1/13 4:17 PM), Davidlohr Bueso wrote: > > While caching the last used vma already does a nice job avoiding > > having to iterate the rbtree in find_vma, we can improve. After > > studying the hit rate on a load of workloads and environ

Re: [PATCH] mm: cache largest vma

2013-11-01 Thread KOSAKI Motohiro
(11/1/13 4:17 PM), Davidlohr Bueso wrote: While caching the last used vma already does a nice job avoiding having to iterate the rbtree in find_vma, we can improve. After studying the hit rate on a load of workloads and environments, it was seen that it was around 45-50% - constant for a standard

[PATCH] mm: cache largest vma

2013-11-01 Thread Davidlohr Bueso
While caching the last used vma already does a nice job avoiding having to iterate the rbtree in find_vma, we can improve. After studying the hit rate on a load of workloads and environments, it was seen that it was around 45-50% - constant for a standard desktop system (gnome3 + evolution + firefo