Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-30 Thread Nick Piggin
Sonny Rao wrote: On Tue, Aug 30, 2005 at 12:53:18PM +1000, Nick Piggin wrote: For testing regular lookups, yeah that's more difficult. For a microbenchmark you can use sparse files, which can be a good trick for testing pagecache performance without the IO. I have a feeling that testing usi

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread Nick Piggin
James Bottomley wrote: On Tue, 2005-08-30 at 10:56 +1000, Nick Piggin wrote: Gang lookup is mainly used on IO paths but also on truncate, which is a reasonably fast path on some workloads (James, this is my suggestion for what you should test - truncate). Actually, I don't think I can test

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread James Bottomley
On Tue, 2005-08-30 at 10:56 +1000, Nick Piggin wrote: > Sonny Rao wrote: > > >On Mon, Aug 29, 2005 at 01:37:48PM +1000, Nick Piggin wrote: > > > >>s/common/only ? > >> > >>But the page tree is indexed by file offset rather than virtual > >>address, and we try to span the file's pagecache with the

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread Andrew Morton
Nick Piggin <[EMAIL PROTECTED]> wrote: > > But of course gang lookup is only useful if a single read() call > asks for more than 1 page - is that a performance critical path? readahead should do gang lookups (or, preferably, find-next, when it's implemented). But nobody got around to it. - To un

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread Nick Piggin
Sonny Rao wrote: On Mon, Aug 29, 2005 at 01:37:48PM +1000, Nick Piggin wrote: s/common/only ? But the page tree is indexed by file offset rather than virtual address, and we try to span the file's pagecache with the smallest possible tree. So it will tend to make the trees taller. I did s

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread James Bottomley
On Mon, 2005-08-29 at 13:37 +1000, Nick Piggin wrote: > But the page tree is indexed by file offset rather than virtual > address, and we try to span the file's pagecache with the smallest > possible tree. So it will tend to make the trees taller. Well, OK, I concede that one. However, my content

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-29 Thread Luben Tuikov
On 08/28/05 23:54, Trond Myklebust wrote: > må den 29.08.2005 Klokka 13:37 (+1000) skreiv Nick Piggin: > >>James Bottomley wrote: >> >>>On Sun, 2005-08-28 at 18:35 -0700, Andrew Morton wrote: >> It does make the tree higher and hence will incur some more cache missing when descending the t

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Trond Myklebust
må den 29.08.2005 Klokka 13:37 (+1000) skreiv Nick Piggin: > James Bottomley wrote: > > On Sun, 2005-08-28 at 18:35 -0700, Andrew Morton wrote: > > >>It does make the tree higher and hence will incur some more cache missing > >>when descending the tree. > > > > > > Actually, I don't think it doe

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Nick Piggin
James Bottomley wrote: On Sun, 2005-08-28 at 18:35 -0700, Andrew Morton wrote: It does make the tree higher and hence will incur some more cache missing when descending the tree. Actually, I don't think it does: the common user is the page tree. Obviously, I've changed nothing on 64 bits,

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread James Bottomley
On Sun, 2005-08-28 at 18:35 -0700, Andrew Morton wrote: > > Well ... It's my opinion (and purely unsubstantiated, I suppose) that > > it's more efficient on 32 bit platforms to do bit operations on 32 bit > > quantities, which is why I changed the radix tree map shift to 5 for > > that case. > > >

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Andrew Morton
James Bottomley <[EMAIL PROTECTED]> wrote: > > On Sun, 2005-08-28 at 17:52 -0700, Andrew Morton wrote: > > > +#if BITS_PER_LONG == 32 > > > +#define RADIX_TREE_MAP_SHIFT 5 > > > +#elif BITS_PER_LONG == 64 > > > ... > > > struct radix_tree_node { > > > - unsigned intcount; > > > void

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread James Bottomley
On Sun, 2005-08-28 at 17:52 -0700, Andrew Morton wrote: > > +#if BITS_PER_LONG == 32 > > +#define RADIX_TREE_MAP_SHIFT 5 > > +#elif BITS_PER_LONG == 64 > > ... > > struct radix_tree_node { > > - unsigned intcount; > > void*slots[RADIX_TREE_MAP_SIZE]; > > - unsigned lo

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Andrew Morton
James Bottomley <[EMAIL PROTECTED]> wrote: > > On Sat, 2005-08-27 at 10:53 -0700, Andrew Morton wrote: > > a) fix radix_tree_gang_lookup() to use find_next_bit() > > > > b) remove radix_tree_node.count > > > > c) Add a new tag field which simply means "present" > > > > d) remove radix_tree_gang_

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread James Bottomley
On Sat, 2005-08-27 at 10:53 -0700, Andrew Morton wrote: > a) fix radix_tree_gang_lookup() to use find_next_bit() > > b) remove radix_tree_node.count > > c) Add a new tag field which simply means "present" > > d) remove radix_tree_gang_lookup() and __lookup() altogether > > e) Implement radix_tr

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Andrew Morton
Linus Torvalds <[EMAIL PROTECTED]> wrote: > > > > On Sun, 28 Aug 2005, James Bottomley wrote: > > > > radix_tree_insert() is reliable from IRQ provided you don't try to use > > radix_tree_preload() and you defined your radix tree gfp flag to be > > GFP_ATOMIC. > > It would be better if it wasn'

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread Linus Torvalds
On Sun, 28 Aug 2005, James Bottomley wrote: > > radix_tree_insert() is reliable from IRQ provided you don't try to use > radix_tree_preload() and you defined your radix tree gfp flag to be > GFP_ATOMIC. It would be better if it wasn't, though. I really don't see why we made it irq-safe, and ta

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-28 Thread James Bottomley
On Sat, 2005-08-27 at 10:53 -0700, Andrew Morton wrote: > I'd say the main naivety in gang lookup is the awkward top-level iteration > algorithm. The way it bales out all the way to the top level of the tree > once __lookup() hits the end of the slots[] array, even though results[] > isn't full ye

Re: [PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-27 Thread Andrew Morton
James Bottomley <[EMAIL PROTECTED]> wrote: > > The current gang lookup is rather naive and slow. I'd say the main naivety in gang lookup is the awkward top-level iteration algorithm. The way it bales out all the way to the top level of the tree once __lookup() hits the end of the slots[] array, e

[PATCH] make radix tree gang lookup faster by using a bitmap search

2005-08-27 Thread James Bottomley
The current gang lookup is rather naive and slow. This patch replaces the integer count with an unsigned long representing the bitmap of occupied elements. We then use that bitmap to find the first occupied entry instead of looping over all the entries from the beginning of the radix node. The p