Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-06 Thread John Lumby
Bruce Momjian wrote: > > On Fri, Nov 2, 2012 at 09:59:08AM -0400, John Lumby wrote: > > However,the OP describes an implementation based on libaio. > > Today what we have (for linux) is librt, which is quite different. > > Well, good thing we didn't switch to using libaio, now that it is gone

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-05 Thread Bruce Momjian
On Fri, Nov 2, 2012 at 09:59:08AM -0400, John Lumby wrote: > Thanks for the mentioning this posting.    Interesting. > However,    the OP describes an implementation based on libaio. > Today what we have (for linux) is librt,  which is quite different. > It is arguable worse than libaio (well act

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-02 Thread John Lumby
Claudio Freire wrote: > > On Thu, Nov 1, 2012 at 10:59 PM, Greg Smith wrote: > > On 11/1/12 6:13 PM, Claudio Freire wrote: > > > >> posix_fadvise what's the trouble there, but the fact that the kernel > >> stops doing read-ahead when a call to posix_fadvise comes. I noticed > >> the performance h

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Claudio Freire
On Thu, Nov 1, 2012 at 10:59 PM, Greg Smith wrote: > On 11/1/12 6:13 PM, Claudio Freire wrote: > >> posix_fadvise what's the trouble there, but the fact that the kernel >> stops doing read-ahead when a call to posix_fadvise comes. I noticed >> the performance hit, and checked the kernel's code. It

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Greg Smith
On 11/1/12 6:13 PM, Claudio Freire wrote: posix_fadvise what's the trouble there, but the fact that the kernel stops doing read-ahead when a call to posix_fadvise comes. I noticed the performance hit, and checked the kernel's code. It effectively changes the prediction mode from sequential to fa

FW: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread John Lumby
Claudio wrote : > > Check the latest patch, it contains heap page prefetching too. > Oh yes I see. I missed that - I was looking in the wrong place. I do have one question about the way you did it : by placing the prefetch heap-page calls in _bt_next, which effectively means inside a call from th

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Claudio Freire
On Thu, Nov 1, 2012 at 2:00 PM, Andres Freund wrote: >> > I agree. I'm a bit hesitant to subscribe to yet another mailing list >> >> FYI you can send messages to linux-kernel without subscribing (there's >> no moderation either). >> >> Subscribing to linux-kernel is like drinking from a firehose :

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Claudio Freire
On Thu, Nov 1, 2012 at 1:37 PM, John Lumby wrote: > > Claudio wrote : >> >> Oops - forgot to effectively attach the patch. >> > > I've read through your patch and the earlier posts by you and Cédric. > > This is very interesting. You chose to prefetch index btree (key-ptr) > pages > whereas

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Andres Freund
On Thursday, November 01, 2012 05:53:39 PM Marti Raudsepp wrote: > On Tue, Oct 30, 2012 at 1:50 AM, Claudio Freire wrote: > > On Mon, Oct 29, 2012 at 7:07 PM, Cédric Villemain > > > > wrote: > >> Well, informing linux hackers may help. > > > > I agree. I'm a bit hesitant to subscribe to yet an

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread Marti Raudsepp
On Tue, Oct 30, 2012 at 1:50 AM, Claudio Freire wrote: > On Mon, Oct 29, 2012 at 7:07 PM, Cédric Villemain > wrote: >> Well, informing linux hackers may help. > > I agree. I'm a bit hesitant to subscribe to yet another mailing list FYI you can send messages to linux-kernel without subscribing (t

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-11-01 Thread John Lumby
Claudio wrote : > > Oops - forgot to effectively attach the patch. > I've read through your patch and the earlier posts by you and Cédric. This is very interesting.      You chose to prefetch index btree (key-ptr) pages whereas I chose to prefetch the data pages pointed to by the key-ptr pages.

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 7:07 PM, Cédric Villemain wrote: >> But it also looks forgotten. Bringing it back to life would mean >> building the latest kernel with that patch included, replicating the >> benchmarks I ran here, sans pg patch, but with patched kernel, and >> reporting the (hopefully equ

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Cédric Villemain
> But it also looks forgotten. Bringing it back to life would mean > building the latest kernel with that patch included, replicating the > benchmarks I ran here, sans pg patch, but with patched kernel, and > reporting the (hopefully equally dramatic) performance improvements in > the kernel ML. Th

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 4:17 PM, Cédric Villemain wrote: >> Ok, this is the best I could come up with, without some real test hardware. >> >> The only improvement I see in single-disk scenarios: >> * Huge speedup of back-sequential index-only scans >> * Marginal speedup on forward index-only s

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Cédric Villemain
> Ok, this is the best I could come up with, without some real test hardware. > > The only improvement I see in single-disk scenarios: > * Huge speedup of back-sequential index-only scans > * Marginal speedup on forward index-only scans (5% or less) > * No discernible difference in heap-incl

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 12:53 PM, Claudio Freire wrote: >> Yes, I've seen that, though I thought it was only an improvement on >> PrefetchBuffer. That patch would interact quite nicely with mine. >> >> I'm now trying to prefetch heap tuples, and I got to a really nice >> place where I get an extra

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Tue, Oct 23, 2012 at 10:54 AM, Claudio Freire wrote: >> Indeed not. I also looked into prefetching for pure index scans for >> b-trees (and extension to use async io). >> http://archives.postgresql.org/message-id/BLU0-SMTP31709961D846CCF4F5EB4C2A3930%40phx.gbl > > Yes, I've seen that, though

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-23 Thread Claudio Freire
On Tue, Oct 23, 2012 at 9:44 AM, John Lumby wrote: >> From: Claudio Freire >> I hope I'm not talking to myself. > > Indeed not. I also looked into prefetching for pure index scans for > b-trees (and extension to use async io). > http://archives.postgresql.org/message-id/BLU0-SMTP31709961D846CC

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-23 Thread John Lumby
> From: Claudio Freire > I hope I'm not talking to myself. Indeed not. I also looked into prefetching for pure index scans for b-trees (and extension to use async io). http://archives.postgresql.org/message-id/BLU0-SMTP31709961D846CCF4F5EB4C2A3930%40phx.gbl I am not where I have a proper se

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-22 Thread Claudio Freire
On Thu, Oct 18, 2012 at 7:42 PM, Claudio Freire wrote: > Fun. That didn't take long. > > With the attached anti-sequential scan patch, and effective_io_concurrency=8: > > > QUERY PLAN > --

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-18 Thread Claudio Freire
On Thu, Oct 18, 2012 at 5:30 PM, Claudio Freire wrote: > Backward: > >QUERY PLAN > -- > GroupAggregat