Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Bruce Momjian
Heikki Linnakangas wrote: > > Should we consider only telling the kernel X pages ahead, meaning when > > we are on page 10 we tell it about page 16? > > Yes. You don't want to fire off thousands of posix_fadvise calls > upfront. That'll just flood the kernel, and it will most likely ignore > any

Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Heikki Linnakangas
Bruce Momjian wrote: Heikki Linnakangas wrote: So it has nothing to do with table size. The fadvise calls need to be (and are) limited by what can be used in the near future, and not for the whole statement. Right, I was sloppy. Instead of table size, what matters is the amount of data the sc

Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Martijn van Oosterhout
On Fri, Mar 28, 2008 at 11:41:58AM -0400, Bruce Momjian wrote: > Should we consider only telling the kernel X pages ahead, meaning when > we are on page 10 we tell it about page 16? It's not so interesting for sequential reads, the kernel can work that out for itself. Disk reads are usually in blo

Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Bruce Momjian
Heikki Linnakangas wrote: > > So it has nothing to do with table size. The fadvise calls need to be > > (and are) > > limited by what can be used in the near future, and not for the whole > > statement. > > Right, I was sloppy. Instead of table size, what matters is the amount > of data the scan

Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Heikki Linnakangas
Zeugswetter Andreas OSB SD wrote: Heikki wrote: It seems that the worst case for this patch is a scan on a table that doesn't fit in shared_buffers, but is fully cached in the OS cache. In that case, the posix_fadvise calls would be a certain waste of time. I think this is a misunderstandin

Re: Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Zeugswetter Andreas OSB SD
Heikki wrote: > It seems that the worst case for this patch is a scan on a table that > doesn't fit in shared_buffers, but is fully cached in the OS cache. In > that case, the posix_fadvise calls would be a certain waste of time. I think this is a misunderstanding, the fadvise is not issued to

Prereading using posix_fadvise (was Re: [HACKERS] Commitfest patches)

2008-03-28 Thread Heikki Linnakangas
Gregory Stark wrote: I described which interfaces worked on Linux and Solaris based on empirical tests. I posted source code for synthetic benchmarks so we could test it on a wide range of hardware. I posted graphs based on empirical results. I posted mathematical formulas analysing just how much