Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-14 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 3:40 PM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: >> So one would expect a write-intensive initial vacuum after a >> PITR-style recovery? > An interesting issue when running wi

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-14 Thread Kevin Grittner
>>> On Fri, Dec 14, 2007 at 1:42 AM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > My feeling is that vacuum's purpose in life is to offload maintenance > cycles from foreground queries, so we should be happy to have it setting > all the hint bits. Absolutely. > If K

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Russell Smith <[EMAIL PROTECTED]> writes: > Given vacuum must then touch every page, is there a win in only setting > hint bits on pages where vacuum has to do some other work on the page? > As vacuum is causing significant IO load for data that may not be > accessed for some time. Well, if va

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Russell Smith
Simon Riggs wrote: On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: What impact would lack of the hint bits have until a vacuum? Vacuum isn't important here. Its the first idiot to read the data that gets hit. Given vacuum must then touch every page, is there a win in only

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 3:40 PM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: >> >>> On Thu, Dec 13, 2007 at 12:12 PM, in message >> <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> >> wrote: >> > On Thu

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 12:12 PM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >> If the hint bit changes are written to the WAL ... > > > >

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 12:12 PM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: >> If the hint bit changes are written to the WAL ... > > They're not. So one would expect a write-intensive initial vacuum aft

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 13:52 -0500, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Simon Riggs wrote: > >> We might be able to have bgwriter set hint bits on dirty blocks, > > > I don't think that works, because the bgwriter has no access to the > > catalogs, therefore it cannot e

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> We might be able to have bgwriter set hint bits on dirty blocks, > I don't think that works, because the bgwriter has no access to the > catalogs, therefore it cannot examine the page contents. To bgwriter, > pages are opaque. An

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 10:11 AM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > > > >> The data was inserted through a Java program using a p

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:11 AM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > >> The data was inserted through a Java program using a prepared >> statement with no indexes on the table. The primary key w

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Alvaro Herrera
Simon Riggs wrote: > We might be able to have bgwriter set hint bits on dirty blocks, but the > success of that would depend upon the transit time of blocks through the > cache, i.e. it might be totally ineffective. So might be just overhead > for the bgwriter and worse, could divert bgwriter atte

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 10:39 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 10:35 AM, in message <[EMAIL PROTECTED]>, > Tom Lane <[EMAIL PROTECTED]> wrote: > > "Kevin Grittner" <[EMAIL PROTECTED]> writes: > >> ... although to a naive user it's not clear what > >> is known at vacuum time

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:35 AM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> ... although to a naive user it's not clear what >> is known at vacuum time that the INSERT into the empty table >> couldn't have inferred. >

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > ... although to a naive user it's not clear what > is known at vacuum time that the INSERT into the empty table > couldn't have inferred. The fact that the INSERT actually committed. regards, tom lane

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:11 AM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > >> The data was inserted through a Java program using a prepared >> statement with no indexes on the table. The primary key w

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > The data was inserted through a Java program using a prepared > statement with no indexes on the table. The primary key was then > added, and now I've started a vacuum. The new table wound up being > the first big table vacuumed, and I

[PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
Yesterday we moved a 300 GB table containing document images (mostly raster-scanned from paper), into a 215 GB PostgreSQL 8.2.5 database which contains the related case management data. (This separation was never "right", since there were links from one to the other, but was necessary under our