On Tue, 2006-10-31 at 11:04 -0500, Tom Lane wrote:
It seems that we're converging on the conclusion that not truncating
clog early is the least bad alternative. This has the advantage of
making things a lot simpler --- we won't need to track minxid at all.
Allow me to summarize what I think
Simon Riggs [EMAIL PROTECTED] writes:
Do we need another GUC? I thought your observation about a PITR slave
having that set lower than its master still remains unresolved.
No, AFAICS that's not an issue in this design. The facts-on-the-ground
are whatever is recorded in pg_class.relvacuumxid,
Tom Lane wrote:
The only alternative I can see is the one Heikki suggested: don't
truncate clog until the freeze horizon. That's safe (given the planned
change to WAL-log tuple freezing) and clean and simple, but a permanent
requirement of 250MB+ for pg_clog would put the final nail in the
Heikki Linnakangas [EMAIL PROTECTED] writes:
I got another idea. If we make sure that vacuum removes any aborted xid
older than OldestXmin from the table, we can safely assume that any xid
the current clog truncation point we are going to be interested in is
committed. Vacuum already
Alvaro Herrera [EMAIL PROTECTED] writes:
Simon Riggs wrote:
Ouch! We did discuss that also. Flushing the buffercache is nasty with
very large caches, so this makes autovacuum much less friendly - and
could take a seriously long time if you enforce the vacuum delay
costings.
Hmm, isn't the
It seems that we're converging on the conclusion that not truncating
clog early is the least bad alternative. This has the advantage of
making things a lot simpler --- we won't need to track minxid at all.
Allow me to summarize what I think has to happen:
* VACUUM will determine a freeze cutoff
On Mon, 2006-10-30 at 20:40 -0500, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
That was understood; in the above example I agree you need to flush. If
you don't pass a truncation point, you don't need to flush whether or
not you actually truncate. So we don't need to flush *every*
Tom Lane [EMAIL PROTECTED] writes:
The added WAL volume should be pretty minimal, because only tuples that have
gone untouched for a long time incur extra work.
That seems like a weak point in the logic. It seems like it would make VACUUM
which is already an i/o hog even more so. Perhaps
Gregory Stark wrote:
Tom Lane [EMAIL PROTECTED] writes:
The added WAL volume should be pretty minimal, because only tuples that have
gone untouched for a long time incur extra work.
That seems like a weak point in the logic. It seems like it would make VACUUM
which is already an i/o
Alvaro Herrera [EMAIL PROTECTED] writes:
Huh, but the log would not be flushed for each operation that the vacuum
logs. Only when it's going to commit.
It strikes me that the vacuum cost delay feature omits to consider
generation of WAL records as a cost factor. It may not be a big problem
We just discussed this in detail with Simon, and it looks like we have
5 (!) different but related problems:
1) The original problem of freeze then crash, leaving too high values in
relminxid and datminxid. If you then run vacuum, it might truncate CLOG
and you lose the commit status of the
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
I think it's premature to start writing
patches until we've decided how this really needs to work.
Not logging hint-bit updates seems safe to me. As long as we have the
clog, the hint-bit is just a hint. The
Heikki Linnakangas wrote:
We just discussed this in detail with Simon, and it looks like we have
5 (!) different but related problems:
Wow, four of them are mine :-(
2) vactuple_get_minxid doesn't take into account xmax's of tuples that
have HEAP_XMAX_INVALID set. That's a problem:
Alvaro Herrera [EMAIL PROTECTED] writes:
Ugh. Is there another solution to this? Say, sync the buffer so that
the hint bits are written to disk?
Yeah. The original design for all this is explained by the notes for
TruncateCLOG:
* When this is called, we know that the database logically
On Mon, 2006-10-30 at 12:05 -0500, Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
Ugh. Is there another solution to this? Say, sync the buffer so that
the hint bits are written to disk?
Yeah. The original design for all this is explained by the notes for
TruncateCLOG:
*
Simon Riggs wrote:
On Mon, 2006-10-30 at 12:05 -0500, Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
Ugh. Is there another solution to this? Say, sync the buffer so that
the hint bits are written to disk?
Yeah. The original design for all this is explained by the notes
Simon Riggs [EMAIL PROTECTED] writes:
ISTM we only need to flush iff the clog would be truncated when we
update relminxid.
Wrong :-( If the relvacuumxid change (not relminxid ... as I said, these
names aren't very transparent) makes it to disk but not all the hint
bits do, you're at risk.
Alvaro Herrera [EMAIL PROTECTED] writes:
In fact I don't understand what's the point about multiple databases vs.
a single database. Surely a checkpoint would flush all buffers in all
databases, no?
Yeah --- all the ones that are dirty *now*. Consider the case where you
vacuum DB X, update
On Mon, 2006-10-30 at 16:58 -0500, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
ISTM we only need to flush iff the clog would be truncated when we
update relminxid.
Wrong :-( If the relvacuumxid change (not relminxid ... as I said, these
names aren't very transparent) makes it
Simon Riggs [EMAIL PROTECTED] writes:
I don't agree: If the truncation points are at 1 million, 2 million etc,
then if we advance the relvacuumxid from 1.2 million to 1.5 million,
then crash, the hints bits for that last vacuum are lost. Sounds bad,
but we have not truncated clog, so there is
On Mon, 2006-10-30 at 19:18 -0500, Tom Lane wrote:
Simon Riggs [EMAIL PROTECTED] writes:
I don't agree: If the truncation points are at 1 million, 2 million etc,
then if we advance the relvacuumxid from 1.2 million to 1.5 million,
then crash, the hints bits for that last vacuum are lost.
Simon Riggs [EMAIL PROTECTED] writes:
That was understood; in the above example I agree you need to flush. If
you don't pass a truncation point, you don't need to flush whether or
not you actually truncate. So we don't need to flush *every* time,
OK, but does that actually do much of anything
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
I would've liked to give freezing a new opcode,
but we've ran out of them (see htup.h).
Hardly ... we have plenty of unused rmgr id's still.
Good point.
The real issue that still has to be resolved is the interaction of all
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
I think it's premature to start writing
patches until we've decided how this really needs to work.
Not logging hint-bit updates seems safe to me. As long as we have the
clog, the hint-bit is just a hint. The problem with freezing
Simon Riggs [EMAIL PROTECTED] writes:
[I've just coded the relcache invalidation WAL logging patch also.]
What? That doesn't make any sense to me.
regards, tom lane
---(end of broadcast)---
TIP 2: Don't 'kill -9' the
On Fri, 2006-10-27 at 12:01 -0400, Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
I think it's premature to start writing
patches until we've decided how this really needs to work.
Not logging hint-bit updates seems safe to me. As long as we have the
On Fri, 2006-10-27 at 22:19 +0100, Simon Riggs wrote:
So we definitely have a nasty problem here.
VACUUM FREEZE is just a loaded gun right now.
Maybe it's OK to say that during WAL replay we keep it
all the way back to the freeze horizon, but I'm not sure how we keep the
system from
Here's a patch for WAL logging tuple freezes in vacuum, per discussion
on pgsql-bugs.
This patch is against CVS head. Should this be backported to stable
branches? I think it should.
After writing the patch, I realized that it needs some thought if
backported, because WAL records of removing
Heikki Linnakangas wrote:
Here's a patch for WAL logging tuple freezes in vacuum, per discussion
on pgsql-bugs.
This patch is against CVS head. Should this be backported to stable
branches? I think it should.
Keep in mind that previous releases do not use the same method for
determining the
Heikki Linnakangas [EMAIL PROTECTED] writes:
I would've liked to give freezing a new opcode,
but we've ran out of them (see htup.h).
Hardly ... we have plenty of unused rmgr id's still.
The real issue that still has to be resolved is the interaction of all
this stuff with PITR scenarios ---
30 matches
Mail list logo
