Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Alvaro Herrera
Tom Lane wrote: What I'm envisioning is that we lose the notion of this is a serializable snapshot that that function currently has, and just give it the rule if MyProc-xmin is currently zero, then set it. Then the only additional mechanism needed is for the snapshot manager to detect when

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Simon Riggs
On Fri, 2008-03-28 at 10:35 -0300, Alvaro Herrera wrote: The problem is that we always consider every transaction's PGPROC-xid in calculating MyProc-xmin. So if you have a long running transaction, it doesn't matter how far beyond the snapshots are -- the value returned by GetOldestXmin will

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Alvaro Herrera
Simon Riggs wrote: On Fri, 2008-03-28 at 10:35 -0300, Alvaro Herrera wrote: The problem is that we always consider every transaction's PGPROC-xid in calculating MyProc-xmin. So if you have a long running transaction, it doesn't matter how far beyond the snapshots are -- the value

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Simon Riggs
On Fri, 2008-03-28 at 11:26 -0300, Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2008-03-28 at 10:35 -0300, Alvaro Herrera wrote: The problem is that we always consider every transaction's PGPROC-xid in calculating MyProc-xmin. So if you have a long running transaction, it

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: As far as I can see, for the purposes of VACUUM we can remove any tuple that was deleted after the old transaction's Xid but before that transaction's Xmin (i.e. all of its live snapshots). This means we get to ignore Xid in GetOldestXmin and in the

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: As far as I can see, for the purposes of VACUUM we can remove any tuple that was deleted after the old transaction's Xid but before that transaction's Xmin (i.e. all of its live snapshots). This means we get to ignore Xid in

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Heikki Linnakangas
Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: As far as I can see, for the purposes of VACUUM we can remove any tuple that was deleted after the old transaction's Xid but before that transaction's Xmin (i.e. all of its live snapshots). This means we get to

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Alvaro Herrera
Heikki Linnakangas wrote: Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: As far as I can see, for the purposes of VACUUM we can remove any tuple that was deleted after the old transaction's Xid but before that transaction's Xmin (i.e. all of its live

Re: [HACKERS] advancing snapshot's xmin

2008-03-28 Thread Heikki Linnakangas
Alvaro Herrera wrote: Heikki Linnakangas wrote: Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: As far as I can see, for the purposes of VACUUM we can remove any tuple that was deleted after the old transaction's Xid but before that transaction's Xmin (i.e. all

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Dimitri Fontaine
Le mercredi 26 mars 2008, Tom Lane a écrit : whenever the number of active snapshots goes to zero Does this ever happen? I mean, if the way to avoid locking contention is to rely on a production system which let the service breathe from time to time, maybe there's something wrong in the

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Heikki Linnakangas [EMAIL PROTECTED] writes: Neil Conway wrote: If we're just updating MyProc-xmin, we only need to acquire ProcArrayLock in shared mode, right? In fact, do you need a lock at all? I think you probably do. GetSnapshotData needs to be

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Tom Lane
Dimitri Fontaine [EMAIL PROTECTED] writes: Le mercredi 26 mars 2008, Tom Lane a écrit : whenever the number of active snapshots goes to zero Does this ever happen? Certainly: between any two commands of a non-serializable transaction. In a serializable transaction the whole thing is a dead

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I think you probably do. GetSnapshotData needs to be confident that the global xmin it computes is = the xmin that any other backend might be about to store into its MyProc-xmin; how can you ensure that if there's no

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Consider a data loading job which has millions of INSERT statements in a file. Currently if you put them all in a transaction it takes a single snapshot and runs them all with the same snapshot. If you reset xmin whenever you have no live snapshots then

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Uhm, yeah, I somehow didn't write was I was thinking. I didn't mean to say we would be taking a new snapshot for each INSERT but that we would be resetting xmin for each INSERT. Whereas currently we only set xmin once when we set the serializable

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Dimitri Fontaine
Le mercredi 26 mars 2008, Tom Lane a écrit : Dimitri Fontaine [EMAIL PROTECTED] writes: Le mercredi 26 mars 2008, Tom Lane a écrit : whenever the number of active snapshots goes to zero Does this ever happen? Certainly: between any two commands of a non-serializable transaction. Oh, it's

Re: [HACKERS] advancing snapshot's xmin

2008-03-26 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Uhm, yeah, I somehow didn't write was I was thinking. I didn't mean to say we would be taking a new snapshot for each INSERT but that we would be resetting xmin for each INSERT. Whereas currently we only set xmin once

[HACKERS] advancing snapshot's xmin

2008-03-25 Thread Alvaro Herrera
Hi, I've finished (hopefully) the code to handle a current list of open snapshots in a transaction. I'm now wondering how to put it to good use ;-) I'm not posting it yet -- first I want to get some feedback on the previous patch I posted,

Re: [HACKERS] advancing snapshot's xmin

2008-03-25 Thread Simon Riggs
On Tue, 2008-03-25 at 17:26 -0300, Alvaro Herrera wrote: I've finished (hopefully) the code to handle a current list of open snapshots in a transaction. I'm now wondering how to put it to good use ;-) I'm not posting it yet -- first I want to get some feedback on the previous patch I

Re: [HACKERS] advancing snapshot's xmin

2008-03-25 Thread Neil Conway
On Tue, 2008-03-25 at 17:26 -0300, Alvaro Herrera wrote: There is one hole here: contention on ProcArrayLock. Basically, for simple transactions we will need to update MyProc after every command. If we're just updating MyProc-xmin, we only need to acquire ProcArrayLock in shared mode, right?

Re: [HACKERS] advancing snapshot's xmin

2008-03-25 Thread Heikki Linnakangas
Neil Conway wrote: On Tue, 2008-03-25 at 17:26 -0300, Alvaro Herrera wrote: There is one hole here: contention on ProcArrayLock. Basically, for simple transactions we will need to update MyProc after every command. If we're just updating MyProc-xmin, we only need to acquire ProcArrayLock in

Re: [HACKERS] advancing snapshot's xmin

2008-03-25 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Neil Conway wrote: If we're just updating MyProc-xmin, we only need to acquire ProcArrayLock in shared mode, right? In fact, do you need a lock at all? I think you probably do. GetSnapshotData needs to be confident that the global xmin it