Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2008-03-13 Thread Simon Riggs
On Wed, 2008-03-12 at 20:13 -0400, Bruce Momjian wrote: Is this a TODO? Tom's reply was: The general topic, yes. The caveats still apply. Nonsense. Main transaction exit also takes an exclusive lock, and is far more likely to be exercised in typical workloads than a subtransaction

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2008-03-12 Thread Bruce Momjian
Is this a TODO? Tom's reply was: Nonsense. Main transaction exit also takes an exclusive lock, and is far more likely to be exercised in typical workloads than a subtransaction abort. In any case: there has still not been any evidence presented by anyone that optimizing

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Simon Riggs
On Wed, 2006-09-13 at 21:45 -0400, Tom Lane wrote: Anyway, given that there's this one nonobvious gotcha, there might be others. My recommendation is that we take this off the open-items list for 8.2 and revisit it in the 8.3 cycle when there's more time. Well, its still 8.3 just... As

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: As discussed in the other thread Final Thoughts for 8.3 on LWLocking and Scalability, XidCacheRemoveRunningXids() is now the only holder of an X lock during normal processing, Nonsense. Main transaction exit also takes an exclusive lock, and is far more

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Simon Riggs
On Tue, 2007-09-11 at 09:58 -0400, Tom Lane wrote: Can we let go of this for 8.3, please? OK, we've moved forward, so its a good place to break. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Anyway, given that there's this one nonobvious gotcha, there might be others. My recommendation is that we take this off the open-items list for 8.2 and revisit it in the 8.3 cycle when there's more time. I wonder if Theo's recent reported problem with 4.3M

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Anyway, given that there's this one nonobvious gotcha, there might be others. My recommendation is that we take this off the open-items list for 8.2 and revisit it in the 8.3 cycle when there's more time. I wonder if

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
I wrote: Yeah, I was just looking at that. Removing useless entries from the child-xid list would presumably help him. Considering we're not even formally in beta yet, I'm probably being too conservative to recommend we not touch it. Actually ... wait a minute. We do not assign an XID to a

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
I wrote: I see a bug though, which is that RecordSubTransactionAbort() calls GetCurrentTransactionId() before having verified that it needs to do anything. This means that we'll generate and then discard an XID uselessly in a failed subxact that didn't touch disk. Well, it would be a bug

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Alvaro Herrera
Tom Lane wrote: I wrote: I see a bug though, which is that RecordSubTransactionAbort() calls GetCurrentTransactionId() before having verified that it needs to do anything. This means that we'll generate and then discard an XID uselessly in a failed subxact that didn't touch disk.

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I think Theo's problem is probably somewhere else, too --- apparently it's not so much that TransactionIdIsCurrentTransactionId takes a long time as that something is calling it lots of times with no check for interrupt. Could it be

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: --- and because the entries are surely added in increasing XID order, such an array could be binary-searched. If they're only added if they write to disk then isn't it possible to add them out of order? Start a child transaction, start a child of that one

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: --- and because the entries are surely added in increasing XID order, such an array could be binary-searched. If they're only added if they write to disk then isn't it possible to add them out of order? Start a

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-13 Thread Tom Lane
I wrote: ... it seems like in the case where RecordSubTransactionCommit detects that the subxact has not stored its XID anywhere, we could immediately remove the XID from the PGPROC array, just as if it had aborted. This would avoid chewing subxid slots for cases such as exception blocks in

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-03 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: I added a subxid array to Snapshot and running subxids are gathered from PGPROC-subxids cache. There are two overflowed case; any of PGPROC-subxids are overflowed or the number of total subxids exceeds pre-allocated buffers. If overflowed, we cannot

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-03 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: OTOH I think we only need to store live Xids and those finished that wrote a WAL record; we can drop subaborted and subcommitted if they didn't. While reviewing this thread, I see Alvaro already had the idea I just came to...

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-28 Thread ITAGAKI Takahiro
Bruce Momjian [EMAIL PROTECTED] wrote: Is there anything to do for 8.2 here? I'm working on Tom's idea. It is not a feature and does not change the on-disk-structures, so I hope it meet the 8.2 deadline... Tom Lane [EMAIL PROTECTED] wrote: I'm wondering about doing something similar to what

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-25 Thread Bruce Momjian
Is there anything to do for 8.2 here? --- ITAGAKI Takahiro wrote: This is an additional information. I wrote: If we want to resolve the probmen fundamentally, we might have to improve SubTrans using a better buffer

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Alvaro Herrera
Tom Lane wrote: I'm wondering about doing something similar to what TransactionIdIsInProgress does, ie, make use of the PGPROC lists of live subtransactions. Suppose that GetSnapshotData copies not only top xids but live subxids into the snapshot, and adds a flag indicating whether the

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I'm wondering about doing something similar to what TransactionIdIsInProgress does, ie, make use of the PGPROC lists of live subtransactions. Well, that sounds awfully more expensive than setting local-to-my-database Xmins as well as

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I'm wondering about doing something similar to what TransactionIdIsInProgress does, ie, make use of the PGPROC lists of live subtransactions. Well, that sounds awfully more expensive than setting

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: I was thinking at what time was the most appropiate to insert or remove an Xid from the cache. We can do without any excl-locking because 1) we already assume the storing of an Xid to be atomic, and 2) no one can be interested in querying for an Xid

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread ITAGAKI Takahiro
This is an additional information. I wrote: If we want to resolve the probmen fundamentally, we might have to improve SubTrans using a better buffer management algorithm or so. The above is maybe wrong. I checked each lwlock of pg_subtrans's buffers. All lwlocks are uniformly acquired and I

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-06 Thread ITAGAKI Takahiro
Tom Lane [EMAIL PROTECTED] wrote: It does not solve, even if it increases the number of NUM_SUBTRANS_BUFFERS. The problem was only postponed. Can you provide a reproducible test case for this? This is the reproducible test case: - Occurs on both 8.1.4 and HEAD. - On smp machine. I used