Re: [HACKERS] Concurrent MERGE

2010-08-06 Thread Simon Riggs
On Thu, 2010-08-05 at 19:39 +0300, Heikki Linnakangas wrote: > On 05/08/10 18:43, Simon Riggs wrote: > > Do we still need me to work on concurrent MERGE, or is that included in > > the current MERGE patch (can't see it), or ... > > It's not in the current MERGE patch. OK, I will work on it, event

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
>> I've never had the deadlock detector successfully deal with the above. >> Let alone a 4-way. > Hm. I have seen 5way deadlocks getting resolved just recently. I can > find the relevant if you find it interesting, but I doubt it is. Ah, I didn't know that it was even *supposed* to resolve larger

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Tom Lane
Josh Berkus writes: >> Hm? Please explain what you're talking about. > Transaction A locks 1 and wants a lock on 2 > Transaction B locks 2 and wants a lock on 3 > Transaction C locks 3 and wants a lock on 1 > I've never had the deadlock detector successfully deal with the above. > Let alone a 4

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Andres Freund
On Thu, Aug 05, 2010 at 03:49:05PM -0700, Josh Berkus wrote: > > > Hm? Please explain what you're talking about. > > Transaction A locks 1 and wants a lock on 2 > Transaction B locks 2 and wants a lock on 3 > Transaction C locks 3 and wants a lock on 1 > > I've never had the deadlock detector succ

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Mark Kirkwood
On 06/08/10 10:49, Josh Berkus wrote: Hm? Please explain what you're talking about. Transaction A locks 1 and wants a lock on 2 Transaction B locks 2 and wants a lock on 3 Transaction C locks 3 and wants a lock on 1 I've never had the deadlock detector successfully deal with the abov

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
> Hm? Please explain what you're talking about. Transaction A locks 1 and wants a lock on 2 Transaction B locks 2 and wants a lock on 3 Transaction C locks 3 and wants a lock on 1 I've never had the deadlock detector successfully deal with the above. Let alone a 4-way. > Not sure I believe thi

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Tom Lane
Josh Berkus writes: > Yes; it's a major project. Our detector works pretty well for deadlocks > which are 2-process locks or even several processes all locking against > the same first process. However, triangular and quadralateral deadlocks > (which I've seen more than once) it completely cannot

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
On 8/5/10 1:59 PM, Kevin Grittner wrote: > Oh, and if deadlocks are that broken, it's a bit scary that we have > let that go. Is it the problem that technically intractable? Yes; it's a major project. Our detector works pretty well for deadlocks which are 2-process locks or even several processe

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
Josh Berkus wrote: > Overall, you're missing the point: there are workarounds for all > of these things now. However, they are *workarounds*, which means > that they are awkward, expensive, and/or hard to administrate; > having predicate locks would make things much easier. Well, if some form

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
Kevin, Overall, you're missing the point: there are workarounds for all of these things now. However, they are *workarounds*, which means that they are awkward, expensive, and/or hard to administrate; having predicate locks would make things much easier. > I don't see how that can be resolved wi

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
Josh Berkus wrote: > Anyway, here's some of the uses I'm thinking of: > > (1) Pre-insert lock: you know that you're going to insert a record > with PK="X" later in a long-running SP, so you want to lock out > other inserts of PK="X" at the beginning of the procedure. Well, if we added a liste

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
On 8/5/10 12:33 PM, Kevin Grittner wrote: > I don't know whether this is the right time to discuss those 9 > different uses, but just so everyone knows, the SIRead locks needed > for the SSI implementation in the current serializable patch have > some characteristics which may be exactly what you

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
Josh Berkus wrote: > Well, we *still* want predicate locking regardless of what MERGE > supports. It's useful in about 9 different ways. I don't know whether this is the right time to discuss those 9 different uses, but just so everyone knows, the SIRead locks needed for the SSI implementatio

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
I wrote: > So... No, it's not directly a problem on the server itself. I just had a thought -- the MERGE code isn't doing anything fancy with snapshots, is it? I haven't been tracking that discussion too closely or read the patch. My previous comments assume that the *snapshot* is stable for

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Josh Berkus
> At 2010 Dev Mtg, we put me down to work on making merge work > concurrently. That was garbled slightly and had me down as working on > predicate locking which is the general solution to the problem. Well, we *still* want predicate locking regardless of what MERGE supports. It's useful in about

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
Chris Browne wrote: > robertmh...@gmail.com (Robert Haas) writes: >> I suspect Kevin's patch will solve it if using a sufficiently >> high transaction isolation level, but something else might be >> needed otherwise. However, I confess to ignorance as to the >> underlying issues? Why is MERGE

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Chris Browne
robertmh...@gmail.com (Robert Haas) writes: > On Thu, Aug 5, 2010 at 11:43 AM, Simon Riggs wrote: >> Looks like MERGE is progressing well. >> >> At 2010 Dev Mtg, we put me down to work on making merge work >> concurrently. That was garbled slightly and had me down as working on >> predicate lockin

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Kevin Grittner
Heikki Linnakangas wrote: >> However, I confess to ignorance as to the underlying >> issues? Why is MERGE worse in this regard than, say, UPDATE? > > MERGE can be used to implement "upsert", where a row is updated if > it exists and inserted if it doesn't. I don't think Kevin's patch > will s

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Heikki Linnakangas
On 05/08/10 18:57, Robert Haas wrote: On Thu, Aug 5, 2010 at 11:43 AM, Simon Riggs wrote: Looks like MERGE is progressing well. At 2010 Dev Mtg, we put me down to work on making merge work concurrently. That was garbled slightly and had me down as working on predicate locking which is the gene

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Heikki Linnakangas
On 05/08/10 18:43, Simon Riggs wrote: Do we still need me to work on concurrent MERGE, or is that included in the current MERGE patch (can't see it), or ... It's not in the current MERGE patch. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mail

Re: [HACKERS] Concurrent MERGE

2010-08-05 Thread Robert Haas
On Thu, Aug 5, 2010 at 11:43 AM, Simon Riggs wrote: > Looks like MERGE is progressing well. > > At 2010 Dev Mtg, we put me down to work on making merge work > concurrently. That was garbled slightly and had me down as working on > predicate locking which is the general solution to the problem. > >

[HACKERS] Concurrent MERGE

2010-08-05 Thread Simon Riggs
Looks like MERGE is progressing well. At 2010 Dev Mtg, we put me down to work on making merge work concurrently. That was garbled slightly and had me down as working on predicate locking which is the general solution to the problem. Do we still need me to work on concurrent MERGE, or is that inc