Re: [HACKERS] foreign key locks

2013-01-23 Thread Alvaro Herrera
I just pushed this patch to the master branch. There was a corresponding catversion bump and pg_control version bump. I have verified that make check-world passes on my machine, as well as isolation tests and pg_upgrade. Tom Lane said at one point this is too complex to maintain. Several times

Re: [HACKERS] foreign key locks

2013-01-19 Thread Simon Riggs
On 18 January 2013 21:01, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-18 15:37:47 -0500, Tom Lane wrote: I doubt it ever came up before. What use is logging only the content of a buffer page? Surely you'd need to know, for example, which

Re: [HACKERS] foreign key locks

2013-01-18 Thread Alvaro Herrera
Alvaro Herrera wrote: Here's version 28 of this patch. The only substantive change here from v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive or LockTupleNoKeyExclusive, depending on whether the key columns are being modified by the update. This needs to go through

Re: [HACKERS] foreign key locks

2013-01-18 Thread Simon Riggs
On 18 January 2013 20:02, Alvaro Herrera alvhe...@2ndquadrant.com wrote: XLOG_HEAP2_LOCK_UPDATED Every xlog record needs to know where to put the block. Compare with XLOG_HEAP_LOCK xlrec.target.node = relation-rd_node; -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] foreign key locks

2013-01-18 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: The reason is that there is an (unknown to me) rule that there must be some data not associated with a buffer: /* * NOTE: We disallow len == 0 because it provides a useful bit of extra * error checking in ReadRecord. This

Re: [HACKERS] foreign key locks

2013-01-18 Thread Andres Freund
On 2013-01-18 15:37:47 -0500, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: The reason is that there is an (unknown to me) rule that there must be some data not associated with a buffer: /* * NOTE: We disallow len == 0 because it provides a useful bit of extra

Re: [HACKERS] foreign key locks

2013-01-18 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-18 15:37:47 -0500, Tom Lane wrote: I doubt it ever came up before. What use is logging only the content of a buffer page? Surely you'd need to know, for example, which relation and page number it is from. It only got to be a length of

Re: [HACKERS] foreign key locks

2013-01-18 Thread Andres Freund
On 2013-01-18 16:01:18 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-18 15:37:47 -0500, Tom Lane wrote: I doubt it ever came up before. What use is logging only the content of a buffer page? Surely you'd need to know, for example, which relation and page

Re: [HACKERS] foreign key locks

2013-01-11 Thread Alvaro Herrera
Andres Freund wrote: On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: Here's version 28 of this patch. The only substantive change here from v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive or LockTupleNoKeyExclusive, depending on whether the key columns are

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 12:11:47 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: Here's version 28 of this patch. The only substantive change here from v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive or

Re: [HACKERS] foreign key locks

2013-01-11 Thread Alvaro Herrera
Andres Freund wrote: No, I was thinking about an update without triggers present. T0: CREATE TABLE tbl(id serial pk, name text unique, data text); T1: BEGIN; -- read committed T1: UPDATE tbl SET name = 'foo' WHERE name = 'blarg'; /* key update of row id = 1 */ T2: BEGIN; -- read

Re: [HACKERS] foreign key locks

2013-01-11 Thread Andres Freund
On 2013-01-11 13:10:49 -0300, Alvaro Herrera wrote: Andres Freund wrote: No, I was thinking about an update without triggers present. T0: CREATE TABLE tbl(id serial pk, name text unique, data text); T1: BEGIN; -- read committed T1: UPDATE tbl SET name = 'foo' WHERE name = 'blarg'; /*

Re: [HACKERS] foreign key locks

2013-01-10 Thread Andres Freund
On 2013-01-10 18:00:40 -0300, Alvaro Herrera wrote: Here's version 28 of this patch. The only substantive change here from v26 is that I've made GetTupleForTrigger() use either LockTupleExclusive or LockTupleNoKeyExclusive, depending on whether the key columns are being modified by the

Re: [HACKERS] foreign key locks

2012-12-22 Thread Erik Rijkers
On Thu, November 29, 2012 17:16, Alvaro Herrera wrote: Here it is. fklocks-26.patch.gz This applies today after removing, not only the infamous catversion.h chunk, but also a file_fdw chunk. (It's not really a problem.) I also wondered if anyone has any perl/python/bash programs handy that

Re: [HACKERS] foreign key locks

2012-12-22 Thread Andres Freund
On 2012-12-22 10:53:47 +0100, Erik Rijkers wrote: On Thu, November 29, 2012 17:16, Alvaro Herrera wrote: Here it is. fklocks-26.patch.gz This applies today after removing, not only the infamous catversion.h chunk, but also a file_fdw chunk. (It's not really a problem.) I also

Re: [HACKERS] foreign key locks

2012-11-29 Thread Alvaro Herrera
Kevin Grittner wrote: Kevin Grittner wrote: Alvaro Herrera wrote: Here's version 24. This no longer applies after the rmgr rm_desc patch. I took a shot at merging those so I could review the patch against HEAD; attached in hopes that it will be useful. Hopefully this isn't too

Re: [HACKERS] foreign key locks - EvalPlanQual interactions

2012-11-29 Thread Andres Freund
On 2012-11-27 12:07:34 -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: Here's version 24. Old review emails still contains some items that didn't lead to any changes. I tried to keep close track of those. To that list I add a couple of things of my own. Here they are, for those

Re: [HACKERS] foreign key locks

2012-11-28 Thread Kevin Grittner
Alvaro Herrera wrote: Here's version 24. This no longer applies after the rmgr rm_desc patch. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] foreign key locks

2012-11-27 Thread Alvaro Herrera
Alvaro Herrera wrote: Here's version 24. Old review emails still contains some items that didn't lead to any changes. I tried to keep close track of those. To that list I add a couple of things of my own. Here they are, for those following along. I welcome suggestions. - Fix the multixact

Re: [HACKERS] foreign key locks

2012-11-20 Thread Alvaro Herrera
Andres Freund wrote: - I find using a default: clause in switches with enum types where everything is expected to be handled like the following a bad idea, this way the compiler won't warn you if youve missed case's which makes changing/extending code harder: switch

Re: [HACKERS] foreign key locks

2012-11-20 Thread Andres Freund
On 2012-11-20 17:36:14 -0300, Alvaro Herrera wrote: Andres Freund wrote: - I find using a default: clause in switches with enum types where everything is expected to be handled like the following a bad idea, this way the compiler won't warn you if youve missed case's which makes

Re: [HACKERS] foreign key locks

2012-11-19 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: * In heap_lock_tuple's XMAX_IS_MULTI case [snip] why is it membermode mode and not membermode = mode? Uh, that's a bug. Fixed. As noticed in the comment above that snippet, there was a deadlock

Re: [HACKERS] foreign key locks

2012-11-19 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: * In heap_lock_tuple's XMAX_IS_MULTI case [snip] why is it membermode mode and not membermode = mode? Uh, that's a bug. Fixed. As noticed in the comment above that snippet, there was a deadlock

Re: [HACKERS] foreign key locks

2012-11-17 Thread Andres Freund
On 2012-11-16 22:31:51 -0500, Noah Misch wrote: On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: Andres is on the verge of convincing me that we need to support singleton FOR SHARE without multixacts due to performance

Re: [HACKERS] foreign key locks

2012-11-17 Thread Noah Misch
On Sat, Nov 17, 2012 at 03:20:20PM +0100, Andres Freund wrote: On 2012-11-16 22:31:51 -0500, Noah Misch wrote: On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: Andres is on the verge of convincing me that we need to

Re: [HACKERS] foreign key locks

2012-11-17 Thread Andres Freund
I agree that tripling FOR SHARE cost is risky. Where is the added cost concentrated? Perchance that multiple belies optimization opportunities. Good question, let me play a bit. Ok, I benchmarked around and from what I see there is no single easy target. The biggest culprits I could find

Re: [HACKERS] foreign key locks

2012-11-17 Thread Noah Misch
On Sat, Nov 17, 2012 at 05:07:18PM +0100, Andres Freund wrote: I agree that tripling FOR SHARE cost is risky. Where is the added cost concentrated? Perchance that multiple belies optimization opportunities. Good question, let me play a bit. Ok, I benchmarked around and from what I

Re: [HACKERS] foreign key locks

2012-11-16 Thread Andres Freund
On 2012-11-14 13:27:26 -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: * In heap_lock_tuple's XMAX_IS_MULTI case [snip] why is it membermode mode and not membermode = mode? Uh, that's a bug. Fixed. As noticed in the comment above that snippet, there was a deadlock

Re: [HACKERS] foreign key locks

2012-11-16 Thread Alvaro Herrera
Noah Misch wrote: On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 (I don't think this is worth a v24 submission). Are you aware of any defects in or unanswered questions of this version

Re: [HACKERS] foreign key locks

2012-11-16 Thread Andres Freund
On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: Noah Misch wrote: On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 (I don't think this is worth a v24 submission). Are you aware

Re: [HACKERS] foreign key locks

2012-11-16 Thread Noah Misch
On Fri, Nov 16, 2012 at 05:31:12PM +0100, Andres Freund wrote: On 2012-11-16 13:17:47 -0300, Alvaro Herrera wrote: Andres is on the verge of convincing me that we need to support singleton FOR SHARE without multixacts due to performance concerns. I don't really see any arguments against

Re: [HACKERS] foreign key locks

2012-11-15 Thread Noah Misch
On Wed, Nov 14, 2012 at 01:27:26PM -0300, Alvaro Herrera wrote: https://github.com/alvherre/postgres/commit/df2847e38198e99f57e52490e1e9391ebb70d770 (I don't think this is worth a v24 submission). Are you aware of any defects in or unanswered questions of this version that would stall your

Re: [HACKERS] foreign key locks

2012-11-14 Thread Alvaro Herrera
Alvaro Herrera wrote: * In heap_lock_tuple's XMAX_IS_MULTI case [snip] why is it membermode mode and not membermode = mode? Uh, that's a bug. Fixed. As noticed in the comment above that snippet, there was a deadlock possible here. Maybe I should add a test to ensure this

Re: [HACKERS] foreign key locks

2012-11-13 Thread Alvaro Herrera
Noah Misch wrote: On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: Not really sure about the proposed syntax, but yes clearly we need some other syntax to mean FOR NON KEY UPDATE. I would rather keep FOR UPDATE to mean what I currently call FOR KEY UPDATE. More proposals

Re: [HACKERS] foreign key locks

2012-11-05 Thread Alvaro Herrera
FWIW I have gotten a lot of feedback about this patch, and since I don't have time right now to produce an updated version, that I'm going to close this as Returned with Feedback, and submit an updated version to the upcoming commitfest. This patch still needs much more review -- for example, as

Re: [HACKERS] foreign key locks

2012-11-05 Thread Andres Freund
On Monday, November 05, 2012 02:37:15 PM Alvaro Herrera wrote: FWIW I have gotten a lot of feedback about this patch, and since I don't have time right now to produce an updated version, that I'm going to close this as Returned with Feedback, and submit an updated version to the upcoming

Re: [HACKERS] foreign key locks

2012-11-05 Thread Dimitri Fontaine
Alvaro Herrera alvhe...@2ndquadrant.com writes: FOR NON KEY UPDATE FOR KEY UPDATE KEY is the default, so FOR UPDATE is a synonym of FOR KEY UPDATE Not really sure about the proposed syntax, but yes clearly we need some other syntax to mean FOR NON KEY UPDATE. I would rather keep FOR

Re: [HACKERS] foreign key locks

2012-10-31 Thread Noah Misch
On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: Andres Freund wrote: On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: Andres Freund and...@2ndquadrant.com wrote: * Is it ok to make FOR UPDATE somewhat weaker than before? In 9.2 and earlier you could be sure

Re: [HACKERS] foreign key locks

2012-10-31 Thread Simon Riggs
On 31 October 2012 19:35, Noah Misch n...@leadboat.com wrote: On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: Andres Freund wrote: On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: Andres Freund and...@2ndquadrant.com wrote: * Is it ok to make FOR UPDATE somewhat

Re: [HACKERS] foreign key locks

2012-10-31 Thread Alvaro Herrera
Simon Riggs wrote: On 31 October 2012 19:35, Noah Misch n...@leadboat.com wrote: On Mon, Oct 29, 2012 at 08:18:33AM -0400, Kevin Grittner wrote: Andres Freund wrote: The point is the introduction of a weaker lock level which can be used by the ri triggers. I don't see any imperative

Re: [HACKERS] foreign key locks

2012-10-31 Thread Noah Misch
On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: Simon Riggs wrote: On 31 October 2012 19:35, Noah Misch n...@leadboat.com wrote: +1. I had not considered this angle during previous reviews, but I agree with Andres's position. Since this patch does not strengthen the

Re: [HACKERS] foreign key locks

2012-10-29 Thread Andres Freund
On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: On 27 October 2012 00:06, Andres Freund and...@2ndquadrant.com wrote: On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: Here is version 22 of this patch. This version contains fixes to issues reported by Andres, as

Re: [HACKERS] foreign key locks

2012-10-29 Thread Simon Riggs
On 29 October 2012 12:27, Andres Freund and...@2ndquadrant.com wrote: This sounds like we need a GUC or table-level default to control whether FOR UPDATE means FOR UPDATE or FOR DELETE I don't like adding a new guc for something that should be solveable with some creative naming. If a new

Re: [HACKERS] foreign key locks

2012-10-29 Thread Kevin Grittner
Andres Freund wrote: On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: Andres Freund and...@2ndquadrant.com wrote: * Is it ok to make FOR UPDATE somewhat weaker than before? In 9.2 and earlier you could be sure that if you FOR UPDATE'ed a row you could delete it. Unless I miss

Re: [HACKERS] foreign key locks

2012-10-28 Thread Simon Riggs
On 27 October 2012 00:06, Andres Freund and...@2ndquadrant.com wrote: On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: Here is version 22 of this patch. This version contains fixes to issues reported by Andres, as well as a rebase to latest master. Ok, I now that pgconf.eu has

Re: [HACKERS] foreign key locks

2012-10-19 Thread Noah Misch
On Thu, Oct 18, 2012 at 04:58:20PM -0300, Alvaro Herrera wrote: Here is version 22 of this patch. This version contains fixes to issues reported by Andres, as well as a rebase to latest master. I scanned this for obvious signs of work left to do. It contains numerous XXX and FIXME comments.

Re: [HACKERS] foreign key locks

2012-10-12 Thread Alvaro Herrera
Alvaro Herrera wrote: Andres Freund wrote: * heap_lock_tuple with mode == LockTupleKeyShare nowait looks like it would wait anyway in heap_lock_updated_tuple_rec Oops. I took a stab at fixing this. However, it is not easy. First you need a way to reproduce the problem, which

Re: [HACKERS] foreign key locks

2012-10-11 Thread Andres Freund
On Friday, August 31, 2012 06:59:51 AM Alvaro Herrera wrote: v21 is merged to latest master. Ok, I am starting to look at this. (working with a git merge of alvherre/fklocks into todays master) In a very first pass as somebody who hasn't followed the discussions in the past I took notice of

Re: [HACKERS] foreign key locks

2012-10-11 Thread Alvaro Herrera
Andres Freund wrote: On Friday, August 31, 2012 06:59:51 AM Alvaro Herrera wrote: v21 is merged to latest master. Ok, I am starting to look at this. (working with a git merge of alvherre/fklocks into todays master) In a very first pass as somebody who hasn't followed the discussions in

Re: [HACKERS] foreign key locks

2012-08-24 Thread Robert Haas
On Wed, Aug 22, 2012 at 5:31 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Patch v19 contains some tweaks. Most notably, 1. if an Xid requests a lock A, and then a lock B which is stronger than A, then record only lock B and forget lock A. This is important for performance, because

Re: [HACKERS] foreign key locks

2012-07-05 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of vie jun 29 19:17:02 -0400 2012: I was only testing migrating from an old version into patched, not same-version upgrades. I think I see what's happening here. Okay, I have pushed the fix to github -- as I suspected, code-wise the fix was simple. I

Re: [HACKERS] foreign key locks

2012-06-29 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of mié jun 27 08:40:58 -0400 2012: Alvaro Herrera wrote: here's fklocks v14, which also merges to new master as there were several other conflicts. It passes make installcheck-world now. Recent commits broke it again, so here's a rebased v15.

Re: [HACKERS] foreign key locks

2012-06-27 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of mié jun 27 08:40:58 -0400 2012: Alvaro Herrera wrote: here's fklocks v14, which also merges to new master as there were several other conflicts. It passes make installcheck-world now. Recent commits broke it again, so here's a rebased v15.

Re: [HACKERS] foreign key locks

2012-06-24 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of sáb jun 23 18:38:10 -0400 2012: Alvaro Herrera wrote: So here's the rebased version. I found a couple problems on `make check-world`. Attached is a patch to fix one of them. The other is on pg_upgrade, pasted below. Ah, I mismerged

Re: [HACKERS] foreign key locks

2012-06-23 Thread Kevin Grittner
Alvaro Herrera wrote: So here's the rebased version. I found a couple problems on `make check-world`. Attached is a patch to fix one of them. The other is on pg_upgrade, pasted below. + pg_upgrade -d /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/data.old -D

Re: [HACKERS] foreign key locks

2012-06-20 Thread Jaime Casanova
On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Hi, This is v12 of the foreign key locks patch. Hi Álvaro, Just noticed that this patch needs a rebase because of the refactoring Tom did in ri_triggers.c -- Jaime Casanova         www.2ndQuadrant.com

Re: [HACKERS] foreign key locks

2012-06-20 Thread Tom Lane
Jaime Casanova ja...@2ndquadrant.com writes: On Thu, Jun 14, 2012 at 11:41 AM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: This is v12 of the foreign key locks patch. Just noticed that this patch needs a rebase because of the refactoring Tom did in ri_triggers.c Hold on a bit before you

Re: [HACKERS] foreign key locks, 2nd attempt

2012-04-05 Thread Peter Geoghegan
On 25 March 2012 09:17, Simon Riggs si...@2ndquadrant.com wrote: The main thing we're waiting on are the performance tests to confirm the lack of regression. I have extensively benchmarked the latest revision of the patch (tpc-b.sql), which I pulled from Alvaro's github account. The benchmark

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-25 Thread Simon Riggs
On Sat, Mar 17, 2012 at 10:45 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Here is v11.  This version is mainly updated to add pg_upgrade support, as discussed.  It also contains the README file that was posted earlier (plus wording fixes per Bruce), a couple of bug fixes, and some

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of lun mar 05 15:28:59 -0300 2012: On Mon, Feb 27, 2012 at 2:47 AM, Robert Haas robertmh...@gmail.com wrote: Regarding performance, the good thing about this patch is that if you have an operation that used to block, it might now not block.  So maybe

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mar mar 06 18:33:13 -0300 2012: The lock modes are correct, appropriate and IMHO have meaningful names. No redesign required here. Not sure about the naming of some of the flag bits however. Feel free to suggest improvements ... I've probably seen

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-17 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mar mar 06 17:28:12 -0300 2012: On Tue, Mar 6, 2012 at 7:39 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: We provide four levels of tuple locking strength: SELECT FOR KEY UPDATE is super-exclusive locking (used to delete tuples and more

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie mar 16 00:04:06 -0300 2012: On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote:

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of vie mar 16 10:36:11 -0300 2012: Now I am confused. Where do you see the word hint used by HEAP_XMAX_EXCL_LOCK and HEAP_XMAX_SHARED_LOCK. These are tuple infomask bits, not hints, meaning they are not optional or there just for performance.

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Robert Haas
On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian br...@momjian.us wrote: On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that some worst case performance tests should be done. Could you

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Bruce Momjian
On Fri, Mar 16, 2012 at 10:40:01AM -0300, Alvaro Herrera wrote: Excerpts from Alvaro Herrera's message of vie mar 16 10:36:11 -0300 2012: Now I am confused. Where do you see the word hint used by HEAP_XMAX_EXCL_LOCK and HEAP_XMAX_SHARED_LOCK. These are tuple infomask bits, not

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Bruce Momjian
On Fri, Mar 16, 2012 at 10:08:07AM -0400, Robert Haas wrote: On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian br...@momjian.us wrote: On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-16 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of vie mar 16 15:22:05 -0300 2012: On Fri, Mar 16, 2012 at 10:08:07AM -0400, Robert Haas wrote: On Thu, Mar 15, 2012 at 11:09 PM, Bruce Momjian br...@momjian.us wrote: On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12,

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas robertmh...@gmail.com wrote: You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid lookups, so I think something more sophisticated is needed to exercise that cost.  Not sure what. I don't think HEAP_XMAX_COMMITTED is much

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 2:15 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 14, 2012 at 6:10 PM, Noah Misch n...@leadboat.com wrote: Well, post-release, the cat is out of the bag: we'll be stuck with this whether the performance characteristics are acceptable or not. That's why we'd

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 14, 2012 at 9:17 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Agreed.  But speaking of that, why exactly do we fsync the multixact SLRU today? Good question.  So far, I can't think of a reason.  

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 1:17 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: As things stand today Can I confirm where we are now? Is there another version of the patch coming out soon? --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support,

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas robertmh...@gmail.com wrote: But that would only make sense if we thought that getting rid of the fsyncs would be more valuable than avoiding the blocking here, and I don't.

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 18:46:44 -0300 2012: On Thu, Mar 15, 2012 at 1:17 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: As things stand today Can I confirm where we are now? Is there another version of the patch coming out soon? Yes, another version

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas robertmh...@gmail.com wrote: But that would only make sense if we thought that getting rid

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue mar 15 19:04:41 -0300 2012: On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of jue mar 15 18:38:53 -0300 2012: On Thu, Mar 15, 2012 at 2:26 AM, Robert Haas

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Simon Riggs
On Thu, Mar 15, 2012 at 10:13 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of jue mar 15 19:04:41 -0300 2012: On Thu, Mar 15, 2012 at 9:54 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of jue mar 15

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Robert Haas
On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas robertmh...@gmail.com wrote: You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid lookups, so I think something more sophisticated is needed to exercise

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 15 21:37:36 -0300 2012: On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas robertmh...@gmail.com wrote: You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Noah Misch
On Thu, Mar 15, 2012 at 08:37:36PM -0400, Robert Haas wrote: On Thu, Mar 15, 2012 at 5:07 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 5:23 PM, Robert Haas robertmh...@gmail.com wrote: You still have HEAP_XMAX_{INVALID,COMMITTED} to reduce the pressure on mxid

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: When there is a single locker in a tuple, we can just store the locking info

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Thu, Mar 15, 2012 at 11:04:06PM -0400, Bruce Momjian wrote: On Tue, Mar 13, 2012 at 02:35:02PM -0300, Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: When there is

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-15 Thread Bruce Momjian
On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that some worst case performance tests should be done. Could you please say what you think the worst cases would be, so those can be

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch n...@leadboat.com wrote: More often than that; each 2-member mxid takes 4 bytes in an offsets file and 10 bytes in a members file.  So, more like one fsync per ~580 mxids.  Note that we already fsync the multixact SLRUs today, so any increase will

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Noah Misch
On Wed, Mar 14, 2012 at 01:23:14PM -0400, Robert Haas wrote: On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch n...@leadboat.com wrote: More often than that; each 2-member mxid takes 4 bytes in an offsets file and 10 bytes in a members file. ?So, more like one fsync per ~580 mxids. ?Note that

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Alvaro Herrera
Excerpts from Noah Misch's message of mié mar 14 19:10:00 -0300 2012: On Wed, Mar 14, 2012 at 01:23:14PM -0400, Robert Haas wrote: On Tue, Mar 13, 2012 at 11:42 PM, Noah Misch n...@leadboat.com wrote: More often than that; each 2-member mxid takes 4 bytes in an offsets file and 10

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Wed, Mar 14, 2012 at 6:10 PM, Noah Misch n...@leadboat.com wrote: Well, post-release, the cat is out of the bag: we'll be stuck with this whether the performance characteristics are acceptable or not. That's why we'd better be as sure as possible before committing to this implementation

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-14 Thread Robert Haas
On Wed, Mar 14, 2012 at 9:17 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Agreed.  But speaking of that, why exactly do we fsync the multixact SLRU today? Good question.  So far, I can't think of a reason.  nextMulti is critical, but we already fsync it with pg_control.  We could

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Bruce Momjian
On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: Here's a first attempt at a README illustrating this. I intend this to be placed in src/backend/access/heap/README.tuplock; the first three paragraphs are stolen from the comment in heap_lock_tuple, so I'd remove those from

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Robert Haas
On Mon, Mar 12, 2012 at 9:24 PM, Noah Misch n...@leadboat.com wrote: When we lock an update-in-progress row, we walk the t_ctid chain and lock all descendant tuples.  They may all have uncommitted xmins.  This is essential to ensure that the final outcome of the updating transaction does not

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mar mar 13 14:00:52 -0300 2012: On Tue, Mar 06, 2012 at 04:39:32PM -0300, Alvaro Herrera wrote: When there is a single locker in a tuple, we can just store the locking info in the tuple itself. We do this by storing the locker's Xid in XMAX, and

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Robert Haas
On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that some worst case performance tests should be done. Could you please say what you think the worst cases would be, so those can be tested? That would avoid wasting time or getting anything backwards.

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-13 Thread Noah Misch
On Tue, Mar 13, 2012 at 01:46:24PM -0400, Robert Haas wrote: On Mon, Mar 12, 2012 at 3:28 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you that some worst case performance tests should be done. Could you please say what you think the worst cases would be, so those can be

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Robert Haas
On Sun, Feb 26, 2012 at 9:47 PM, Robert Haas robertmh...@gmail.com wrote: Regarding performance, the good thing about this patch is that if you have an operation that used to block, it might now not block.  So maybe multixact-related operation is a bit slower than before, but if it allows you

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Simon Riggs
On Mon, Mar 12, 2012 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: In other words, we'd entirely avoid needing to make mxacts crash-safe, and we'd avoid most of the extra SLRU lookups that the current implementation requires; they'd only be needed when (and for as long as) the locked

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Robert Haas
On Mon, Mar 12, 2012 at 1:50 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Mar 12, 2012 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: In other words, we'd entirely avoid needing to make mxacts crash-safe, and we'd avoid most of the extra SLRU lookups that the current

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Simon Riggs
On Mon, Mar 12, 2012 at 6:14 PM, Robert Haas robertmh...@gmail.com wrote: Considering that nobody's done any work to resolve the uncertainty about whether the worst-case performance characteristics of this patch are acceptable, and considering further that it was undergoing massive code churn

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-12 Thread Noah Misch
On Mon, Mar 12, 2012 at 01:28:11PM -0400, Robert Haas wrote: I spent some time thinking about this over the weekend, and I have an observation, and an idea. Here's the observation: I believe that locking a tuple whose xmin is uncommitted is always a noop, because if it's ever possible for a

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-07 Thread Gokulakannan Somasundaram
I feel sad, that i followed this topic very late. But i still want to put forward my views. Have we thought on the lines of how Robert has implemented relation level locks. In short it should go like this a) The locks for enforcing Referential integrity should be taken only when the rarest of the

Re: [HACKERS] foreign key locks, 2nd attempt

2012-03-07 Thread Simon Riggs
On Wed, Mar 7, 2012 at 9:24 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: I feel sad, that i followed this topic very late. But i still want to put forward my views. Have we thought on the lines of how Robert has implemented relation level locks. In short it should go like this a)

  1   2   >