Re: [COMMITTERS] pgsql: Add a test for transition table usage in FOR EACH ROW trigger.

2017-05-16 Thread Kevin Grittner
On Tue, May 16, 2017 at 5:18 PM, Kevin Grittner wrote: > I swear I did a `make check-world` before committing! Then I spotted a typo in a comment, fixed the comment, and didn't re-run. Sorry. -- Kevin Grittner VMware vCenter Server https://www.vmware.com/ -- Sent via pgsql-co

Re: [COMMITTERS] pgsql: Add a test for transition table usage in FOR EACH ROW trigger.

2017-05-16 Thread Kevin Grittner
On Tue, May 16, 2017 at 4:32 PM, Andres Freund wrote: > On 2017-05-16 21:12:08 +0000, Kevin Grittner wrote: >> Add a test for transition table usage in FOR EACH ROW trigger. > > The buildfarm does not like this one: > https://buildfarm.postgresql.org/cgi-bin/show_failures.pl

[COMMITTERS] pgsql: Revert "Add a test for transition table usage in FOR EACH ROW tr

2017-05-16 Thread Kevin Grittner
Revert "Add a test for transition table usage in FOR EACH ROW trigger." This reverts commit 4a03f935b3438de27ee00d9e562ffe4e225978a9. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a19ea9c6601bfb06dfd9f4c1060550dbc3f7bde1 Modified Files -- src/test/reg

[COMMITTERS] pgsql: Add a test for transition table usage in FOR EACH ROW trigger.

2017-05-16 Thread Kevin Grittner
Add a test for transition table usage in FOR EACH ROW trigger. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4a03f935b3438de27ee00d9e562ffe4e225978a9 Modified Files -- src/test/regress/expected/sanity_check.out | 1 + src/test/regress/expected/triggers

[COMMITTERS] pgsql: Add GUCs for predicate lock promotion thresholds.

2017-04-07 Thread Kevin Grittner
Add GUCs for predicate lock promotion thresholds. Defaults match the fixed behavior of prior releases, but now DBAs have better options to tune serializable workloads. It might be nice to be able to set this per relation, but that part will need to wait for another release. Author: Dagfinn Ilmar

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 8:26 PM, Euler Taveira wrote: > 2017-04-07 13:06 GMT-03:00 Kevin Grittner : >> >> ERROR: cannot drop table t1 column t1id because other objects depend on >> it >> DETAIL: view v1 depends on table t1 column t1id >> HINT: Use DROP ... CASC

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-04-07 Thread Kevin Grittner
On Thu, Apr 6, 2017 at 6:10 PM, Kevin Grittner wrote: > * About JOINs and dropped columns: although the parser never includes an > * already-dropped column in a JOIN RTE's alias var list, it is possible for > * such a list in a stored rule to include references to dropped col

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-04-06 Thread Kevin Grittner
On Thu, Apr 6, 2017 at 5:20 PM, Kevin Grittner wrote: > I'll commit this fix first so I don't hold up Andres or break any > picky buildfarm critters Done. > and then see whether I can't manage to get > the tests to cover this code. The function in question is on

[COMMITTERS] pgsql: Fix the RTE_NAMEDTUPLESTORE case in get_rte_attribute_is_dropped

2017-04-06 Thread Kevin Grittner
Fix the RTE_NAMEDTUPLESTORE case in get_rte_attribute_is_dropped(). Problems pointed out by Andres Freund and Thomas Munro. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/255efa241f460ee4f4c4c98c8cdd7457807f3af9 Modified Files -- src/backend/parser/pars

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-04-06 Thread Kevin Grittner
I don't hold up Andres or break any picky buildfarm critters and then see whether I can't manage to get the tests to cover this code. Thanks! On Thu, Apr 6, 2017 at 5:16 PM, Andres Freund wrote: > On 2017-04-06 17:03:20 -0500, Kevin Grittner wrote: >> Out of curiosity, what co

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-04-06 Thread Kevin Grittner
rentheses to ensure people read that correctly. Out of curiosity, what compiler or setting catches this? -- Kevin Grittner -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add isolation test for SERIALIZABLE READ ONLY DEFERRABLE.

2017-04-05 Thread Kevin Grittner
Add isolation test for SERIALIZABLE READ ONLY DEFERRABLE. This improves code coverage and lays a foundation for testing similar issues in a distributed environment. Author: Thomas Munro Reviewed-by: Michael Paquier Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/4

[COMMITTERS] pgsql: Follow-on cleanup for the transition table patch.

2017-04-04 Thread Kevin Grittner
transition tables were only supported in C and PL/pgSQL triggers, and correct some ommissions. Thomas Munro with some work by Kevin Grittner (mostly docs) Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/5ebeb579b9b281dba5f8415b2fbda86fdae7b366 Modified Files

[COMMITTERS] pgsql: Fix two undocumented parameters to functions from ENR patch.

2017-04-01 Thread Kevin Grittner
Fix two undocumented parameters to functions from ENR patch. On ProcessUtility document the parameter, to match others. On CreateCachedPlan drop the queryEnv parameter. It was not referenced within the function, and had been added on the assumption that with some unknown future usage of QueryEnv

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-03-31 Thread Kevin Grittner
On Sat, Apr 1, 2017 at 12:21 AM, Kevin Grittner wrote: > On Sat, Apr 1, 2017 at 12:01 AM, Tom Lane wrote: > >> rhinoceros says you missed contrib/sepgsql. > > Yeah, I saw that and have pushed an attempt to fix. That blind fix seemed to work. >> (And while I'm bitch

Re: [COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-03-31 Thread Kevin Grittner
there. > (And while I'm bitching, you definitely failed to update ProcessUtility's > header comment, which like most significant functions takes some pains > to describe all the arguments.) Will fix. -- Kevin Grittner -- Sent via pgsql-committers mailing list (pgsql-commit

[COMMITTERS] pgsql: Try to fix breakage of sepgsql hooks by ENR patch.

2017-03-31 Thread Kevin Grittner
Try to fix breakage of sepgsql hooks by ENR patch. Turned up by buildfarm animal rhinoceros. Fixing blind. Will have to wait for next run by rhinoceros to know whether it worked. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/01fd6f8f2d15a9369768921d6fc95ac4817794

[COMMITTERS] pgsql: Add transition table support to plpgsql.

2017-03-31 Thread Kevin Grittner
Add transition table support to plpgsql. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others Branch -- master Details --- http://git.postgresql.org/pg/commitdiff

[COMMITTERS] pgsql: Add infrastructure to support EphemeralNamedRelation references.

2017-03-31 Thread Kevin Grittner
to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with

[COMMITTERS] pgsql: Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix

2016-12-13 Thread Kevin Grittner
Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix. When there is both a serialization failure and a unique violation, throw the former rather than the latter. When initially pushed, this was viewed as a feature to assist application framework developers, so that they could more acc

[COMMITTERS] pgsql: Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix

2016-12-13 Thread Kevin Grittner
Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix. When there is both a serialization failure and a unique violation, throw the former rather than the latter. When initially pushed, this was viewed as a feature to assist application framework developers, so that they could more acc

[COMMITTERS] pgsql: Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix

2016-12-13 Thread Kevin Grittner
Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix. When there is both a serialization failure and a unique violation, throw the former rather than the latter. When initially pushed, this was viewed as a feature to assist application framework developers, so that they could more acc

[COMMITTERS] pgsql: Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix

2016-12-13 Thread Kevin Grittner
Back-patch fcff8a575198478023ada8a48e13b50f70054766 as a bug fix. When there is both a serialization failure and a unique violation, throw the former rather than the latter. When initially pushed, this was viewed as a feature to assist application framework developers, so that they could more acc

[COMMITTERS] pgsql: Improve tab completion for CREATE TRIGGER.

2016-11-04 Thread Kevin Grittner
Improve tab completion for CREATE TRIGGER. This includes support for the new REFERENCING clause. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/927d7bb6b120a2ca09a164898f887eb850b7a329 Modified Files -- src/bin/psql/tab-complete.c | 34 +

[COMMITTERS] pgsql: Implement syntax for transition tables in AFTER triggers.

2016-11-04 Thread Kevin Grittner
Implement syntax for transition tables in AFTER triggers. This is infrastructure for the complete SQL standard feature. No support is included at this point for execution nodes or PLs. The intent is to add that soon. As this patch leaves things, standard syntax can create tuplestores to contain

[COMMITTERS] pgsql: Fix recent commit for tab-completion of database template.

2016-09-12 Thread Kevin Grittner
Fix recent commit for tab-completion of database template. The details of commit 52803098ab26051c0c9802f3c19edffc90de8843 were based on a misunderstanding of the role inheritance allowing use of a database for a template. While the CREATEDB privilege is not inherited, the database ownership is pr

[COMMITTERS] pgsql: psql tab completion for CREATE DATABASE ... TEMPLATE ...

2016-09-11 Thread Kevin Grittner
psql tab completion for CREATE DATABASE ... TEMPLATE ... Sehrope Sarkuni, reviewed by Merlin Moncure & Vitaly Burovoy with some editing by me Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/52803098ab26051c0c9802f3c19edffc90de8843 Modified Files -- src/b

[COMMITTERS] pgsql: Improve tab completion for BEGIN & START|SET TRANSACTION.

2016-09-01 Thread Kevin Grittner
Improve tab completion for BEGIN & START|SET TRANSACTION. Andreas Karlsson with minor change by me for SET TRANSACTION SNAPSHOT. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/76f9dd4fa82270899f7b56b002b5d34226dc99d8 Modified Files -- src/bin/psql/tab-c

[COMMITTERS] pgsql: Remove unnecessary #include.

2016-08-24 Thread Kevin Grittner
Remove unnecessary #include. Accidentally added in 8b65cf4c5edabdcae45ceaef7b9ac236879aae50. Pointed out by Álvaro Herrera Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/5cd3864075622b203d530f1a710818777859304e Modified Files -- src/include/storage/buf

[COMMITTERS] pgsql: Remove unnecessary #include.

2016-08-24 Thread Kevin Grittner
Remove unnecessary #include. Accidentally added in 8b65cf4c5edabdcae45ceaef7b9ac236879aae50. Pointed out by Álvaro Herrera Branch -- REL9_6_STABLE Details --- http://git.postgresql.org/pg/commitdiff/eaae83c123f5e8e103abbbe822fe73b791d9d5c9 Modified Files -- src/include/stor

[COMMITTERS] pgsql: Add OldSnapshotTimeMapLock to wait_event table in docs.

2016-08-03 Thread Kevin Grittner
Add OldSnapshotTimeMapLock to wait_event table in docs. Ashutosh Sharma with minor fixes by me. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/c93d8737be47667091b36f5852fd706146514008 Modified Files -- doc/src/sgml/monitoring.sgml | 6 +- 1 file chan

[COMMITTERS] pgsql: Add comment & docs about no vacuum truncation with sto.

2016-07-19 Thread Kevin Grittner
Add comment & docs about no vacuum truncation with sto. Omission noted by Andres Freund. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/1c15aac53f3421fd38ae145118d3204f055ba955 Modified Files -- doc/src/sgml/config.sgml | 9 + src/backen

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-16 Thread Kevin Grittner
um, tuptoaster.c:1945 Thanks! That's something I should be able to work with. Unfortunately, I am going to be on vacation, so I won't have any results until sometime after 28 June. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-commit

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-16 Thread Kevin Grittner
On Thu, Jun 16, 2016 at 1:19 PM, Kevin Grittner wrote: > On Thu, Jun 16, 2016 at 11:54 AM, Andres Freund wrote: >> On 2016-06-16 12:43:34 -0400, Robert Haas wrote: >>> Maybe it would help if you lay out the whole sequence of events, like: >>> >>> S1: Does

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-16 Thread Kevin Grittner
e. > > I presume it'd be something like: > > Assuming a 'toasted' table, which contains one row, with a 1GB field. > > S1: BEGIN REPEATABLE READ; > S1: SELECT SUM(length(one_gb_record)) FROM toasted; > S2: DELETE FROM toasted; > AUTOVAC: vacuum toasted's toa

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-16 Thread Kevin Grittner
liest still-active transaction ID. If it did we would have all sorts of weird problems. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-16 Thread Kevin Grittner
ny benefit. And we certainly don't want to add some new way to prune TOAST tuples which might still have referencing heap tuples; that could provide a route to *create* the problem you describe. I am on vacation tomorrow (Friday the 17th) through Monday the 27th, so I will be unable to r

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 5:34 PM, Andres Freund wrote: > On 2016-06-15 16:58:25 -0500, Kevin Grittner wrote: >> On Wed, Jun 15, 2016 at 3:25 PM, Andres Freund wrote: >>> On 2016-06-15 14:24:58 -0500, Kevin Grittner wrote: >>>> On Wed, Jun 15, 2016 at 2:20 PM, Andre

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 2:40 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: >> On Wed, Jun 15, 2016 at 1:59 PM, Alvaro Herrera >> wrote: > >> > We actually go quite some lengths to support this case, even when it's >> > the opinion of many that we s

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 3:25 PM, Andres Freund wrote: > On 2016-06-15 14:24:58 -0500, Kevin Grittner wrote: >> On Wed, Jun 15, 2016 at 2:20 PM, Andres Freund wrote: >> >> > We might fetch a toast tuple which >> > since have been re-purposed for a datum of a dif

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 2:20 PM, Andres Freund wrote: > We might fetch a toast tuple which > since have been re-purposed for a datum of a different type. How would that happen? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-comm

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 1:59 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: >> On Wed, Jun 15, 2016 at 1:50 PM, Robert Haas wrote: >> >>> The expression index case is the one to worry about; if there is a >>> problem, that's where it is. What bother

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
error inside the expression which is trying to generate the index value. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
IMMUTABLE when it actually isn't, and then updating the referenced rows (which is very likely to render the index corrupted), that statistics might also become stale. They might. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committe

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
only using values from the row itself. In such a case you would have had to mark a function as IMMUTABLE which depends on table contents. I say you get to keep both pieces. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 1:29 PM, Robert Haas wrote: > On Wed, Jun 15, 2016 at 1:44 PM, Kevin Grittner > wrote:>> So what happens in this scenario: >>> 1. ANALYZE runs really slowly - maybe the user-defined function it's >>> running for the expression ind

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-15 Thread Kevin Grittner
On Wed, Jun 15, 2016 at 10:46 AM, Robert Haas wrote: > On Sat, Jun 11, 2016 at 11:29 AM, Kevin Grittner wrote: >> I have reviewed the code and run tests to try to find something >> here which could be considered a bug, without finding any problem. >> When reading pages for

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-11 Thread Kevin Grittner
On Tue, May 24, 2016 at 4:10 PM, Robert Haas wrote: > On Tue, May 24, 2016 at 3:48 PM, Kevin Grittner wrote: >> On Tue, May 24, 2016 at 12:00 PM, Andres Freund wrote: >>> On 2016-05-24 11:24:44 -0500, Kevin Grittner wrote: >>>> On Fri, May 6, 2016 at 8:28 PM, Kevin

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-10 Thread Kevin Grittner
On Fri, Jun 10, 2016 at 10:26 AM, Robert Haas wrote: > On Fri, Jun 10, 2016 at 10:45 AM, Kevin Grittner wrote: >> Since vacuum calls the pruning function, and not the other way >> around, the name you suggest would be technically more correct. >> Committed using "Pruni

[COMMITTERS] pgsql: Rename local variable for consistency.

2016-06-10 Thread Kevin Grittner
Rename local variable for consistency. Pointed out by Robert Haas. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/13761bccb177022c8c0dabc08f3e9acb491b1c96 Modified Files -- src/backend/catalog/index.c | 8 1 file changed, 4 insertions(+), 4 dele

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-10 Thread Kevin Grittner
ggest would be technically more correct. Committed using "Pruning" instead of "Vacuum" in both new macros. I have closed the CREATE INDEX versus "snapshot too old" issue in the "PostgreSQL 9.6 Open Items" Wiki page. -- Kevin Grittner EDB: http://www.enterprised

[COMMITTERS] pgsql: Fix interaction between CREATE INDEX and "snapshot too old".

2016-06-10 Thread Kevin Grittner
Fix interaction between CREATE INDEX and "snapshot too old". Since indexes are created without valid LSNs, an index created while a snapshot older than old_snapshot_threshold existed could cause queries to return incorrect results when those old snapshots were used, if any relevant rows had been s

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-09 Thread Kevin Grittner
[Thanks to Robert to stepping up to keep this moving while I was down yesterday with a minor injury. I'm back on it today.] On Wed, Jun 8, 2016 at 3:11 PM, Robert Haas wrote: > On Wed, Jun 8, 2016 at 4:04 PM, Kevin Grittner wrote: >> -- connection 1 >> drop table if exists

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-08 Thread Kevin Grittner
The fix prevents use of such an index until all snapshots early enough to have a problem have been released. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make cha

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-08 Thread Kevin Grittner
On Tue, Jun 7, 2016 at 10:40 AM, Robert Haas wrote: > On Sat, Jun 4, 2016 at 4:21 PM, Kevin Grittner wrote: >> the minimal patch to fix behavior in this area would be: >> >> diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c >> index 31a1438..

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-04 Thread Kevin Grittner
On Fri, Jun 3, 2016 at 4:24 PM, Kevin Grittner wrote: > Consequently, causing the index to be ignored in planning when > using the old index That last line should have read "using an old snapshot" > is not a nice optimization, but necessary for > correctness. We already

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-03 Thread Kevin Grittner
On Fri, May 27, 2016 at 10:35 AM, Kevin Grittner wrote: > On Tue, May 24, 2016 at 4:10 PM, Robert Haas wrote: >> [ANALYZE of index with expression may fail to update statistics >> if ANALYZE runs longer than old_snapshot_threshold] >> If we can get away with it, it would

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-06-03 Thread Kevin Grittner
On Fri, May 27, 2016 at 10:18 AM, Kevin Grittner wrote: > On Fri, May 27, 2016 at 9:58 AM, Kevin Grittner wrote: >> On Tue, May 24, 2016 at 4:56 PM, Andres Freund wrote: > >>> If an old session with >= repeatable read accesses a clustered >>> table (after the cl

[COMMITTERS] pgsql: Add new snapshot fields to serialize/deserialize functions.

2016-06-03 Thread Kevin Grittner
Add new snapshot fields to serialize/deserialize functions. The "snapshot too old" condition was not being recognized when using a copied snapshot, since the original timestamp and lsn were not being passed along. Noticed when testing the combination of "snapshot too old" with parallel query exec

[COMMITTERS] pgsql: C comment improvement & typo fix.

2016-06-02 Thread Kevin Grittner
C comment improvement & typo fix. Thomas Munro Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/4edb7bd2fd6a48f6104c73551423cb208e13e529 Modified Files -- src/include/access/nbtree.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- Sent via pgsq

[COMMITTERS] pgsql: Fix btree mark/restore bug.

2016-06-02 Thread Kevin Grittner
Fix btree mark/restore bug. Commit 2ed5b87f96d473962ec5230fd820abfeaccb2069 introduced a bug in mark/restore, in an attempt to optimize repeated restores to the same page. This caused an assertion failure during a merge join which fed directly from an index scan, although the impact would not be

[COMMITTERS] pgsql: Fix btree mark/restore bug.

2016-06-02 Thread Kevin Grittner
Fix btree mark/restore bug. Commit 2ed5b87f96d473962ec5230fd820abfeaccb2069 introduced a bug in mark/restore, in an attempt to optimize repeated restores to the same page. This caused an assertion failure during a merge join which fed directly from an index scan, although the impact would not be

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-31 Thread Kevin Grittner
On Tue, May 31, 2016 at 10:03 AM, Robert Haas wrote: > On Fri, May 27, 2016 at 10:58 AM, Kevin Grittner wrote: >>> As far as I can see normal index builds will allow concurrent hot >>> prunes and everything; since those only require page-level >>> exclusive loc

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
On Tue, May 24, 2016 at 4:10 PM, Robert Haas wrote: > On Tue, May 24, 2016 at 3:48 PM, Kevin Grittner wrote: >> On Tue, May 24, 2016 at 12:00 PM, Andres Freund wrote: >>> On 2016-05-24 11:24:44 -0500, Kevin Grittner wrote: >>>> On Fri, May 6, 2016 at 8:28 PM, Kevin

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
On Fri, May 27, 2016 at 9:58 AM, Kevin Grittner wrote: > On Tue, May 24, 2016 at 4:56 PM, Andres Freund wrote: >> If an old session with >= repeatable read accesses a clustered >> table (after the cluster committed), they'll now not see any >> errors, because all

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-27 Thread Kevin Grittner
t is new LSNs that trigger errors; if the page has not been written recently the LSN is old and there is no error. I think you may be seeing problems based on getting the basics of this backwards. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsq

[COMMITTERS] pgsql: Update doc text to reflect new column in MVCC phenomena table.

2016-05-25 Thread Kevin Grittner
Update doc text to reflect new column in MVCC phenomena table. Scott Wehrenberg Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/627e360358e3beb67cd2f54393835f979c5e30b7 Modified Files -- doc/src/sgml/mvcc.sgml | 3 ++- 1 file changed, 2 insertions(+), 1 d

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
On Tue, May 24, 2016 at 4:09 PM, Kevin Grittner wrote: > On Tue, May 24, 2016 at 3:54 PM, Andres Freund wrote: >> It appears that concurrent index builds are currently broken >> from a quick skim? > > Either you don't understand this feature very well, or I don'

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
to work up a test case for this. If you have one, please send it along to me. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.po

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
chunks of code and asking why an error isn't thrown there without showing that you get bad results otherwise, or at least some plausible argument why you might. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
On Tue, May 24, 2016 at 12:00 PM, Andres Freund wrote: > On 2016-05-24 11:24:44 -0500, Kevin Grittner wrote: >> On Fri, May 6, 2016 at 8:28 PM, Kevin Grittner wrote: >>> On Fri, May 6, 2016 at 7:48 PM, Andres Freund wrote: >> >>>> That comment reminds me o

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
ng that would cause the ANALYZE command in my test to use a snapshot other than the one from the REPEATABLE READ transaction in which it was run? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (pgsql-committers@postg

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-24 Thread Kevin Grittner
On Fri, May 6, 2016 at 8:28 PM, Kevin Grittner wrote: > On Fri, May 6, 2016 at 7:48 PM, Andres Freund wrote: >> That comment reminds me of a question I had: Did you consider the effect >> of this patch on analyze? It uses a snapshot, and by memory you've not >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-06 Thread Kevin Grittner
On Fri, May 6, 2016 at 7:48 PM, Andres Freund wrote: > On 2016-05-06 19:43:24 -0500, Kevin Grittner wrote: >> It's disappointing that I am not getting more consistent numbers, >> but NUMA can be hard to manage that way. > > FWIW, in my experience, unless you disable au

[COMMITTERS] pgsql: Mitigate "snapshot too old" performance regression on NUMA

2016-05-06 Thread Kevin Grittner
mments and whitespace. Ants Aasma, with cosmetic adjustments suggested by Andres Freund Reviewed by Kevin Grittner and Andres Freund Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/7e3da1c4737fd6582e12c80983987e4d2cbc1d17 Modified Files -- src/backend/

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-06 Thread Kevin Grittner
On Fri, May 6, 2016 at 5:07 PM, Andres Freund wrote: > On 2016-05-06 14:18:22 -0500, Kevin Grittner wrote: >> I rebased the patch Ants posted (attached), and am running >> benchmarks on a cthulhu (a big NUMA machine with 8 memory nodes). >> Normally I wouldn't post

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-06 Thread Kevin Grittner
ll showing similar numbers. If the BF chokes on it I would revert this optimization attempt. Thoughts? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company update-old-snapshot-map-once-per-tick-v2.patch Description: binary/octet-stream -- Sent via pgsql-com

[COMMITTERS] pgsql: Fix hash index vs "snapshot too old" problemms

2016-05-06 Thread Kevin Grittner
Fix hash index vs "snapshot too old" problemms Hash indexes are not WAL-logged, and so do not maintain the LSN of index pages. Since the "snapshot too old" feature counts on detecting error conditions using the LSN of a table and all indexes on it, this makes it impossible to safely do early vacu

[COMMITTERS] pgsql: Add a few entries to the tail of time mapping, to see old values

2016-04-29 Thread Kevin Grittner
Add a few entries to the tail of time mapping, to see old values. Without a few entries beyond old_snapshot_threshold, the lookup would often fail, resulting in the more aggressive pruning or vacuum being skipped often enough to matter. This was very clearly shown by a python test script posted b

[COMMITTERS] pgsql: Fix C comment typo and redundant test

2016-04-25 Thread Kevin Grittner
Fix C comment typo and redundant test Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/e65953be4f540dce31f17db2934ee58365077272 Modified Files -- src/backend/utils/time/snapmgr.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) -- Sent via pg

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-21 Thread Kevin Grittner
On Thu, Apr 21, 2016 at 2:10 PM, Ants Aasma wrote: > On Thu, Apr 21, 2016 at 5:16 PM, Kevin Grittner wrote: >> Could you provide enough to make that a self-contained >> reproducible test case [?] > [provided] Thanks! I have your test case running, and it is not immediately

[COMMITTERS] pgsql: Include snapmgr.h in blscan.c

2016-04-21 Thread Kevin Grittner
Include snapmgr.h in blscan.c Windows builds on buildfarm are failing because old_snapshot_threshold is not found in the bloom filter contrib module. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/7cb1db1d9599f0a09d6920d2149d956ef6d88b0e Modified Files

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-21 Thread Kevin Grittner
on with a transaction ID and an xmin which prevented cleanup from advancing). I'll be happy to look at your case and see whether it's another such case or some bug, but it seems a waste to reverse engineer or rewrite parts of the test case to do so. -- Kevin Grittner EDB: http://w

[COMMITTERS] pgsql: Inline initial comparisons in TestForOldSnapshot()

2016-04-21 Thread Kevin Grittner
Inline initial comparisons in TestForOldSnapshot() Even with old_snapshot_threshold = -1 (which disables the "snapshot too old" feature), performance regressions were seen at moderate to high concurrency. For example, a one-socket, four-core system running 200 connections at saturation could see

[COMMITTERS] pgsql: Revert no-op changes to BufferGetPage()

2016-04-20 Thread Kevin Grittner
Revert no-op changes to BufferGetPage() The reverted changes were intended to force a choice of whether any newly-added BufferGetPage() calls needed to be accompanied by a test of the snapshot age, to support the "snapshot too old" feature. Such an accompanying test is needed in about 7% of the c

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-19 Thread Kevin Grittner
On Tue, Apr 19, 2016 at 10:14 AM, Robert Haas wrote: > On Tue, Apr 19, 2016 at 11:11 AM, Kevin Grittner wrote: >> On Tue, Apr 19, 2016 at 9:57 AM, Amit Kapila wrote: >>> On Sun, Apr 17, 2016 at 2:26 AM, Andres Freund wrote: >>>> >>>> On 2016-04-16 16:

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-19 Thread Kevin Grittner
s > enabled. Thanks, Amit -- I think something along those lines is the right solution to the scaling issues when the feature is enabled. For now I'm focusing on the back-patching issues and the performance regression when the feature is disabled, but I'll shift focus to this once th

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 3:47 PM, Andres Freund wrote: > On 2016-04-13 15:21:31 -0500, Kevin Grittner wrote: >> What is the kernel on which these tests were run? > > 3.16. I can upgrade to 4.4 if necessary. No, I'm not aware of any problems from 3.8 on. > But I still bel

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
can we confirm that this is a real difference in what is at master's HEAD? Of course, I'm still scheduled to test on bare metal machines in a couple days, on two different architectures, so we'll have a few more data points after that. -- Kevin Grittner EDB: http://www.enterprise

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > I'll run with -1 once the current (longer) run has finished. Just for the record, were any of the other results purporting to be with the feature "off" also actually running with the feature set for its fastest possible

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
t there is a significant effect even with it > turned off. No evidence of that has been provided. -1 is off; 0 is for testing very fast expiration. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-committers mailing list (p

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-12 Thread Kevin Grittner
On Tue, Apr 12, 2016 at 2:53 PM, Kevin Grittner wrote: > Readonly with client and job counts matching scale. Single-socket i7, BTW. >> A lot of this will be different between >> single-socket and multi-socket servers; as soon as you have the latter >> the likelihood of cont

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-12 Thread Kevin Grittner
On Tue, Apr 12, 2016 at 2:28 PM, Andres Freund wrote: > On 2016-04-12 14:17:12 -0500, Kevin Grittner wrote: >> Well, something is different between your environment and mine, >> since I saw no difference at scale 100 and 2.2% at scale 200. > > In a readonly test or r/w? Rea

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-12 Thread Kevin Grittner
On Tue, Apr 12, 2016 at 1:56 PM, Andres Freund wrote: > On 2016-04-12 13:44:00 -0500, Kevin Grittner wrote: >> On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: >>> On 2016-04-12 16:49:25 +0000, Kevin Grittner wrote: >>>> On a big NUMA machine with 1000

Re: [COMMITTERS] pgsql: Add the "snapshot too old" feature

2016-04-12 Thread Kevin Grittner
On Mon, Apr 11, 2016 at 12:31 PM, Kevin Grittner wrote: > On Mon, Apr 11, 2016 at 8:20 AM, Alexander Korotkov >> So, for read-only benchmark this patch introduces more than 5 times >> regression on big machine. > > I did not schedule a saturation test on a big machine. I

Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-12 Thread Kevin Grittner
On Tue, Apr 12, 2016 at 12:38 PM, Andres Freund wrote: > On 2016-04-12 16:49:25 +0000, Kevin Grittner wrote: >> On a big NUMA machine with 1000 connections in saturation load >> there was a performance regression due to spinlock contention, for >> acquiring values which w

[COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-12 Thread Kevin Grittner
Avoid extra locks in GetSnapshotData if old_snapshot_threshold < 0 On a big NUMA machine with 1000 connections in saturation load there was a performance regression due to spinlock contention, for acquiring values which were never used. Just fill with dummy values if we're not going to use them.

[COMMITTERS] pgsql: Use static inline function for BufferGetPage()

2016-04-11 Thread Kevin Grittner
Use static inline function for BufferGetPage() I was initially concerned that the some of the hundreds of references to BufferGetPage() where the literal BGP_NO_SNAPSHOT_TEST were passed might not optimize as well as a macro, leading to some hard-to-find performance regressions in corner cases. I

[COMMITTERS] pgsql: Make oldSnapshotControl a pointer to a volatile structure

2016-04-11 Thread Kevin Grittner
Make oldSnapshotControl a pointer to a volatile structure It was incorrectly declared as a volatile pointer to a non-volatile structure. Eliminate the OldSnapshotControl struct definition; it is really not needed. Pointed out by Tom Lane. While at it, add OldSnapshotControlData to pgindent's li

Re: [COMMITTERS] pgsql: Add the "snapshot too old" feature

2016-04-11 Thread Kevin Grittner
ly sufficient. I am looking at this and the misuse of "volatile" now. Are you able to easily test that or should I book time on one (or more) of our big machines on my end? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-com

  1   2   3   >