Re: [COMMITTERS] pgsql: Remove unreachable code in expression evaluation.

2017-03-25 Thread Andres Freund 
On 2017-03-26 00:14:30 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > Remove unreachable code in expression evaluation.
> > The previous code still contained expression evaluation time support
> > for CaseExprs without a defresult.  But transformCaseExpr() creates a
> > default expression if necessary.
> 
> BTW, now that I think about it, I wonder if a better answer wouldn't
> be to leave the execExpr code as it stood, and remove the code in
> transformCaseExpr that pre-creates a default expression.  You'd end
> up with identical code at runtime, and eliminate some parse/rewrite/plan
> overhead for dealing with that subexpression.
> 
> It's possible there's some code somewhere else that expects defresult
> to always be non-null, but I can't think what, or why that would be
> a good thing to assume.

There's some relationship with the type inferrence code in
transformCaseExpr().  I don't think it'd be hard to change, however -
just move resultexprs = lcons(newc->defresult, resultexprs); into the
else branch of if defresult == NULL.

- Andres


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Remove unreachable code in expression evaluation.

2017-03-25 Thread Tom Lane 
Andres Freund  writes:
> Remove unreachable code in expression evaluation.
> The previous code still contained expression evaluation time support
> for CaseExprs without a defresult.  But transformCaseExpr() creates a
> default expression if necessary.

BTW, now that I think about it, I wonder if a better answer wouldn't
be to leave the execExpr code as it stood, and remove the code in
transformCaseExpr that pre-creates a default expression.  You'd end
up with identical code at runtime, and eliminate some parse/rewrite/plan
overhead for dealing with that subexpression.

It's possible there's some code somewhere else that expects defresult
to always be non-null, but I can't think what, or why that would be
a good thing to assume.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Blindly attempt to fix sepgsql tests #2.

2017-03-25 Thread Andres Freund 
Blindly attempt to fix sepgsql tests #2.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/d253b0f6e3ea829bbc898040477cac6775faab67

Modified Files
--
contrib/sepgsql/expected/alter.out | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Blindly attempt to fix sepgsql tests.

2017-03-25 Thread Andres Freund 
Blindly attempt to fix sepgsql tests.

Due to b8d7f053c5c some permission checks are now happening even on
empty tables, and some of the checks move around.

Discussion: 
https://postgr.es/m/95bdb608-093c-160f-c6be-983a36ccd...@joeconway.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/83bbcb04ab3bce94282907d09f0e152f4ea4050b

Modified Files
--
contrib/sepgsql/expected/alter.out | 3 +++
contrib/sepgsql/expected/ddl.out   | 1 +
contrib/sepgsql/expected/misc.out  | 2 +-
3 files changed, 5 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Faster expression evaluation and targetlist projection.

2017-03-25 Thread Michael Paquier 
On Sun, Mar 26, 2017 at 7:58 AM, David Rowley
 wrote:
> On 26 March 2017 at 09:55, Thomas Munro  wrote:
>> On Sun, Mar 26, 2017 at 11:11 AM, Andres Freund  wrote:
>>> Faster expression evaluation and targetlist projection.
>>>
>>> This replaces the old, recursive tree-walk based evaluation, with
>>> non-recursive, opcode dispatch based, expression evaluation.
>>> Projection is now implemented as part of expression evaluation.
>>>
>>> This both leads to significant performance improvements, and makes
>>> future just-in-time compilation of expressions easier.
>>
>> This is a huge achievement.  Congratulations!
>
> Agreed. Congratulations!

Hurray. Congratulations.
-- 
Michael


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remove unreachable code in expression evaluation.

2017-03-25 Thread Andres Freund 
Remove unreachable code in expression evaluation.

The previous code still contained expression evaluation time support
for CaseExprs without a defresult.  But transformCaseExpr() creates a
default expression if necessary.

Author: Andres Freund
Discussion: https://postgr.es/m/4834.1490480...@sss.pgh.pa.us

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/ad46a2aa79f9ee79aecd889d0ace9ecc08dae984

Modified Files
--
src/backend/executor/execExpr.c | 20 ++--
1 file changed, 6 insertions(+), 14 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Faster expression evaluation and targetlist projection.

2017-03-25 Thread David Rowley 
On 26 March 2017 at 09:55, Thomas Munro  wrote:
> On Sun, Mar 26, 2017 at 11:11 AM, Andres Freund  wrote:
>> Faster expression evaluation and targetlist projection.
>>
>> This replaces the old, recursive tree-walk based evaluation, with
>> non-recursive, opcode dispatch based, expression evaluation.
>> Projection is now implemented as part of expression evaluation.
>>
>> This both leads to significant performance improvements, and makes
>> future just-in-time compilation of expressions easier.
>
> This is a huge achievement.  Congratulations!

Agreed. Congratulations!


-- 
 David Rowley   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Faster expression evaluation and targetlist projection.

2017-03-25 Thread Peter Geoghegan 
On Sat, Mar 25, 2017 at 3:55 PM, Thomas Munro
 wrote:
> This is a huge achievement.  Congratulations!

I also think it's excellent. Well done to all involved.

-- 
Peter Geoghegan


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Faster expression evaluation and targetlist projection.

2017-03-25 Thread Thomas Munro 
On Sun, Mar 26, 2017 at 11:11 AM, Andres Freund  wrote:
> Faster expression evaluation and targetlist projection.
>
> This replaces the old, recursive tree-walk based evaluation, with
> non-recursive, opcode dispatch based, expression evaluation.
> Projection is now implemented as part of expression evaluation.
>
> This both leads to significant performance improvements, and makes
> future just-in-time compilation of expressions easier.

This is a huge achievement.  Congratulations!

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: git rm execQual.c

2017-03-25 Thread Tom Lane 
git rm execQual.c

Should have been in commit b8d7f053c5c2bf2a7e8734fe3327f6a8bc711755,
but passing the patch back and forth as a patch seems to have dropped
that metadata.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/8acf08c68dd8b9fa1793f6ccf9e216cdec63fb49

Modified Files
--
src/backend/executor/execQual.c | 0
1 file changed, 0 insertions(+), 0 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Faster expression evaluation and targetlist projection.

2017-03-25 Thread Andres Freund 
Faster expression evaluation and targetlist projection.

This replaces the old, recursive tree-walk based evaluation, with
non-recursive, opcode dispatch based, expression evaluation.
Projection is now implemented as part of expression evaluation.

This both leads to significant performance improvements, and makes
future just-in-time compilation of expressions easier.

The speed gains primarily come from:
- non-recursive implementation reduces stack usage / overhead
- simple sub-expressions are implemented with a single jump, without
  function calls
- sharing some state between different sub-expressions
- reduced amount of indirect/hard to predict memory accesses by laying
  out operation metadata sequentially; including the avoidance of
  nearly all of the previously used linked lists
- more code has been moved to expression initialization, avoiding
  constant re-checks at evaluation time

Future just-in-time compilation (JIT) has become easier, as
demonstrated by released patches intended to be merged in a later
release, for primarily two reasons: Firstly, due to a stricter split
between expression initialization and evaluation, less code has to be
handled by the JIT. Secondly, due to the non-recursive nature of the
generated "instructions", less performance-critical code-paths can
easily be shared between interpreted and compiled evaluation.

The new framework allows for significant future optimizations. E.g.:
- basic infrastructure for to later reduce the per executor-startup
  overhead of expression evaluation, by caching state in prepared
  statements.  That'd be helpful in OLTPish scenarios where
  initialization overhead is measurable.
- optimizing the generated "code". A number of proposals for potential
  work has already been made.
- optimizing the interpreter. Similarly a number of proposals have
  been made here too.

The move of logic into the expression initialization step leads to some
backward-incompatible changes:
- Function permission checks are now done during expression
  initialization, whereas previously they were done during
  execution. In edge cases this can lead to errors being raised that
  previously wouldn't have been, e.g. a NULL array being coerced to a
  different array type previously didn't perform checks.
- The set of domain constraints to be checked, is now evaluated once
  during expression initialization, previously it was re-built
  every time a domain check was evaluated. For normal queries this
  doesn't change much, but e.g. for plpgsql functions, which caches
  ExprStates, the old set could stick around longer.  The behavior
  around might still change.

Author: Andres Freund, with significant changes by Tom Lane,
changes by Heikki Linnakangas
Reviewed-By: Tom Lane, Heikki Linnakangas
Discussion: 
https://postgr.es/m/20161206034955.bh33paeralxbt...@alap3.anarazel.de

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/b8d7f053c5c2bf2a7e8734fe3327f6a8bc711755

Modified Files
--
contrib/postgres_fdw/postgres_fdw.c   |2 +-
src/backend/bootstrap/bootstrap.c |2 +-
src/backend/catalog/index.c   |   43 +-
src/backend/catalog/partition.c   |3 +-
src/backend/catalog/toasting.c|2 +-
src/backend/commands/analyze.c|   10 +-
src/backend/commands/explain.c|2 +-
src/backend/commands/indexcmds.c  |4 +-
src/backend/commands/prepare.c|4 +-
src/backend/commands/tablecmds.c  |   25 +-
src/backend/commands/trigger.c|   10 +-
src/backend/executor/Makefile |7 +-
src/backend/executor/README   |  176 +-
src/backend/executor/execExpr.c   | 2665 +++
src/backend/executor/execExprInterp.c | 3525 +++
src/backend/executor/execIndexing.c   |   20 +-
src/backend/executor/execMain.c   |   39 +-
src/backend/executor/execQual.c   | 5313 -
src/backend/executor/execSRF.c|  924 +
src/backend/executor/execScan.c   |   10 +-
src/backend/executor/execUtils.c  |  338 +-
src/backend/executor/functions.c  |2 +-
src/backend/executor/nodeAgg.c|   34 +-
src/backend/executor/nodeBitmapHeapscan.c |   17 +-
src/backend/executor/nodeCtescan.c|8 +-
src/backend/executor/nodeCustom.c |8 +-
src/backend/executor/nodeForeignscan.c|   15 +-
src/backend/executor/nodeFunctionscan.c   |   19 +-
src/backend/executor/nodeGather.c |8 +-
src/backend/executor/nodeGatherMerge.c|8 +-
src/backend/executor/nodeGroup.c  |   12 +-
src/backend/executor/nodeHash.c   |   12 +-
src/backend/executor/nodeHashjoin.c   |   44 +-
src/backend/executor/nodeIndexonlyscan.c  |   15 +-
src/backend/executor/nodeIndexscan.c  |   24 +-
src/backend/executor/nodeMergejoin.c  |   33 +-
src/backend/executor/nodeModifyTable.c|

[COMMITTERS] pgsql: Re-adhere to policy of no more than 20 tests per parallel group.

2017-03-25 Thread Tom Lane 
Re-adhere to policy of no more than 20 tests per parallel group.

As explained at the head of parallel_schedule, we place an arbitrary limit
of 20 test cases per parallel group.  Commit c7a9fa399 overlooked this.

Least messy solution seems to be to move the "comments" test to the next
group, since it doesn't really belong in a group of datatype tests anyway.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/7d3957e53ebf26fc8d72dee1dacc2c827cc07caa

Modified Files
--
src/test/regress/parallel_schedule | 4 ++--
src/test/regress/serial_schedule   | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remember to drop roles created by regression tests.

2017-03-25 Thread Tom Lane 
Remember to drop roles created by regression tests.

Commit e3920ac82 created "regress_subscription_user2" in subscription.sql,
but forgot to drop it, causing the regression tests to fail if run twice
without re-initdb'ing.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/2cdc1389a25bc62b36a6f83d5d095acb99e3277c

Modified Files
--
src/test/regress/expected/subscription.out | 1 +
src/test/regress/sql/subscription.sql  | 1 +
2 files changed, 2 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-25 Thread Masahiko Sawada 
On Sun, Mar 26, 2017 at 1:37 AM, Fujii Masao  wrote:
> On Mon, Mar 6, 2017 at 9:37 PM, Masahiko Sawada  wrote:
>> On Fri, Mar 3, 2017 at 10:50 PM, Simon Riggs  wrote:
>>> Allow vacuums to report oldestxmin
>>>
>>> Allow VACUUM and Autovacuum to report the oldestxmin value they
>>> used while cleaning tables, helping to make better sense out of
>>> the other statistics we report in various cases.
>>>
>>> Branch
>>> --
>>> master
>>>
>>> Details
>>> ---
>>> http://git.postgresql.org/pg/commitdiff/9eb344faf54a849898d9be012ddfa8204cfeb57c
>>>
>>> Modified Files
>>> --
>>> src/backend/commands/vacuumlazy.c | 9 +
>>> 1 file changed, 5 insertions(+), 4 deletions(-)
>>>
>>>
>>
>> Should we change the example in vacuum.sgml file as well? Attached patch.
>
> "tuples" in the above should be "row versions"?
> We should review not only this line but also all the lines in the example
> of VERBOSE output, I think.

Right. These verbose log messages are out of date. I ran
VACUUM(VERBOSE, ANALYZE) with same scenario as current example as
possible. Attached patch updates verbose log messages.


Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


fix_vacuum_example_in_doc_v2.patch
Description: Binary data

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Allow vacuums to report oldestxmin

2017-03-25 Thread Fujii Masao 
On Mon, Mar 6, 2017 at 9:37 PM, Masahiko Sawada  wrote:
> On Fri, Mar 3, 2017 at 10:50 PM, Simon Riggs  wrote:
>> Allow vacuums to report oldestxmin
>>
>> Allow VACUUM and Autovacuum to report the oldestxmin value they
>> used while cleaning tables, helping to make better sense out of
>> the other statistics we report in various cases.
>>
>> Branch
>> --
>> master
>>
>> Details
>> ---
>> http://git.postgresql.org/pg/commitdiff/9eb344faf54a849898d9be012ddfa8204cfeb57c
>>
>> Modified Files
>> --
>> src/backend/commands/vacuumlazy.c | 9 +
>> 1 file changed, 5 insertions(+), 4 deletions(-)
>>
>>
>
> Should we change the example in vacuum.sgml file as well? Attached patch.

"tuples" in the above should be "row versions"?
We should review not only this line but also all the lines in the example
of VERBOSE output, I think.

Regards,

-- 
Fujii Masao


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add COMMENT and SECURITY LABEL support for publications and subs

2017-03-25 Thread Stephen Frost 
Peter,

* Peter Eisentraut (pete...@gmx.net) wrote:
> Add COMMENT and SECURITY LABEL support for publications and subscriptions

Isn't this missing psql tab completion, and pg_dump support?  And
regression tests for the latter?

Thanks!

Stephen


signature.asc
Description: Digital signature

[COMMITTERS] pgsql: Add cleanup to new test cases

2017-03-25 Thread Peter Eisentraut 
Add cleanup to new test cases

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/cf366e97ff2f74663754085ee2386bbee39b3edf

Modified Files
--
src/test/modules/dummy_seclabel/expected/dummy_seclabel.out | 2 ++
src/test/modules/dummy_seclabel/sql/dummy_seclabel.sql  | 3 +++
2 files changed, 5 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Report catalog_xmin separately in hot_standby_feedback

2017-03-25 Thread Simon Riggs 
Report catalog_xmin separately in hot_standby_feedback

If the upstream walsender is using a physical replication slot, store the
catalog_xmin in the slot's catalog_xmin field. If the upstream doesn't use a
slot and has only a PGPROC entry behaviour doesn't change, as we store the
combined xmin and catalog_xmin in the PGPROC entry.

Author: Craig Ringer

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/5737c12df0581b3298e3e9586bdef170811ce176

Modified Files
--
doc/src/sgml/protocol.sgml |  33 ++-
src/backend/replication/walreceiver.c  |  43 ++--
src/backend/replication/walsender.c| 108 +++--
src/backend/storage/ipc/procarray.c|  12 ++-
src/include/storage/proc.h |   5 +
src/include/storage/procarray.h|  11 +++
.../recovery/t/010_logical_decoding_timelines.pl   |  38 +++-
7 files changed, 199 insertions(+), 51 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers