Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Peter Geoghegan]

On Thu, Nov 9, 2017 at 4:53 PM, Andres Freund  wrote:
> Primarily because it's not an anti-corruption tool. I'd be surprised if
> there weren't ways to corrupt the page using these corruptions that
> aren't detected by it.

It's very hard to assess the risk of missing something that's actually
detectable with total confidence, but I think that the check is actually
very thorough.

> But even if it were, I don't think there's
> enough information to do so in the general case. You very well can end
> up with pages where subsequent hot pruning has removed a good bit of the
> direct evidence of this bug.

Sure, but maybe those are cases that can't get any worse anyway. So the
question of avoiding making it worse doesn't arise.

> But I'm not really sure why the error detection capabilities of matter
> much for the principal point I raised, which is how much work we need to
> do to not further worsen the corruption.

You're right. Just trying to put the risk in context, and to understand the
extent of the concern that you have.

-- 
Peter Geoghegan

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Andres Freund]

On 2017-11-09 16:45:07 -0800, Peter Geoghegan wrote:
> On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund  wrote:
> >> Actually, on second thought, I take that back -- I don't think that
> >> REINDEXing will even finish once a HOT chain is broken by the bug.
> >> IndexBuildHeapScan() actually does quite a good job of making sure
> >> that HOT chains are sane, which is how the enhanced amcheck notices
> >> the bug here in practice.
> >
> > I think that's too optimistic.
> 
> Why? Because the "find the TID of the root" logic in
> IndexBuildHeapScan()/heap_get_root_tuples() won't reliably find the
> actual root (it might be some other HOT chain root following TID
> recycling by VACUUM)?

Primarily because it's not an anti-corruption tool. I'd be surprised if
there weren't ways to corrupt the page using these corruptions that
aren't detected by it.  But even if it were, I don't think there's
enough information to do so in the general case.  You very well can end
up with pages where subsequent hot pruning has removed a good bit of the
direct evidence of this bug.

But I'm not really sure why the error detection capabilities of matter
much for the principal point I raised, which is how much work we need to
do to not further worsen the corruption.

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Peter Geoghegan]

On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund  wrote:
>> Actually, on second thought, I take that back -- I don't think that
>> REINDEXing will even finish once a HOT chain is broken by the bug.
>> IndexBuildHeapScan() actually does quite a good job of making sure
>> that HOT chains are sane, which is how the enhanced amcheck notices
>> the bug here in practice.
>
> I think that's too optimistic.

Why? Because the "find the TID of the root" logic in
IndexBuildHeapScan()/heap_get_root_tuples() won't reliably find the
actual root (it might be some other HOT chain root following TID
recycling by VACUUM)?

Assuming that's what you meant: I would have thought that the
xmin/xmax matching within heap_get_root_tuples() makes the sanity
checking fairly reliable in practice.

-- 
Peter Geoghegan


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Peter Geoghegan]

On Thu, Nov 9, 2017 at 4:17 PM, Andres Freund  wrote:
>> I don't follow you here. Why would REINDEXing make the rows that
>> should be dead disappear again, even for a short period of time?
>
> It's not the REINDEX that makes them reappear.

Of course. I was just trying to make sense of what you said.

> It's the second
> vacuum. The reindex part was about $user trying to fix the problem...
> As you need two vacuums with appropriate cutoffs to hit the "rows
> revive" problem, that'll often in practice not happen immediately.

This explanation clears things up, though.

-- 
Peter Geoghegan


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Andres Freund]

On 2017-11-09 16:02:17 -0800, Peter Geoghegan wrote:
> > What I'm currently wondering about is how much we need to harden
> > postgres against such existing corruption. If e.g. the hot chains are
> > broken somebody might have reindexed thinking the problem is fixed - but
> > if they then later vacuum everything goes to shit again, with dead rows
> > reappearing.
> 
> I don't follow you here. Why would REINDEXing make the rows that
> should be dead disappear again, even for a short period of time?

It's not the REINDEX that makes them reappear. It's the second
vacuum. The reindex part was about $user trying to fix the problem...
As you need two vacuums with appropriate cutoffs to hit the "rows
revive" problem, that'll often in practice not happen immediately.


> Actually, on second thought, I take that back -- I don't think that
> REINDEXing will even finish once a HOT chain is broken by the bug.
> IndexBuildHeapScan() actually does quite a good job of making sure
> that HOT chains are sane, which is how the enhanced amcheck notices
> the bug here in practice.

I think that's too optimistic.

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Peter Geoghegan]

On Thu, Nov 9, 2017 at 2:24 PM, Andres Freund  wrote:
> Attached is a version of the already existing regression test that both
> reproduces the broken hot chain (and thus failing index lookups) and
> then also the tuple reviving.  I don't see any need for letting this run
> with arbitrary permutations.

I thought that the use of every possible permutation was excessive,
myself. It left us with an isolation test that didn't precisely
describe the behavior that is tested. What you came up with seems far,
far better, especially because of the comments you included. The mail
message-id references seem to add a lot, too.

> What I'm currently wondering about is how much we need to harden
> postgres against such existing corruption. If e.g. the hot chains are
> broken somebody might have reindexed thinking the problem is fixed - but
> if they then later vacuum everything goes to shit again, with dead rows
> reappearing.

I don't follow you here. Why would REINDEXing make the rows that
should be dead disappear again, even for a short period of time? It
might do so for index scans, I suppose, but not for sequential scans.
Are you concerned about a risk of somebody not noticing that
sequential scans are still broken?

Actually, on second thought, I take that back -- I don't think that
REINDEXing will even finish once a HOT chain is broken by the bug.
IndexBuildHeapScan() actually does quite a good job of making sure
that HOT chains are sane, which is how the enhanced amcheck notices
the bug here in practice. (Before this bug was discovered, I would
have expected amcheck to catch problems like it slightly later, during
the Bloom filter probe for that HOT chain...but, in fact, it never
gets there with corruption from this bug in practice, AFAIK.)

-- 
Peter Geoghegan


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add hash partitioning.

[Robert Haas]

Add hash partitioning.

Hash partitioning is useful when you want to partition a growing data
set evenly.  This can be useful to keep table sizes reasonable, which
makes maintenance operations such as VACUUM faster, or to enable
partition-wise join.

At present, we still depend on constraint exclusion for partitioning
pruning, and the shape of the partition constraints for hash
partitioning is such that that doesn't work.  Work is underway to fix
that, which should both improve performance and make partitioning
pruning work with hash partitioning.

Amul Sul, reviewed and tested by Dilip Kumar, Ashutosh Bapat, Yugo
Nagata, Rajkumar Raghuwanshi, Jesper Pedersen, and by me.  A few
final tweaks also by me.

Discussion: 
http://postgr.es/m/CAAJ_b96fhpJAP=albetmelk1uni_gfzd938zgenhf49qgdt...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/1aba8e651ac3e37e1d2d875842de1e0ed22a651e

Modified Files
--
doc/src/sgml/ddl.sgml|  28 +-
doc/src/sgml/ref/alter_table.sgml|   7 +
doc/src/sgml/ref/create_table.sgml   |  85 +++-
src/backend/catalog/partition.c  | 682 ---
src/backend/commands/tablecmds.c |  48 +-
src/backend/nodes/copyfuncs.c|   2 +
src/backend/nodes/equalfuncs.c   |   2 +
src/backend/nodes/outfuncs.c |   2 +
src/backend/nodes/readfuncs.c|   2 +
src/backend/optimizer/path/joinrels.c|  12 +-
src/backend/parser/gram.y|  76 ++-
src/backend/parser/parse_utilcmd.c   |  29 +-
src/backend/utils/adt/ruleutils.c|  15 +-
src/backend/utils/cache/relcache.c   |  26 +-
src/bin/psql/tab-complete.c  |   2 +-
src/include/catalog/catversion.h |   2 +-
src/include/catalog/partition.h  |   3 +
src/include/catalog/pg_proc.h|   4 +
src/include/nodes/parsenodes.h   |   8 +-
src/test/regress/expected/alter_table.out|  62 +++
src/test/regress/expected/create_table.out   |  78 ++-
src/test/regress/expected/insert.out |  46 ++
src/test/regress/expected/partition_join.out |  81 
src/test/regress/expected/update.out |  29 ++
src/test/regress/sql/alter_table.sql |  64 +++
src/test/regress/sql/create_table.sql|  51 +-
src/test/regress/sql/insert.sql  |  33 ++
src/test/regress/sql/partition_join.sql  |  32 ++
src/test/regress/sql/update.sql  |  28 ++
src/tools/pgindent/typedefs.list |   1 +
30 files changed, 1420 insertions(+), 120 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

[Andres Freund]

On 2017-11-04 06:15:00 -0700, Andres Freund wrote:
> The reason for that is that I hadn't yet quite figured out how the bug I
> described in the commit message (and the previously committed testcase)
> would cause that. I was planning to diagnose / experiment more with this
> and write an email if I couldn't come up with an explanation.   The
> committed test does *not* actually trigger that.
> 
> The reason I couldn't quite figure out how the problem triggers is that
> [ long explanation ]

Attached is a version of the already existing regression test that both
reproduces the broken hot chain (and thus failing index lookups) and
then also the tuple reviving.  I don't see any need for letting this run
with arbitrary permutations.

Thanks to whoever allowed isolationtester permutations to go over
multiple lines and allow comments. I was wondering about adding that as
a feature just to discover it's already there ;)


What I'm currently wondering about is how much we need to harden
postgres against such existing corruption. If e.g. the hot chains are
broken somebody might have reindexed thinking the problem is fixed - but
if they then later vacuum everything goes to shit again, with dead rows
reappearing.  There's no way we can fix hot chains after the fact, but
preventing dead rows from reapparing seems important.  A minimal version
of that is fairly easy - we slap a bunch of if if
!TransactionIdDidCommit() elog(ERROR) at various code paths. But that'll
often trigger clog access errors when the problem occurred - if we want
to do better we need to pass down relfrozenxid/relminmxid to a few
functions.  I'm inclined to do so, but it'll make the patch larger...

Comments?

Greetings,

Andres Freund
# Test for interactions of tuple freezing with dead, as well as recently-dead
# tuples using multixacts via FOR KEY SHARE.
setup
{
  DROP TABLE IF EXISTS tab_freeze;
  CREATE TABLE tab_freeze (
id int PRIMARY KEY,
name char(3),
x int);
  INSERT INTO tab_freeze VALUES (1, '111', 0);
  INSERT INTO tab_freeze VALUES (3, '333', 0);
}

teardown
{
  DROP TABLE tab_freeze;
}

session "s1"
step "s1_begin" { BEGIN; }
step "s1_update"{ UPDATE tab_freeze SET x = x + 1 WHERE id = 3; }
step "s1_commit"{ COMMIT; }
step "s1_vacuum"{ VACUUM FREEZE tab_freeze; }
step "s1_selectone" {
BEGIN;
SET LOCAL enable_seqscan = false;
SET LOCAL enable_bitmapscan = false;
SELECT * FROM tab_freeze WHERE id = 3;
COMMIT;
}
step "s1_selectall" { SELECT * FROM tab_freeze ORDER BY name, id; }

session "s2"
step "s2_begin" { BEGIN; }
step "s2_key_share" { SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE; 
}
step "s2_commit"{ COMMIT; }
step "s2_vacuum"{ VACUUM FREEZE tab_freeze; }

session "s3"
step "s3_begin" { BEGIN; }
step "s3_key_share" { SELECT id FROM tab_freeze WHERE id = 3 FOR KEY SHARE; 
}
step "s3_commit"{ COMMIT; }
step "s3_vacuum"{ VACUUM FREEZE tab_freeze; }

# This permutation verfies that a previous bug
# https://postgr.es/m/e5711e62-8fdf-4dca-a888-c200bf6b5...@amazon.com
# https://postgr.es/m/20171102112019.33wb7g5wp4zpj...@alap3.anarazel.de
# is not reintroduced. We used to make wrong pruning / freezing
# decision for multixacts, which could lead to a) broken hot chains b)
# dead rows being revived.
permutation "s1_begin" "s2_begin" "s3_begin" # start transactions
   "s1_update" "s2_key_share" "s3_key_share" # have xmax be a multi with an 
updater, updater being oldest xid
   "s1_update" # create additional row version that has multis
   "s1_commit" "s2_commit" # commit both updater and share locker
   "s2_vacuum" # due to bug in freezing logic, we used to *not* prune updated 
row, and then froze it
   "s1_selectone" # if hot chain is broken, the row can't be found via index 
scan
   "s3_commit" # commit remaining open xact
   "s2_vacuum" # pruning / freezing in broken hot chains would unset xmax, 
reviving rows
   "s1_selectall" # show borkedness

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remove junk left from DSSSL to XSL conversion

[Peter Eisentraut]

Remove junk left from DSSSL to XSL conversion

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4ea3c243b98da6d526960586e9acc6d22283ffa2

Modified Files
--
doc/src/sgml/Makefile | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Remove junk left from DSSSL to XSL conversion

[Peter Eisentraut]

Remove junk left from DSSSL to XSL conversion

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/e7397f015c9589f95f5f5b48d7a274b2f1628971

Modified Files
--
doc/src/sgml/Makefile | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Refactor permissions checks for large objects.

[Tom Lane]

Refactor permissions checks for large objects.

Up to now, ACL checks for large objects happened at the level of
the SQL-callable functions, which led to CVE-2017-7548 because of a
missing check.  Push them down to be enforced in inv_api.c as much
as possible, in hopes of preventing future bugs.  This does have the
effect of moving read and write permission errors to happen at lo_open
time not loread or lowrite time, but that seems acceptable.

Michael Paquier and Tom Lane

Discussion: 
https://postgr.es/m/cab7npqrhmnoybetnc_2ejsuzsm00z+bwkv9sy6tnvsd5gwt...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/ae20b23a9e7029f31ee902da08a464d968319f56

Modified Files
--
src/backend/catalog/objectaddress.c|   2 +-
src/backend/libpq/be-fsstubs.c |  88 +--
src/backend/storage/large_object/inv_api.c | 108 +++--
src/backend/utils/misc/guc.c   |  12 ++--
src/include/libpq/be-fsstubs.h |   5 --
src/include/storage/large_object.h |  13 ++--
6 files changed, 117 insertions(+), 111 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Restrict lo_import()/lo_export() via SQL permissions not hard-wi

[Tom Lane]

Restrict lo_import()/lo_export() via SQL permissions not hard-wired checks.

While it's generally unwise to give permissions on these functions to
anyone but a superuser, we've been moving away from hard-wired permission
checks inside functions in favor of using the SQL permission system to
control access.  Bring lo_import() and lo_export() into compliance with
that approach.

In particular, this removes the manual configuration option
ALLOW_DANGEROUS_LO_FUNCTIONS.  That dates back to 1999 (commit 4cd4a54c8);
it's unlikely anyone has used it in many years.  Moreover, if you really
want such behavior, now you can get it with GRANT ... TO PUBLIC instead.

Michael Paquier

Discussion: 
https://postgr.es/m/cab7npqrhmnoybetnc_2ejsuzsm00z+bwkv9sy6tnvsd5gwt...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/5ecc0d738e5864848bbc2d1d97e56d5846624ba2

Modified Files
--
src/backend/catalog/system_views.sql | 10 ++
src/backend/libpq/be-fsstubs.c   | 16 
src/include/catalog/catversion.h |  2 +-
src/include/pg_config_manual.h   | 10 --
src/test/regress/expected/privileges.out | 10 ++
src/test/regress/sql/privileges.sql  |  2 ++
6 files changed, 19 insertions(+), 31 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in ALTER SYSTEM output.

[Tom Lane]

Fix typo in ALTER SYSTEM output.

The header comment written into postgresql.auto.conf by ALTER SYSTEM
should match what initdb put there originally.

Feike Steenbergen

Discussion: 
https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1da48a9a6b9dced66610b70f94f1e5418afc1f8c

Modified Files
--
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in ALTER SYSTEM output.

[Tom Lane]

Fix typo in ALTER SYSTEM output.

The header comment written into postgresql.auto.conf by ALTER SYSTEM
should match what initdb put there originally.

Feike Steenbergen

Discussion: 
https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4384f8a51b8b0a2eaac3f2f9d25825cd5f5f4b7a

Modified Files
--
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in ALTER SYSTEM output.

[Tom Lane]

Fix typo in ALTER SYSTEM output.

The header comment written into postgresql.auto.conf by ALTER SYSTEM
should match what initdb put there originally.

Feike Steenbergen

Discussion: 
https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/6c3a7ba5bb0f960ed412b1c36e815f53347b3d79

Modified Files
--
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in ALTER SYSTEM output.

[Tom Lane]

Fix typo in ALTER SYSTEM output.

The header comment written into postgresql.auto.conf by ALTER SYSTEM
should match what initdb put there originally.

Feike Steenbergen

Discussion: 
https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b2ad6f9e2614f80e6dd88fe584e3aefac52cc454

Modified Files
--
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in ALTER SYSTEM output.

[Tom Lane]

Fix typo in ALTER SYSTEM output.

The header comment written into postgresql.auto.conf by ALTER SYSTEM
should match what initdb put there originally.

Feike Steenbergen

Discussion: 
https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3c966ce65317e8c72f4368b98dd19e753a9eae73

Modified Files
--
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix bogus logic for checking executables' versions within pg_upg

[Tom Lane]

Fix bogus logic for checking executables' versions within pg_upgrade.

Somebody messed up a refactoring here.  As it stood, we'd check pg_ctl's
--version output twice for each cluster.  Worse, the first check for the
new cluster's version happened before we'd done any validate_exec checks
there, breaking the check ordering the code intended.

A. Akenteva

Discussion: https://postgr.es/m/f9266a85d918a3cf3a386b5148aee...@postgrespro.ru

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/115a7075633899408fc1ab85f3bc2e5deeaa5e45

Modified Files
--
src/bin/pg_upgrade/exec.c | 9 -
1 file changed, 4 insertions(+), 5 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix bogus logic for checking executables' versions within pg_upg

[Tom Lane]

Fix bogus logic for checking executables' versions within pg_upgrade.

Somebody messed up a refactoring here.  As it stood, we'd check pg_ctl's
--version output twice for each cluster.  Worse, the first check for the
new cluster's version happened before we'd done any validate_exec checks
there, breaking the check ordering the code intended.

A. Akenteva

Discussion: https://postgr.es/m/f9266a85d918a3cf3a386b5148aee...@postgrespro.ru

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/9be95ef156e7c2ae0924300acddd483504fa33b3

Modified Files
--
src/bin/pg_upgrade/exec.c | 9 -
1 file changed, 4 insertions(+), 5 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/9aa6a1b29b0238802583a3b358142752a77c253d

Modified Files
--
configure| 58 --
configure.in |  8 ++--
2 files changed, 6 insertions(+), 60 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/28d177e3f95e3aef72f53e9e55096b350d1d5d9b

Modified Files
--
configure| 58 --
configure.in |  8 ++--
2 files changed, 6 insertions(+), 60 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/20d9adab60754ac71b0b500c91c45e12e940b3ce

Modified Files
--
configure| 58 --
configure.in |  8 ++--
2 files changed, 6 insertions(+), 60 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1772c5c6eee7f3eeaa0e485f67e9cd92f165e1cc

Modified Files
--
configure| 58 --
configure.in |  8 ++--
2 files changed, 6 insertions(+), 60 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3b04eb98f5f330215b67e7818ef136d991f9a16e

Modified Files
--
configure| 58 --
configure.in |  8 ++--
2 files changed, 6 insertions(+), 60 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Revert "Allow --with-bonjour to work with non-macOS implementati

[Tom Lane]

Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour."

Upon further review, our Bonjour code doesn't actually work with the
Avahi not-too-compatible compatibility library.  While you can get it
to work on non-macOS platforms if you link to Apple's own mDNSResponder
code, there don't seem to be many people who care about that.  Leaving in
the AC_SEARCH_LIBS call seems more likely to encourage people to build
broken configurations than to do anything very useful.

Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead.

Discussion: 
https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/89df643c3b3350de1b0e0d5cddd2f439972ca00f

Modified Files
--
configure| 91 
configure.in |  8 --
2 files changed, 6 insertions(+), 93 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/dfc015dcf46c1996bd7ed5866e9e045d258604b3

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL9_2_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/eda780281c9c09599d12e783c51905078674b2e8

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Make json{b}_populate_recordset() use the right tuple descriptor

[Tom Lane]

Make json{b}_populate_recordset() use the right tuple descriptor.

json{b}_populate_recordset() used the tuple descriptor created from the
query-level AS clause without worrying about whether it matched the actual
input record type.  If it didn't, that would usually result in a crash,
though disclosure of server memory contents seems possible as well, for a
skilled attacker capable of issuing crafted SQL commands.  Instead, use
the query-supplied descriptor only when there is no input tuple to look at,
and otherwise get a tuple descriptor based on the input tuple's own type
marking.  The core code will detect any type mismatch in the latter case.

Michael Paquier and Tom Lane, per a report from David Rowley.
Back-patch to 9.3 where this functionality was introduced.

Security: CVE-2017-15098

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c0c8807ded2f59c25b375998ef24ff09994563a1

Modified Files
--
src/backend/utils/adt/jsonfuncs.c  | 58 --
src/test/regress/expected/json.out | 13 +
src/test/regress/sql/json.sql  |  5 
3 files changed, 55 insertions(+), 21 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Make json{b}_populate_recordset() use the right tuple descriptor

[Tom Lane]

Make json{b}_populate_recordset() use the right tuple descriptor.

json{b}_populate_recordset() used the tuple descriptor created from the
query-level AS clause without worrying about whether it matched the actual
input record type.  If it didn't, that would usually result in a crash,
though disclosure of server memory contents seems possible as well, for a
skilled attacker capable of issuing crafted SQL commands.  Instead, use
the query-supplied descriptor only when there is no input tuple to look at,
and otherwise get a tuple descriptor based on the input tuple's own type
marking.  The core code will detect any type mismatch in the latter case.

Michael Paquier and Tom Lane, per a report from David Rowley.
Back-patch to 9.3 where this functionality was introduced.

Security: CVE-2017-15098

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d5fe5fb232a4e57c66779b8b6acdcb458e9b8b9c

Modified Files
--
src/backend/utils/adt/jsonfuncs.c   | 36 +---
src/test/regress/expected/json.out  | 13 +
src/test/regress/expected/jsonb.out | 13 +
src/test/regress/sql/json.sql   |  6 ++
src/test/regress/sql/jsonb.sql  |  6 ++
5 files changed, 63 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/ed546dd06195c27d92ae46f2bb1a74261bcb49e6

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/6b0b983f793b5c9ea68167a408ef58d40b942596

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/29d06705194396d46004a48bf56e741c79269790

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL9_2_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/203b965f275061894621a5a359213ac77558d33f

Modified Files
--
doc/src/sgml/release-9.2.sgml | 25 +
1 file changed, 25 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b7d6f75072b3569d7d2acce04669e72086b547cb

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Make json{b}_populate_recordset() use the right tuple descriptor

[Tom Lane]

Make json{b}_populate_recordset() use the right tuple descriptor.

json{b}_populate_recordset() used the tuple descriptor created from the
query-level AS clause without worrying about whether it matched the actual
input record type.  If it didn't, that would usually result in a crash,
though disclosure of server memory contents seems possible as well, for a
skilled attacker capable of issuing crafted SQL commands.  Instead, use
the query-supplied descriptor only when there is no input tuple to look at,
and otherwise get a tuple descriptor based on the input tuple's own type
marking.  The core code will detect any type mismatch in the latter case.

Michael Paquier and Tom Lane, per a report from David Rowley.
Back-patch to 9.3 where this functionality was introduced.

Security: CVE-2017-15098

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c30f082d2767c22cefb8875dcb1932e5ed338db6

Modified Files
--
src/backend/utils/adt/jsonfuncs.c   | 39 ++---
src/test/regress/expected/json.out  | 13 +
src/test/regress/expected/jsonb.out | 13 +
src/test/regress/sql/json.sql   |  6 ++
src/test/regress/sql/jsonb.sql  |  6 ++
5 files changed, 66 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1f23d1cd21ed46dba882729bedd9c40b71995989

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Make json{b}_populate_recordset() use the right tuple descriptor

[Tom Lane]

Make json{b}_populate_recordset() use the right tuple descriptor.

json{b}_populate_recordset() used the tuple descriptor created from the
query-level AS clause without worrying about whether it matched the actual
input record type.  If it didn't, that would usually result in a crash,
though disclosure of server memory contents seems possible as well, for a
skilled attacker capable of issuing crafted SQL commands.  Instead, use
the query-supplied descriptor only when there is no input tuple to look at,
and otherwise get a tuple descriptor based on the input tuple's own type
marking.  The core code will detect any type mismatch in the latter case.

Michael Paquier and Tom Lane, per a report from David Rowley.
Back-patch to 9.3 where this functionality was introduced.

Security: CVE-2017-15098

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/38e825632be777a6ea4a39796e121c39728403b3

Modified Files
--
src/backend/utils/adt/jsonfuncs.c   | 36 +---
src/test/regress/expected/json.out  | 13 +
src/test/regress/expected/jsonb.out | 13 +
src/test/regress/sql/json.sql   |  6 ++
src/test/regress/sql/jsonb.sql  |  6 ++
5 files changed, 63 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/fb3930ab1fdb53ad842307a47ddaa1fed4e85d5c

Modified Files
--
doc/src/sgml/release-9.2.sgml | 25 +
doc/src/sgml/release-9.3.sgml | 42 ++
2 files changed, 67 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3f80895723037c0d1c684dbdd50b7e03453df90f

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e7bae63e02dee20fdcbad2664d4722c80febf8a1

Modified Files
--
doc/src/sgml/release-9.2.sgml | 25 +
doc/src/sgml/release-9.3.sgml | 42 ++
doc/src/sgml/release-9.4.sgml | 42 ++
3 files changed, 109 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/7b4c179b70a59ad2dbd5c928ce8fc84629da0237

Modified Files
--
doc/src/sgml/release-9.2.sgml | 25 +++
doc/src/sgml/release-9.3.sgml | 42 
doc/src/sgml/release-9.4.sgml | 42 
doc/src/sgml/release-9.5.sgml | 75 ++-
4 files changed, 183 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/50abeafc74a812d2449ec49dc16e99baf8c5023a

Modified Files
--
doc/src/sgml/release-10.sgml  | 108 +-
doc/src/sgml/release-9.2.sgml |  25 ++
doc/src/sgml/release-9.3.sgml |  42 
doc/src/sgml/release-9.4.sgml |  42 
doc/src/sgml/release-9.5.sgml |  75 -
doc/src/sgml/release-9.6.sgml |  75 -
6 files changed, 364 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add tests for json{b}_populate_recordset() crash case.

[Tom Lane]

Add tests for json{b}_populate_recordset() crash case.

The problem reported as CVE-2017-15098 was already resolved in HEAD by
commit 37a795a60, but let's add the relevant test cases anyway.

Michael Paquier and Tom Lane, per a report from David Rowley.

Security: CVE-2017-15098

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/b574228715f0fd77ed1f4f084603cff9e757e992

Modified Files
--
src/test/regress/expected/json.out  | 13 +
src/test/regress/expected/jsonb.out | 13 +
src/test/regress/sql/json.sql   |  6 ++
src/test/regress/sql/jsonb.sql  |  6 ++
4 files changed, 38 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Make json{b}_populate_recordset() use the right tuple descriptor

[Tom Lane]

Make json{b}_populate_recordset() use the right tuple descriptor.

json{b}_populate_recordset() used the tuple descriptor created from the
query-level AS clause without worrying about whether it matched the actual
input record type.  If it didn't, that would usually result in a crash,
though disclosure of server memory contents seems possible as well, for a
skilled attacker capable of issuing crafted SQL commands.  Instead, use
the query-supplied descriptor only when there is no input tuple to look at,
and otherwise get a tuple descriptor based on the input tuple's own type
marking.  The core code will detect any type mismatch in the latter case.

Michael Paquier and Tom Lane, per a report from David Rowley.
Back-patch to 9.3 where this functionality was introduced.

Security: CVE-2017-15098

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/70846ee0597b4aabc11ffe252eb972de6f77a021

Modified Files
--
src/backend/utils/adt/jsonfuncs.c   | 36 +---
src/test/regress/expected/json.out  | 13 +
src/test/regress/expected/jsonb.out | 13 +
src/test/regress/sql/json.sql   |  6 ++
src/test/regress/sql/jsonb.sql  |  6 ++
5 files changed, 63 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/87b2ebd352c4afe1ded0841604b59a3afbae97d1

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: start-scripts: switch to $PGUSER before opening $PGLOG.

[Noah Misch]

start-scripts: switch to $PGUSER before opening $PGLOG.

By default, $PGUSER has permission to unlink $PGLOG.  If $PGUSER
replaces $PGLOG with a symbolic link, the server will corrupt the
link-targeted file by appending log messages.  Since these scripts open
$PGLOG as root, the attack works regardless of target file ownership.

"make install" does not install these scripts anywhere.  Users having
manually installed them in the past should repeat that process to
acquire this fix.  Most script users have $PGLOG writable to root only,
located in $PGDATA.  Just before updating one of these scripts, such
users should rename $PGLOG to $PGLOG.old.  The script will then recreate
$PGLOG with proper ownership.

Reviewed by Peter Eisentraut.  Reported by Antoine Scemama.

Security: CVE-2017-12172

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b5002976804cfd42ada725b30cff324ebd3e9638

Modified Files
--
contrib/start-scripts/freebsd| 4 ++--
contrib/start-scripts/linux  | 4 ++--
contrib/start-scripts/osx/PostgreSQL | 8 
3 files changed, 8 insertions(+), 8 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/045a1f38bd46f5b50e145470095f461cc41c

Modified Files
--
src/backend/catalog/pg_constraint.c   | 99 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint.h   |  2 +
src/test/regress/expected/privileges.out  | 18 +-
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 21 ++-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 197 insertions(+), 13 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Last-minute updates for release notes.

[Tom Lane]

Last-minute updates for release notes.

Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d69c0710a68068c7a415aaefd2c7d51f3197fe38

Modified Files
--
doc/src/sgml/release-9.2.sgml | 25 +++
doc/src/sgml/release-9.3.sgml | 42 
doc/src/sgml/release-9.4.sgml | 42 
doc/src/sgml/release-9.5.sgml | 75 ++-
doc/src/sgml/release-9.6.sgml | 75 ++-
5 files changed, 257 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers