== PostgreSQL Weekly News - November 12 2017 == PostgreSQL security releases 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, and 9.2.24 are out. Please read the announcement below and upgrade at the next available downtime. https://www.postgresql.org/about/news/1801/
PGConf.Brazil 2018 will take place in São Paulo, Brazil on August 3-4 2018. The CfP will open soon. http://pgconf.com.br Nordic PGDay 2018 will be held in Oslo, Norway, at the Radisson Blu Hotel Nydalen, on March 13, 2018. The CfP is open through December 31, 2017 at https://2018.nordicpgday.org/cfp/ pgDay Paris 2018 will be held in Paris, France at the Espace Saint-Martin, on March 15 2018. The CfP is open until December 31, 2017. http://2018.pgday.paris/callforpapers/ == PostgreSQL Jobs for November == http://archives.postgresql.org/pgsql-jobs/2017-11/ == PostgreSQL Local == PGConf Local: Seattle will be held November 13 - 14, 2017. https://www.pgconf.us/#Seattle2017 PGDay Australia 2017 will be held on November 17 in Melbourne. http://2017.pgday.com.au/ PostgreSQL Session will take place November 17th, 2017, in Paris, France. http://www.postgresql-sessions.org/en/9/start PGDAY Brasilia 2017 will take place December 2, 2017. The CfP is open at http://www.pgdaybrasilia.com.br/submissao-de-palestra/ PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is now open at https://www.pgconf.us/conferences/Austin2017 PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo, Japan. http://www.pgconf.asia/EN/2017/ FOSDEM PGDay 2018, a one day conference held before the main FOSDEM event will be held in Brussels, Belgium, on Feb 2nd, 2018. https://2018.fosdempgday.org/ Prague PostgreSQL Developer Day 2018 (P2D2 2018) is a two-day conference that will be held on February 14-15 2018 in Prague, Czech Republic. The CfP is open until January 5, 2018 at https://p2d2.cz/callforpapers http://www.p2d2.cz/ PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka. Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017. http://pgconf.in/ PostgreSQL@SCaLE is a two day, two track event which takes place on March 8-9, 2018, at Pasadena Convention Center, as part of SCaLE 16X. The CfP is open through October 31, 2017 at http://www.socallinuxexpo.org/scale/16x/cfp PGConf APAC 2018 will be held in Singapore March 22-24, 2018. The CfP is open at http://2018.pgconfapac.org/cfp through December 4, 2017. http://2018.pgconfapac.org/ The German-speaking PostgreSQL Conference 2018 will take place on April 13th, 2018 in Berlin. The CfP is open until January 09, 2018 at http://2018.pgconf.de/de/callforpapers.html and the conference site is at http://2018.pgconf.de/ PGCon 2018 will take place in Ottawa on May 29 - June 2018. The CfP goes out soon. https://www.pgcon.org/2018/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English language ones to da...@fetter.org, German language to p...@pgug.de, Italian language to p...@itpug.org. == Applied Patches == Tom Lane pushed: - Release notes for 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, 9.2.24. In the v10 branch, also back-patch the effects of 1ff01b390 and c29c57890 on these files, to reduce future maintenance issues. (I'd do it further back, except that the 9.X branches differ anyway due to xlog-to-wal link tag renaming.) https://git.postgresql.org/pg/commitdiff/b35b185bf705c4dbaf21198c81b3d85f4a96804a - Fix version numbering foulups exposed by 10.1. configure computed PG_VERSION_NUM incorrectly. (Coulda sworn I tested that logic back when, but it had an obvious thinko.) pg_upgrade had not been taught about the new dispensation with just one part in the major version number. Both things accidentally failed to fail with 10.0, but with 10.1 we got the wrong results. Per buildfarm. https://git.postgresql.org/pg/commitdiff/d0c80c17f1a6d0b93d2ca14fe47d83b131ce9108 - Fix unportable usage of <ctype.h> functions. isdigit(), isspace(), etc are likely to give surprising results if passed a signed char. We should always cast the argument to unsigned char to avoid that. Error in commit 63d6b97fd, found by buildfarm member gaur. Back-patch to 9.3, like that commit. https://git.postgresql.org/pg/commitdiff/d1f9ac5b100dbc4da02f0f209a2e7730bd5e83e9 - Fix unportable spelling of int64 constant. Per buildfarm member pademelon. https://git.postgresql.org/pg/commitdiff/92a1834dd88e56e823ac6641313a2f077a8af72e - Fix two violations of the ResourceOwnerEnlarge/Remember protocol. The point of having separate ResourceOwnerEnlargeFoo and ResourceOwnerRememberFoo functions is so that resource allocation can happen in between. Doing it in some other order is just wrong. OpenTemporaryFile() did open(), enlarge, remember, which would leak the open file if the enlarge step ran out of memory. Because fd.c has its own layer of resource-remembering, the consequences look like they'd be limited to an intratransaction FD leak, but it's still not good. IncrBufferRefCount() did enlarge, remember, incr-refcount, which would blow up if the incr-refcount step ever failed. It was safe enough when written, but since the introduction of PrivateRefCountHash, I think the assumption that no error could happen there is pretty shaky. The odds of real problems from either bug are probably small, but still, back-patch to supported branches. Thomas Munro and Tom Lane, per a comment from Andres Freund https://git.postgresql.org/pg/commitdiff/c5269472ea9bb4a6fbb8a0510f7d676d725933ab - Doc: fix erroneous example. The grammar requires these options to appear the other way 'round. jo...@posteo.de Discussion: https://postgr.es/m/78933bd0-45ce-690e-b832-a328dd1a5...@posteo.de https://git.postgresql.org/pg/commitdiff/bd65e0c62486e6108a7dc824f918754a13072f7a - Allow --with-bonjour to work with non-macOS implementations of Bonjour. On macOS the relevant functions require no special library, but elsewhere we need to pull in libdns_sd. Back-patch to supported branches. No docs change since the docs do not suggest that this is a Mac-only feature. Luke Lonergan Discussion: https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com https://git.postgresql.org/pg/commitdiff/9b9cb3c4534d717c1c95758670198ebbf8a20af2 - Last-minute updates for release notes. Security: CVE-2017-12172, CVE-2017-15098, CVE-2017-15099 https://git.postgresql.org/pg/commitdiff/92d830f4bff643953a09563abaa106af42625207 - Add tests for json{b}_populate_recordset() crash case. The problem reported as CVE-2017-15098 was already resolved in HEAD by commit 37a795a60, but let's add the relevant test cases anyway. Michael Paquier and Tom Lane, per a report from David Rowley. Security: CVE-2017-15098 https://git.postgresql.org/pg/commitdiff/b574228715f0fd77ed1f4f084603cff9e757e992 - Revert "Allow --with-bonjour to work with non-macOS implementations of Bonjour.". Upon further review, our Bonjour code doesn't actually work with the Avahi not-too-compatible compatibility library. While you can get it to work on non-macOS platforms if you link to Apple's own mDNSResponder code, there don't seem to be many people who care about that. Leaving in the AC_SEARCH_LIBS call seems more likely to encourage people to build broken configurations than to do anything very useful. Hence, remove the AC_SEARCH_LIBS call and put in a warning comment instead. Discussion: https://postgr.es/m/2d8331c5-d64f-44c1-8717-63edc6eaf...@brightforge.com https://git.postgresql.org/pg/commitdiff/20d9adab60754ac71b0b500c91c45e12e940b3ce - Fix bogus logic for checking executables' versions within pg_upgrade. Somebody messed up a refactoring here. As it stood, we'd check pg_ctl's --version output twice for each cluster. Worse, the first check for the new cluster's version happened before we'd done any validate_exec checks there, breaking the check ordering the code intended. A. Akenteva Discussion: https://postgr.es/m/f9266a85d918a3cf3a386b5148aee...@postgrespro.ru https://git.postgresql.org/pg/commitdiff/9be95ef156e7c2ae0924300acddd483504fa33b3 - Fix typo in ALTER SYSTEM output. The header comment written into postgresql.auto.conf by ALTER SYSTEM should match what initdb put there originally. Feike Steenbergen Discussion: https://postgr.es/m/CAK_s-G0KcKdO=0hqzkwb3s+tqzuuhwwqmf5bdsmoo9ftx75...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/6c3a7ba5bb0f960ed412b1c36e815f53347b3d79 - Restrict lo_import()/lo_export() via SQL permissions not hard-wired checks. While it's generally unwise to give permissions on these functions to anyone but a superuser, we've been moving away from hard-wired permission checks inside functions in favor of using the SQL permission system to control access. Bring lo_import() and lo_export() into compliance with that approach. In particular, this removes the manual configuration option ALLOW_DANGEROUS_LO_FUNCTIONS. That dates back to 1999 (commit 4cd4a54c8); it's unlikely anyone has used it in many years. Moreover, if you really want such behavior, now you can get it with GRANT ... TO PUBLIC instead. Michael Paquier Discussion: https://postgr.es/m/cab7npqrhmnoybetnc_2ejsuzsm00z+bwkv9sy6tnvsd5gwt...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/5ecc0d738e5864848bbc2d1d97e56d5846624ba2 - Refactor permissions checks for large objects. Up to now, ACL checks for large objects happened at the level of the SQL-callable functions, which led to CVE-2017-7548 because of a missing check. Push them down to be enforced in inv_api.c as much as possible, in hopes of preventing future bugs. This does have the effect of moving read and write permission errors to happen at lo_open time not loread or lowrite time, but that seems acceptable. Michael Paquier and Tom Lane Discussion: https://postgr.es/m/cab7npqrhmnoybetnc_2ejsuzsm00z+bwkv9sy6tnvsd5gwt...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ae20b23a9e7029f31ee902da08a464d968319f56 - Tighten test in contrib/bloom/t/001_wal.pl. Make bloom WAL test compare psql output text, not just result codes; this was evidently the intent all along, but it was mis-coded. In passing, make sure we will notice any failure in setup steps. Alexander Korotkov, reviewed by Michael Paquier and Masahiko Sawada Discussion: https://postgr.es/m/capphfdtohpdq9rc5mdwjxq+3vsbnw534kv_5o65dtqrsdvj...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/7e60e678615b1f803ac73faee71cca79ec310d1d Noah Misch pushed: - Add a temp-install prerequisite to "check"-like targets not having one. Makefile.global assigns this prerequisite to every target named "check", but similar targets must mention it explicitly. Affected targets failed, tested $PATH binaries, or tested a stale temporary installation. The src/test/modules examples worked properly when called as "make -C src/test/modules/$FOO check", but "make -j" allowed the test to start before the temporary installation was in place. Back-patch to 9.5, where commit dcae5faccab64776376d354decda0017c648bb53 introduced the shared temp-install. https://git.postgresql.org/pg/commitdiff/c66b438db62748000700c9b90b585e756dd54141 - start-scripts: switch to $PGUSER before opening $PGLOG. By default, $PGUSER has permission to unlink $PGLOG. If $PGUSER replaces $PGLOG with a symbolic link, the server will corrupt the link-targeted file by appending log messages. Since these scripts open $PGLOG as root, the attack works regardless of target file ownership. "make install" does not install these scripts anywhere. Users having manually installed them in the past should repeat that process to acquire this fix. Most script users have $PGLOG writable to root only, located in $PGDATA. Just before updating one of these scripts, such users should rename $PGLOG to $PGLOG.old. The script will then recreate $PGLOG with proper ownership. Reviewed by Peter Eisentraut. Reported by Antoine Scemama. Security: CVE-2017-12172 https://git.postgresql.org/pg/commitdiff/dfc015dcf46c1996bd7ed5866e9e045d258604b3 - Ignore XML declaration in xpath_internal(), for UTF8 databases. When a value contained an XML declaration naming some other encoding, this function interpreted UTF8 bytes as the named encoding, yielding mojibake. xml_parse() already has similar logic. This would be necessary but not sufficient for non-UTF8 databases, so preserve behavior there until the xpath facility can support such databases comprehensively. Back-patch to 9.3 (all supported versions). Pavel Stehule and Noah Misch Discussion: https://postgr.es/m/CAFj8pRC-dM=tT=qkgi+achkm+gwpmjyoayguufxvumcxkdg...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/2918fcedbf2b2adab688a7849ecce4556ef912ac - Fix previous commit's test, for non-UTF8 databases with non-XML builds. To ensure stable output, catch one more configuration-specific error. Back-patch to 9.3, like the commit that added the test. https://git.postgresql.org/pg/commitdiff/4b865aee2582292a42a8e58247a41b46f5aa8a82 - Make connect/test1 independent of localhost IPv6. Since commit 868898739a8da9ab74c105b8349b7b5c711f265a, it has assumed "localhost" resolves to both ::1 and 127.0.0.1. We gain nothing from that assumption, and it does not hold in a default installation of Red Hat Enterprise Linux 5. Back-patch to 9.3 (all supported versions). https://git.postgresql.org/pg/commitdiff/34baf8a00b018caf7269134cf9b461266e66d9a7 - Add post-2010 ecpg tests to checktcp. This suite had been a proper superset of the regular ecpg test suite, but the three newest tests didn't reach it. To make this less likely to recur, delete the extra schedule file and pass the TCP-specific test on the command line. Back-patch to 9.3 (all supported versions). https://git.postgresql.org/pg/commitdiff/0b7e76eb2b142d0b4a2a831e7fa1fac44820f52c Simon Riggs pushed: - Exclude pg_internal.init from BASE_BACKUP. Add docs to explain this for other backup mechanisms Author: David Steele <da...@pgmasters.net> Reviewed-by: Petr Jelinek <petr.jeli...@2ndquadrant.com> et al https://git.postgresql.org/pg/commitdiff/98267ee83e068afa8bd85c8cc710ab2144e711fa - Remove secondary checkpoint. Previously server reserved WAL for last two checkpoints, which used too much disk space for small servers. Bumps PG_CONTROL_VERSION Author: Simon Riggs <si...@2ndquadrant.com> Reviewed-by: Michael Paquier <michael.paqu...@gmail.com> https://git.postgresql.org/pg/commitdiff/4b0d28de06b28e57c540fca458e4853854fbeaf8 Robert Haas pushed: - Fix typo in comment. Masahiko Sawada Discussion: http://postgr.es/m/CAD21AoDrf5AOpZ-mX-j6O=zfnffkatdhkv3o1x2ess2nbxa...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/698e5254377baffb95774d47872100997f135cba - Fix incorrect comment. Etsuro Fujita Discussion: http://postgr.es/m/5a05728e.4050...@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/b9941d3468505aea8bfdd74840b753ed27b9d29f - Account for the effect of lossy pages when costing bitmap scans. Dilip Kumar, reviewed by Alexander Kumenkov, Amul Sul, and me. Some final adjustments by me. Discussion: http://postgr.es/m/cafitn-sytquoxq4spuhtv0z9gd0si3yxzgv_pqaamx8qboo...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/5edc63bda68a77c4d38f0cbeae1c4271f9ef4100 - Add hash partitioning. Hash partitioning is useful when you want to partition a growing data set evenly. This can be useful to keep table sizes reasonable, which makes maintenance operations such as VACUUM faster, or to enable partition-wise join. At present, we still depend on constraint exclusion for partitioning pruning, and the shape of the partition constraints for hash partitioning is such that that doesn't work. Work is underway to fix that, which should both improve performance and make partitioning pruning work with hash partitioning. Amul Sul, reviewed and tested by Dilip Kumar, Ashutosh Bapat, Yugo Nagata, Rajkumar Raghuwanshi, Jesper Pedersen, and by me. A few final tweaks also by me. Discussion: http://postgr.es/m/CAAJ_b96fhpJAP=albetmelk1uni_gfzd938zgenhf49qgdt...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/1aba8e651ac3e37e1d2d875842de1e0ed22a651e Peter Eisentraut pushed: - Expand empty end tag. https://git.postgresql.org/pg/commitdiff/6e1e4c0d2f757677b03b6f164487a582c7dfab33 - Put markup in the right place. https://git.postgresql.org/pg/commitdiff/4497f2f3b30fa5cd48898033c351bfcf01ce73e2 - Change TRUE/FALSE to true/false. The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paqu...@gmail.com> https://git.postgresql.org/pg/commitdiff/2eb4a831e5fb5d8fc17e13aea56e04af3efe27b4 - Remove junk left from DSSSL to XSL conversion. https://git.postgresql.org/pg/commitdiff/e7397f015c9589f95f5f5b48d7a274b2f1628971 - Add -wnet to SP invocations. This causes a warning when accidentally backpatching an XML-style empty-element tag like <xref linkend="abc"/>. https://git.postgresql.org/pg/commitdiff/9a8d3c4eeaf34966056a41a263c6e2ca4d5e4012 - Add some const decorations to prototypes. Reviewed-by: Fabien COELHO <coe...@cri.ensmp.fr> https://git.postgresql.org/pg/commitdiff/0e1539ba0d0a43de06c6e0572a565e73b9472538 - Fix some null pointer dereferences in LDAP auth code. An LDAP URL without a host name such as "ldap://"; or without a base DN such as "ldap://localhost"; would cause a crash when reading pg_hba.conf. If no binddn is configured, an error message might end up trying to print a null pointer, which could crash on some platforms. Author: Thomas Munro <thomas.mu...@enterprisedb.com> Reviewed-by: Michael Paquier <michael.paqu...@gmail.com> https://git.postgresql.org/pg/commitdiff/0c98d0dd5c85ce0c8485ae1a8351a26b83c4338b Dean Rasheed pushed: - Always require SELECT permission for ON CONFLICT DO UPDATE. The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT permission on the columns of the arbiter index, but it failed to check for that in the case of an arbiter specified by constraint name. In addition, for a table with row level security enabled, it failed to check updated rows against the table's SELECT policies when the update path was taken (regardless of how the arbiter index was specified). Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced. Security: CVE-2017-15099 https://git.postgresql.org/pg/commitdiff/87b2ebd352c4afe1ded0841604b59a3afbae97d1 == Pending Patches == Edmund Horner sent in a patch to add some tab completion to the SELECT target list in psql. Ashutosh Bapat sent in a patch to improve \d+ output of a partitioned table and separate default partition from rest of the partitions in psql. Noah Misch sent in a patch to fix a race to build pg_isolation_regress in "make -j check-world". David Rowley sent in a patch to remove useless DISTINCT clauses. Thomas Munro sent in a patch atop the statement-level rollback patch to document it better. Konstantin Knizhnik sent in another revision of a patch to optimize secondary index lookups. Raúl Marín Rodríguez sent in three more revisions of a patch to add pow() to pgbench. Asim Praveen sent in two more revisions of a patch to change incorrect calls to PageGetLSN to BufferGetLSNAtomic and assert that locks are properly held in PageGetLSN. Kyotaro HORIGUCHI and Pavel Stěhule traded patches to add default namespaces for XPath expressions. Amit Langote and David Rowley traded patches to prune partitions faster. Haribabu Kommi sent in another revision of a patch to create the infrastructure for pluggable storage. Nathan Bossart sent in two more revisions of a patch to add new logging for VACUUM and ANALYZE. Haribabu Kommi sent in another revision of a patch to add a pg_stat_wal_write statistics view. Haribabu Kommi sent in two more revisions of a patch to refactor handling of database attributes between pg_dump and pg_dumpall. Tom Lane sent in a patch to speed up compactify_tuples, this time without a sort step. Alexander Korotkov sent in two more revisions of a patch to fix the bloom WAL tap test. Thomas Munro sent in a patch to add planning counters to pg_stat_statements. Michaël Paquier and Fabrízio de Royes Mello traded patches to add session_start and session_end hooks. Dmitry Dolgov and Artur Zakirov traded patches to add generic subscripting, implementing those for arrays and JSON with same. Masahiko Sawada sent in a patch to remove a duplicate setting in test/recovery/Makefile. Kyotaro HORIGUCHI sent in another revision of a patch to restrict the maximum keep segments by repslots. Amit Khandekar sent in two more revisions of a patch to enable UPDATEs of a partition key to move tuples to the appropriate partition. Etsuro Fujita sent in a patch to reorder header files in alphabetical order. Robert Haas sent in another revision of a patch to implement parallel append. Amit Kapila sent in another revision of a patch to ensure that parallel paths include tlist cost. Beena Emerson sent in another revision of a patch to implement runtime partiton pruning. Pavel Stěhule sent in two more revisions of a patch to add \graw to psql. Michaël Paquier sent in a patch to remove ALLOW_DANGEROUS_LO_FUNCTIONS for LO-related superuser checks, replace superuser checks of large object import/export by ACL checks, and move ACL checks for large objects when opening them. Huong Dangminh sent in a patch to fix an issue where user-defined data types weren't working right under logical replication. Mark Rofail sent in another revision of a patch to implement foreign key arrays. David Rowley sent in a patch to add a mention of CREATE STATISTICS to the event trigger docs. Robert Haas sent in another revision of a patch to speed up processing at Gather nodes. Graham Leggett sent in a patch to log SSL certificate verification errors. Robert Haas and Amit Kapila traded patches to parallelize queries containing initplans. Pavel Stěhule sent in another revision of a patch to add a SERVER_VERSION variable to psql. Robert Haas sent in a patch to ensure that GatherMerge pushes tlists down. Thomas Munro sent in a patch to add a parallel_leader_participation GUC. Andrey Borodin sent in another revision of a patch to add GiST VACUUM. -- Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-announce
