== PostgreSQL Weekly News - February 28 2016 == == PostgreSQL Product News ==
Ora2PG 17.0 a migration system for moving from Oracle and MySQL to PostgreSQL, released. http://ora2pg.darold.net/ pgBadger 8.0, a parallel PostgreSQL log analyzer written in Perl, released: https://github.com/dalibo/pgbadger/blob/master/ChangeLog Development: https://github.com/dalibo/pgbadger/ pgbouncer 1.7.2, a light-weight connection pooler for PostgreSQL, released. https://pgbouncer.github.io/changelog.html == PostgreSQL Jobs for February == http://archives.postgresql.org/pgsql-jobs/2016-02/ == PostgreSQL Local == The first pan-Asian PostgreSQL conference will be held March 17-19, 2016 in Singapore. Registration is open. http://2016.pgday.asia/ Nordic PGDay 2016 is a one day one track conference which will be held in Helsinki, Finland, on March 17, 2016. Registration is still open. http://2016.nordicpgday.org/ PGDay Paris 2016, on March 31 in Paris, has opened registration. http://www.pgday.paris/registration/ (scroll down for English) The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon, France. The CfP is open until February 29, 2016 at call-for-paper <AT> postgresql-sessions <DOT> org. PGConf US 2016 will take place April 18-20, 2016 in NYC. Registration is open. http://www.pgconf.us/2016/ LinuxFest Northwest will take place April 23-24, 2016 at Bellingham Technical College in Bellingham, Washington, USA. The CfP is now open. http://www.linuxfestnorthwest.org/2016/present FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina. The CfP is still open. https://2016.foss4g-na.org/cfp PGCon 2016 will be held May 17-21, 2016 in Ottawa. http://www.pgcon.org/ This year's Swiss PGDay will be held on June 24, 2016 at the University of Applied Sciences in Rapperswil (Switzerland). The CfP is open. http://www.pgday.ch/ "5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016. The CfP is open until February 28th, 2016. http://5432meet.us/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to da...@fetter.org, German language to p...@pgug.de, Italian language to p...@itpug.org. Spanish language to p...@arpug.com.ar. == Applied Patches == Andres Freund pushed: - Fix wrong keysize in PrivateRefCountHash creation. In 4b4b680c3 I accidentally used sizeof(PrivateRefCountArray) instead of sizeof(PrivateRefCountEntry) when creating the refcount overflow hashtable. As the former is bigger than the latter, this luckily only resulted in a slightly increased memory usage when many buffers are pinned in a backend. Reported-By: Takashi Horikawa Discussion: 73fa3881462c614096f815f75628afcd035a4...@bpxm01gp.gisp.nec.co.jp Backpatch: 9.5, where thew new ref count infrastructure was introduced http://git.postgresql.org/pg/commitdiff/ea56b06cf77a6932a74f9d4ec6c950a333d1527d Tom Lane pushed: - Remove redundant PGPROC.lockGroupLeaderIdentifier field. We don't really need this field, because it's either zero or redundant with PGPROC.pid. The use of zero to mark "not a group leader" is not necessary since we can just as well test whether lockGroupLeader is NULL. This does not save very much, either as to code or data, but the simplification seems worthwhile anyway. http://git.postgresql.org/pg/commitdiff/73bf8715aa7430bd003516bde448507fbe789c05 - Create a function to reliably identify which sessions block which others. This patch introduces "pg_blocking_pids(int) returns int[]", which returns the PIDs of any sessions that are blocking the session with the given PID. Historically people have obtained such information using a self-join on the pg_locks view, but it's unreasonably tedious to do it that way with any modicum of correctness, and the addition of parallel queries has pretty much broken that approach altogether. (Given some more columns in the view than there are today, you could imagine handling parallel-query cases with a 4-way join; but ugh.) The new function has the following behaviors that are painful or impossible to get right via pg_locks: 1. Correctly understands which lock modes block which other ones. 2. In soft-block situations (two processes both waiting for conflicting lock modes), only the one that's in front in the wait queue is reported to block the other. 3. In parallel-query cases, reports all sessions blocking any member of the given PID's lock group, and reports a session by naming its leader process's PID, which will be the pg_backend_pid() value visible to clients. The motivation for doing this right now is mostly to fix the isolation tests. Commit 38f8bdcac4982215beb9f65a19debecaf22fd470 lobotomized isolationtester's is-it-waiting query by removing its ability to recognize nonconflicting lock modes, as a crude workaround for the inability to handle soft-block situations properly. But even without the lock mode tests, the old query was excessively slow, particularly in CLOBBER_CACHE_ALWAYS builds; some of our buildfarm animals fail the new deadlock-hard test because the deadlock timeout elapses before they can probe the waiting status of all eight sessions. Replacing the pg_locks self-join with use of pg_blocking_pids() is not only much more correct, but a lot faster: I measure it at about 9X faster in a typical dev build with Asserts, and 3X faster in CLOBBER_CACHE_ALWAYS builds. That should provide enough headroom for the slower CLOBBER_CACHE_ALWAYS animals to pass the test, without having to lengthen deadlock_timeout yet more and thus slow down the test for everyone else. http://git.postgresql.org/pg/commitdiff/52f5d578d6c29bf254e93c69043b817d4047ca67 Noah Misch pushed: - MSVC: Clean tmp_check directory of pg_controldata test suite. Back-patch to 9.4, where the suite was introduced. http://git.postgresql.org/pg/commitdiff/41635887835d54715a2fd4745dbf0fdb99f57f4f - Clean the last few TAP suite tmp_check directories. Back-patch to 9.5, where the suites were introduced. http://git.postgresql.org/pg/commitdiff/25924ac47abde5330ca502e23796e9d37cd1ae68 Robert Haas pushed: - Enable parallelism for prepared statements and extended query protocol. Parallel query can't handle running a query only partially rather than to completion. However, there seems to be no way to run a statement prepared via SQL PREPARE other than to completion, so we can enable it there without a problem. The situation is more complicated for the extend query protocol. libpq seems to provide no way to send an Execute message with a non-zero rowcount, but some other client might. If that happens, and a parallel plan was chosen, we'll execute the parallel plan without using any workers, which may be somewhat inefficient but should still work. Hopefully this won't be a problem; users can always set max_parallel_degree=0 to avoid choosing parallel plans in the first place. Amit Kapila, reviewed by me. http://git.postgresql.org/pg/commitdiff/57a6a72b6bc98f3003e87bc31de4b9c2c89fe019 - Add new FDW API to test for parallel-safety. This is basically a bug fix; the old code assumes that a ForeignScan is always parallel-safe, but for postgres_fdw, for example, this is definitely false. It should be true for file_fdw, though, since a worker can read a file from the filesystem just as well as any other backend process. Original patch by Thomas Munro. Documentation, and changes to the comments, by me. http://git.postgresql.org/pg/commitdiff/35746bc348b6bf1f690fe17f4f80cfb68e22f504 - On second thought, disable parallelism for prepared statements. CREATE TABLE .. AS EXECUTE can turn an apparently read-only query into a write operation, which parallel query can't handle. It's a bit of a shame that requires us to avoid parallel query for queries prepared via PREPARE in all cases, but for right now it does. http://git.postgresql.org/pg/commitdiff/7bea19d0a9d3e6975418ffe685fb510bd31ab434 - Respect TEMP_CONFIG when running contrib regression tests. Thomas Munro http://git.postgresql.org/pg/commitdiff/9117985b6ba9beda4f280f596035649fc23b6233 Álvaro Herrera pushed: - Fix typos Backpatch to: 9.4 http://git.postgresql.org/pg/commitdiff/343f709c0691abdc779bec91fa3587dda259808e - Add README in src/test and src/test/modules. Author: Craig Ringer Reviewed by: Michaël Paquier http://git.postgresql.org/pg/commitdiff/bda0b081984011ba5347bf3eecc95b71833de082 - Add POD docs to PostgresNode. Also, the dump_info method got split into another method that returns the stuff as a string instead of just printing it to stdout. Add a new README in src/test/perl too. Author: Craig Ringer Reviewed by: Michaël Paquier http://git.postgresql.org/pg/commitdiff/e64009303d4e2434abafbdafe4d571cc4f279d39 - Move some code from RewindTest into PostgresNode. Some code in the RewindTest test suite is more generally useful than just for that suite, so put it where other test suites can reach it. Some postgresql.conf parameters change their default values when a cluster is initialized with 'allows_streaming' than the previous behavior; most notably, autovacuum is no longer turned off. (Also, we no longer call pg_ctl promote with -w, but that flag doesn't actually do anything in promote so there's no behavior change.) Author: Michael Paquier http://git.postgresql.org/pg/commitdiff/89ac7004dadf4116d9b180bb5ff64b64bfce94b1 - Apply last revision of recovery patch. I applied the previous-to-last revision of Michaël's submitted patch instead of the last; these two tweaks pointed out by Craig were left out of the previous commit by accident. http://git.postgresql.org/pg/commitdiff/74d58425c70f9f7f1cb5d7f428ceaa1b56aa081d - Add a test framework for recovery. This long-awaited framework is an expansion of the existing PostgresNode stuff to support additional features for recovery testing; the recovery tests included in this commit are a starting point that cover some of the recovery features we have. More scripts are expected to be added later. Author: Michaël Paquier, a bit of help from Amir Rohan. Reviewed by: Amir Rohan, Stas Kelvich, Kyotaro Horiguchi, Victor Wagner, Craig Ringer, Álvaro Herrera. Discussion: http://www.postgresql.org/message-id/CAB7nPqTf7V6rswrFa=q_rrWeETUWagP=h8lx8xaov2jcxw0...@mail.gmail.com Discussion: http://www.postgresql.org/message-id/trinity-b4a8035d-59af-4c42-a37e-258f0f28e44a-1443795007012@3capp-mailcom-lxa08 http://git.postgresql.org/pg/commitdiff/49148645f7f30f461b09618b5342b37425f33b22 - Add isolationtester spec for old heapam.c bug. In 0e5680f4737a, I fixed a bug in heapam that caused spurious deadlocks when multiple updates concurrently attempted to modify the old version of an updated tuple whose new version was key-share locked. I proposed an isolationtester spec file that reproduced the bug, but back then isolationtester wasn't mature enough to be able to run it. Now that 38f8bdcac498 is in the tree, we can have this spec file too. Discussion: https://www.postgresql.org/message-id/20141212205254.GC1768%40alvh.no-ip.org http://git.postgresql.org/pg/commitdiff/c9578135f769072e2597b88402f256a398279c91 Andrew Dunstan pushed: - Respect TEMP_CONFIG when pg_regress_check and friends are called. This reverts commit 9117985b6ba9beda4f280f596035649fc23b6233 in favor of a more general solution. http://git.postgresql.org/pg/commitdiff/87cc6b57a911bf439ff9e53f1eddba58e9749ffe - Allow multiple --temp-config arguments to pg_regress. This means that if, for example, TEMP_CONFIG is set and a Makefile explicitly sets a temp-config file, both will now be used. Patch from John Gorman. http://git.postgresql.org/pg/commitdiff/26fdff1b8f76f3c47d4e19be7c4aef3cdcd3393c == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Amit Kapila sent in another revision of a patch to extend pg_stat_activity with wait_type_event. Corey Huinker sent in another revision of a patch to add \gexec to psql. Rushabh Lathia sent in another revision of a patch to help fix some slowness in FDW DML. Michaël Paquier sent in another revision of a patch to add new authentication methods with SCRAM as one example. Vitaly Burovoy sent in a patch to fix handling of negative years. Ashutosh Bapat sent in another revision of a patch to push sorted joins down to FDWs. Fujii Masao sent in a patch to add tab completion in psql to CREATE USER MAPPING. Michaël Paquier sent in a pair of patches, one which extends XLogInsert() with an extra argument for flags, the other which introduces XLogInsertExtended with this extra argument and leaves XLogInsert() alone. Michaël Paquier sent in another revision of a patch to fix a potential data loss bug on ext4 filesystems. Craig Ringer sent in another revision of a patch to implement failover slots. Tomas Vondra and Mark Dilger traded patches to improve GROUP BY estimation. Tomas Vondra and Kyotaro HORIGUCHI traded patches to allow index-only scans with partial indexes. Jim Nasby sent in another revision of a patch to convert PL/Tcl from strings to objects. Thomas Munro sent in two more revisions of a patch to make the PostgreSQL in parallel mode safer on PPC. Teodor Sigaev sent in two more revisions of a patch to fix a GIN corruption bug. Vitaly Burovoy sent in a patch to fix some overflows in timstamp[tz]. Iacob Catalin and Pavel Stěhule traded patches to add an ereport function to PL/PythonU. Mithun Cy sent in another revision of a patch to cache data in GetSnapshotData(). Kyotaro HORIGUCHI and SAWADA Masahiko traded patches to support N>1 synchronous standby servers. Petr Jelínek sent in another revision of a patch to add generic WAL messages. Julien Rouhaud sent in a patch to ensure that the CREATE OPERATOR CLASS documentation mentions that BRIN indexes also support the STORAGE parameter. Kyotaro HORIGUCHI sent in a patch to fix wrong comments for PQmblen() and PQdsplen(). Kyotaro HORIGUCHI sent in a patch to fix identifier completion with multibyte characters. Peter Eisentraut sent in a patch to add table qualifications to some tags in pg_dump. Kyotaro HORIGUCHI sent in another revision of a patch to add "IF [NOT] EXISTS" support to psql's tab completion. Michaël Paquier sent in four more revisions of a patch to add new regression tests for recovery, etc. Peter Eisentraut sent in a patch to introduce new configuration parameters syslog_sequence_numbers and syslog_split_lines. Amit Kapila sent in another revision of a patch to speed up CLOG access. Vitaly Burovoy sent in a patch to allow infinite values in to_timestamp. Pavel Stěhule sent in another revision of a patch to add a raw format to COPY. Vinayak Pokure sent in another revision of a patch to add a vacuum progress checker. Roma Sokolov sent in two revisions of a patch to fix DROP OPERATOR to reset links to itself on commutator and negator. Ivan Kartyshov sent in two revisions of a patch to add a pg_oldest_xlog_location() function. Simon Riggs sent in two revisions of a patch to fix an issue with relcache invalidation on a physical replica. Amit Langote sent in a patch to fix a typo in src/backend/utils/init/postinit.c. Joe Conway sent in two more revisions of a patch to add control data functions. Tom Lane sent in a WIP patch to path-ify the upper planner. And there was much rejoicing. Jim Nasby sent in a patch to improve error handling in PL/Tcl. -- Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-announce
