== PostgreSQL Weekly News - November 15 2015 == PostgreSQL 9.5 Beta 2 released. Test! http://www.postgresql.org/about/news/1625/
== PostgreSQL Product News == pg_wintools, a PostgreSQL extension for interfacing with Windows OS, released. https://github.com/jmguazzo/pg_wintools == PostgreSQL Jobs for November == http://archives.postgresql.org/pgsql-jobs/2015-11/ == PostgreSQL Local == PGConf Silicon Valley 2015 is November 17-18 at the South San Francisco Convention Center. http://www.pgconfsv.com PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto Alegre, Rio Grande do Sul, on November 18, 19 and 20. Registration is open. http://pgbr.postgresql.org.br/2015/en/ Postgres Conference China 2015 will be on November 20-22, 2015 in Beijing. http://postgres2015.eventdove.com/ (Chinese) http://postgrescluster2015.eventdove.com/ (English) PGConf.DE will be held on November 26-27, 2015 in Hamburg, Germany, at the Lindner Hotel am Michel. http://2015.pgconf.de/ PostgreSQL@SCaLE is a two day, two track event which takes place on Jan. 21-22, 2016, at Pasadena Convention Center, as part of SCaLE 14X. https://www.socallinuxexpo.org/scale/14x/cfp == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to da...@fetter.org, German language to p...@pgug.de, Italian language to p...@itpug.org. Spanish language to p...@arpug.com.ar. == Applied Patches == Andres Freund pushed: - Add paragraph about ON CONFLICT interaction with partitioning. Author: Peter Geoghegan and Andres Freund Discussion: cam3swzscpwzq-7ejc77vwqzz1go8gnmurq1qqdq3wrn7abw...@mail.gmail.com, cahgqgwfucwwsu7dtc2ardrk73ztyr_jy5cpoyts+k8xkj92...@mail.gmail.com Backpatch: 9.5, where UPSERT was introduced http://git.postgresql.org/pg/commitdiff/c31f1dc55992bc8b2c35c0ff99795547d4df323a Robert Haas pushed: - Add a dummy return statement to TupleQueueRemap. This is unreachable for multiple reasons, but per Amit Kapila the Windows compiler he is using still thinks we can get there. http://git.postgresql.org/pg/commitdiff/89ff5c7f7512a919e76407b4f8269e5b0b39dd2b - Fix rebasing mistake in nodeGather.c The patches committed as 6e71dd7ce9766582da453f493bc371d64977282f and 3a1f8611f2582df0a16bcd35caed2e1526387643 were developed in parallel but dependent on each other in a way that I failed to notice. This patch to fix the problem was prepared by Amit Kapila. http://git.postgresql.org/pg/commitdiff/bf3d0156311f0d8fa7705d8de0d07c4e0298051a - Comment update. Adjust to account for 5fc4c26db5120bd90348b6ee3101fcddfdf54800. Etsuro Fujita http://git.postgresql.org/pg/commitdiff/5c90a2ffddd2504a7441e80a22ba0fe8d484291a - Add outfuncs.c support for GatherPath. I dunno how commit 3bd909b220930f21d6e15833a17947be749e7fde missed this, but it evidently did. http://git.postgresql.org/pg/commitdiff/f764ecd81b2a8a1e9000d43a73ca5eec8e8008bc - Make sequential scans parallel-aware. In addition, this path fills in a number of missing bits and pieces in the parallel infrastructure. Paths and plans now have a parallel_aware flag indicating whether whatever parallel-aware logic they have should be engaged. It is believed that we will need this flag for a number of path/plan types, not just sequential scans, which is why the flag is generic rather than part of the SeqScan structures specifically. Also, execParallel.c now gives parallel nodes a chance to initialize their PlanState nodes from the DSM during parallel worker startup. Amit Kapila, with a fair amount of adjustment by me. Review of previous patch versions by Haribabu Kommi and others. http://git.postgresql.org/pg/commitdiff/f0661c4e8c44c0ec7acd4ea7c82e85b265447398 - Generate parallel sequential scan plans in simple cases. Add a new flag, consider_parallel, to each RelOptInfo, indicating whether a plan for that relation could conceivably be run inside of a parallel worker. Right now, we're pretty conservative: for example, it might be possible to defer applying a parallel-restricted qual in a worker, and later do it in the leader, but right now we just don't try to parallelize access to that relation. That's probably the right decision in most cases, anyway. Using the new flag, generate parallel sequential scan plans for plain baserels, meaning that we now have parallel sequential scan in PostgreSQL. The logic here is pretty unsophisticated right now: the costing model probably isn't right in detail, and we can't push joins beneath Gather nodes, so the number of plans that can actually benefit from this is pretty limited right now. Lots more work is needed. Nevertheless, it seems time to enable this functionality so that all this code can actually be tested easily by users and developers. Note that, if you wish to test this functionality, it will be necessary to set max_parallel_degree to a value greater than the default of 0. Once a few more loose ends have been tidied up here, we might want to consider changing the default value of this GUC, but I'm leaving it alone for now. Along the way, fix a bug in cost_gather: the previous coding thought that a Gather node's transfer overhead should be costed on the basis of the relation size rather than the number of tuples that actually need to be passed off to the leader. Patch by me, reviewed in earlier versions by Amit Kapila. http://git.postgresql.org/pg/commitdiff/80558c1f5aa109d08db0fbd76a6d370f900628a8 - Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me. http://git.postgresql.org/pg/commitdiff/a05dc4d7fd57d4ae084c1f0801973e5c1a1aa26e - Make idle backends exit if the postmaster dies. Letting backends continue to run if the postmaster has exited prevents PostgreSQL from being restarted, which in many environments is catastrophic. Worse, if some other backend crashes, we no longer have any protection against shared memory corruption. So, arrange for them to exit instead. We don't want to expend many cycles on this, but including postmaster death in the set of things that we wait for when a backend is idle seems cheap enough. Rajeev Rastogi and Robert Haas http://git.postgresql.org/pg/commitdiff/ac1d7945f866b1928c2554c0f80fd52d7f977772 - Move each SLRU's lwlocks to a separate tranche. This makes it significantly easier to identify these lwlocks in LWLOCK_STATS or Trace_lwlocks output. It's also arguably better from a modularity standpoint, since lwlock.c no longer needs to know anything about the LWLock needs of the higher-level SLRU facility. Ildus Kurbangaliev, reviewd by Álvaro Herrera and by me. http://git.postgresql.org/pg/commitdiff/fe702a7b3f9f2bc5bf6d173166d7d55226af82c8 - libpq: Notice errors a backend may have sent just before dying. At least since the introduction of Hot Standby, the backend has sometimes sent fatal errors even when no client query was in progress, assuming that the client would receive it. However, pqHandleSendFailure was not in sync with this assumption, and only tries to catch notices and notifies. Add a parseInput call to the loop there to fix. Andres Freund suggested the fix. Comments are by me. Reviewed by Michael Paquier. http://git.postgresql.org/pg/commitdiff/c3e7c24a1d60dc6ad56e2a0723399f1570c54224 - Remove accidentally-committed debugging code. Amit Kapila http://git.postgresql.org/pg/commitdiff/179c97bf587df710f98aa3ce2d95f503b0757e15 Tom Lane pushed: - Improve our workaround for 'TeX capacity exceeded' in building PDF files. In commit a5ec86a7c787832d28d5e50400ec96a5190f2555 I wrote a quick hack that reduced the number of TeX string pool entries created while converting our documentation to PDF form. That held the fort for awhile, but as of HEAD we're back up against the same limitation. It turns out that the original coding of \FlowObjectSetup actually results in *three* string pool entries being generated for every "flow object" (that is, potential cross-reference target) in the documentation, and my previous hack only got rid of one of them. With a little more care, we can reduce the string count to one per flow object plus one per actually-cross-referenced flow object (about 115000 + 5000 as of current HEAD); that should work until the documentation volume roughly doubles from where it is today. As a not-incidental side benefit, this change also causes pdfjadetex to stop emitting unreferenced hyperlink anchors (bookmarks) into the PDF file. It had been making one willy-nilly for every flow object; now it's just one per actually-cross-referenced object. This results in close to a 2X savings in PDF file size. We will still want to run the output through "jpdftweak" to get it to be compressed; but we no longer need removal of unreferenced bookmarks, so we might be able to find a quicker tool for that step. Although the failure only affects HEAD and US-format output at the moment, 9.5 cannot be more than a few pages short of failing likewise, so it will inevitably fail after a few rounds of minor-version release notes. I don't have a lot of faith that we'll never hit the limit in the older branches; and anyway it would be nice to get rid of jpdftweak across the board. Therefore, back-patch to all supported branches. http://git.postgresql.org/pg/commitdiff/944b41fc00600b74f518005ac314cc222bf6abd5 - Add missing "static" qualifier. Per buildfarm member pademelon. http://git.postgresql.org/pg/commitdiff/b05ae27e9a57a1c8f1f44720c23c4828106b7205 - Docs: fix misleading example. Commit 8457d0beca731bf0 introduced an example which, while not incorrect, failed to exhibit the behavior it meant to describe, as a result of omitting an E'' prefix that needed to be there. Noticed and fixed by Peter Geoghegan. I (tgl) failed to resist the temptation to wordsmith nearby text a bit while at it. http://git.postgresql.org/pg/commitdiff/7b6fb76349fccc38f0ef96166f207c6acd21968c - Be more noisy about "wrong number of nailed relations" initfile problems. In commit 5d1ff6bd559ea8df1b7302e245e690b01b9a4fa4 I added some logic to relcache.c to try to ensure that the regression tests would fail if we made a mistake about which relations belong in the relcache init files. I'm quite sure I tested that, but I must have done so only for the non-shared-catalog case, because a report from Adam Brightwell showed that the regression tests still pass just fine if we bollix the shared-catalog init file in the way this code was supposed to catch. The reason is that that file gets loaded before we do client authentication, so the WARNING is not sent to the client, only to the postmaster log, where it's far too easily missed. The least Rube Goldbergian answer to this is to put an Assert(false) after the elog(WARNING). That will certainly get developers' attention, while not breaking production builds' ability to recover from corner cases with similar symptoms. Since this is only of interest to developers, there seems no need for a back-patch, even though the previous commit went into all branches. http://git.postgresql.org/pg/commitdiff/da3751c8ea6f833d8e69fbdd168d29d328674398 - Improve documentation around autovacuum-related storage parameters. These were discussed in three different sections of the manual, which unsurprisingly had diverged over time; and the descriptions of individual variables lacked stylistic consistency even within each section (and frequently weren't in very good English anyway). Clean up the mess, and remove some of the redundant information in hopes that future additions will be less likely to re-introduce inconsistency. For instance I see no need for maintenance.sgml to include its very own list of all the autovacuum storage parameters, especially since that list was already incomplete. http://git.postgresql.org/pg/commitdiff/6404751ce91fb02999e46700a1ac62b3c6319abf - Do a round of copy-editing on the 9.5 release notes. Also fill in the previously empty "major enhancements" list. YMMV as to which items should make the cut, but it's past time we had something more than a placeholder here. (I meant to get this done before beta2 was wrapped, but got distracted by PDF build problems. Better late than never.) http://git.postgresql.org/pg/commitdiff/39b9978d9cac34ad799a5fa3ff3846f3e0372b0a - Fix unwanted flushing of libpq's input buffer when socket EOF is seen. In commit 210eb9b743c0645d I centralized libpq's logic for closing down the backend communication socket, and made the new pqDropConnection routine always reset the I/O buffers to empty. Many of the call sites previously had not had such code, and while that amounted to an oversight in some cases, there was one place where it was intentional and necessary *not* to flush the input buffer: pqReadData should never cause that to happen, since we probably still want to process whatever data we read. This is the true cause of the problem Robert was attempting to fix in c3e7c24a1d60dc6a, namely that libpq no longer reported the backend's final ERROR message before reporting "server closed the connection unexpectedly". But that only accidentally fixed it, by invoking parseInput before the input buffer got flushed; and very likely there are timing scenarios where we'd still lose the message before processing it. To fix, pass a flag to pqDropConnection to tell it whether to flush the input buffer or not. On review I think flushing is actually correct for every other call site. Back-patch to 9.3 where the problem was introduced. In HEAD, also improve the comments added by c3e7c24a1d60dc6a. http://git.postgresql.org/pg/commitdiff/c405918858c09d4264ffc5a3c73d57dc3efb0213 - Improve type numeric's calculations for ln(), log(), exp(), pow(). Set the "rscales" for intermediate-result calculations to ensure that suitable numbers of significant digits are maintained throughout. The previous coding hadn't thought this through in any detail, and as a result could deliver results with many inaccurate digits, or in the worst cases even fail with divide-by-zero errors as a result of losing all nonzero digits of intermediate results. In exp_var(), get rid entirely of the logic that separated the calculation into integer and fractional parts: that was neither accurate nor particularly fast. The existing range-reduction method of dividing by 2^n can be applied across the full input range instead of only 0..1, as long as we are careful to set an appropriate rscale for each step. Also fix the logic in mul_var() for shortening the calculation when the caller asks for fewer output digits than an exact calculation would require. This bug doesn't affect simple multiplications since that code path asks for an exact result, but it does contribute to accuracy issues in the transcendental math functions. In passing, improve performance of mul_var() a bit by forcing the shorter input to be on the left, thus reducing the number of iterations of the outer loop and probably also reducing the number of carry-propagation steps needed. This is arguably a bug fix, but in view of the lack of field complaints, it does not seem worth the risk of back-patching. Dean Rasheed http://git.postgresql.org/pg/commitdiff/7d9a4737c268f61fb8800957631f12d3f13be218 - Fix ruleutils.c's dumping of whole-row Vars in ROW() and VALUES() contexts. Normally ruleutils prints a whole-row Var as "foo.*". We already knew that that doesn't work at top level of a SELECT list, because the parser would treat the "*" as a directive to expand the reference into separate columns, not a whole-row Var. However, Joshua Yanovski points out in bug #13776 that the same thing happens at top level of a ROW() construct; and some nosing around in the parser shows that the same is true in VALUES(). Hence, apply the same workaround already devised for the SELECT-list case, namely to add a forced cast to the appropriate rowtype in these cases. (The alternative of just printing "foo" was rejected because it is difficult to avoid ambiguity against plain columns named "foo".) Back-patch to all supported branches. http://git.postgresql.org/pg/commitdiff/7745bc352a82bd588be986479c7aabc3b076a375 Álvaro Herrera pushed: - vacuumdb: don't prompt for passwords over and over. Having the script prompt for passwords over and over was a preexisting problem when it processed multiple databases or when it processed multiple analyze stages, but the parallel mode introduced in commit a179232047 made it worse. Fix the annoyance by keeping a copy of the password used by the first connection that requires one. Since users can (currently) only have a single password, there's no need for more complex arrangements (such as remembering one password per database). Per bug #13741 reported by Eric Brown. Patch authored and cross-reviewed by Haribabu Kommi and Michael Paquier, slightly tweaked by Álvaro Herrera. Discussion: http://www.postgresql.org/message-id/20151027193919.931.54...@wrigleys.postgresql.org Backpatch to 9.5, where parallel vacuumdb was introduced. http://git.postgresql.org/pg/commitdiff/83dec5a712af251af15effbf781ddaedc3bf6b3b Stephen Frost pushed: - Correct sepgsql docs with regard to RLS The sepgsql docs included a comment that PG doesn't support RLS. That is only true for versions prior to 9.5. Update the docs for 9.5 and master to say that PG supports RLS but that sepgsql does not yet. Pointed out by Heikki. Back-patch to 9.5 http://git.postgresql.org/pg/commitdiff/42aa1c032e3520d608a1e89abbf36409b022d4b7 Bruce Momjian pushed: - pg_upgrade: properly detect file copy failure on Windows. Previously, file copy failures were ignored on Windows due to an incorrect return value check. Report by Manu Joye Backpatch through 9.1 http://git.postgresql.org/pg/commitdiff/025106e314627f5aacdf72fca52ba0ed985d696e - Fix spelling error in postgresql.conf. Report by Greg Clough http://git.postgresql.org/pg/commitdiff/e57646e96284d4bad79f87a5c1dfbcd8f5434020 == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Adam Brightwell sent in two revisions of a patch to bootstrap pg_shseclabel in relcache initialization. Kaigai Kouhei sent in another revision of a patch to reconcile Foreign join pushdown with EvalPlanQual. Ashutosh Bapat sent in another revision of a patch to add a transaction manager for foreign servers. Marco Nenciarini sent in a patch to fix an incompatibility between 9.3's WAL stream and pg_receivelog. Kaigai Kouhei sent in a patch for each of 9.5 and git master to allow CustomScan on copyfuncs. Pavel Stěhule sent in another revision of a patch to allow psql to send multiple SQL statements via the command line. Pavel Stěhule and Catalin Iacob traded patches to add an ereport function to PL/PythonU. Artur Zakirov sent in another revision of a patch to enhance the performance of Hunspell dictionaries in full-text search. Peter Geoghegan sent in a patch to fix some erroneous documentation of char(n). Michael Paquier sent in another revision of a patch to add trigonometric functions which take degree values as input. Andres Freund sent in another revision of a patch to control continuous flushing of checkpoints. Thomas Munro sent in two revisions of a patch to create a "causal reads" capability. Peter Eisentraut sent in another revision of a patch to handle missing pieces of compilation infrastructure more gracefully. Michael Paquier sent in another revision of a patch to show tuple data in pageinspect. Craig Ringer sent in a patch to implement pg_logical, a plugin based on logical WAL streams. Kyotaro HORIGUCHI sent in a patch to allow using regexes in frontend code, in particular tab completion for psql. Thomas Munro sent in another revision of a patch to simplify tab completion in psql. Alexander Korotkov sent in another revision of a patch to rework the access method interface. Peter Geoghegan sent in a patch to expand SortSupport to char(n), bytea, and alternative opclasses. Kaigai Kouhei sent in a patch to add serialization support to CustomScan. SAWADA Masahiko sent in two more revisions of a patch to add a "frozen" bit to the visibility map. Amit Kapila sent in a patch to fix an issue in the parallel seq scan code. Robert Haas sent in a patch to implement Append in parallel. Jeff Janes and Tom Lane traded patches to speed up set_rtable_names. Amit Kapila sent in a patch to fix some issues in parallel seq scan. Robert Haas sent in a patch to tranche-ify buffer locks. Haribabu Kommi sent in another revision of a patch to skip ALTER x SET SCHEMA if the schema didn't change. -- Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-announce