== PostgreSQL Weekly News - October 15 2017 == PGConf APAC 2018 will be held in Singapore March 22-24, 2018. The CfP is open at http://2018.pgconfapac.org/cfp through December 4, 2017. http://2018.pgconfapac.org/
PGDAY Brasilia 2017 will take place December 2, 2017. The CfP is open at http://www.pgdaybrasilia.com.br/submissao-de-palestra/ == PostgreSQL Product News == psqlODBC 10.00.0000 released. https://odbc.postgresql.org/docs/release.html Ajqvue Version 1.15, a java-based UI which supports PostgreSQL, released. http://ajqvue.com pglogical 2.1, a logical-WAL-based replication system for PostgreSQL, released. https://www.2ndquadrant.com/en/resources/pglogical/ repmgr 4.0 beta1, a replication manager for PostgreSQL, released. http://repmgr.org/release-notes-4.0beta1.html pg_repack 1.4.2, an extension for removing bloat, released. http://reorg.github.io/pg_repack/ sql_insert_writer v0.1.0, a Python3 utility to build legible skeletons for INSERT and INSERT FROM statements, released. https://github.com/18F/sql_insert_writer == PostgreSQL Jobs for October == http://archives.postgresql.org/pgsql-jobs/2017-10/ == PostgreSQL Local == PostgreSQL Conference Europe 2017 will be held on October 24-27, 2017 in the Warsaw Marriott Hotel, in Warsaw, Poland. https://2017.pgconf.eu/ pgday.Seoul 2017 will be held in Seoul, South Korea on November 4, 2017. Korean language information is here: http://pgday.postgresql.kr/ 2ndQuadrant PostgreSQL Conference 2017 (2Q PGConf, for short) will be hosted on November 6th & 7th in New York City, and November 9th in Chicago. http://www.2qpgconf.com/ PGConf Local: Seattle will be held November 13 - 14, 2017. https://www.pgconf.us/#Seattle2017 PGDay Australia 2017 will be held on November 17 in Melbourne. http://2017.pgday.com.au/ PostgreSQL Session will take place November 17th, 2017, in Paris, France. http://www.postgresql-sessions.org/en/9/start PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is now open at https://www.pgconf.us/conferences/Austin2017 PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo, Japan. http://www.pgconf.asia/EN/2017/ PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka. Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017. http://pgconf.in/ PostgreSQL@SCaLE is a two day, two track event which takes place on March 8-9, 2018, at Pasadena Convention Center, as part of SCaLE 16X. The CfP is open through October 31, 2017 at http://www.socallinuxexpo.org/scale/16x/cfp == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English language ones to da...@fetter.org, German language to p...@pgug.de, Italian language to p...@itpug.org. == Applied Patches == Andres Freund pushed: - Reduce memory usage of targetlist SRFs. Previously nodeProjectSet only released memory once per input tuple, rather than once per returned tuple. If the computation of an individual returned tuple requires a lot of memory, that can lead to problems. Instead change things so that the expression context can be reset once per output tuple, which requires a new memory context to store SRF arguments in. This is a longstanding issue, but was hard to fix before 9.6, due to the way tSRFs where evaluated. But it's fairly easy to fix now. We could backpatch this into 10, but given there've been fewc omplaints that doesn't seem worth the risk so far. Reported-By: Lucas Fairchild Author: Andres Freund, per discussion with Tom Lane Discussion: https://postgr.es/m/4514.1507318...@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/84ad4b036d975ad1be0f52251bac3a06463c9811 - Fix pnstrdup() to not memcpy() the maximum allowed length. The previous behaviour was dangerous if the length passed wasn't the size of the underlying buffer, but the maximum size of the underlying buffer. Author: Andres Freund Discussion: https://postgr.es/m/20161003215524.mwz5p45pcverr...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/82c117cb90e6b6b79f06d61eb1ddf06e94e75b60 - Rewrite strnlen replacement implementation from 8a241792f96. The previous placement of the fallback implementation in libpgcommon was problematic, because libpqport functions need strnlen functionality. Move replacement into libpgport. Provide strnlen() under its posix name, instead of pg_strnlen(). Fix stupid configure bug, executing the test only when compiled with threading support. Author: Andres Freund Discussion: https://postgr.es/m/e1e1gr2-0005fb...@gemulon.postgresql.org https://git.postgresql.org/pg/commitdiff/fffd651e83ccbd6191a76be6ec7c6b1b27888fde - Prevent idle in transaction session timeout from sometimes being ignored. The previous coding in ProcessInterrupts() could lead to idle_in_transaction_session_timeout being ignored, when statement_timeout occurred earlier. The problem was that ProcessInterrupts() would return before processing the transaction timeout if QueryCancelPending was set while QueryCancelHoldoffCount != 0 - which is the case when reading new commands from the client. Ergo when the idle transaction timeout would hit. Fix that by removing the early return. Alternatively the transaction timeout code could have been moved up, but that early return seems like an issue that could hit other cases too. Author: Lukas Fittl Bug: #14821 Discussion: https://www.postgresql.org/message-id/20170921010956.17345.61461%40wrigleys.postgresql.org https://www.postgresql.org/message-id/CAP53PkxQnv3OWJpyNPGJYT62uY=n1=2cf_lpc6gvofnc0-g...@mail.gmail.com Backpatch: 9.6-, where idle_in_transaction_session_timeout was introduced. https://git.postgresql.org/pg/commitdiff/f676616651c83b14e1d879fbfabdd3ab2dc70bbe - Add more efficient functions to pqformat API. There's three prongs to achieve greater efficiency here: 1) Allow reusing a stringbuffer across pq_beginmessage/endmessage, with the new pq_beginmessage_reuse/endmessage_reuse. This can be beneficial both because it avoids allocating the initial buffer, and because it's more likely to already have an correctly sized buffer. 2) Replacing pq_sendint() with pq_sendint$width() inline functions. Previously unnecessary and unpredictable branches in pq_sendint() were needed. Additionally the replacement functions are implemented more efficiently. pq_sendint is now deprecated, a separate commit will convert all in-tree callers. 3) Add pq_writeint$width(), pq_writestring(). These rely on sufficient space in the StringInfo's buffer, avoiding individual space checks & potential individual resizing. To allow this to be used for strings, expose mbutil.c's MAX_CONVERSION_GROWTH. Followup commits will make use of these facilities. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/1de09ad8eb1fa673ee7899d6dfbb2b49ba204818 - Allow to avoid NUL-byte management for stringinfos and use in format.c. In a lot of the places having appendBinaryStringInfo() maintain a trailing NUL byte wasn't actually meaningful, e.g. when appending an integer which can contain 0 in one of its bytes. Removing this yields some small speedup, but more importantly will be more consistent when providing faster variants of pq_sendint etc. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/70c2d1be2b1e1efa8ef38a92b443fa290a9558dd - Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/f2dec34e19d3969ddd616e671fe9a7b968bec812 - Work around overly strict restrict checks by MSVC. Apparently MSVC requires a * before a restrict in a variable declaration, even if the adorned type already is a pointer, just via typedef. As reported by buildfarm animal woodlouse. Author: Andres Freund Discussion: https://postgr.es/m/20171012001320.4putagiruueht...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/060b069984a69ff0255ce318f10681c553613bef - Temporary attempt at a workaround for further MSVC restrict build failures. It appears some versions of msvc use __declspec(restrict) in stdlib.h and subsidiary headers. Including those after defining 'restrict' to '__restrict' doesn't work. Try to get the buildfarm green to see whether there's further problems, by including stdlib.h just before said define. https://git.postgresql.org/pg/commitdiff/36b4b91ba07843406d5a30106facb59d8275c6de - Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/31079a4a8e66e56e48bad94d380fa6224e9ffa0d - Add configure infrastructure to detect support for C99's restrict. Will be used in later commits improving performance for a few key routines where information about aliasing allows for significantly better code generation. This allows to use the C99 'restrict' keyword without breaking C89, or for that matter C++, compilers. If not supported it's defined to be empty. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/0b974dba2d6b5581ce422ed883209de46f313fb6 - Use C99 restrict via pg_restrict, rather than restrict directly. Unfortunately using 'restrict' plainly causes problems with MSVC, which supports restrict only as '__restrict'. Defining 'restrict' to '__restrict' unfortunately causes a conflict with MSVC's usage of __declspec(restrict) in headers. Therefore define pg_restrict to the appropriate keyword instead, and replace existing usages. This replaces the temporary workaround introduced in 36b4b91ba078. Author: Andres Freund Discussion: https://postgr.es/m/2656.1507830...@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/91d5f1a4a3e8aea2a6488243bac55806160408fb - Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all. SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbek...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/4c119fbcd49ba882791c7b99a1e934b985468e9f - Force "restrict" not to be used when compiling with xlc. Per buildfarm animal Hornet and followup manual testing by Noah Misch, it appears xlc miscompiles code using "restrict" in at least some cases. Allow disabling restrict usage with FORCE_DISABLE_RESTRICT=yes in template files, and do so for aix/xlc. Author: Andres Freund and Tom Lane Discussion: https://postgr.es/m/1820.1507918...@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/d133982d598c7e6208d16cb4fc0b552151796603 - Improve sys/catcache performance. The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Reviewed-By: Robert Haas Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7l...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/141fd1b66ce6e3d10518d66d4008bd368f1505fd - Add pg_noinline macro to c.h. Forcing a function not to be inlined can be useful if it's the slow-path of a performance critical function, or should be visible in profiles to allow for proper cost attribution. Author: Andres Freund Discussion: https://postgr.es/m/20170914061207.zxotvyopetm7l...@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/a0247e7a11bb9f5fd55694b594a3906b7bd05881 Peter Eisentraut pushed: - Remove unused documentation file. https://git.postgresql.org/pg/commitdiff/71c75ddfbb277362bf62dc5b1645c3903e16bc34 - Use lower-case SGML attribute values. for DocBook XML compatibility https://git.postgresql.org/pg/commitdiff/44b3230e821e7a0cc4e9438d1c27305d533edacc - Fix whitespace. https://git.postgresql.org/pg/commitdiff/e9e0f78bdeaee6e1e24544fd564cf0907f6a2134 - Improve LDAP cleanup code in error paths. After calling ldap_unbind_s() we probably shouldn't try to use the LDAP connection again to call ldap_get_option(), even if it failed. The OpenLDAP man page for ldap_unbind[_s] says "Once it is called, the connection to the LDAP server is closed, and the ld structure is invalid." Otherwise, as a general rule we should probably call ldap_unbind() before returning in all paths to avoid leaking resources. It is unlikely there is any practical leak problem since failure to authenticate currently results in the backend exiting soon afterwards. Author: Thomas Munro Reviewed-By: Alvaro Herrera, Peter Eisentraut Discussion: https://postgr.es/m/20170914141205.eup4kxzlkagtmfac%40alvherre.pgsql https://git.postgresql.org/pg/commitdiff/1feff99fe4576d4685c14dff18d1f845a1456f10 - Attempt to fix LDAP build. Apparently, an older spelling of LDAP_OPT_DIAGNOSTIC_MESSAGE is LDAP_OPT_ERROR_STRING, so fall back to that one. https://git.postgresql.org/pg/commitdiff/7d1b8e7591690fb68cc53553e0f13b537b5455dc - Log diagnostic messages if errors occur during LDAP auth. Diagnostic messages seem likely to help users diagnose root causes more easily, so let's report them as errdetail. Author: Thomas Munro Reviewed-By: Ashutosh Bapat, Christoph Berg, Alvaro Herrera, Peter Eisentraut Discussion: https://postgr.es/m/CAEepm=2_dA-SYpFdmNVwvKsEBXOUj=k4ookovhmvj6jnmdt...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/cf1238cd9763f0a6e3454ddf75ac56ff722f18ee - Reinstate genhtml --prefix option for non-vpath builds. In c3d9a66024a93e6d0380bdd1b18cb03a67216b72, the genhtml --prefix option was removed to get slightly better behavior for vpath builds. genhtml would then automatically pick a suitable prefix. However, for non-vpath builds, this makes the coverage output dependent on the length of the path where the source code happens to be, leading to confusingly arbitrary results. So put the --prefix option back for non-vpath builds. https://git.postgresql.org/pg/commitdiff/5f340cb30ce2f0d9f272840b0d977b0a4b854f0b - doc: Postgres -> PostgreSQL. https://git.postgresql.org/pg/commitdiff/d8794fd7c337a2285f46b23d348c9826afff69eb Tom Lane pushed: - Add missing clean step to src/test/modules/brin/Makefile. I noticed the tmp_check subdirectory wasn't getting cleaned up after a check-world run. Apparently pgxs.mk will only do this for you if you've defined REGRESS. The only other src/test/modules Makefile that does not set that is snapshot_too_old, and it does it like this. https://git.postgresql.org/pg/commitdiff/fa5e119dc71ada8d023deadcb36dbfae328f8902 - Regenerate configure script. Not sure how fffd651e83ccbd6191a76be6ec7c6b1b27888fde ended up probing for "strnlenfrak" rather than "strnlen". My autoconf doesn't do that ... https://git.postgresql.org/pg/commitdiff/f4128ab466aac639387a5dade6647621c87bbb3f - Fix low-probability loss of NOTIFY messages due to XID wraparound. Up to now async.c has used TransactionIdIsInProgress() to detect whether a notify message's source transaction is still running. However, that function has a quick-exit path that reports that XIDs before RecentXmin are no longer running. If a listening backend is doing nothing but listening, and not running any queries, there is nothing that will advance its value of RecentXmin. Once 2 billion transactions elapse, the RecentXmin check causes active transactions to be reported as not running. If they aren't committed yet according to CLOG, async.c decides they aborted and discards their messages. The timing for that is a bit tight but it can happen when multiple backends are sending notifies concurrently. The net symptom therefore is that a sufficiently-long-surviving listen-only backend starts to miss some fraction of NOTIFY traffic, but only under heavy load. The only function that updates RecentXmin is GetSnapshotData(). A brute-force fix would therefore be to take a snapshot before processing incoming notify messages. But that would add cycles, as well as contention for the ProcArrayLock. We can be smarter: having taken the snapshot, let's use that to check for running XIDs, and not call TransactionIdIsInProgress() at all. In this way we reduce the number of ProcArrayLock acquisitions from one per message to one per notify interrupt; that's the same under light load but should be a benefit under heavy load. Light testing says that this change is a wash performance-wise for normal loads. I looked around for other callers of TransactionIdIsInProgress() that might be at similar risk, and didn't find any; all of them are inside transactions that presumably have already taken a snapshot. Problem report and diagnosis by Marko Tiikkaja, patch by me. Back-patch to all supported branches, since it's been like this since 9.0. Discussion: https://postgr.es/m/20170926182935.14128.65...@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/118e99c3d71efbea85341697a447d84bbfb54f18 - Doc: fix missing explanation of default object privileges. The GRANT reference page, which lists the default privileges for new objects, failed to mention that USAGE is granted by default for data types and domains. As a lesser sin, it also did not specify anything about the initial privileges for sequences, FDWs, foreign servers, or large objects. Fix that, and add a comment to acldefault() in the probably vain hope of getting people to maintain this list in future. Noted by Laurenz Albe, though I editorialized on the wording a bit. Back-patch to all supported branches, since they all have this behavior. Discussion: https://postgr.es/m/1507620895.4152.1.ca...@cybertec.at https://git.postgresql.org/pg/commitdiff/28605968322b70a7efe1cc89595d1cfc557d80b9 - Remove unnecessary PG_TRY overhead for CurrentResourceOwner changes. resowner/README contained advice to use a PG_TRY block to restore the old CurrentResourceOwner value anywhere that that variable is transiently changed. That advice was only inconsistently followed, however, and on reflection it seems like unnecessary overhead. We don't bother with such a convention for transient CurrentMemoryContext changes, on the grounds that any (sub)transaction abort will start out by resetting CurrentMemoryContext to what it wants. But the same is true of CurrentResourceOwner, so there seems no need to treat it differently. Hence, remove PG_TRY blocks that exist only to restore CurrentResourceOwner before re-throwing the error. There are a couple of places that restore it along with some other actions, and I left those alone; the restore is probably unnecessary but no noticeable gain will result from removing it. Discussion: https://postgr.es/m/5236.1507583...@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/5fa6b0d102eb8ccd15c4963ee9841baec50df45e - Prevent sharing transition states between ordered-set aggregates. This ought to work, but the built-in OSAs are not capable of coping, because their final-functions destructively modify their transition state (specifically, the contained tuplesort object). That was fine when those functions were written, but commit 804163bc2 moved the goalposts without telling orderedsetaggs.c. We should fix the built-in OSAs to support this, but it will take a little work, especially if we don't want to sacrifice performance in the normal non-shared-state case. Given that it took a year after 9.6 release for anyone to notice this bug, we should not prioritize sharable-state over nonsharable-state performance. And a proper fix is likely to be more complicated than we'd want to back-patch, too. Therefore, let's just put in this stop-gap patch to prevent nodeAgg.c from choosing to use shared state for OSAs. We can revert it in HEAD when we get a better fix. Report from Lukas Eder, diagnosis by me, patch by David Rowley. Back-patch to 9.6 where the problem was introduced. Discussion: https://postgr.es/m/cab4elo5rzhoamut9xsf72ozbendllxzksk07fisvsujnzb8...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/52328727bea4d9f95af9622e4624b9d1492df88e - Doc: fix typo in release notes. Ioseph Kim Discussion: https://postgr.es/m/e7a79f91-8244-5bcb-afcc-96c817e86...@postgresql.kr https://git.postgresql.org/pg/commitdiff/0a047a1e3ef852884278b1324df73e359972c43a - Add port/strnlen support to libpq and ecpg Makefiles. In the wake of fffd651e8, any makefile that pulls in snprintf.c from src/port/ needs to be prepared to pull in strnlen.c as well. Per buildfarm. https://git.postgresql.org/pg/commitdiff/46912d9b1504cfaede1b22811039028a75f76ab8 - Fix AggGetAggref() so it won't lie to aggregate final functions. If we merge the transition calculations for two different aggregates, it's reasonable to assume that the transition function should not care which of those Aggref structs it gets from AggGetAggref(). It is not reasonable to make the same assumption about an aggregate final function, however. Commit 804163bc2 broke this, as it will pass whichever Aggref was first associated with the transition state in both cases. This doesn't create an observable bug so far as the core system is concerned, because the only existing uses of AggGetAggref() are in ordered-set aggregates that happen to not pay attention to anything but the input properties of the Aggref; and besides that, we disabled sharing of transition calculations for OSAs yesterday. Nonetheless, if some third-party code were using AggGetAggref() in a normal aggregate, they would be entitled to call this a bug. Hence, back-patch the fix to 9.6 where the problem was introduced. In passing, improve some of the comments about transition state sharing. Discussion: https://postgr.es/m/cab4elo5rzhoamut9xsf72ozbendllxzksk07fisvsujnzb8...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/305cf1fd7239e0ffa9ae4ff54a7c66f36432c741 - Rely on sizeof(typename) rather than sizeof(variable) in pqformat.h. In each of the pq_writeintN functions, the three uses of sizeof() should surely all be consistent. I started out to make them all sizeof(ni), but on reflection let's make them sizeof(typename) instead. That's more like our usual style elsewhere, and it's just barely possible that the failures buildfarm member hornet has shown since 4c119fbcd went in are caused by the compiler getting confused about sizeof() a parameter that it's optimizing away. In passing, improve a couple of comments. Discussion: https://postgr.es/m/e1e2rml-0002do...@gemulon.postgresql.org https://git.postgresql.org/pg/commitdiff/5229db6c6f92515afcd698cf5d5badc12ffe6bc2 - Improve implementation of CRE-stack-flattening in map_variable_attnos(). I (tgl) objected to the obscure implementation introduced in commit 1c497fa72. This one seems a bit less action-at-a-distance-y, at the price of repeating a few lines of code. Improve the comments about what the function is doing, too. Amit Khandekar, whacked around a bit more by me Discussion: https://postgr.es/m/caj3gd9egytyhuh0ntmxm8-1m3rvdqebatygc-cuntyf7tkn...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/73937119bfd07a140da4817f5ca949351942ffdc - Explicitly track whether aggregate final functions modify transition state. Up to now, there's been hard-wired assumptions that normal aggregates' final functions never modify their transition states, while ordered-set aggregates' final functions always do. This has always been a bit limiting, and in particular it's getting in the way of improving the built-in ordered-set aggregates to allow merging of transition states. Therefore, let's introduce catalog and CREATE AGGREGATE infrastructure that lets the finalfn's behavior be declared explicitly. There are now three possibilities for the finalfn behavior: it's purely read-only, it trashes the transition state irrecoverably, or it changes the state in such a way that no more transfn calls are possible but the state can still be passed to other, compatible finalfns. There are no examples of this third case today, but we'll shortly make the built-in OSAs act like that. This change allows user-defined aggregates to explicitly disclaim support for use as window functions, and/or to prevent transition state merging, if their implementations cannot handle that. While it was previously possible to handle the window case with a run-time error check, there was not any way to prevent transition state merging, which in retrospect is something commit 804163bc2 should have provided for. But better late than never. In passing, split out pg_aggregate.c's extern function declarations into a new header file pg_aggregate_fn.h, similarly to what we've done for some other catalog headers, so that pg_aggregate.h itself can be safe for frontend files to include. This lets pg_dump use the symbolic names for relevant constants. Discussion: https://postgr.es/m/4834.1507849...@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/4de2d4fba38f4f7aff7f95401eb43a6cd05a6db4 - gcc's support for __attribute__((noinline)) hasn't been around forever. Buildfarm member gaur says it wasn't there in 2.95.3. Guess that 3.0 and later have it. https://git.postgresql.org/pg/commitdiff/82aff8d3369754282114cb0fff92a342b2864e75 Robert Haas pushed: - Fix mistakes in comments. Masahiko Sawada Discussion: http://postgr.es/m/cad21aobsfysmhd6_sl9in3n_foaa+opbl5jg55dxu1chauj...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/20d210bf5bb0d5ae37c727d364cfd810c367704a - pg_stat_statements: Widen query IDs from 32 bits to 64 bits. This takes advantage of the infrastructure introduced by commit 81c5e46c490e2426db243eada186995da5bb0ba7 to greatly reduce the likelihood that two different queries will end up with the same query ID. It's still possible, of course, but whereas before it the chances of a collision reached 25% around 50,000 queries, it will now take more than 3 billion queries. Backward incompatibility: Because the type exposed at the SQL level is int8, users may now see negative query IDs in the pg_stat_statements view (and also, query IDs more than 4 billion, which was the old limit). Patch by me, reviewed by Michael Paquier and Peter Geoghegan. Discussion: http://postgr.es/m/ca+tgmobg_kp4cbkfmsznuaam1gww6hhrnizc0kjrmooeynz...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/cff440d368690f94fbda1a475277e90ea2263843 - Fix logical replication to fire BEFORE ROW DELETE triggers. Before, that would fail to happen unless a BEFORE ROW UPDATE trigger was also present. Noted by me while reviewing a patch from Masahiko Sawada, who also wrote this patch. Reviewed by Petr Jelinek. Discussion: http://postgr.es/m/ca+tgmobazvcxdug8y_mqkbk7nz-vhbdlvjm354kefozpuzm...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/360fd1a7b2fe779cc9e696b813b12f6a8e83b558 - Synchronize error messages. Commits 6476b26115f3ef25a9cd87880e0ac5ec5f7a05f6 and 14f67a8ee282ebc0de78e773fbd597f460ab4a54 didn't use quite the same error message for what is basically the same situation. Amit Langote, pared back a bit by me. Discussion: http://postgr.es/m/54dc76d0-3b5b-ba5a-27dc-fb31a3975...@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/ad4a7ed0996ee044ee7291559deddf9842d8bbf7 - Use ResultRelInfo ** rather than ResultRelInfo * for tuple routing. The previous convention doesn't lend itself to creating ResultRelInfos lazily, as we already do in ExecGetTriggerResultRel. This patch doesn't make anything lazier than before, but the pending patch for UPDATE tuple routing proposes to do so (and there might be other opportunities as well). Amit Khandekar with some adjustments by me. Discussion: http://postgr.es/m/ca+tgmoypvp9lyf6vufa5dwxs4c--x6loj2y36bsjaytp62e...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/60f7c0abef0327648c02795312d1679c66586fbb - Avoid coercing a whole-row variable that is already coerced. Marginal efficiency and beautification hack. I'm not sure whether this case ever arises currently, but the pending patch for update tuple routing will cause it to arise. Amit Khandekar Discussion: http://postgr.es/m/caj3gd9cazfppe7-wwubabpcq4_0subkipfd1+0r5_dkvnwo...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/1c497fa72df7593d8976653538da3d0ab033207f - Fix possible crash with Parallel Bitmap Heap Scan. If a Parallel Bitmap Heap scan's chain of leftmost descendents includes a BitmapOr whose first child is a BitmapAnd, the prior coding would mistakenly create a non-shared TIDBitmap and then try to perform shared iteration. Report by Tomas Vondra. Patch by Dilip Kumar. Discussion: http://postgr.es/m/50e89684-8ad9-dead-8767-c9545bafd...@2ndquadrant.com https://git.postgresql.org/pg/commitdiff/6393613b6a1e0feae3d22af608397b252cee5b58 Álvaro Herrera pushed: - Infer functional dependency past RelabelType. Vars hidden within a RelabelType would not be detected as compatible with some functional dependency. Repair by properly ignoring the RelabelType. Author: David Rowley Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/CAKJS1f-y-UEy=rsbxynbogiw1fkmr_lvoysgl9qoc36mlec...@mail.gmail.com https://git.postgresql.org/pg/commitdiff/e9ef11ac8bb2acc2d2462fc17ec3291a959589e7 Joe Conway pushed: - Add missing options to pg_regress help() output. A few command line options accepted by pg_regress were not being output by help(), including --help itself. Add that one, as well as --version and --bindir, and the corresponding short options for the first two. We could consider this for backpatching, but it did not seem worthwhile and no one else advocated for it, so apply only to master for now. Author: Joe Conway Reviewed-By: Tom Lane Discussion: https://postgr.es/m/dd519469-06d7-2662-83ef-c926f6c4f0f1%40joeconway.com https://git.postgresql.org/pg/commitdiff/b81eba6a650186dc35b6a1fb8bde320d9c29055d == Pending Patches == Nathan Bossart sent in another revision of a patch to add more logging to VACUUM and ANALYZE. Amul Sul sent in three more revisions of a patch to add hash partitioning. Ashutosh Bapat sent in a patch to support partition-wise join for dummy partitioned relation. Amit Kapila sent in another revision of a patch to parallize queries containing initplans. Jeevan Chalke sent in another revision of a patch to implement partition-wise aggregation/grouping. David Rowley sent in two revisions of a patch to remove left joins with a DISTINCT clause, as the DISTINCT makes the LEFT JOIN redundant. Julien Rouhaud and Thomas Munro traded patches to fix an oversight in EphemeralNamedRelation support. Rushabh Lathia sent in another revision of a patch to add parallel B-tree index build sorting. Michaël Paquier sent in another revision of a patch to refactor the routine to test connection to an SSL server, support channel binding 'tls-unique' in SCRAM, add connection parameters "saslname" and "saslchannelbinding", and implement channel binding tls-server-end-point for SCRAM. Masahiko Sawada and Aleksander Alekseev traded patches to ensure that updated columns are not null in logical replication when they shouldn't be. Laurenz Albe sent in a patch to document that PUBLIC has USAGE privileges on newly created types. Amit Khandekar sent in another revision of a patch to implement parallel append. Ashutosh Bapat sent in another revision of a patch to modify the bound comparision functions to accept members of PartitionKey, add partition-wise join for 1:1, 1:0, 0:1 partition matching, and add tests for same. Yugo Nagata sent in a patch to implement lockable views. Tom Lane sent in a patch to remove Windows warnings from VS 2017 by removing MemSet. Dilip Kumar sent in another revision of a patch to improve bitmap costing for lossy pages. Pavel Stěhule sent in another revision of a patch to PL/pgsql to allow forcing either a generic or a custom plan. Peter Eisentraut sent in a patch to replace GrantObjectType with ObjectType. Jeevan Chalke sent in a patch to add a cost_append function which determines and returns the cost of an Append node. Amit Kapila sent in a patch to handle redundant ConvertRowtypeExpr nodes. Álvaro Herrera sent in a patch to refactor the relkind check in DefineIndex from an ugly rat's nest of 'if' statements to a switch statement. Amit Kapila sent in a patch to fix parallel safety for extern params, fix simple expr interaction gather, and fix parallel mode nested execution. Tomas Vondra sent in a PoC patch to enable parallel execution for cursors explicitly. Pavel Stěhule sent in another revision of a patch to enable default namespaces for XPath expressions. -- Sent via pgsql-announce mailing list (pgsql-announce@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-announce
