[HACKERS] Wire protocol compression

2016-04-20 Thread Shay Rojansky
I know this has been discussed before ( http://postgresql.nabble.com/Compression-on-SSL-links-td2261205.html, http://www.postgresql.org/message-id/BANLkTi=Ba1ZCmBuwwn7M1wvPFioT=6n...@mail.gmail.com), but it seems to make sense to revisit this in 2016. Since CRIME in 2012, AFAIK compression with en

Re: [HACKERS] EXPLAIN VERBOSE with parallel Aggregate

2016-04-20 Thread Tom Lane
David Rowley writes: > I'd like to admit that I'm a bit confused as to why > generate_function_name(), when it already knows the funcid, bothers to > call func_get_detail(), which performs a search for the function based > on the name and argument types, to find the function, most likely with > th

Re: [HACKERS] EXPLAIN VERBOSE with parallel Aggregate

2016-04-20 Thread David Rowley
On 18 April 2016 at 02:22, Tom Lane wrote: > David Rowley writes: >> Note that I've done nothing for the weird schema prefixing problem I >> mentioned. I think I'd need input on the best way to solve this. If >> it's actually a problem. > > It sure looks like a problem to me. Maybe it's only cos

Re: [HACKERS] Fix of doc for synchronous_standby_names.

2016-04-20 Thread Amit Langote
On 2016/04/21 12:25, Kyotaro HORIGUCHI wrote: > At Wed, 20 Apr 2016 23:07:41 -0400, Robert Haas wrote: >> On Sun, Apr 17, 2016 at 11:56 PM, Kyotaro HORIGUCHI wrote: >>> There is no mechanism to enforce uniqueness. In case of duplicates one of the matching standbys will be considered as >>

Re: [HACKERS] Should XLogInsert() be done only inside a critical section?

2016-04-20 Thread Michael Paquier
On Thu, Apr 21, 2016 at 5:44 AM, Tom Lane wrote: > Anyway, I went through our tree and added START/END_CRIT_SECTION calls > around all XLogInsert calls that could currently be reached without one; > see attached. Since this potentially breaks third-party code I would > not propose back-patching i

Re: [HACKERS] PGCTLTIMEOUT in pg_regress, or skink versus the clock

2016-04-20 Thread Tom Lane
Noah Misch writes: > On Wed, Apr 20, 2016 at 06:38:56PM -0400, Tom Lane wrote: >> I am thinking that we missed a bet in commit 2ffa86962077c588 >> et al, and that pg_regress's hard-wired 60-second start timeout ought to >> be overridable from an environment variable just as pg_ctl's timeout is. >>

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-20 Thread Robert Haas
On Tue, Mar 29, 2016 at 10:20 AM, Ashutosh Bapat wrote: >> I think the reason for that is in foreign_join_ok. This in that function: >> >> wrongly pulls up remote_conds from joining relations in the FULL JOIN >> case. I think we should not pull up such conditions in the FULL JOIN case. >> > > Ri

Re: [HACKERS] Fix of doc for synchronous_standby_names.

2016-04-20 Thread Kyotaro HORIGUCHI
At Wed, 20 Apr 2016 23:07:41 -0400, Robert Haas wrote in > On Sun, Apr 17, 2016 at 11:56 PM, Kyotaro HORIGUCHI > wrote: > > Hello, now the synchronous_standby_names can teach to ensure more > > then one synchronous standbys. But the doc for it seems assuming > > only one synchronous standby. >

Re: [HACKERS] Fix of doc for synchronous_standby_names.

2016-04-20 Thread Robert Haas
On Sun, Apr 17, 2016 at 11:56 PM, Kyotaro HORIGUCHI wrote: > Hello, now the synchronous_standby_names can teach to ensure more > then one synchronous standbys. But the doc for it seems assuming > only one synchronous standby. > >> There is no mechanism to enforce uniqueness. In case of >> duplicat

Re: [HACKERS] "parallel= " information is not coming in pg_dumpall for create aggregate

2016-04-20 Thread Robert Haas
On Mon, Apr 18, 2016 at 10:47 AM, Fabrízio de Royes Mello wrote: >> I checked in PG 9.6 , if we create an aggregate function with saying - >> parallel=safe/restricted/unsafe and then take >> a pg_dumpall of the entire cluster , "parallel= " is missing from create >> aggregate syntax >> >> Steps to

Re: [HACKERS] pg_stat_activity crashes

2016-04-20 Thread Kyotaro HORIGUCHI
Hello, At Wed, 20 Apr 2016 15:14:16 +0200, Petr Jelinek wrote in <571780a8.4070...@2ndquadrant.com> > I noticed sporadic segfaults when selecting from pg_stat_activity on > current HEAD. > > The culprit is the 53be0b1add7064ca5db3cd884302dfc3268d884e commit > which added more wait info into the

Re: [HACKERS] pg_dump dump catalog ACLs

2016-04-20 Thread Stephen Frost
Noah, On Wednesday, April 20, 2016, Noah Misch wrote: > On Wed, Apr 20, 2016 at 11:12:44AM -0400, Stephen Frost wrote: > > * Noah Misch (n...@leadboat.com ) wrote: > > > On Sun, Apr 17, 2016 at 11:02:28PM -0400, Noah Misch wrote: > > > > (3) pg_dumpall became much slower around the time of these

Re: [HACKERS] EXPLAIN VERBOSE with parallel Aggregate

2016-04-20 Thread Robert Haas
On Tue, Apr 19, 2016 at 9:22 PM, Noah Misch wrote: > On Sun, Apr 17, 2016 at 10:22:24AM -0400, Tom Lane wrote: >> David Rowley writes: >> > On 16 April 2016 at 04:27, Tom Lane wrote: >> >> +1 for the latter, if we can do it conveniently. I think exposing >> >> the names of the aggregate impleme

Re: [HACKERS] pg_dump dump catalog ACLs

2016-04-20 Thread Noah Misch
On Wed, Apr 20, 2016 at 11:12:44AM -0400, Stephen Frost wrote: > * Noah Misch (n...@leadboat.com) wrote: > > On Sun, Apr 17, 2016 at 11:02:28PM -0400, Noah Misch wrote: > > > (3) pg_dumpall became much slower around the time of these commits. On > > > one > > > machine (POWER7 3.55 GHz), a pg_dum

Re: [HACKERS] PGCTLTIMEOUT in pg_regress, or skink versus the clock

2016-04-20 Thread Noah Misch
On Wed, Apr 20, 2016 at 06:38:56PM -0400, Tom Lane wrote: > I am thinking that we missed a bet in commit 2ffa86962077c588 > et al, and that pg_regress's hard-wired 60-second start timeout ought to > be overridable from an environment variable just as pg_ctl's timeout is. > It might as well be the s

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Peter Geoghegan
On Wed, Apr 20, 2016 at 6:43 PM, Alvaro Herrera wrote: > The brin.sql test does that ... I actually copied brin.sql when creating regression tests for external sorting, primarily because I wanted to test a variety of collations, without having any control of what they happen to be on the target.

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-20 Thread Noah Misch
On Wed, Apr 20, 2016 at 02:03:16PM +0900, Michael Paquier wrote: > On Wed, Apr 20, 2016 at 1:39 PM, Noah Misch wrote: > > On Tue, Apr 19, 2016 at 02:42:24AM -0300, Alvaro Herrera wrote: > >> This thread seems to have stalled. I thought we were going to consider > >> these patches for 9.6. > > > >

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Apr 21, 2016 at 7:13 AM, Peter Geoghegan wrote: > > On Wed, Apr 20, 2016 at 2:49 PM, Robert Haas wrote: > >> Committed. But I think the regression test needs more thought, so I > >> left that out. > > > > It would be nice if there was a fuzz testing infrastructur

Re: [HACKERS] Disallow unique index on system columns

2016-04-20 Thread Tom Lane
Eric Ridge writes: > I've got an extension that's actually a custom Access Method, and for > reasons that are probably too boring to go into here, it requires that the > first column in the index be a function that takes the ctid. Ie, something > akin to: >CREATE INDEX idx ON table (my_func('

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-20 Thread Ants Aasma
On Tue, Apr 19, 2016 at 6:11 PM, Kevin Grittner wrote: > On Tue, Apr 19, 2016 at 9:57 AM, Amit Kapila wrote: >> On Sun, Apr 17, 2016 at 2:26 AM, Andres Freund wrote: >>> >>> On 2016-04-16 16:44:52 -0400, Noah Misch wrote: >>> > That is more controversial than the potential ~2% regression for >>>

Re: [HACKERS] Disallow unique index on system columns

2016-04-20 Thread Eric Ridge
On Sat, Apr 16, 2016 at 12:14 PM Tom Lane wrote: > > Pushed. I moved the check into DefineIndex, as that's where user-facing > complaints about indexes generally ought to be. > If you're planning on back-patching this, please don't. :) It'll literally ruin my life. I've got an extension that

[HACKERS] PGCTLTIMEOUT in pg_regress, or skink versus the clock

2016-04-20 Thread Tom Lane
Buildfarm member skink has failed three times recently like this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2016-04-15%2001%3A20%3A44 the relevant part of that being pg_regress: postmaster did not respond within 60 seconds Examine /home/andres/build/buildfarm/REL9_5_STABLE/

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Michael Paquier
On Thu, Apr 21, 2016 at 7:13 AM, Peter Geoghegan wrote: > On Wed, Apr 20, 2016 at 2:49 PM, Robert Haas wrote: >> Committed. But I think the regression test needs more thought, so I >> left that out. > > It would be nice if there was a fuzz testing infrastructure that > verified that parallel pla

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Peter Geoghegan
On Wed, Apr 20, 2016 at 2:49 PM, Robert Haas wrote: > Committed. But I think the regression test needs more thought, so I > left that out. It would be nice if there was a fuzz testing infrastructure that verified that parallel plans produce the same answer as serial plans. -- Peter Geoghegan

Re: [HACKERS] Avoid parallel full and right join paths.

2016-04-20 Thread Robert Haas
On Tue, Apr 19, 2016 at 10:21 AM, Mithun Cy wrote: > Tests: > create table mytab(x int,x1 char(9),x2 varchar(9)); > create table mytab1(y int,y1 char(9),y2 varchar(9)); > insert into mytab values (generate_series(1,5),'aa','aaa'); > insert into mytab1 values (generate_series(1,1),'aa','aaa

Re: [HACKERS] Proposal: Remove regress-python3-mangle.mk

2016-04-20 Thread Tom Lane
Yury Zhuravlev writes: > Tom Lane wrote: >> Also, I would bet that those tools would not know anything >> about converting PL language names between 'plpythonu' and 'plpython2u' >> and 'plpython3u'; so even if we used one of them, we would still need a >> layer pretty similar to what we have. > I

[HACKERS] Should XLogInsert() be done only inside a critical section?

2016-04-20 Thread Tom Lane
Over in <17456.1460832...@sss.pgh.pa.us> I speculated about whether we should be enforcing that WAL insertion happen only inside critical sections. We don't currently, and a survey of the backend says that there are quite a few calls that aren't inside critical sections. But there are at least two

Re: [HACKERS] Proposal: Remove regress-python3-mangle.mk

2016-04-20 Thread Yury Zhuravlev
Tom Lane wrote: Also, I would bet that those tools would not know anything about converting PL language names between 'plpythonu' and 'plpython2u' and 'plpython3u'; so even if we used one of them, we would still need a layer pretty similar to what we have. I do not see this as a problem because

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-04-20 Thread Tom Lane
Andres Freund writes: > max_parallel_degree currently defaults to 0. I think we should enable > it by default for at least the beta period. Otherwise we're primarily > going to get reports back after the release. > Then, at the end of beta, we can decide what the default should be. +1, but let'

Re: [HACKERS] Defaults for replication/backup

2016-04-20 Thread Magnus Hagander
On Sun, Feb 14, 2016 at 9:58 AM, Magnus Hagander wrote: > > > On Sun, Feb 14, 2016 at 2:00 AM, Robert Haas > wrote: > >> On Sat, Feb 13, 2016 at 11:31 AM, Andres Freund >> wrote: >> > On 2016-02-13 11:10:58 -0500, Tom Lane wrote: >> >> Magnus Hagander writes: >> >> > The big thing is, IIRC, th

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-04-20 Thread Andreas Joseph Krogh
På onsdag 20. april 2016 kl. 19:46:31, skrev Andres Freund mailto:and...@anarazel.de>>: Hi, max_parallel_degree currently defaults to 0.  I think we should enable it by default for at least the beta period. Otherwise we're primarily going to get reports back after the release. Then, at the en

[HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-04-20 Thread Andres Freund
Hi, max_parallel_degree currently defaults to 0. I think we should enable it by default for at least the beta period. Otherwise we're primarily going to get reports back after the release. Then, at the end of beta, we can decide what the default should be. - Andres -- Sent via pgsql-hackers

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-20 Thread Magnus Hagander
On Wed, Apr 20, 2016 at 1:12 AM, Fujii Masao wrote: > On Sun, Apr 17, 2016 at 1:22 AM, Magnus Hagander > wrote: > > On Wed, Apr 13, 2016 at 4:07 AM, Noah Misch wrote: > >> > >> On Tue, Apr 12, 2016 at 10:08:23PM +0200, Magnus Hagander wrote: > >> > On Tue, Apr 12, 2016 at 8:39 AM, Noah Misch >

Re: [HACKERS] Declarative partitioning

2016-04-20 Thread Amit Langote
Hi Ildar, On Wed, Apr 20, 2016 at 11:46 PM, Ildar Musin wrote: > Thanks for clarification! I tried the updated patch, now it works correctly. Great, thanks! > I encountered another problem that concerns expressions as partitioning key. > Probably there is still some work in progress. But if it

Re: [HACKERS] pg_dump dump catalog ACLs

2016-04-20 Thread Stephen Frost
Noah, all, * Noah Misch (n...@leadboat.com) wrote: > On Sun, Apr 17, 2016 at 11:02:28PM -0400, Noah Misch wrote: > > On Tue, Apr 05, 2016 at 05:50:18PM -0400, Stephen Frost wrote: > > > I'll be doing more testing, review and clean-up (there are some > > > whitespace only changes in the later patch

Re: [HACKERS] Declarative partitioning

2016-04-20 Thread Ildar Musin
Hi Amit, On 20.04.2016 13:28, Amit Langote wrote: On 2016/04/19 23:52, Amit Langote wrote: On Tue, Apr 19, 2016 at 11:26 PM, Alexander Korotkov Another question is that it might be NOT what users expect from that. From the syntax side it very looks like defining something boxes regions for tw

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-20 Thread Andres Freund
On 2016-04-19 20:27:31 +0530, Amit Kapila wrote: > On Sun, Apr 17, 2016 at 2:26 AM, Andres Freund wrote: > > > > On 2016-04-16 16:44:52 -0400, Noah Misch wrote: > > > That is more controversial than the potential ~2% regression for > > > old_snapshot_threshold=-1. Alvaro[2] and Robert[3] are okay

Re: [HACKERS] snapshot too old, configured by time

2016-04-20 Thread Kevin Grittner
On Tue, Apr 19, 2016 at 12:49 PM, Alvaro Herrera wrote: > Well, it seems I'm outvoted. The no-op change to attempt to force an explicit choice of whether to test for snapshot age after calling BufferGetPage() has been reverted. This eliminates about 500 back-patching pain points in 65 files. I

Re: [HACKERS] Proposal: Remove regress-python3-mangle.mk

2016-04-20 Thread Tom Lane
Yury Zhuravlev writes: > Noah Misch wrote: >> I may not understand that second sentence. We have multiple buildfarm >> members >> verifying the python2 case and multiple members verifying the python3 case. > I wrote about special python3 features what not testing today. We testing > Python3 as

[HACKERS] pg_stat_activity crashes

2016-04-20 Thread Petr Jelinek
Hi, I noticed sporadic segfaults when selecting from pg_stat_activity on current HEAD. The culprit is the 53be0b1add7064ca5db3cd884302dfc3268d884e commit which added more wait info into the pg_stat_get_activity(). More specifically, the following code is broken: + pro

Re: [HACKERS] Proposal: Remove regress-python3-mangle.mk

2016-04-20 Thread Yury Zhuravlev
Noah Misch wrote: Considering we have 2756 lines of Python test SQL and just thirteen lines of sed to mangle them, the current method is scaling nicely. What pitfalls hides this approach? Convert python2 code to python3 it is really bad practice. I may not understand that second sentence. We

Re: [HACKERS] FATAL: could not send end-of-streaming message to primary: no COPY in progress

2016-04-20 Thread Kyotaro HORIGUCHI
At Wed, 20 Apr 2016 16:16:40 +0900, Fujii Masao wrote in > On Thu, Mar 31, 2016 at 9:15 AM, Thomas Munro > wrote: > > Hi hackers, > > > > If you shut down a primary server, a standby that is streaming from it > > says54: > > > > LOG: replication terminated by primary server > > DETAIL: End o

Re: [HACKERS] Re: BUG #13685: Archiving while idle every archive_timeout with wal_level hot_standby

2016-04-20 Thread Michael Paquier
On Mon, Apr 4, 2016 at 3:22 PM, Michael Paquier wrote: > The approach introducing the concept of WAL progress addresses the > problem of unnecessary checkpoints and of unnecessary standby > snapshots. If we take the problem only from the angle of RUNNING_XACTS > the current logic used to check if

Re: [HACKERS] Description of ForeignPath

2016-04-20 Thread Amit Langote
Fujita-san, On 2016/04/20 16:20, Etsuro Fujita wrote: > On 2016/04/18 17:31, Amit Langote wrote: >> Is the following description now outdated: >> >> "ForeignPath represents a potential scan of a foreign table" >> >> Considering that there now exists FdwRoutine.GetForeignJoinPaths() whose >> produ

Re: [HACKERS] Description of ForeignPath

2016-04-20 Thread Etsuro Fujita
On 2016/04/18 17:31, Amit Langote wrote: Is the following description now outdated: "ForeignPath represents a potential scan of a foreign table" Considering that there now exists FdwRoutine.GetForeignJoinPaths() whose product is nothing else but a ForeignPath, should it now say (patch attached)

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-20 Thread Kyotaro HORIGUCHI
At Wed, 20 Apr 2016 11:51:09 +0900, Masahiko Sawada wrote in > On Mon, Apr 18, 2016 at 2:15 PM, Kyotaro HORIGUCHI > wrote: > > At Fri, 15 Apr 2016 17:36:57 +0900, Masahiko Sawada > > wrote in > > > >> How about if check_hook just parses parameter in > >> CurrentMemoryContext(i.g., T_AllocSe

Re: [HACKERS] FATAL: could not send end-of-streaming message to primary: no COPY in progress

2016-04-20 Thread Fujii Masao
On Thu, Mar 31, 2016 at 9:15 AM, Thomas Munro wrote: > Hi hackers, > > If you shut down a primary server, a standby that is streaming from it says54: > > LOG: replication terminated by primary server > DETAIL: End of WAL reached on timeline 1 at 0/14F4B68. > FATAL: could not send end-of-streami