Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Kyotaro HORIGUCHI
At Fri, 15 Apr 2016 08:52:56 +0530, Amit Kapila wrote in > On Thu, Apr 14, 2016 at 1:10 PM, Masahiko Sawada > wrote: > > > > On Thu, Apr 14, 2016 at 1:11 PM, Kyotaro HORIGUCHI > > wrote: > > > At Thu, 14 Apr 2016 12:42:06 +0900, Fujii Masao > wrote in > > > >> > Yes, this is what I was tryin

[HACKERS] Rework help interface of vcregress.pl

2016-04-14 Thread Michael Paquier
Hi all, (Windows-only be careful) Horiguchi-san has mentioned yesterday (http://www.postgresql.org/message-id/20160414.172539.34325458.horiguchi.kyot...@lab.ntt.co.jp) that we are missing a couple of modes in vcregress.pl in its help message: modulescheck, bincheck, recoverycheck. The help messag

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 10:44 AM, Etsuro Fujita wrote: > On 2016/04/13 21:50, Michael Paquier wrote: >> On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas >> wrote: >>> On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita >>> wrote: > > How about we encapsulate the while (PQisBusy(...)) loop into a

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Fujii Masao
On Fri, Apr 15, 2016 at 1:12 PM, Michael Paquier wrote: > On Fri, Apr 15, 2016 at 12:57 PM, Fujii Masao wrote: >> On Fri, Apr 15, 2016 at 12:39 PM, Michael Paquier >> wrote: >>> On Fri, Apr 15, 2016 at 12:16 PM, Fujii Masao wrote: On Thu, Apr 14, 2016 at 8:41 PM, Michael Paquier wrot

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Apr 2016 21:05:40 +0900, Michael Paquier wrote in > > IPC::Run is not installed on Active Perl on my environment and > > Active state seems to be saying that IPC-Run cannot be compiled > > on Windows. ppm doesn't show IPC-Run. Is there any means to do > > TAP test other than this way?

[HACKERS] parallel query vs extensions

2016-04-14 Thread Jeff Janes
I think there are a lot of extensions which create functions which could benefit from being declared parallel safe. But how does one go about doing that? create extension xml2; select xml_valid(filler),count(*) from pgbench_accounts group by 1; Time: 3205.830 ms alter function xml_valid (text)

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Petr Jelinek
On 14/04/16 20:46, Robert Haas wrote: I believe logical replication is a fundamental database technology that should be considered just as much within the score of the core product as physical replication, parallel query, or UPSERT. Agreed, I believed we need this for very long time as well (pg

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread David Rowley
On 15 April 2016 at 13:43, David Rowley wrote: > The attached patch just disallows any index containing a system > column, apart from OID. Seems I only did half the job as I forgot to think to check for system columns that are hidden away inside expressions or predicates. The attached fixes that

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Michael Paquier
On Fri, Apr 15, 2016 at 12:57 PM, Fujii Masao wrote: > On Fri, Apr 15, 2016 at 12:39 PM, Michael Paquier > wrote: >> On Fri, Apr 15, 2016 at 12:16 PM, Fujii Masao wrote: >>> On Thu, Apr 14, 2016 at 8:41 PM, Michael Paquier >>> wrote: On Thu, Apr 14, 2016 at 5:17 PM, Masahiko Sawada

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Fujii Masao
On Fri, Apr 15, 2016 at 12:39 PM, Michael Paquier wrote: > On Fri, Apr 15, 2016 at 12:16 PM, Fujii Masao wrote: >> On Thu, Apr 14, 2016 at 8:41 PM, Michael Paquier >> wrote: >>> On Thu, Apr 14, 2016 at 5:17 PM, Masahiko Sawada >>> wrote: On Thu, Apr 14, 2016 at 1:22 PM, Michael Paquier >>

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Petr Jelinek
On 14/04/16 20:14, Robert Haas wrote: On Wed, Apr 13, 2016 at 8:54 PM, Craig Ringer wrote: On 14 April 2016 at 00:48, Robert Haas wrote: I think this would be a good topic to discuss at PGCon. I'm not going to be able to be at PGCon, and I don't think Petr is either. That's unfortunate in t

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Michael Paquier
On Fri, Apr 15, 2016 at 12:16 PM, Fujii Masao wrote: > On Thu, Apr 14, 2016 at 8:41 PM, Michael Paquier > wrote: >> On Thu, Apr 14, 2016 at 5:17 PM, Masahiko Sawada >> wrote: >>> On Thu, Apr 14, 2016 at 1:22 PM, Michael Paquier >>> wrote: Yes. I'd prefer avoid a hardcoded sleep and have s

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Amit Kapila
On Thu, Apr 14, 2016 at 1:10 PM, Masahiko Sawada wrote: > > On Thu, Apr 14, 2016 at 1:11 PM, Kyotaro HORIGUCHI > wrote: > > At Thu, 14 Apr 2016 12:42:06 +0900, Fujii Masao wrote in > >> > Yes, this is what I was trying to explain to Fujii-san upthread and I have > >> > also verified that the sa

Re: [HACKERS] Memory leak when querying GIN indexes

2016-04-14 Thread Tom Lane
Julien Rouhaud writes: > My colleague Adrien reported me a memory leak in GIN indexes while doing > some benchmark on several am. > ... > I'm not at all familiar with GIN code, but naive attached patch seems to > fix the issue and not break anything. I can reproduce this issue up to 9.4. Yes, thi

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Fujii Masao
On Thu, Apr 14, 2016 at 8:41 PM, Michael Paquier wrote: > On Thu, Apr 14, 2016 at 5:17 PM, Masahiko Sawada > wrote: >> On Thu, Apr 14, 2016 at 1:22 PM, Michael Paquier >> wrote: >>> Yes. I'd prefer avoid a hardcoded sleep and have something that relies >>> on lookups of pg_stat_replication thou

[HACKERS] more parallel query documentation

2016-04-14 Thread Robert Haas
As previously threatened, I have written some user documentation for parallel query. I put it up here: https://wiki.postgresql.org/wiki/Parallel_Query This is not totally comprehensive and I can think of a few more details that could be added, but it's a pretty good high-level overview of what g

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-14 Thread Andres Freund
On 2016-04-14 22:46:01 -0400, Tom Lane wrote: > > Hm. Doing so I found the following in 9.3: > > > /home/andres/src/postgresql-9.3/src/bin/pg_dump/parallel.c:561:23: error: > > initializer for aggregate is not a compile-time constant > > [-Werror,-Wc99-extensions] > > int

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-14 Thread Tom Lane
Andres Freund writes: > On 2016-04-13 23:31:21 -0700, Andres Freund wrote: >> I'm also putting up an animal with clang that uses >> CFLAGS='-std=c89 -Wc99-extensions -Werror=c99-extensions' >> which actually catches this. > Hm. Doing so I found the following in 9.3: > /home/andres/src/postgresql

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread David Rowley
On 15 April 2016 at 13:56, Tom Lane wrote: > David Rowley writes: >> On 15 April 2016 at 13:30, Andres Freund wrote: >>> What'd be the point of indexing ctid, and why would it be correct? >>> Wouldn't, hm, HOT break it? > >> I don't personally see the point. > > An index on ctid is useless by de

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-14 Thread Andres Freund
On 2016-04-13 23:31:21 -0700, Andres Freund wrote: > I'm also putting up an animal with clang that uses > CFLAGS='-std=c89 -Wc99-extensions -Werror=c99-extensions' > which actually catches this. Hm. Doing so I found the following in 9.3: /home/andres/src/postgresql-9.3/src/bin/pg_dump/parallel.c:

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread Tom Lane
David Rowley writes: > On 15 April 2016 at 13:30, Andres Freund wrote: >> What'd be the point of indexing ctid, and why would it be correct? >> Wouldn't, hm, HOT break it? > I don't personally see the point. An index on ctid is useless by definition: if you know the ctid of a tuple, you can jus

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread David Rowley
On 15 April 2016 at 13:30, Andres Freund wrote: > On 2016-04-15 13:26:35 +1200, David Rowley wrote: >> On 15 April 2016 at 13:02, Tom Lane wrote: >> > David Rowley writes: >> >> I proposed a fix over there, but it didn't go anywhere, probably >> >> because Tom and Andres discussed just disallowi

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread Andres Freund
On 2016-04-15 13:26:35 +1200, David Rowley wrote: > On 15 April 2016 at 13:02, Tom Lane wrote: > > David Rowley writes: > >> I proposed a fix over there, but it didn't go anywhere, probably > >> because Tom and Andres discussed just disallowing unique indexes on > >> system columns altogether. So

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread David Rowley
On 15 April 2016 at 13:02, Tom Lane wrote: > David Rowley writes: >> I proposed a fix over there, but it didn't go anywhere, probably >> because Tom and Andres discussed just disallowing unique indexes on >> system columns altogether. So, the attached patch does just that, and >> also fixes up th

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread Andres Freund
On 2016-04-14 21:02:26 -0400, Tom Lane wrote: > David Rowley writes: > > I proposed a fix over there, but it didn't go anywhere, probably > > because Tom and Andres discussed just disallowing unique indexes on > > system columns altogether. So, the attached patch does just that, and > > also fixes

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Craig Ringer
On 15 April 2016 at 02:14, Robert Haas wrote: > That's a bummer. I suppose you won't be at NYC either? > I won't. I'm not sure about Petr. It's over 24h travel, and costly, from Perth. So conference attendance is unfortunately something I have to pick and choose about. -- Craig Ringer

Re: [HACKERS] Disallow unique index on system columns

2016-04-14 Thread Tom Lane
David Rowley writes: > I proposed a fix over there, but it didn't go anywhere, probably > because Tom and Andres discussed just disallowing unique indexes on > system columns altogether. So, the attached patch does just that, and > also fixes up the replica identity bugs too, as it's still possibl

[HACKERS] Disallow unique index on system columns

2016-04-14 Thread David Rowley
Hi, Over in [1], while I was aiming to fix some incorrect formatting in an error message, Tom noticed that the code in that area was much more broken than I had thought. Basically, if you do; postgres=# create table t (a int) with oids; CREATE TABLE postgres=# create unique index t_oid_idx on t(

Re: [HACKERS] Draft release notes for next week's releases

2016-04-14 Thread Peter Geoghegan
On Thu, Apr 14, 2016 at 4:42 PM, Peter Geoghegan wrote: > * We should look into using the ucol_nextSortKeyPart() API: > > http://userguide.icu-project.org/collation/architecture#TOC-Partial-sort-keys Another more rich API we could immediately put to good use is the ICU strcoll() variant that does

Re: [HACKERS] Draft release notes for next week's releases

2016-04-14 Thread Peter Geoghegan
On Tue, Mar 29, 2016 at 5:18 AM, Teodor Sigaev wrote: > It's based on https://people.freebsd.org/~girgen/postgresql-icu/readme.html > work, and it was migrated to 9.5 with abbrevation keys support. > Patch in current state is not ready to commit, of course. Cool. Some quick observations on this:

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-14 Thread Michael Paquier
On Fri, Apr 15, 2016 at 12:13 AM, Tom Lane wrote: > Michael Paquier writes: >> On Thu, Apr 14, 2016 at 9:38 AM, Michael Paquier >> wrote: >>> IO::Socket::INET is another option, but I am not seeing it in perl < >>> 5.12, and that's not part of ActivePerl, which makes life harder on >>> Windows.

[HACKERS] Obsolete comment within fmgr.c

2016-04-14 Thread Peter Geoghegan
Attached patch removes obsolete comment from fmgr.c. The code the comment refers to was already removed by commit 5ea86e6e65dd2da3e9a3464484985d48328e7fe3. -- Peter Geoghegan remove_obsolete_comment.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Broken API specification for walrcv_receive

2016-04-14 Thread Thomas Munro
On Fri, Apr 15, 2016 at 4:28 AM, Tom Lane wrote: > I noticed thanks to a buildfarm warning that replication/walreceiver.h > now declares the walrcv_receive hook as > > typedef int (*walrcv_receive_type) (char **buffer, int *wait_fd); > > This is flat out wrong, isn't it? wait_fd needs to be of ty

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-14 Thread Alexander Korotkov
On Thu, Apr 14, 2016 at 5:35 AM, Andres Freund wrote: > On 2016-04-14 07:59:07 +0530, Amit Kapila wrote: > > What you want to see by prewarming? > > Prewarming appears to greatly reduce the per-run variance on that > machine, making it a lot easier to get meaningful results. Thus it'd > make it

[HACKERS] Re: Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

2016-04-14 Thread Stephen Frost
Jeff (Reviving an old thread for 2014...) * Jeff Davis (pg...@j-davis.com) wrote: > On Thu, 2014-07-10 at 23:43 -0700, Jeff Davis wrote: > > On Mon, 2014-07-07 at 01:21 -0700, Jeff Davis wrote: > > > On Sun, 2014-07-06 at 21:11 -0700, Jeff Davis wrote: > > > > On Wed, 2014-04-16 at 12:50 +0100, N

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Tom Lane wrote: > I wrote: > > My feeling is that what we should do is undo the change to use OT_SQLID, > > and in indexOfColumn() perform a downcasing/dequoting conversion that > > duplicates what OT_SQLID does in psqlscanslash.l. > > Here's an updated patch that does it that way, and al

[HACKERS] Memory leak when querying GIN indexes

2016-04-14 Thread Julien Rouhaud
Hello, My colleague Adrien reported me a memory leak in GIN indexes while doing some benchmark on several am. Here is a test case to reproduce the issue: CREATE TABLE test AS ( SELECT t FROM generate_series(now(), now() + interval '10 day', '1 second') AS d(t) CROSS JOIN generate_ser

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread David Steele
On 4/14/16 3:01 PM, Andres Freund wrote: > On 2016-04-14 14:55:37 -0400, Robert Haas wrote: >> On Thu, Apr 14, 2016 at 11:12 AM, Andres Freund wrote: >>> On 2016-04-14 13:43:34 +0530, Ashutosh Sharma wrote: I tried performing pg_basebackup after creating a symbolic link for pg_replslot,

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread David Steele
On 4/14/16 2:55 PM, Robert Haas wrote: > On Thu, Apr 14, 2016 at 11:12 AM, Andres Freund wrote: >> On 2016-04-14 13:43:34 +0530, Ashutosh Sharma wrote: >>> I tried performing pg_basebackup after creating a symbolic link for >>> pg_replslot, pg_stat_tmp, pg_log and pg_clog in the source directory >

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Andres Freund
On 2016-04-14 14:55:37 -0400, Robert Haas wrote: > On Thu, Apr 14, 2016 at 11:12 AM, Andres Freund wrote: > > On 2016-04-14 13:43:34 +0530, Ashutosh Sharma wrote: > >> I tried performing pg_basebackup after creating a symbolic link for > >> pg_replslot, pg_stat_tmp, pg_log and pg_clog in the sourc

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Robert Haas
On Thu, Apr 14, 2016 at 11:12 AM, Andres Freund wrote: > On 2016-04-14 13:43:34 +0530, Ashutosh Sharma wrote: >> I tried performing pg_basebackup after creating a symbolic link for >> pg_replslot, pg_stat_tmp, pg_log and pg_clog in the source directory > > That's not supported, and I strongly susp

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Magnus Hagander
On Thu, Apr 14, 2016 at 8:44 PM, Alvaro Herrera wrote: > Magnus Hagander wrote: > > On Thu, Apr 14, 2016 at 8:20 PM, Ashutosh Sharma > > wrote: > > > > > I was just curious to know how would "*pg_basebackup*" behave if we do > > > create a symbolic link for directories other than pg_xlog/pg_tbls

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Robert Haas
On Thu, Apr 14, 2016 at 11:26 AM, Simon Riggs wrote: > 1) "more deeply into core" > I'm open to doing that for some parts of the code, if there is benefit. At > present, an extension has exactly the same attributes as an in-core > solution, so I don't currently see any benefit in doing so. Could y

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Alvaro Herrera
Magnus Hagander wrote: > On Thu, Apr 14, 2016 at 8:20 PM, Ashutosh Sharma > wrote: > > > I was just curious to know how would "*pg_basebackup*" behave if we do > > create a symbolic link for directories other than pg_xlog/pg_tblspc. > > However it is clearly mentioned in the documentation of pg_b

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Magnus Hagander
On Thu, Apr 14, 2016 at 8:20 PM, Ashutosh Sharma wrote: > Hi, > > I was just curious to know how would "*pg_basebackup*" behave if we do > create a symbolic link for directories other than pg_xlog/pg_tblspc. > However it is clearly mentioned in the documentation of pg_basebackup that > if a Symb

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Ashutosh Sharma
Hi, I was just curious to know how would "*pg_basebackup*" behave if we do create a symbolic link for directories other than pg_xlog/pg_tblspc. However it is clearly mentioned in the documentation of pg_basebackup that if a Symbolic link for the directories other than pg_tblspc and pg_xlog is cre

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Robert Haas
On Wed, Apr 13, 2016 at 8:54 PM, Craig Ringer wrote: > On 14 April 2016 at 00:48, Robert Haas wrote: >> I think this would be a good topic to discuss at PGCon. > > I'm not going to be able to be at PGCon, and I don't think Petr is either. > That's unfortunate in terms of planning for this. That'

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-04-14 Thread Robert Haas
On Wed, Apr 13, 2016 at 11:22 PM, Amit Kapila wrote: > Yes, I also think that this particular issue can be closed. However I felt > that the observation related to performance variation is still present as I > never need to perform prewarm or anything else to get consistent results > during my wo

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-04-14 Thread Robert Haas
On Thu, Apr 14, 2016 at 2:00 PM, Pavel Stehule wrote: >>> You don't need a separate shared memory segment. You might want to >>> have a look at what we did for pldebugger: >>> >>> http://git.postgresql.org/gitweb/?p=pldebugger.git;a=commitdiff;h=14c6caf08bb14a7404a8241e47a80ef5f97e451e >>> >>> I

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-04-14 Thread Pavel Stehule
2016-04-12 15:15 GMT+02:00 Pavel Stehule : > > > 2016-04-12 14:51 GMT+02:00 Robert Haas : > >> On Tue, Apr 12, 2016 at 4:18 AM, Pavel Stehule >> wrote: >> > If there will be simple way, how to fix it, then I'll fix my >> extensions. But >> > new API is working only when the extension has own shar

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread David Steele
On 4/14/16 1:33 PM, Tom Lane wrote: > David Steele writes: >> On 4/14/16 7:16 AM, Andreas Karlsson wrote: >>> I am personally not a fan of the pg_get_Xdef() functions due to their >>> heavy reliance on the syscache which feels rather unsafe in combination >>> with concurrent DDL. > >> As far as I

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Tom Lane
David Steele writes: > On 4/14/16 7:16 AM, Andreas Karlsson wrote: >> I am personally not a fan of the pg_get_Xdef() functions due to their >> heavy reliance on the syscache which feels rather unsafe in combination >> with concurrent DDL. > As far as I know pg_dump share locks everything before i

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread David Steele
On 4/14/16 7:16 AM, Andreas Karlsson wrote: > On 04/14/2016 12:22 PM, Craig Ringer wrote: >> I'd find a pg_get_tabledef(...) built-in function more interesting for >> this particular purpose than pg_dump as a library would be. We already >> have pg_get_viewdef(...), pg_get_functiondef(...) etc. >

Re: [HACKERS] has_language_privilege returns incorrect answer for non-superuser

2016-04-14 Thread Tom Lane
Alvaro Herrera writes: > Joe Conway wrote: >> I noticed today that has_language_privilege() returns incorrect answer >> for non-superuser, e.g.: >> >> 8<--- >> select has_language_privilege('nobody', >> 'plperlu', >> 'usage'); >> has_language_privil

Re: [HACKERS] has_language_privilege returns incorrect answer for non-superuser

2016-04-14 Thread Alvaro Herrera
Joe Conway wrote: > I noticed today that has_language_privilege() returns incorrect answer > for non-superuser, e.g.: > > 8<--- > select has_language_privilege('nobody', > 'plperlu', > 'usag

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Joshua D. Drake
On 04/14/2016 08:57 AM, Simon Riggs wrote: On 14 April 2016 at 16:52, Andres Freund mailto:and...@anarazel.de>> wrote: > If we aren't going to talk about changes in advance, then multiple > commit rights is a recipe for conflict, not a better way at all. Obviously we should discuss

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Joshua D. Drake
On 04/14/2016 08:52 AM, Andres Freund wrote: I think there's quite a bit of work needed all around. The output plugin's protocol needs some work, the whole catalog infrastructure and management in pglogical itself needs a lot of work. +1 and let's not forget a need for comprehensive documentat

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Andres Freund
On 2016-04-14 13:16:20 +0200, Andreas Karlsson wrote: > I am personally not a fan of the pg_get_Xdef() functions due to their heavy > reliance on the syscache which feels rather unsafe in combination with > concurrent DDL. I'm not sure I find that convincing: The state portrayed by the syscache is

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Joshua D. Drake
On 04/14/2016 08:26 AM, Simon Riggs wrote: On 13 April 2016 at 17:48, Robert Haas mailto:robertmh...@gmail.com>> wrote: On Wed, Apr 13, 2016 at 4:38 AM, Simon Riggs mailto:si...@2ndquadrant.com>> wrote: > Anyway, who agrees with the overall design of pglogical and who does not? I ha

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Jim Nasby
On 4/14/16 6:16 AM, Andreas Karlsson wrote: On 04/14/2016 12:22 PM, Craig Ringer wrote: I'd find a pg_get_tabledef(...) built-in function more interesting for this particular purpose than pg_dump as a library would be. We already have pg_get_viewdef(...), pg_get_functiondef(...) etc. I am pers

[HACKERS] Broken API specification for walrcv_receive

2016-04-14 Thread Tom Lane
I noticed thanks to a buildfarm warning that replication/walreceiver.h now declares the walrcv_receive hook as typedef int (*walrcv_receive_type) (char **buffer, int *wait_fd); This is flat out wrong, isn't it? wait_fd needs to be of type pgsocket, which is a different width from "int" on Window

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

2016-04-14 Thread Andres Freund
On 2016-04-14 11:50:58 -0400, Robert Haas wrote: > On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: > > We've recently discussed a very similar issue around > > http://www.postgresql.org/message-id/20160227002958.peftvmcx4dxwe...@alap3.anarazel.de > > > > Unfortunately Simon over in that thre

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Tom Lane
I wrote: > My feeling is that what we should do is undo the change to use OT_SQLID, > and in indexOfColumn() perform a downcasing/dequoting conversion that > duplicates what OT_SQLID does in psqlscanslash.l. Here's an updated patch that does it that way, and also adopts Christoph's documentation s

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Andres Freund
Hi, On 2016-04-14 08:52:00 -0700, Andres Freund wrote: > > Petr is more than capable for managing bug fixes and I would like to see > > him have his chance to demonstrate his skill and get a shot at being a > > committer, just as you did. Having one person manage a feature seems like > > the best

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Simon Riggs
On 14 April 2016 at 16:52, Andres Freund wrote: > > If we aren't going to talk about changes in advance, then multiple > > commit rights is a recipe for conflict, not a better way at all. > > Obviously we should discuss design bits, I don't see how that's a > conflict. > If everything is going

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Andres Freund
Hi, On 2016-04-14 16:42:10 +0100, Simon Riggs wrote: > What intense work is required? pglogical already works, it just requires > review. I've done a round of reviews in January, still not all points are addressed of yet (due to time limits afaik). At that pace it's unrealistic to get anything i

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

2016-04-14 Thread Robert Haas
On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund wrote: > We've recently discussed a very similar issue around > http://www.postgresql.org/message-id/20160227002958.peftvmcx4dxwe...@alap3.anarazel.de > > Unfortunately Simon over in that thread disagreed there about fixing > this by always emitting a

Re: [HACKERS] SET ROLE and reserved roles

2016-04-14 Thread Robert Haas
On Wed, Apr 13, 2016 at 6:53 PM, Stephen Frost wrote: > Requiring that SET ROLE be allowed will mean that many more paths must > be checked and adjusted, such as in all of the CreateObject statements > and potentionally many other paths that I'm not thinking of here, not > the least of which are a

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Simon Riggs
On 14 April 2016 at 02:05, Andres Freund wrote: > Hi, > > On 2016-04-13 09:38:39 +0100, Simon Riggs wrote: > > If we want this in 9.7 > > I desperately want logical replication for 9.7. And I'm planning to put > in a good chunk of work to make that happen in some way. Good, thanks. I'm happy to

Re: [HACKERS] Problems with huge_pages and IBM Power8

2016-04-14 Thread Andres Freund
On 2016-04-14 12:05:39 +0200, Bernd Helmle wrote: > > > --On 12. April 2016 13:20:10 -0700 Andres Freund wrote: > > >> When working with huge_pages, we initially got this error. > >> > >> munmap(0x3efbe400) failed: Invalid argument > > > > *munmap*, not mmap failed? that's odd; because th

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Simon Riggs
On 13 April 2016 at 17:48, Robert Haas wrote: > On Wed, Apr 13, 2016 at 4:38 AM, Simon Riggs > wrote: > > Anyway, who agrees with the overall design of pglogical and who does not? > > I haven't spent very much time on it yet. I tend to prefer the idea > of integrating it more deeply into core a

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-14 Thread Tom Lane
Michael Paquier writes: > On Thu, Apr 14, 2016 at 9:38 AM, Michael Paquier > wrote: >> IO::Socket::INET is another option, but I am not seeing it in perl < >> 5.12, and that's not part of ActivePerl, which makes life harder on >> Windows. Socket is available on both. Does that address your concer

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-04-14 Thread Andres Freund
Hi, On 2016-04-14 13:43:34 +0530, Ashutosh Sharma wrote: > I tried performing pg_basebackup after creating a symbolic link for > pg_replslot, pg_stat_tmp, pg_log and pg_clog in the source directory That's not supported, and I strongly suspect that you're goint to hit more than just this issue. T

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-14 Thread Alexander Korotkov
On Thu, Apr 14, 2016 at 12:23 AM, Andres Freund wrote: > On 2016-04-13 16:05:25 -0500, Kevin Grittner wrote: > > OK, thanks. I can't think of anything else to ask for at this > > point. If you feel that you have enough to press for some > > particular course of action, go for it. > > I think we

Re: [HACKERS] Pglogical questions and problems

2016-04-14 Thread Joshua D. Drake
On 04/13/2016 06:05 PM, Andres Freund wrote: Hi, On 2016-04-13 09:38:39 +0100, Simon Riggs wrote: If we want this in 9.7 I desperately want logical replication for 9.7. And I'm planning to put in a good chunk of work to make that happen in some way. I don't see a need for it to be in -core

Re: [HACKERS] Wrong definition of pgwin32_bind.

2016-04-14 Thread Tom Lane
Michael Paquier writes: > On Thu, Apr 14, 2016 at 5:15 PM, Kyotaro HORIGUCHI > wrote: >> The third parameter should be int, not int*. It might have been >> confused with pgwin32_accept. > Nice catch. David Rowley reported this a bit earlier. regards, tom lane -- Sent

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Tom Lane
"Daniel Verite" writes: > Christoph Berg wrote: >> If there's no way out, what about changing it the other way, i.e. >> breaking the case where the column is named by a number? That seems >> much less of a problem in practice. > I don't think it would be acceptable. I had figured it would be ;-)

Re: [HACKERS] SEGFAULT in CREATE EXTENSION related pg_init_privs

2016-04-14 Thread Pavel Stehule
Hi 2016-04-14 14:26 GMT+02:00 Stephen Frost : > Pavel, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > I am trying to prepare orafce for PostgreSQL 9.6. > > > > I can successfully compile this extension, but the statement CREATE > > EXTENSION fails on segfault > > Just the latest off of

Re: [HACKERS] Declarative partitioning

2016-04-14 Thread Ashutosh Bapat
Hi Amit, Here are some random comments. You said that you were about to post a new patch, so you might have already taken care of those comments. But anyway here they are. 1. Following patches do not apply cleanly for me. 0001 0003 - conflicts at #include for partition.h in rel.h. 0004 -

Re: [HACKERS] SEGFAULT in CREATE EXTENSION related pg_init_privs

2016-04-14 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > I am trying to prepare orafce for PostgreSQL 9.6. > > I can successfully compile this extension, but the statement CREATE > EXTENSION fails on segfault Just the latest off of https://github.com/orafce/orafce ? I took a quick look through

Re: [HACKERS] Wrong definition of pgwin32_bind.

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 5:15 PM, Kyotaro HORIGUCHI wrote: > Hello. > > VC complains that pgwin32_bind() had a parameter differs in > levels of indirection from definition. pgwin32_bind() in > src/backend/port/win32/socket.c is defined as the following. > >> int pgwin32_bind(SOCKET s, struct sockad

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Christoph Berg wrote: > If there's no way out, what about changing it the other way, i.e. > breaking the case where the column is named by a number? That seems > much less of a problem in practice. I don't think it would be acceptable. But there's still the option of keeping the dedicated

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 5:48 PM, Kyotaro HORIGUCHI wrote: > At Thu, 14 Apr 2016 17:25:39 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20160414.172539.34325458.horiguchi.kyot...@lab.ntt.co.jp> >> Hello, >> >> At Thu, 14 Apr 2016 13:24:34 +0900, Michael Paquier >> wrote in >>

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 5:25 PM, Kyotaro HORIGUCHI wrote: > diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl > index 3d14544..08e2acc 100644 > --- a/src/tools/msvc/vcregress.pl > +++ b/src/tools/msvc/vcregress.pl > @@ -548,6 +548,6 @@ sub usage > { > print STDERR >

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-14 Thread Ashutosh Bapat
Thanks Robert for the patch. > OK, here's a patch. What I did is: > > 1. For a regular FDW scan, zero the xmin, xmax, and cid of the tuple > before returning it from postgres_fdw, so that we don't expose the > datum-tuple fields. I can't see any reason this isn't safe, but I > might be missin

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 5:17 PM, Masahiko Sawada wrote: > On Thu, Apr 14, 2016 at 1:22 PM, Michael Paquier > wrote: >> Yes. I'd prefer avoid a hardcoded sleep and have something that relies >> on lookups of pg_stat_replication though, because there is no way to >> be sure that a sleep will ever b

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-14 Thread Michael Paquier
On Thu, Apr 14, 2016 at 9:38 AM, Michael Paquier wrote: > IO::Socket::INET is another option, but I am not seeing it in perl < > 5.12, and that's not part of ActivePerl, which makes life harder on > Windows. Socket is available on both. Does that address your concerns? And this gives the patch at

Re: [HACKERS] standalone backend PANICs during recovery

2016-04-14 Thread Bernd Helmle
--On 5. April 2016 21:50:17 -0400 Robert Haas wrote: > If nobody's going to commit this right away, this should be added to > the next CommitFest so we don't forget it. Done. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Andreas Karlsson
On 04/14/2016 12:22 PM, Craig Ringer wrote: I'd find a pg_get_tabledef(...) built-in function more interesting for this particular purpose than pg_dump as a library would be. We already have pg_get_viewdef(...), pg_get_functiondef(...) etc. I am personally not a fan of the pg_get_Xdef() functio

Re: [HACKERS] Problems with huge_pages and IBM Power8

2016-04-14 Thread Bert
hmpf; are you sure? I just checked on our own rhel7 system, and RemoveIPC is set to 'no' by default.. On Tue, Apr 12, 2016 at 10:26 PM, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-12 21:58:14 +0200, reiner peterke wrote: > >> Looking for some insight into this issue. the error from

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Craig Ringer
On 14 April 2016 at 18:16, Jakob Egger wrote: > Would anybody else be interested in a pg_dump library? > Yes. It comes up a lot, but so far nobody's had the time and energy to actually do the required refactoring. I'm the developer of a PostgreSQL GUI client, and I am looking for ways to > inte

[HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Jakob Egger
Would anybody else be interested in a pg_dump library? I've found a thread from 2013 related to the idea, but the discussion came to nothing. Thread started here: http://www.postgresql.org/message-id/71e01949.2e16b.13df4707405.coremail.shuai900...@126.com My Motivation: I'm the developer of a

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Christoph Berg wrote: > > I don't quite see how to work around that, short of simply > > removing the possibility of addressing columns by their > > numbers. [...] > That would be bad news, given that \crosstabview is meant for > interactive use where these number shortcuts are much more

Re: [HACKERS] Problems with huge_pages and IBM Power8

2016-04-14 Thread Bernd Helmle
--On 12. April 2016 13:20:10 -0700 Andres Freund wrote: >> When working with huge_pages, we initially got this error. >> >> munmap(0x3efbe400) failed: Invalid argument > > *munmap*, not mmap failed? that's odd; because there the hugepagesize > shouldn't have much of an influence. If somet

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Christoph Berg
Re: Daniel Verite 2016-04-14 > I don't quite see how to work around that, short of simply > removing the possibility of addressing columns by their > numbers. Which maybe is a bit sad for the end user, I'm not > sure, but ISTM that's a logical consequence of abandoning > the dedicated parser for c

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Tom Lane wrote: > > That would be OK with me; it's certainly less of a hack than what's > > there now. (I went back and forth about how much effort to put into > > dealing with the colon syntax; I think the version I have in my patch > > would be all right, but it's not perfect.) > > Her

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Christoph Berg
Re: Tom Lane 2016-04-14 <15673.1460592...@sss.pgh.pa.us> > Here's a patch along those lines. Any objections? > \crosstabview [ > colV > ! [ colH > ! [ colD > ! [ scolH > ! ] ] ] ] Maybe use "sortcolH" to make it immediately

Re: [HACKERS] SEGFAULT in CREATE EXTENSION related pg_init_privs

2016-04-14 Thread Pavel Stehule
2016-04-14 10:49 GMT+02:00 Peter Geoghegan : > On Thu, Apr 14, 2016 at 1:27 AM, Pavel Stehule > wrote: > > (gdb) bt > > #0 heap_deform_tuple (tuple=tuple@entry=0x1d87e90, > > tupleDesc=tupleDesc@entry=0x7f1dab9525b0, values=values@entry=0x1d87a28, > > isnull=isnull@entry=0x1d882d8 "\177\177

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Apr 2016 17:25:39 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20160414.172539.34325458.horiguchi.kyot...@lab.ntt.co.jp> > Hello, > > At Thu, 14 Apr 2016 13:24:34 +0900, Michael Paquier > wrote in > > > On Thu, Apr 14, 2016 at 11:45 AM, Amit Kapila > > wrote: > > >

Re: [HACKERS] SEGFAULT in CREATE EXTENSION related pg_init_privs

2016-04-14 Thread Peter Geoghegan
On Thu, Apr 14, 2016 at 1:27 AM, Pavel Stehule wrote: > (gdb) bt > #0 heap_deform_tuple (tuple=tuple@entry=0x1d87e90, > tupleDesc=tupleDesc@entry=0x7f1dab9525b0, values=values@entry=0x1d87a28, > isnull=isnull@entry=0x1d882d8 "\177\177\177\177\177~\177\177") at > heaptuple.c:881 The memory a

  1   2   >