Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-11-13 Thread Mithun Cy
On Mon, Nov 14, 2016 at 11:31 AM, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > PGconn->target_server_type is not freed in freePGconn(). Thanks, will fix in new patch. >Could you add PGTARGETSERVERTYPE environment variable? Like other variables, it will ease testing, since

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-11-13 Thread Mithun Cy
On Fri, Nov 11, 2016 at 7:33 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > We tend to use the term "primary" instead of "master". Thanks, I will use primary instead of master in my next patch. >Will this work with logical replication? I have not tested with logical

[HACKERS] Re: [PATCH] PostgresNode.pm enhancements, pg_lsn helper, and some more recovery tests

2016-11-13 Thread Craig Ringer
On 14 November 2016 at 15:01, Craig Ringer wrote: > These three are related enough, and all only touch the TAP framework, > so I've bundled them in a series. CF entry https://commitfest.postgresql.org/12/883/ -- Craig Ringer

[HACKERS] [PATCH] PostgresNode.pm enhancements, pg_lsn helper, and some more recovery tests

2016-11-13 Thread Craig Ringer
Every test I write with the TAP framework for recovery seems to need to wait for one node to catch up to another or examine replication slot state. So: attached is a patch to add helper methods for these tasks. Also add a new pg_lsn.pm helper class to deal with PostgreSQL's awkward representation

Re: [HACKERS] 9.6 TAP tests and extensions

2016-11-13 Thread Craig Ringer
On 27 October 2016 at 00:42, Robert Haas wrote: > On Wed, Oct 26, 2016 at 7:17 AM, Andres Freund wrote: >> On 2016-09-23 16:04:32 -0400, Tom Lane wrote: >>> Looking back over the thread, I see that you also proposed installing >>> isolationtester and

Re: [HACKERS] [PATCH] Allow TAP tests to be run individually

2016-11-13 Thread Craig Ringer
On 11 November 2016 at 18:13, Michael Paquier wrote: > On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer wrote: >> Please backpatch to at least 9.6 since it's trivial and we seem to be >> doing that for TAP. 9.5 and 9.4 would be nice too :) > > Yes

Re: [HACKERS] Declarative partitioning - another take

2016-11-13 Thread Amit Langote
On 2016/11/04 0:49, Robert Haas wrote: > On Thu, Nov 3, 2016 at 7:46 AM, wrote: >> El 2016-10-28 07:53, Amit Langote escribió: >>> @@ -6267,6 +6416,12 @@ ATAddForeignKeyConstraint(AlteredTableInfo *tab, >>> Relation rel, >>> * Validity checks (permission checks

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-13 Thread Michael Paquier
On Sat, Nov 12, 2016 at 9:01 PM, Andres Freund wrote: > On 2016-11-11 16:42:43 +0900, Michael Paquier wrote: > >> + * This takes also >> + * advantage to avoid 8-byte torn reads on some platforms by using the >> + * fact that each insert lock is located on the same cache line.

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-11-13 Thread Tsunakawa, Takayuki
Hi, Mithun Before going deeper into the patch, let me give you some findings. (1) PGconn->target_server_type is not freed in freePGconn(). (2) Could you add PGTARGETSERVERTYPE environment variable? Like other variables, it will ease testing, since users can change the behavior without

Re: [HACKERS] Declarative partitioning - another take

2016-11-13 Thread Amit Langote
I forgot to quote your comments in the email I sent on Friday [1], with new patches that do take care of the following comments. On 2016/11/11 4:04, Robert Haas wrote: > On Thu, Nov 10, 2016 at 7:40 AM, Amit Langote >> >> OK, "partition key" and "partitioning method" it is then. Source code >>

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-13 Thread Michael Paquier
On Mon, Nov 14, 2016 at 12:49 PM, Kyotaro HORIGUCHI wrote: > At Sat, 12 Nov 2016 10:28:56 +0530, Amit Kapila > wrote in >> I think it is good to check the performance

Re: [HACKERS] Minor improvement to delete.sgml

2016-11-13 Thread Etsuro Fujita
On 2016/10/19 2:51, Robert Haas wrote: On Fri, Oct 14, 2016 at 12:05 AM, Etsuro Fujita wrote: I think it's better to mention that an alias is needed for the target table specified in the USING clause of a DELETE statement, to set up a self-join, as the

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-13 Thread Kyotaro HORIGUCHI
Hello, thank you for the comment. At Sat, 12 Nov 2016 10:28:56 +0530, Amit Kapila wrote in > On Tue, Nov 8, 2016 at 5:18 PM, Kyotaro HORIGUCHI > wrote: > > Hello, > >

Re: [HACKERS] sequences and pg_upgrade

2016-11-13 Thread Peter Eisentraut
On 11/2/16 2:34 AM, Michael Paquier wrote: > I had a look at those fresh patches, and 0001 looks like a good thing. > This makes the separation between sequences and table data dump > cleaner. I ran some tests with pg_upgrade and 0002, and things are > clear. And +1 for the way done in the patch,

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-11-13 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Great, committed. There's still potentially more work to be done here, > because my patch omits some features that were present in Victor's original > submission, like setting the

Re: [HACKERS] Physical append-only tables

2016-11-13 Thread Alvaro Herrera
Magnus Hagander wrote: > But then consider the same table. Except rows are typically updated once or > twice when they are new, and *then* go read only. And we also have a > process that at some point deletes *some* old rows (but not all - in fact, > only a small portion). > > In this case, the

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-11-13 Thread Tomas Vondra
On 11/12/2016 08:12 PM, Andres Freund wrote: Hi, Subject: [PATCH 1/2] slab allocator diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c index 6ad7e7d..520f295 100644 --- a/src/backend/replication/logical/reorderbuffer.c +++

Re: [HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

2016-11-13 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Ashutosh Bapat > I have changed some comments around this block. See attached patch. > Let me know if that looks good. Thanks, it looks good. Regards Takayuki Tsunakawa -- Sent via

Re: [HACKERS] ExplainOneQuery_hook ignored for EXPLAIN EXECUTE

2016-11-13 Thread Tom Lane
Andres Freund writes: > On 2016-11-13 17:20:05 -0500, Tom Lane wrote: >> Why do you care? It's a pretty specialized hook. > Citus currently uses it to output additional information for distributed > queries. I suspect we'll instead, for now, have to intercept EXPLAIN as > a

Re: [HACKERS] ExplainOneQuery_hook ignored for EXPLAIN EXECUTE

2016-11-13 Thread Andres Freund
On 2016-11-13 17:20:05 -0500, Tom Lane wrote: > Andres Freund writes: > > I don't quite know what the hook in $subject was originally designed > > for, but right now it has the problem that it's not invoked for EXPLAIN > > EXECUTE. That's because ExplainExecuteQuery directly

Re: [HACKERS] ExplainOneQuery_hook ignored for EXPLAIN EXECUTE

2016-11-13 Thread Tom Lane
Andres Freund writes: > I don't quite know what the hook in $subject was originally designed > for, but right now it has the problem that it's not invoked for EXPLAIN > EXECUTE. That's because ExplainExecuteQuery directly calls > ExplainOnePlan instead of

Re: [HACKERS] Logical Replication WIP

2016-11-13 Thread Steve Singer
On 10/31/2016 06:38 AM, Petr Jelinek wrote: On 31/10/16 00:52, Steve Singer wrote: There are some fundamental issues with initial sync that need to be discussed on list but this one is not known. I'll try to convert this to test case (seems like useful one) and fix it, thanks for the report.

Re: [HACKERS] Improving RLS planning

2016-11-13 Thread Tom Lane
Stephen Frost writes: > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> +1 for that terminology and no renaming of fields. > Agreed. Here's an updated version of the RLS planning patch that gets rid of the incorrect interaction with Query.hasRowSecurity and adjusts

Re: [HACKERS] Why PostgreSQL doesn't implement a semi sync replication?

2016-11-13 Thread 余森彬
在 2016/11/12 上午11:27, Craig Ringer 写道: On 12 November 2016 at 02:12, Petr Jelinek wrote: On 11/11/16 16:03, Francisco Olarte wrote: On Fri, Nov 11, 2016 at 4:40 AM, 余森彬 wrote: As we know, the synchronous commit process is blocked while

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-11-13 Thread Emre Hasegeli
> We can remove the fuzz factor altogether but I think we also > should provide a means usable to do similar things. At least "is > a point on a line" might be useless for most cases without any > fuzzing feature. (Nevertheless, it is a problem only when it is > being used to do that:) If we don't

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-13 Thread Rahila Syed
>ECHO_HIDDEN differs from the generic boolean case because it also >accepts "noexec" and pset.echo_hidden is an enum, not a boolean. When >considering refactoring echo_hidden_hook() to call >generic_boolean_hook() instead of ParseVariableBool() after >having established that the value is not

[HACKERS] ExplainOneQuery_hook ignored for EXPLAIN EXECUTE

2016-11-13 Thread Andres Freund
Hi, I don't quite know what the hook in $subject was originally designed for, but right now it has the problem that it's not invoked for EXPLAIN EXECUTE. That's because ExplainExecuteQuery directly calls ExplainOnePlan instead of ExplainOneQuery_hook. Unfortunately that's not entirely trivial

Re: [HACKERS] Physical append-only tables

2016-11-13 Thread Thomas Munro
On Mon, Nov 14, 2016 at 4:45 AM, Magnus Hagander wrote: > For a scenario like this, would it make sense to have an option that could > be set on an individual table making it physical append only? Basically > VACUUM would run as normal and clean up the old space when rows are

Re: [HACKERS] Tackling JsonPath support

2016-11-13 Thread Pavel Stehule
Hi 2016-11-13 15:14 GMT+01:00 Christian Convey : > Hi Pavel, > > Can I check a few assumptions about what you're suggesting for this task? > > * Our ultimate goal is to give Postgres an implementation of the functions > "JSON_EXISTS", "JSON_VALUE", and "JSON_QUERY"

Re: [HACKERS] proposal: psql \setfileref

2016-11-13 Thread Pavel Stehule
Hi I am sending a initial implementation of psql content commands. This design is reaction to Tom's objections against psql file ref variables. This design is more cleaner, more explicit and more practical - import can be in one step. Now supported commands are: r - read file without any

Re: [HACKERS] Contains and is contained by operators of inet datatypes

2016-11-13 Thread Andreas Karlsson
On 11/13/2016 01:21 PM, Emre Hasegeli wrote: Thank you for the review. New version is attached. Nice, I am fine with this version of the patch. Setting it to ready for committer! Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Tackling JsonPath support

2016-11-13 Thread Pavel Stehule
2016-11-13 18:13 GMT+01:00 Tom Lane : > Christian Convey writes: > > * Our ultimate goal is to give Postgres an implementation of the > functions > > "JSON_EXISTS", "JSON_VALUE", and "JSON_QUERY" which fully comply with the > > SQL standards. > > *

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Tom Lane
Andres Freund writes: > On 2016-11-13 11:23:09 -0500, Tom Lane wrote: >> We can't use CREATE FUNCTION as the representation in the .bki file, >> because of the circularities involved (you can't fill pg_proc before >> pg_type nor vice versa). But I think Peter was suggesting

Re: [HACKERS] Tackling JsonPath support

2016-11-13 Thread Tom Lane
Christian Convey writes: > * Our ultimate goal is to give Postgres an implementation of the functions > "JSON_EXISTS", "JSON_VALUE", and "JSON_QUERY" which fully comply with the > SQL standards. > * The best representation of those standards is found here: [1]. > [1] >

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Andres Freund
On 2016-11-13 11:23:09 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2016-11-13 00:20:22 -0500, Peter Eisentraut wrote: > >> Then we're not very far away from just using CREATE FUNCTION SQL commands. > > > Well, those do a lot of syscache lookups, which in turn do

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Andres Freund
On 2016-11-13 11:11:37 -0500, Tom Lane wrote: > 1. Are we going to try to keep these things in the .h files, or split > them out? I'd like to get them out, as that eliminates both the need > to keep the things looking like macro calls, and the need for the data > within the macro call to be at

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Tom Lane
Andres Freund writes: > On 2016-11-13 00:20:22 -0500, Peter Eisentraut wrote: >> Then we're not very far away from just using CREATE FUNCTION SQL commands. > Well, those do a lot of syscache lookups, which in turn do lookups for > functions... We can't use CREATE FUNCTION as

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Tom Lane
Andrew Dunstan writes: > I'm not convinced the line prefix part is necessary, though. What I'm > thinking of is something like this: > PROCDATA( oid=1242 name=boolin isstrict=t volatile=i parallel=s nargs=1 > rettype=bool argtypes="cstring" src=boolin ); We could go

[HACKERS] Physical append-only tables

2016-11-13 Thread Magnus Hagander
Scenario: Large table, *mostly* insert+select only, either a sequence or a timestamp column set to current time. This is an ideal usecase for a BRIN index. Awesome. But then consider the same table. Except rows are typically updated once or twice when they are new, and *then* go read only. And

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Andrew Dunstan
On 11/13/2016 04:54 AM, Andres Freund wrote: On 2016-11-12 12:30:45 -0500, Andrew Dunstan wrote: On 11/11/2016 11:10 AM, Tom Lane wrote: boolin: OID=1242 proname=boolin proargtypes="cstring" prorettype=bool boolin: prosrc=boolin provolatile=i proparallel=s I have written a little perl

Re: [HACKERS] Tackling JsonPath support

2016-11-13 Thread Christian Convey
Hi Pavel, Can I check a few assumptions about what you're suggesting for this task? * Our ultimate goal is to give Postgres an implementation of the functions "JSON_EXISTS", "JSON_VALUE", and "JSON_QUERY" which fully comply with the SQL standards. * The best representation of those standards is

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-11-13 Thread Emre Hasegeli
> The reason for that is that you forgot to edit an alternative > exptect file, which matches for en_US locale. Now I understand. Thank you for the explanation. > But the test doesn't for collation and the only difference > between the alternative expect files comes from the difference of >

Re: [HACKERS] Contains and is contained by operators of inet datatypes

2016-11-13 Thread Emre Hasegeli
> - I am not convinced that your changes to the descriptions of the operators > necessarily make things clearer. For example "is contained by and smaller > network (subnet)" only mentions subnets and not IP-addresses. I was trying to avoid confusion. <@ is the "contained by" operator which is

Re: [HACKERS] Logical Replication WIP

2016-11-13 Thread Andres Freund
On 2016-11-13 00:40:12 -0500, Peter Eisentraut wrote: > On 11/12/16 2:18 PM, Andres Freund wrote: > >>> I also wonder if we want an easier to > >>> > > extend form of pubinsert/update/delete (say to add pubddl, > >>> > > pubtruncate, > >>> > > pub ... without changing the schema). > >>> > > >

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

2016-11-13 Thread Andres Freund
On 2016-11-13 00:20:22 -0500, Peter Eisentraut wrote: > On 11/11/16 11:10 AM, Tom Lane wrote: > > boolin: OID=1242 proname=boolin proargtypes="cstring" prorettype=bool > > boolin: prosrc=boolin provolatile=i proparallel=s > > Then we're not very far away from just using CREATE FUNCTION SQL

Re: [HACKERS] pg_dump, pg_dumpall and data durability

2016-11-13 Thread Andres Freund
Hi, On 2016-11-08 18:18:01 -0500, Tom Lane wrote: > I think this might be better addressed by adding something to backup.sgml > pointing out that you'd better fsync or sync your backups before assuming > that they can't be lost. How does a normal user do that? I don't think there's a