Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-14 Thread Andres Freund
On April 14, 2017 9:42:41 PM PDT, Tom Lane wrote: >Per >https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae&dt=2017-04-15%2004%3A00%3A02 > >2017-04-15 04:31:21.657 GMT [16792] FATAL: could not reattach to >shared memory (key=6280001, addr=0x7f692fece000): Invalid argument > >Presu

[HACKERS] OK, so culicidae is *still* broken

2017-04-14 Thread Tom Lane
Per https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae&dt=2017-04-15%2004%3A00%3A02 2017-04-15 04:31:21.657 GMT [16792] FATAL: could not reattach to shared memory (key=6280001, addr=0x7f692fece000): Invalid argument Presumably, this is the same issue we've seen on Windows where t

Re: [HACKERS] PANIC in pg_commit_ts slru after crashes

2017-04-14 Thread Pavan Deolasee
On Sat, Apr 15, 2017 at 12:53 AM, Jeff Janes wrote: > > In the first statement executed after crash recovery, I sometimes get this > error: > > PANIC: XX000: could not access status of transaction 207580505 > DETAIL: Could not read from file "pg_commit_ts/1EF0" at offset 131072: > Success. > LO

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-14 Thread Tom Lane
Andreas Karlsson writes: > Looked some at this and what take time now for me seems to mainly be > these four things (out of a total runtime of 560 ms). > 1. setup_conversion:140 ms > 2. select_default_timezone: 90 ms > 3. bootstrap_template1: 80 ms > 4. setup_schema: 65

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-14 Thread Noah Misch
On Fri, Apr 14, 2017 at 09:38:32PM +0200, Magnus Hagander wrote: > On Fri, Apr 14, 2017 at 9:36 PM, Peter Eisentraut > wrote: > > On 4/14/17 14:45, Magnus Hagander wrote: > > > Attached is a patch that can be applied to pgweb which should fix all > > > of > > > this. > Indeed. Fix pushe

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-04-14 Thread Tom Lane
Petr Jelinek writes: > So this is what I came up with on plane. Generalized the loading > functionality into load_library_function which that can load either > known postgres functions or call load_external_function. I've had more than enough of seeing buildfarm failures from culicidae, so I whac

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-14 Thread Noah Misch
On Wed, Apr 05, 2017 at 09:51:02PM -0400, Noah Misch wrote: > On Thu, Apr 06, 2017 at 12:48:56AM +0900, Fujii Masao wrote: > > > On Mon, Dec 19, 2016 at 09:49:58PM +0900, Fujii Masao wrote: > > >> (2) > > >> There will be still many source comments and documentations that > > >> we need to update,

Re: [HACKERS] snapbuild woes

2017-04-14 Thread Petr Jelinek
Hi, here is updated patch (details inline). On 13/04/17 20:00, Petr Jelinek wrote: > Thanks for looking at this! > > On 13/04/17 02:29, Andres Freund wrote: >> Hi, >> On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: >> >>> From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001 >>

[HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

2017-04-14 Thread Erik Rijkers
Testing logical replication, with the following patches on top of yesterday's master: 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch + 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+ 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch + 0004-Fix-xl_r

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-14 Thread Gavin Flower
On 15/04/17 13:44, Andreas Karlsson wrote: On 04/14/2017 11:54 PM, Tom Lane wrote: I failed to resist the temptation to poke at this, and found that indeed nothing seems to break if we just use one transaction for the whole processing of postgres.bki. So I've pushed a patch that does that. We'

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-14 Thread Andreas Karlsson
On 04/14/2017 11:54 PM, Tom Lane wrote: I failed to resist the temptation to poke at this, and found that indeed nothing seems to break if we just use one transaction for the whole processing of postgres.bki. So I've pushed a patch that does that. We're definitely down to the point where worryi

Re: [HACKERS] Different table schema in logical replication crashes

2017-04-14 Thread Petr Jelinek
On 14/04/17 17:33, Peter Eisentraut wrote: > On 4/14/17 08:49, Petr Jelinek wrote: >>> Are we prepared to support different schemas in v10? Or should we >>> disallow it for v10 and add a TODO? >>> >> >> Ah nuts, yes it's supposed to be supported, we seem to not initialize >> cstate->range_table in

Re: [HACKERS] Allowing extended stats on foreign and partitioned tables

2017-04-14 Thread Peter Eisentraut
On 4/10/17 06:18, Ashutosh Bapat wrote: > This isn't exactly about this particular thread. But I noticed, that > after we introduced RELKIND_PARTITIONED_TABLE, we required to change a > number of conditions to include this relkind. We missed some places in > initial commits and fixed those later. I

Re: [HACKERS] Shouldn't duplicate addition to publication be a no-op?

2017-04-14 Thread Peter Eisentraut
On 4/13/17 06:23, Amit Langote wrote: > create table bar (a int); > create publication mypub for table bar; > alter publication mypub add table bar; > ERROR: relation "bar" is already member of publication "mypub" > > 2nd command should be a no-op, IMHO. We generally require a IF NOT EXISTS in t

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Rod Taylor
On Fri, Apr 14, 2017 at 9:09 AM, Stephen Frost wrote: > Rod, > > * Rod Taylor (rod.tay...@gmail.com) wrote: > > My actual use-case involves a range. Most users can see and manipulate > the > > record when CURRENT_TIMESTAMP is within active_period. Some users > > (staff/account admins) can see rec

Re: [HACKERS] minor typo in client-auth.sgml

2017-04-14 Thread Peter Eisentraut
On 4/14/17 18:26, Jaime Casanova wrote: > I was reading about SCRAM and the changes in the docs and found that > in this sentence (in doc/src/sgml/client-auth.sgml:925) there is a > missing comma (,) when listing the "password-based authentication > methods": > > """ > The password-based authe

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Tom Lane
Peter Eisentraut writes: > If we're talking about making things easier to understand, wouldn't a > random user rather know what a WAL "location" is instead of a WAL "LSN"? I wouldn't object to standardizing on "location" instead of "lsn" in the related function and column names. What I don't lik

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Peter Eisentraut
On 4/14/17 11:36, Bruce Momjian wrote: > Yeah, this area is complex enough so any consistency we can add helps. If we're talking about making things easier to understand, wouldn't a random user rather know what a WAL "location" is instead of a WAL "LSN"? -- Peter Eisentraut http://w

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Peter Eisentraut
On 4/14/17 04:28, Kyotaro HORIGUCHI wrote: > =# select distinct attname from pg_attribute where attname like '%lsn%'; >attname > - > confirmed_flush_lsn > latest_end_lsn > local_lsn > receive_start_lsn > received_lsn > remote_lsn > restart_lsn > srsublsn >

[HACKERS] minor typo in client-auth.sgml

2017-04-14 Thread Jaime Casanova
Hi, I was reading about SCRAM and the changes in the docs and found that in this sentence (in doc/src/sgml/client-auth.sgml:925) there is a missing comma (,) when listing the "password-based authentication methods": """ The password-based authentication methods are scram md5 and password.

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-14 Thread Tom Lane
Andres Freund writes: > On 2017-04-13 14:05:43 -0400, Tom Lane wrote: >> Hm. That ties into something I was looking at yesterday. The only >> reason that function is called so much is that bootstrap mode runs a >> separate transaction for *each line of the bki file* (cf do_start, >> do_end in bo

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

2017-04-14 Thread Petr Jelinek
On 13/04/17 19:31, Fujii Masao wrote: > On Fri, Apr 14, 2017 at 1:28 AM, Peter Eisentraut > wrote: >> On 4/10/17 13:28, Fujii Masao wrote: >>> src/backend/replication/logical/launcher.c >>> * Worker started and attached to our shmem. This check is safe >>> * because only

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-14 Thread Magnus Hagander
On Fri, Apr 14, 2017 at 9:36 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/14/17 14:45, Magnus Hagander wrote: > > Attached is a patch that can be applied to pgweb which should fix > all of > > this. > > > > > > > > Applied, thanks. > > Oops, one too many. > > Ind

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-14 Thread Peter Eisentraut
On 4/14/17 14:45, Magnus Hagander wrote: > Attached is a patch that can be applied to pgweb which should fix all of > this. > > > > Applied, thanks. Oops, one too many. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Tra

Re: [HACKERS] SCRAM authentication, take three

2017-04-14 Thread Peter Eisentraut
On 4/11/17 01:10, Heikki Linnakangas wrote: > That question won't arise in practice. Firstly, if the server can do > scram-sha-256-plus, it presumably can also do scram-sha-512-plus. Unless > there's a change in the way the channel binding works, such that the > scram-sha-512-plus variant needs

Re: [HACKERS] Wincrypt.h vs wincrypt.h

2017-04-14 Thread Magnus Hagander
On Fri, Apr 14, 2017 at 9:27 PM, Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > > Commit fe0a0b59 included this line: > > #include > > On Windows file names are not case sensitive, so the "W" works fine, but > as I discovered this morning, if you're cross-compiling on Linux it > ma

[HACKERS] Wincrypt.h vs wincrypt.h

2017-04-14 Thread Andrew Dunstan
Commit fe0a0b59 included this line: #include On Windows file names are not case sensitive, so the "W" works fine, but as I discovered this morning, if you're cross-compiling on Linux it matters very much, and mingw-w64 ships headers with lower case file names, in this case "wincrypt.h". The

[HACKERS] PANIC in pg_commit_ts slru after crashes

2017-04-14 Thread Jeff Janes
In the first statement executed after crash recovery, I sometimes get this error: PANIC: XX000: could not access status of transaction 207580505 DETAIL: Could not read from file "pg_commit_ts/1EF0" at offset 131072: Success. LOCATION: SlruReportIOError, slru.c:918 STATEMENT: create temporary t

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-14 Thread Peter Eisentraut
On 4/11/17 09:03, Magnus Hagander wrote: > I would expect most enterprise customers who care about MITM protection > are already using either TLS or ipsec to cover that already. They have > benefit from the other parts of SCRAM, but they've already solved those > problems. Yeah, I think if you're

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-14 Thread Petr Jelinek
On 14/04/17 21:05, Peter Eisentraut wrote: > On 4/14/17 14:23, Petr Jelinek wrote: >> Ah yes looking at the code, it does exactly that (on master only). Means >> that backport will be necessary. > > I think these two sentences are contradicting each other. > Hehe, didn't realize master will be t

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-14 Thread Peter Eisentraut
On 4/14/17 14:23, Petr Jelinek wrote: > Ah yes looking at the code, it does exactly that (on master only). Means > that backport will be necessary. I think these two sentences are contradicting each other. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Interval for launching the table sync worker

2017-04-14 Thread Peter Eisentraut
On 4/13/17 18:09, Petr Jelinek wrote: > It would have the > disadvantage that if some tables were consistently failing, no other > tables could get synchronized as the amount of workers is limited. OTOH > the approach chosen in the patch will first try all tables and only then > start rate limiting

Re: [HACKERS] Interval for launching the table sync worker

2017-04-14 Thread Peter Eisentraut
On 4/13/17 06:23, Masahiko Sawada wrote: > Attached the latest patch. It didn't actually necessary to change > GetSubscriptionNotReadyRelations. I just changed the logic refreshing > the sync table state list. > Please give me feedback. I think this is a reasonable direction, but do we really need

Re: [HACKERS] Logical replication and inheritance

2017-04-14 Thread Peter Eisentraut
On 4/13/17 01:00, Noah Misch wrote: >> Relative to some of the other open items I'm involved in, I consider >> this a low priority, so I'm not working on it right now. > > By what day should the community look for your next update? Tuesday -- Peter Eisentraut http://www.2ndQuadrant

Re: [HACKERS] Logical replication and inheritance

2017-04-14 Thread Peter Eisentraut
On 4/13/17 06:48, Amit Langote wrote: > That is an important consideration because of pg_dump. See below: > > create table foo (a int); > create table bar () inherits (foo); > create publication mypub for table foo; -- bar is added too. > > $ pg_dump -s | psql -e test > > ALTER PUBLICATION myp

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-14 Thread Andreas Karlsson
On 04/13/2017 06:13 PM, Tom Lane wrote: I've pushed this with some mostly-cosmetic adjustments: Thanks! I like your adjustments. There's certainly lots more that could be done in the genbki code, but I think all we can justify at this stage of the development cycle is to get the low-hanging f

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-14 Thread Magnus Hagander
On Sat, Apr 8, 2017 at 3:52 AM, Bruce Momjian wrote: > On Fri, Mar 24, 2017 at 07:01:46AM +0100, Fabien COELHO wrote: > > > > Hello Peter, > > > > >I think the fix belongs into the web site CSS, so there is nothing to > > >commit into PostgreSQL here. > > > > Indeed, the changes were only for the

Re: [HACKERS] logical replication worker and statistics

2017-04-14 Thread Peter Eisentraut
On 4/14/17 12:09, Petr Jelinek wrote: > Indeed, if catchup phase didn't happen (because tablesync was faster > than apply) then the commit handler is never called so all the changes > made by copy would be forgotten. Also the tablesync worker might exit > from process_syncing_tables() call which is

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-14 Thread Petr Jelinek
On 14/04/17 19:33, Fujii Masao wrote: > On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek > wrote: >> On 12/04/17 15:55, Fujii Masao wrote: >>> Hi, >>> >>> When I shut down the publisher while I repeated creating and dropping >>> the subscription in the subscriber, the publisher emitted the following

Re: [HACKERS] Tuplesort merge pre-reading

2017-04-14 Thread Peter Geoghegan
On Fri, Apr 14, 2017 at 5:57 AM, Robert Haas wrote: > I don't think there's any one fixed answer, because increasing the > number of tapes reduces I/O by adding CPU cost, and visca versa. Sort of, but if you have to merge hundreds of runs (a situation that should be quite rare), then you should b

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-04-14 Thread Peter Eisentraut
On 4/13/17 17:52, Petr Jelinek wrote: > On 12/04/17 06:10, Peter Eisentraut wrote: >> On 3/24/17 10:49, Petr Jelinek wrote: >>> Rebase after table copy patch got committed. >> >> You could please sent another rebase of this, perhaps with some more >> documentation, as discussed downthread. >> >> Al

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-14 Thread Fujii Masao
On Fri, Apr 14, 2017 at 10:33 PM, Petr Jelinek wrote: > On 12/04/17 15:55, Fujii Masao wrote: >> Hi, >> >> When I shut down the publisher while I repeated creating and dropping >> the subscription in the subscriber, the publisher emitted the following >> PANIC error during shutdown checkpoint. >>

Re: [HACKERS] Make defines in pg_config_manual.h conditional?

2017-04-14 Thread Tom Lane
Andres Freund writes: > ATM the defines in pg_config_manual.h are largely unconditional - which > means the file has to be edited instead of being able to override things > via CFLAGS/COPT. Is there any good reason for doing so? YES. If you do things like that, the installed version of pg_confi

[HACKERS] Make defines in pg_config_manual.h conditional?

2017-04-14 Thread Andres Freund
Hi, ATM the defines in pg_config_manual.h are largely unconditional - which means the file has to be edited instead of being able to override things via CFLAGS/COPT. Is there any good reason for doing so? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-14 Thread Daniel Verite
Fabien COELHO wrote: > Pavel also suggested some support for TEXT, although I would like to see a > use case. That could be another extension to the engine. SQLSTATE is text. Also when issuing "psql -v someoption=value -f script", it's reasonable to want to compare :someoptionvar to 'so

Re: [HACKERS] logical replication worker and statistics

2017-04-14 Thread Petr Jelinek
On 12/04/17 05:41, Peter Eisentraut wrote: > On 4/10/17 13:06, Stas Kelvich wrote: >> >>> On 10 Apr 2017, at 19:50, Peter Eisentraut >>> wrote: >>> >>> On 4/10/17 05:49, Stas Kelvich wrote: Here is small patch to call statistics in logical worker. Originally i thought that stat co

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-14 Thread Jaime Casanova
On 5 April 2017 at 13:32, Pavan Deolasee wrote: > > Ok. I've extensively updated the README to match the current state of > affairs. Updated patch set attached. Hi Pavan, I run a test on current warm patchset, i used pgbench with a scale of 20 and a fillfactor of 90 and then start the pgbench ru

Re: [HACKERS] Different table schema in logical replication crashes

2017-04-14 Thread Petr Jelinek
On 14/04/17 17:33, Peter Eisentraut wrote: > On 4/14/17 08:49, Petr Jelinek wrote: >>> Are we prepared to support different schemas in v10? Or should we >>> disallow it for v10 and add a TODO? >>> >> >> Ah nuts, yes it's supposed to be supported, we seem to not initialize >> cstate->range_table i

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-14 Thread Peter Eisentraut
On 4/14/17 11:35, Bruce Momjian wrote: > I think the only open issue is the JavaScript CSS font fix that has a > posted patch here: > > https://www.postgresql.org/message-id/20170408015201.ga18...@momjian.us The open item is specifically about this issue. -- Peter Eisentraut

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Fri, Apr 14, 2017 at 4:28 AM, Kyotaro HORIGUCHI > > wrote: > >> Similariliy, these columns may need renaming. > > > Personally, I would be inclined not to tinker with this, not just > > because we're after freeze but because it

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Bruce Momjian
On Fri, Apr 14, 2017 at 10:22:36AM -0400, Tom Lane wrote: > Robert Haas writes: > > On Fri, Apr 14, 2017 at 4:28 AM, Kyotaro HORIGUCHI > > wrote: > >> Similariliy, these columns may need renaming. > > > Personally, I would be inclined not to tinker with this, not just > > because we're after fre

Re: [HACKERS] Interval for launching the table sync worker

2017-04-14 Thread Masahiko Sawada
On Fri, Apr 14, 2017 at 9:14 PM, Petr Jelinek wrote: > On 14/04/17 12:18, Masahiko Sawada wrote: >> On Fri, Apr 14, 2017 at 7:09 AM, Petr Jelinek >> wrote: >>> On 13/04/17 12:23, Masahiko Sawada wrote: On Thu, Apr 13, 2017 at 11:53 AM, Masahiko Sawada wrote: > On Wed, Apr 12, 2017

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-14 Thread Bruce Momjian
On Fri, Apr 14, 2017 at 11:25:48AM -0400, Peter Eisentraut wrote: > On 4/14/17 01:49, Noah Misch wrote: > > On Wed, Apr 05, 2017 at 01:43:42PM -0400, Peter Eisentraut wrote: > >> On 4/5/17 02:56, Noah Misch wrote: > >>> On Thu, Mar 23, 2017 at 11:21:39PM -0400, Peter Eisentraut wrote: > I thin

Re: [HACKERS] Different table schema in logical replication crashes

2017-04-14 Thread Peter Eisentraut
On 4/14/17 08:49, Petr Jelinek wrote: >> Are we prepared to support different schemas in v10? Or should we >> disallow it for v10 and add a TODO? >> > > Ah nuts, yes it's supposed to be supported, we seem to not initialize > cstate->range_table in tablesync which causes this bug. The CopyState > s

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-14 Thread Peter Eisentraut
On 4/14/17 01:49, Noah Misch wrote: > On Wed, Apr 05, 2017 at 01:43:42PM -0400, Peter Eisentraut wrote: >> On 4/5/17 02:56, Noah Misch wrote: >>> On Thu, Mar 23, 2017 at 11:21:39PM -0400, Peter Eisentraut wrote: I think the fix belongs into the web site CSS, so there is nothing to commit

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-14 Thread Peter Eisentraut
On 4/7/17 21:52, Bruce Momjian wrote: > The fix, as Fabien identified, is to conditionally inject additional CSS > to be _more_ specific than the first CSS and set the font-size to a > simple '1em' so the first CSS is not called twice. I don't think > 'important!' is necessary but it would be good

Re: [HACKERS] Small patch for pg_basebackup argument parsing

2017-04-14 Thread Pierre Ducroquet
On Friday, April 14, 2017 8:59:03 AM CEST Robert Haas wrote: > On Fri, Apr 14, 2017 at 2:32 AM, Pierre Ducroquet wrote: > > On Friday, April 14, 2017 8:44:37 AM CEST Michael Paquier wrote: > >> On Fri, Apr 14, 2017 at 6:32 AM, Pierre Ducroquet > > > > wrote: > >> > Yesterday while doing a few p

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 14, 2017 at 4:28 AM, Kyotaro HORIGUCHI > wrote: >> Similariliy, these columns may need renaming. > Personally, I would be inclined not to tinker with this, not just > because we're after freeze but because it doesn't seem like an > improvement to me. Referring

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-04-14 Thread Petr Jelinek
On 12/04/17 02:32, Robert Haas wrote: > On Tue, Apr 11, 2017 at 8:11 PM, Michael Paquier > wrote: >> On Wed, Apr 12, 2017 at 3:40 AM, Tom Lane wrote: >>> I notice looking at pg_stat_activity that the logical replication launcher >>> sets its application_name to "logical replication launcher". Th

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-14 Thread Petr Jelinek
On 12/04/17 15:55, Fujii Masao wrote: > Hi, > > When I shut down the publisher while I repeated creating and dropping > the subscription in the subscriber, the publisher emitted the following > PANIC error during shutdown checkpoint. > > PANIC: concurrent transaction log activity while database

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Apr 14, 2017 at 9:09 AM, Stephen Frost wrote: > > I agree that the documentation could be improved here. > > This does not seem like it's only a documentation problem. There > seems to be no principled reason why a grant for ALL sho

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 9:09 AM, Stephen Frost wrote: > I agree that the documentation could be improved here. This does not seem like it's only a documentation problem. There seems to be no principled reason why a grant for ALL shouldn't have exactly the same effect as one grant per relevant op

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 4:28 AM, Kyotaro HORIGUCHI wrote: > Similariliy, these columns may need renaming. > > s=# select attname, attrelid::regclass from pg_attribute where attname like > '%location%'; > attname | attrelid > -+- > sent_location

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Stephen Frost
Rod, * Rod Taylor (rod.tay...@gmail.com) wrote: > My actual use-case involves a range. Most users can see and manipulate the > record when CURRENT_TIMESTAMP is within active_period. Some users > (staff/account admins) can see recently dead records too. And a 3rd group > (senior staff) have no time

Re: [HACKERS] [POC] hash partitioning

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 4:23 AM, Yugo Nagata wrote: > On Thu, 13 Apr 2017 16:40:29 -0400 > Robert Haas wrote: >> On Fri, Mar 17, 2017 at 7:57 AM, Yugo Nagata wrote: >> > I also understanded that my design has a problem during pg_dump and >> > pg_upgrade, and that some information to identify the

Re: [HACKERS] Minor typo in partition.c

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 3:43 AM, Simon Riggs wrote: > Oh, just looks very different to what we discussed, so I presumed more > changes were coming. I waited quite a while for you to review Amit's patches on that thread, but as you never did, I eventually picked it up. > Where shall I mention BRI

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Petr Jelinek
On 14/04/17 14:30, Michael Paquier wrote: > On Fri, Apr 14, 2017 at 9:19 PM, Petr Jelinek > wrote: >> I am not quite sure adding more GUCs is all that great option. When >> writing the patches I was wondering if we should perhaps rename the >> wal_receiver_timeout and wal_retrieve_retry_interval t

Re: [HACKERS] Small patch for pg_basebackup argument parsing

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 2:32 AM, Pierre Ducroquet wrote: > On Friday, April 14, 2017 8:44:37 AM CEST Michael Paquier wrote: >> On Fri, Apr 14, 2017 at 6:32 AM, Pierre Ducroquet > wrote: >> > Yesterday while doing a few pg_basebackup, I realized that the integer >> > parameters were not properly c

Re: [HACKERS] Tuplesort merge pre-reading

2017-04-14 Thread Robert Haas
On Fri, Apr 14, 2017 at 1:19 AM, Peter Geoghegan wrote: > Interestingly enough, I think that Knuth was pretty much spot on with > his "sweet spot" of 7 tapes, even if you have modern hardware. Commit > df700e6 (where the sweet spot of merge order 7 was no longer always > used) was effective becaus

Re: [HACKERS] Different table schema in logical replication crashes

2017-04-14 Thread Petr Jelinek
On 13/04/17 05:04, Euler Taveira wrote: > Hi, > > If a certain table has different schemas and the subscriber table has an > unmatched column with a not null constraint, the logical replication > crashes with the above stack trace. > > [snip] > > Are we prepared to support different schemas in

Re: [HACKERS] Logical replication and inheritance

2017-04-14 Thread Petr Jelinek
On 14/04/17 06:14, Amit Langote wrote: > On 2017/04/14 10:57, Petr Jelinek wrote: >> I don't think inheritance and partitioning should behave the same in >> terms of logical replication. > > I see. > >> >> For me the current behavior with inherited tables is correct. > > OK. > > By the way, wha

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Rod Taylor
On Fri, Apr 14, 2017 at 7:41 AM, Rod Taylor wrote: > > > > On Thu, Apr 13, 2017 at 5:31 PM, Stephen Frost wrote: > >> Rod, all, >> >> * Joe Conway (m...@joeconway.com) wrote: >> > On 04/13/2017 01:31 PM, Stephen Frost wrote: >> > > * Robert Haas (robertmh...@gmail.com) wrote: >> > >> On Thu, Apr

Re: [HACKERS] Allowing extended stats on foreign and partitioned tables

2017-04-14 Thread Alvaro Herrera
Noah Misch wrote: > On Mon, Apr 10, 2017 at 09:39:22PM +1200, David Rowley wrote: > > While reviewing extended stats I noticed that it was possible to > > create extended stats on many object types, including sequences. I > > mentioned that this should be disallowed. Statistics were then changed >

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Masahiko Sawada
On Fri, Apr 14, 2017 at 9:19 PM, Petr Jelinek wrote: > On 14/04/17 12:57, Masahiko Sawada wrote: >> Hi, >> >> I noticed that the logical replication launcher uses >> wal_retrieve_retry_interval as a interval of launching logical >> replication worker process. This behavior is not documented and I

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 9:19 PM, Petr Jelinek wrote: > I am not quite sure adding more GUCs is all that great option. When > writing the patches I was wondering if we should perhaps rename the > wal_receiver_timeout and wal_retrieve_retry_interval to something that > makes more sense for both phys

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 8:28 PM, Craig Ringer wrote: > There's no point advertising scram-512 if only -256 can work for 'bob' > because that's what we have in pg_authid. The possibility to have multiple verifiers has other benefits than that, password rolling being one. We may want to revisit tha

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-14 Thread Stephen Frost
* Michael Paquier (michael.paqu...@gmail.com) wrote: > On Fri, Apr 14, 2017 at 8:03 PM, Stephen Frost wrote: > > Some of those were specifically left around to test those code paths. > > I'm not sure if these were those or not though, Andrew was the one who > > reviewed the various pg_dumpall call

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 8:03 PM, Stephen Frost wrote: > Some of those were specifically left around to test those code paths. > I'm not sure if these were those or not though, Andrew was the one who > reviewed the various pg_dumpall calls to add --no-sync in places. Well, Andrew has pushed the pa

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Petr Jelinek
On 14/04/17 12:57, Masahiko Sawada wrote: > Hi, > > I noticed that the logical replication launcher uses > wal_retrieve_retry_interval as a interval of launching logical > replication worker process. This behavior is not documented and I > guess this is no longer consistent with what its name mean

Re: [HACKERS] Interval for launching the table sync worker

2017-04-14 Thread Petr Jelinek
On 14/04/17 12:18, Masahiko Sawada wrote: > On Fri, Apr 14, 2017 at 7:09 AM, Petr Jelinek > wrote: >> On 13/04/17 12:23, Masahiko Sawada wrote: >>> On Thu, Apr 13, 2017 at 11:53 AM, Masahiko Sawada >>> wrote: On Wed, Apr 12, 2017 at 11:46 PM, Peter Eisentraut wrote: > On 4/12/17 0

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Stephen Frost
Rod, * Rod Taylor (rod.tay...@gmail.com) wrote: > Then there is a bug in the simpler statement which happily lets you give > away records. > > CREATE POLICY simple_all ON t TO simple USING (value > 0) WITH CHECK (true); > > SET session authorization simple; > SELECT * FROM t; > UPDATE t SET valu

Re: [HACKERS] Row Level Security UPDATE Confusion

2017-04-14 Thread Rod Taylor
On Thu, Apr 13, 2017 at 5:31 PM, Stephen Frost wrote: > Rod, all, > > * Joe Conway (m...@joeconway.com) wrote: > > On 04/13/2017 01:31 PM, Stephen Frost wrote: > > > * Robert Haas (robertmh...@gmail.com) wrote: > > >> On Thu, Apr 6, 2017 at 4:05 PM, Rod Taylor > wrote: > > >> > I'm a little conf

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-14 Thread Craig Ringer
On 14 Apr. 2017 10:44, "Michael Paquier" wrote: On Fri, Apr 14, 2017 at 1:37 AM, Heikki Linnakangas wrote: > On 04/13/2017 05:53 AM, Michael Paquier wrote: >> +* Parse the list of SASL authentication mechanisms in the >> +* AuthenticationSASL message, and select the best mechanism that w

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-14 Thread Stephen Frost
Michael, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Thu, Apr 6, 2017 at 7:48 AM, Michael Paquier > wrote: > > On Wed, Apr 5, 2017 at 10:24 PM, Stephen Frost wrote: > >> * Michael Paquier (michael.paqu...@gmail.com) wrote: > >>> 1) Initialize the old cluster and start it. > >>> 2)

[HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Masahiko Sawada
Hi, I noticed that the logical replication launcher uses wal_retrieve_retry_interval as a interval of launching logical replication worker process. This behavior is not documented and I guess this is no longer consistent with what its name means. I think that we should either introduce a new GUC

Re: [HACKERS] Interval for launching the table sync worker

2017-04-14 Thread Masahiko Sawada
On Fri, Apr 14, 2017 at 7:09 AM, Petr Jelinek wrote: > On 13/04/17 12:23, Masahiko Sawada wrote: >> On Thu, Apr 13, 2017 at 11:53 AM, Masahiko Sawada >> wrote: >>> On Wed, Apr 12, 2017 at 11:46 PM, Peter Eisentraut >>> wrote: On 4/12/17 00:48, Masahiko Sawada wrote: > On Wed, Apr 12, 2

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-14 Thread Kyotaro HORIGUCHI
At Mon, 10 Apr 2017 19:26:11 +1200, David Rowley wrote in > ... and of course the other functions matching *wal*location* > > My thoughts here are that we're already breaking backward > compatibility of these functions for PG10, so thought we might want to > use this as an opportunity to fix th

Re: [HACKERS] [POC] hash partitioning

2017-04-14 Thread Yugo Nagata
On Thu, 13 Apr 2017 16:40:29 -0400 Robert Haas wrote: > On Fri, Mar 17, 2017 at 7:57 AM, Yugo Nagata wrote: > > I also understanded that my design has a problem during pg_dump and > > pg_upgrade, and that some information to identify the partition > > is required not depending the command order.

Re: [HACKERS] Minor typo in partition.c

2017-04-14 Thread Amit Langote
On 2017/04/14 16:43, Simon Riggs wrote: > On 14 April 2017 at 08:39, Amit Langote wrote: >> On 2017/04/14 16:25, Simon Riggs wrote: >>> On 14 April 2017 at 08:13, Amit Langote >>> wrote: >>> Attached patch gets rid of a "is has". >>> >>> Yes, its a typo, but doesn't add missing information

Re: [HACKERS] Minor typo in partition.c

2017-04-14 Thread Simon Riggs
On 14 April 2017 at 08:39, Amit Langote wrote: > On 2017/04/14 16:25, Simon Riggs wrote: >> On 14 April 2017 at 08:13, Amit Langote >> wrote: >> >>> Attached patch gets rid of a "is has". >> >> Yes, its a typo, but doesn't add missing information or change the meaning. >> >> It should be fixed,

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-14 Thread Simon Riggs
On 13 April 2017 at 18:47, Fujii Masao wrote: > But on second thought, I don't think that reporting NULL as the priority when > quorum-based sync replication is used is less confusing. When there is async > standby, we report 0 as its priority when synchronous_standby_names is empty > or a priori

Re: [HACKERS] Minor typo in partition.c

2017-04-14 Thread Amit Langote
On 2017/04/14 16:25, Simon Riggs wrote: > On 14 April 2017 at 08:13, Amit Langote wrote: > >> Attached patch gets rid of a "is has". > > Yes, its a typo, but doesn't add missing information or change the meaning. > > It should be fixed, but could I suggest we include that in your next > lot of

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-14 Thread Kyotaro HORIGUCHI
At Fri, 14 Apr 2017 10:47:46 +0900, Masahiko Sawada wrote in > On Fri, Apr 14, 2017 at 9:38 AM, Michael Paquier > wrote: > > On Fri, Apr 14, 2017 at 2:47 AM, Fujii Masao wrote: > >> I'm thinking that it's less confusing to report always 0 as the priority of > >> async standby whatever the sett

Re: [HACKERS] Minor typo in partition.c

2017-04-14 Thread Simon Riggs
On 14 April 2017 at 08:13, Amit Langote wrote: > Attached patch gets rid of a "is has". Yes, its a typo, but doesn't add missing information or change the meaning. It should be fixed, but could I suggest we include that in your next lot of doc changes, rather than keep making single minor chang

[HACKERS] Minor typo in partition.c

2017-04-14 Thread Amit Langote
Attached patch gets rid of a "is has". Thanks, Amit diff --git a/src/backend/catalog/partition.c b/src/backend/catalog/partition.c index ab891f6ef2..e0d2665a91 100644 --- a/src/backend/catalog/partition.c +++ b/src/backend/catalog/partition.c @@ -549,7 +549,7 @@ RelationBuildPartitionDesc(Relation