Re: [HACKERS] pageinspect and hash indexes

2017-03-22 Thread Amit Kapila
On Thu, Mar 23, 2017 at 10:02 AM, Ashutosh Sharma wrote: >>> >> >> 1. >> @@ -70,6 +70,17 @@ verify_hash_page(bytea *raw_page, int flags) >> pageopaque = (HashPageOpaque) PageGetSpecialPointer(page); >> + >> + /* Check if it is an unused hash page. */ >> + if (pageopaque->hasho_flag == LH_UNUSED_

Re: [HACKERS] Measuring replay lag

2017-03-22 Thread Simon Riggs
On 23 March 2017 at 01:02, Thomas Munro wrote: > Thanks! Please find attached v7, which includes a note we can point > at when someone asks why it doesn't show 00:00:00, as requested. Thanks. Now I look harder the handling for logical lag seems like it would be problematic in many cases. It's

Re: [HACKERS] ANALYZE command progress checker

2017-03-22 Thread Haribabu Kommi
On Wed, Mar 22, 2017 at 8:11 PM, vinayak wrote: > > On 2017/03/21 21:25, Haribabu Kommi wrote: > > > > On Tue, Mar 21, 2017 at 3:41 PM, vinayak > wrote: > >> Thank you for testing the patch on Windows platform. >> >> > Thanks for the updated patch. > > It works good for a normal relation. But f

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-22 Thread Masahiko Sawada
On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: > On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: >> We already have BTPageOpaqueData.btpo, a union whose contained type >> varies based on the page being dead. We could just do the same with >> some other field in that struct, and t

[HACKERS] dblink module printing unnamed connection (with commit acaf7ccb94)

2017-03-22 Thread Rushabh Lathia
Hi All, DBLINK contrib module started showing :"unnamed" connection name. Consider the below test: postgres=# CREATE ROLE alice NOSUPERUSER NOCREATEDB NOCREATEROLE LOGIN PASSWORD 'wonderland'; CREATE ROLE postgres=# GRANT EXECUTE ON FUNCTION dblink_connect_u(text,text) to alice; GRANT postgres=#

Re: [WIP] RE: [HACKERS] DECLARE STATEMENT setting up a connection in ECPG

2017-03-22 Thread Haribabu Kommi
On Wed, Mar 22, 2017 at 7:57 PM, Ideriha, Takeshi < ideriha.take...@jp.fujitsu.com> wrote: > Hi, thank you very much for reviewing. > Attached is v6 patch. > > >There was a minor conflict in applying 004_declareXX patch. > > I rebased 004_declareStmt_test_v6.patch. > > >Some comments in 001_declar

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 12:41, Andres Freund wrote: > On 2017-03-23 12:14:02 +0800, Craig Ringer wrote: >> On 23 March 2017 at 09:39, Andres Freund wrote: >> > I still think decoding-on-standby is simply not the right approach as >> > the basic/first HA approach for logical rep. It's a nice later-on

Re: [HACKERS] Hash support for grouping sets

2017-03-22 Thread Andrew Gierth
> "Andres" == Andres Freund writes: >> - Assert(newphase == 0 || newphase == aggstate->current_phase + 1); >> + Assert(newphase <= 1 || newphase == aggstate->current_phase + 1); Andres> I think this somewhere in the file header needs an expanded Andres> explanation about what these "

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-22 Thread Andrew Borodin
2017-03-22 22:48 GMT+05:00 Teodor Sigaev : > hasEmptyChild? and hasNonEmptyChild (BTW, isAnyNonempy has missed 't') Yes, I think this naming is good. It's clear what's in common in these flags and what's different. > And if the whole posting tree is empty,then we could mark root page as leaf > an

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-22 Thread Amit Kapila
On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila wrote: > On Tue, Mar 21, 2017 at 6:47 PM, Pavan Deolasee > wrote: >> >>> >> >> Please find attached rebased patches. >> > > Few comments on 0005_warm_updates_v18.patch: > Few more comments on 0005_warm_updates_v18.patch: 1. @@ -234,6 +241,25 @@ index_

Re: [HACKERS] scram and \password

2017-03-22 Thread Michael Paquier
On Wed, Mar 22, 2017 at 8:54 PM, Heikki Linnakangas wrote: > On 03/17/2017 05:38 AM, Michael Paquier wrote: >> >> Regression tests are proving to be useful here (it would be nice to >> get those committed first!). I am noticing that this patch breaks >> connection for users with cleartext or md5-h

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Andres Freund
On 2017-03-23 12:14:02 +0800, Craig Ringer wrote: > On 23 March 2017 at 09:39, Andres Freund wrote: > > I still think decoding-on-standby is simply not the right approach as > > the basic/first HA approach for logical rep. It's a nice later-on > > feature. But that's an irrelevant aside. > > I

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Rafia Sabih
On Thu, Mar 23, 2017 at 5:23 AM, Dilip Kumar wrote: > On Wed, Mar 22, 2017 at 10:33 PM, Robert Haas wrote: >> So couldn't we actually make this test !fcache->returnsSet || !es->lazyEval? >> That would let us allow parallel execution for all non-set-returning >> functions, and also for set-returni

Re: [HACKERS] pageinspect and hash indexes

2017-03-22 Thread Ashutosh Sharma
>> I think it is not just happening for freed overflow but also for newly >> allocated bucket page. It would be good if we could mark freed >> overflow page as UNUSED page rather than just initialising it's header >> portion and leaving the page type in special area as it is. Attached >> is the pa

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >> slots on standbys in, so we can maintain slots on stand

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-22 Thread Ashutosh Bapat
> >> I don't think we will get away by supporting just scan paths, since >> the inner side of lateral join can be any paths not just scan path. Or >> you are suggesting that we disable partition-wise lateral join and >> support reparameterization of only scan paths? > > I think if you can do a stra

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 11:25, Craig Ringer wrote: > Amended patch attached, with added TAP test included. Managed to omit it, sigh. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 011_crash_recovery.pl Description: Perl p

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 09:39, Andres Freund wrote: > We can't just assume that snapbuild is going to work correctly when it's > prerequisites - pinned xmin horizon - isn't working. Makes sense. >> What do _you_ see as the minimum acceptable way to achieve the ability >> for a logical decoding clie

Re: [HACKERS] Hash support for grouping sets

2017-03-22 Thread Mark Dilger
> On Mar 22, 2017, at 8:09 AM, Mark Dilger wrote: > > >> On Mar 22, 2017, at 7:55 AM, Andrew Gierth >> wrote: >> >> [snip] >> >> This thread seems to have gone quiet - is it time for me to just go >> ahead and commit the thing anyway? Anyone else want to weigh in? > > Sorry for the delay.

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-22 Thread Amit Kapila
On Thu, Mar 23, 2017 at 8:43 AM, Amit Kapila wrote: > On Wed, Mar 22, 2017 at 3:39 PM, Ashutosh Sharma > wrote: >> Hi, >> >> On Wed, Mar 22, 2017 at 8:41 AM, Amit Kapila wrote: >> >> To fix this, I think we should pass 'REGBUF_KEEP_DATA' while >> registering the buffer. Something like this, >>

Re: [HACKERS] Hash support for grouping sets

2017-03-22 Thread Andrew Gierth
> "Andres" == Andres Freund writes: Andres> Changes to advance_aggregates() are, in my experience, quite Andres> likely to have performance effects. This needs some Andres> performance tests. [...] Andres> Looks like it could all be noise, but it seems worthwhile to Andres> look into s

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 01:41, Robert Haas wrote: > On Wed, Mar 22, 2017 at 3:13 AM, Simon Riggs wrote: >>> Changes made per discussion. >> >> This looks good to me also. Does what we need it to do. >> >> I was a little worried by possible performance of new lock, but its >> not intended to be run fr

Re: [HACKERS] Radix tree for character conversion

2017-03-22 Thread Kyotaro HORIGUCHI
At Tue, 21 Mar 2017 13:10:48 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170321.131048.150321071.horiguchi.kyot...@lab.ntt.co.jp> > At Fri, 17 Mar 2017 13:03:35 +0200, Heikki Linnakangas > wrote in <01efd334-b839-0450-1b63-f2dea9326...@iki.fi> > > On 03/17/2017 07:19 AM, Kyotaro

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-22 Thread Amit Kapila
On Wed, Mar 22, 2017 at 3:39 PM, Ashutosh Sharma wrote: > Hi, > > On Wed, Mar 22, 2017 at 8:41 AM, Amit Kapila wrote: > > To fix this, I think we should pass 'REGBUF_KEEP_DATA' while > registering the buffer. Something like this, > > - XLogRegisterBuffer(0, buf, REGBUF_STAND

Re: [HACKERS] pageinspect and hash indexes

2017-03-22 Thread Amit Kapila
On Mon, Mar 20, 2017 at 6:53 PM, Ashutosh Sharma wrote: > > I think it is not just happening for freed overflow but also for newly > allocated bucket page. It would be good if we could mark freed > overflow page as UNUSED page rather than just initialising it's header > portion and leaving the pa

Re: [HACKERS] bug/oversight in TestLib.pm and PostgresNode.pm

2017-03-22 Thread Michael Paquier
On Thu, Mar 23, 2017 at 12:51 AM, Erik Rijkers wrote: > While trying to test pgbench's stderr (looking for 'creating tables' in > output of the initialisation step) I ran into these two bugs (or perhaps > better 'oversights'). + if (defined $expected_stderr) { + like($stderr, $expected_s

Re: [HACKERS] Measuring replay lag

2017-03-22 Thread Thomas Munro
On Wed, Mar 15, 2017 at 8:15 PM, Ian Barwick wrote: >> 2. Recognise when the last reported write/flush/apply LSN from the >> standby == end of WAL on the sending server, and show lag times of >> 00:00:00 in all three columns. I consider this entirely bogus: it's >> not an actual measurement that

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 02:08, Simon Riggs wrote: > And of course, we might return "subcommitted" also, which could > technically also be an abort in some cases, so we'd need to do a wait > loop on that. Users generally don't see subxact IDs, so it wasn't something I was overly concerned by. Most no

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Andres Freund
On 2017-03-23 09:14:07 +0800, Craig Ringer wrote: > On 23 March 2017 at 07:31, Andres Freund wrote: > > On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: > > >> I was thinking that by disallowing snapshot use and output plugin > >> invocation we'd avoid the need to support cancellation on recover

Re: [HACKERS] UPDATE of partition key

2017-03-22 Thread Amit Langote
Hi Amit, Thanks for the updated patch. On 2017/03/23 3:09, Amit Khandekar wrote: > Attached is v2 patch which implements the above optimization. Would it be better to have at least some new tests? Also, there are a few places in the documentation mentioning that such updates cause error, which

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

2017-03-22 Thread Lukas Fittl
On Sat, Mar 18, 2017 at 12:46 PM, Tom Lane wrote: > > So it turns out this discussion just reinvented the alternative that > Lukas had in his 0002 proposal. Are there any remaining objections > to proceeding with that approach? > Thanks for reviewing - updated patch attached, comments below. >

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 07:31, Andres Freund wrote: > On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: >> I was thinking that by disallowing snapshot use and output plugin >> invocation we'd avoid the need to support cancellation on recovery >> conflicts, etc, simplifying things considerably. > > Th

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 00:17, Andres Freund wrote: > On 2017-03-22 15:59:42 +, Simon Riggs wrote: >> On 22 March 2017 at 13:06, Andres Freund wrote: >> >> >> The parts I think are important for Pg10 are: >> > >> >> * Ability to create logical slots on replicas >> > >> > Doesn't this also imply r

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-22 Thread Amit Langote
> On Tue, Mar 21, 2017 at 10:37 AM, Robert Haas wrote: >> On Tue, Mar 21, 2017 at 9:49 AM, Robert Haas wrote: >>> On Tue, Mar 21, 2017 at 5:05 AM, Amit Langote >>> wrote: Attached updated patches. >>> >>> Committed 0001 after removing a comma. >> >> Regarding 0002, Thanks for the review.

Re: [HACKERS] Measuring replay lag

2017-03-22 Thread Thomas Munro
On Thu, Mar 23, 2017 at 12:12 AM, Simon Riggs wrote: > On 22 March 2017 at 11:03, Thomas Munro wrote: > >> Hah. Apologies for the delay -- I will post a patch with >> documentation as requested within 24 hours. > > Thanks very much. I'll reserve time to commit it tomorrow, all else being > good

Re: [HACKERS] Parallel Append implementation

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 4:49 AM, Amit Khandekar wrote: > Attached is the updated patch that handles the changes for all the > comments except the cost changes part. Details about the specific > changes are after the cost-related points discussed below. > > For non-partial paths, I was checking fol

Re: [HACKERS] Hash support for grouping sets

2017-03-22 Thread Andres Freund
Hi, > +/* > + * Switch to phase "newphase", which must either be 0 or 1 (to reset) or > * current_phase + 1. Juggle the tuplesorts accordingly. > */ > static void > initialize_phase(AggState *aggstate, int newphase) > { > - Assert(newphase == 0 || newphase == aggstate->current_phase + 1

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
David, * David Steele (da...@pgmasters.net) wrote: > On 3/22/17 3:45 PM, Robert Haas wrote: > >On Wed, Mar 22, 2017 at 3:24 PM, David Steele wrote: > >>>One of the reasons to go with the LSN is that we would actually be > >>>maintaining what happens when the WAL files are 16MB in size. > >>> > >>

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-22 Thread Peter Geoghegan
On Wed, Mar 22, 2017 at 3:17 AM, Thomas Munro wrote: >> If I follow the new code correctly, then it doesn't matter that you've >> unlink()'d to take care of the more obvious resource management chore. >> You can still have a reference leak like this, if I'm not mistaken, >> because you still have

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Dilip Kumar
On Wed, Mar 22, 2017 at 10:33 PM, Robert Haas wrote: > So couldn't we actually make this test !fcache->returnsSet || !es->lazyEval? > That would let us allow parallel execution for all non-set-returning > functions, and also for set-returning functions that end up with > es->lazyEval set to false.

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Andres Freund
On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: > On 22 March 2017 at 21:06, Andres Freund wrote: > > Hi, > > > > On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: > >> > 0002 should be doable as a whole this release, I have severe doubts that > >> > 0003 as a whole has a chance for 10 - the cod

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 22 March 2017 at 21:06, Andres Freund wrote: > Hi, > > On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: >> > 0002 should be doable as a whole this release, I have severe doubts that >> > 0003 as a whole has a chance for 10 - the code is in quite a raw shape, >> > there's a significant number o

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-22 Thread Michael Paquier
On Thu, Mar 23, 2017 at 1:24 AM, Robert Haas wrote: > I mean, your argument boils down to "somebody might want to > deliberately hide things from pg_stat_activity". But that's not > really a mode we support in general, and supporting it only for > certain cases doesn't seem like something that th

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
Hi Robert, On 3/22/17 3:45 PM, Robert Haas wrote: On Wed, Mar 22, 2017 at 3:24 PM, David Steele wrote: One of the reasons to go with the LSN is that we would actually be maintaining what happens when the WAL files are 16MB in size. David's initial expectation was this for 64MB WAL files: 000

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread David Steele
On 3/22/17 4:42 PM, Peter Eisentraut wrote: On 3/22/17 15:14, Stephen Frost wrote: -SELECT * FROM pg_stop_backup(false); +SELECT * FROM pg_stop_backup(false [, true ]); I think that it's better to get rid of "[" and "]" from the above because IMO this should be the command example that users ac

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
On 3/22/17 3:39 PM, Peter Eisentraut wrote: On 3/22/17 15:37, Peter Eisentraut wrote: If changing WAL sizes catches on, I do think we should keep thinking about a new format for a future release, I think that means that I'm skeptical about changing the default size right now. I think if we d

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Elvis Pranskevichus
On Wednesday, March 22, 2017 4:28:18 PM EDT Robert Haas wrote: > On Wed, Mar 22, 2017 at 4:00 PM, Peter Eisentraut > > I think we could use "in_recovery", which would be consistent with > > existing naming. > > True. Ironically, that was the name I originally used. I'll update the patch. > (J

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread Stephen Frost
* Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 3/22/17 15:14, Stephen Frost wrote: > >> -SELECT * FROM pg_stop_backup(false); > >> +SELECT * FROM pg_stop_backup(false [, true ]); > >> > >> I think that it's better to get rid of "[" and "]" from the above because > >> IMO this sho

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread Peter Eisentraut
On 3/22/17 15:14, Stephen Frost wrote: >> -SELECT * FROM pg_stop_backup(false); >> +SELECT * FROM pg_stop_backup(false [, true ]); >> >> I think that it's better to get rid of "[" and "]" from the above because >> IMO this should be the command example that users actually can run. > Using the '[' a

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 4:00 PM, Peter Eisentraut wrote: > On 3/22/17 14:09, Robert Haas wrote: >>> The opposite means primary. I can flip the GUC name to "is_primary", if >>> that's clearer. >> Hmm, I don't find that clearer. "hot standby" has a very specific >> meaning; "primary" isn't vague,

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
* Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > The question is, which property is more useful to preserve: matching > LSN, or having a mostly consecutive numbering. > > Actually, I would really really like to have both, but if I had to pick > one, I'd lean 55% toward consecutive nu

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Peter Eisentraut
On 3/22/17 14:09, Robert Haas wrote: >> The opposite means primary. I can flip the GUC name to "is_primary", if >> that's clearer. > Hmm, I don't find that clearer. "hot standby" has a very specific > meaning; "primary" isn't vague, but I would say it's less specific. The problem I have is that

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 8:46 AM, Ashutosh Bapat wrote: > Here's set of updated patches rebased on > 1148e22a82edc96172fc78855da392b6f0015c88. > > I have fixed all the issues reported till now. I don't understand why patch 0001 ends up changing every existing test for RELOPT_JOINREL anywhere in th

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 3:24 PM, David Steele wrote: >> One of the reasons to go with the LSN is that we would actually be >> maintaining what happens when the WAL files are 16MB in size. >> >> David's initial expectation was this for 64MB WAL files: >> >> 00010040 >> 00010

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Peter Eisentraut
On 3/22/17 15:37, Peter Eisentraut wrote: > If changing WAL sizes catches on, I do think we should keep thinking > about a new format for a future release, I think that means that I'm skeptical about changing the default size right now. -- Peter Eisentraut http://www.2ndQuadrant.com

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Peter Eisentraut
On 3/22/17 15:09, Stephen Frost wrote: > David's initial expectation was this for 64MB WAL files: > > 00010040 > 00010080 > 000100CO > 00010001 > > Which both matches the LSN *and* keeps the file names the same when > they're 16MB.

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
* David Steele (da...@pgmasters.net) wrote: > On 3/22/17 3:09 PM, Stephen Frost wrote: > >* Robert Haas (robertmh...@gmail.com) wrote: > >>On Wed, Mar 22, 2017 at 1:49 PM, Stephen Frost wrote: > >>>Then perhaps we do need to be thinking of moving this to PG11 instead of > >>>exposing an option tha

Re: [HACKERS] Report the number of skipped frozen pages by manual VACUUM

2017-03-22 Thread Fujii Masao
On Wed, Mar 15, 2017 at 7:51 PM, Masahiko Sawada wrote: > On Wed, Mar 15, 2017 at 1:09 PM, Yugo Nagata wrote: >> On Fri, 10 Mar 2017 20:08:42 +0900 >> Masahiko Sawada wrote: >> >>> On Fri, Mar 10, 2017 at 3:58 PM, Jim Nasby wrote: >>> > On 3/6/17 8:34 PM, Masahiko Sawada wrote: >>> >> >>> >> I

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David Steele
On 3/22/17 3:09 PM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Mar 22, 2017 at 1:49 PM, Stephen Frost wrote: Then perhaps we do need to be thinking of moving this to PG11 instead of exposing an option that users will start to use which will result in WAL naming t

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-22 Thread Mithun Cy
On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > > > On Wed, Mar 22, 2017 at 4:53 PM, Mithun Cy > wrote: > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validatio

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread Stephen Frost
Fujii, * Fujii Masao (masao.fu...@gmail.com) wrote: > On Thu, Mar 23, 2017 at 12:37 AM, Stephen Frost wrote: > > * David Steele (da...@pgmasters.net) wrote: > >> On 3/21/17 2:34 PM, Fujii Masao wrote: > >> >The patch basically looks good to me, but one comment is; > >> >backup.sgml (at least the

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 22, 2017 at 1:49 PM, Stephen Frost wrote: > > Then perhaps we do need to be thinking of moving this to PG11 instead of > > exposing an option that users will start to use which will result in WAL > > naming that'll be confusing an

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-03-22 Thread Oleg Bartunov
On Wed, Mar 22, 2017 at 8:04 PM, Arseny Sher wrote: > > While I admire your fearlessness, I think the chances of you being > > able to bring a project of this type to a successful conclusion are > > remote. Here is what I said about this topic previously: > > > > http://postgr.es/m/CA+Tgmoa=kzHJ

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-22 Thread Pavan Deolasee
On Wed, Mar 22, 2017 at 4:53 PM, Mithun Cy wrote: > On Wed, Mar 22, 2017 at 3:44 PM, Pavan Deolasee > wrote: > > > > This looks quite weird to me. Obviously these numbers are completely > > non-comparable. Even the time for VACUUM FULL goes up with every run. > > > > May be we can blame it on AW

Re: [HACKERS] Possible regression with gather merge.

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 3:39 AM, Rushabh Lathia wrote: > Looking at the explain analyze output of both the plan, its clear that GM > taking longer as its using external merge dist for the sort, where as > another plan perform top-N heapsort. For normal sort path, it can consider > the limit as bou

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-22 Thread Fujii Masao
On Thu, Mar 23, 2017 at 12:37 AM, Stephen Frost wrote: > David, all, > > * David Steele (da...@pgmasters.net) wrote: >> On 3/21/17 2:34 PM, Fujii Masao wrote: >> >The patch basically looks good to me, but one comment is; >> >backup.sgml (at least the description for "Making a non-exclusive >> >low

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Elvis Pranskevichus
On Wednesday, March 22, 2017 2:17:27 PM EDT Jaime Casanova wrote: > On 18 March 2017 at 14:01, Elvis Pranskevichus wrote: > > On Saturday, March 18, 2017 3:33:16 AM EDT Michael Paquier wrote: > >> Why adding a good chunk of code instead of using > >> pg_is_in_recovery(), > >> which switches to fal

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Jaime Casanova
On 18 March 2017 at 14:01, Elvis Pranskevichus wrote: > On Saturday, March 18, 2017 3:33:16 AM EDT Michael Paquier wrote: >> >> Why adding a good chunk of code instead of using pg_is_in_recovery(), >> which switches to false once a server exits recovery? > > That requires polling the database cont

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 2:08 PM, Simon Riggs wrote: > On 22 March 2017 at 17:41, Robert Haas wrote: >> +if (TransactionIdIsCurrentTransactionId(xid)) >> +status = gettext_noop("in progress"); >> +else if (TransactionIdDidCommit(xid)) >> +status = gettext_no

Re: [HACKERS] UPDATE of partition key

2017-03-22 Thread Amit Khandekar
On 17 March 2017 at 16:07, Amit Khandekar wrote: > On 6 March 2017 at 15:11, Amit Langote wrote: >> But that starts to sound less attractive when one realizes that that will occur for every row that wants to move. >>> >>> If we manage to call ExecSetupPartitionTupleRouting() during exec

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 8:25 AM, Elvis Pranskevichus wrote: > On Tuesday, March 21, 2017 11:50:38 PM EDT Peter Eisentraut wrote: >> On 3/17/17 13:56, Elvis Pranskevichus wrote: >> > Currently, clients wishing to know when the server exits hot standby >> > have to resort to polling, which is often

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Simon Riggs
On 22 March 2017 at 17:41, Robert Haas wrote: > +if (TransactionIdIsCurrentTransactionId(xid)) > +status = gettext_noop("in progress"); > +else if (TransactionIdDidCommit(xid)) > +status = gettext_noop("committed"); > +else if (TransactionIdDidAbort

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 1:49 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Mar 22, 2017 at 1:22 PM, Stephen Frost wrote: >> > To put this in another light, had this issue been brought up post >> > feature-freeze, your definition would mean that we would only h

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread David G. Johnston
On Wed, Mar 22, 2017 at 9:51 AM, Stephen Frost wrote: > Robert, > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Wed, Mar 22, 2017 at 12:22 PM, Stephen Frost > wrote: > > > While I understand that you'd like to separate the concerns between > > > changing the renaming scheme and changing

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 22, 2017 at 1:22 PM, Stephen Frost wrote: > > To put this in another light, had this issue been brought up post > > feature-freeze, your definition would mean that we would only have the > > option to either revert the patch entir

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-22 Thread Teodor Sigaev
No, second conditional code will be called for any subtree, which contains totally empty subtree. That check !isRoot covers case when the entire posting tree should be erased: we cannot just quit out of recursive cleanup, we have to make a scan here, starting from root. Oh, I see Probably, vari

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix and simplify check for whether we're running as Windows serv

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 10:13 AM, Alvaro Herrera wrote: > Heikki Linnakangas wrote: >> I did some archeology, and found CheckTokenMembership() in MinGW's w32api >> packages version 3.14 >> (https://sourceforge.net/projects/mingw/files/MinGW/Base/w32api/w32api-3.14/, >> in include/winbase.h). Accor

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 1:22 PM, Stephen Frost wrote: > To put this in another light, had this issue been brought up post > feature-freeze, your definition would mean that we would only have the > option to either revert the patch entirely or to live with the poor > naming scheme. Yeah, and I abs

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 3:13 AM, Simon Riggs wrote: >> Changes made per discussion. > > This looks good to me also. Does what we need it to do. > > I was a little worried by possible performance of new lock, but its > not intended to be run frequently, so its OK. Agreed. Reviewing 0002: +

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-22 Thread Andres Freund
On 2017-03-22 13:15:41 -0400, Tom Lane wrote: > BTW, I'm fairly concerned by what you did in nodeTidscan.c, ie delaying > compile of the TID expressions until TidListCreate. I think that probably > fails for cases involving, eg, subplans in the expressions; we need > subplans to get linked to the

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 22, 2017 at 12:51 PM, Stephen Frost wrote: > > This would clearly be an adjustment to the submitted patch, which > > happens regularly during the review and commit process and is part of > > the commitfest process, so I don't agre

Re: [HACKERS] patch proposal

2017-03-22 Thread David Steele
On 3/21/17 8:45 PM, Venkata B Nagothi wrote: On Tue, Mar 21, 2017 at 8:46 AM, David Steele Backup recovery starts from the checkpoint in the backup_label, not from the checkpoint in pg_control. The original checkpoint that started the backup is generally overwritten in pg_control by the end

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 9:59 AM, Ashutosh Bapat wrote: >> In my testing last week, reparameterize_path_by_child() was essential >> for nested loops to work properly, even without LATERAL. Without it, >> the parameterized path ends up containing vars that reference the >> parent varno instead of t

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-22 Thread Tom Lane
BTW, I'm fairly concerned by what you did in nodeTidscan.c, ie delaying compile of the TID expressions until TidListCreate. I think that probably fails for cases involving, eg, subplans in the expressions; we need subplans to get linked to the parent node, and this way won't do it till (maybe) too

Re: [HACKERS] RADIUS fallback servers

2017-03-22 Thread Magnus Hagander
On Mon, Mar 6, 2017 at 8:14 PM, Adam Brightwell < adam.brightw...@crunchydata.com> wrote: > On Mon, Mar 6, 2017 at 10:24 AM, Adam Brightwell > wrote: > >>> I wonder if removing the complexity of maintaining two separate lists > >>> for the server and port would be a better/less complex approach.

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 12:51 PM, Stephen Frost wrote: > This would clearly be an adjustment to the submitted patch, which > happens regularly during the review and commit process and is part of > the commitfest process, so I don't agree that holding it to new-feature > level is correct when it's

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-03-22 Thread Arseny Sher
> While I admire your fearlessness, I think the chances of you being > able to bring a project of this type to a successful conclusion are > remote. Here is what I said about this topic previously: > > http://postgr.es/m/CA+Tgmoa=kzhj+twxyq+vku21nk3prkrjsdbhjubn7qvc8uk...@mail.gmail.com Well, as

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 3:00 AM, Rafia Sabih wrote: > On Wed, Mar 22, 2017 at 12:55 AM, Robert Haas > wrote: > > On Tue, Mar 21, 2017 at 6:36 AM, Dilip Kumar > wrote: > >> How about taking the decision of execute_once based on > >> fcache->returnsSet instead of based on lazyEval? > >> > >> chan

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 22, 2017 at 12:22 PM, Stephen Frost wrote: > > While I understand that you'd like to separate the concerns between > > changing the renaming scheme and changing the default and enabling this > > option, I don't agree that they can

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-03-22 Thread Simon Riggs
On 22 March 2017 at 03:42, Haribabu Kommi wrote: > > > On Wed, Mar 22, 2017 at 1:53 PM, Seki, Eiji > wrote: >> >> >> Thank you for your review, again. >> >> I think your proposals are better, so I reflected them. > > > Thanks for the updated patch. Patch looks good to me. > I marked it as "ready

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 12:22 PM, Stephen Frost wrote: > While I understand that you'd like to separate the concerns between > changing the renaming scheme and changing the default and enabling this > option, I don't agree that they can or should be independently > considered. Well, I don't under

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Andres Freund
On 2017-03-22 15:59:42 +, Simon Riggs wrote: > On 22 March 2017 at 13:06, Andres Freund wrote: > > >> The parts I think are important for Pg10 are: > > > >> * Ability to create logical slots on replicas > > > > Doesn't this also imply recovery conflicts on DROP DATABASE? > > Not needed until

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-03-22 Thread Peter Geoghegan
On Wed, Mar 22, 2017 at 5:44 AM, Robert Haas wrote: >> Actually, that's quite possible with the design I came up with. > > I don't think it is. What sequence of calls do the APIs you've > proposed would accomplish that goal? I don't see anything in this > patch set that would permit anything oth

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 12:20 PM, Robert Haas wrote: > On Wed, Mar 22, 2017 at 1:31 AM, Michael Paquier > wrote: >> Okay, switched as ready for committer. One note for the committer >> though: keeping the calls of pgstat_bestart() out of >> BackgroundWorkerInitializeConnection() and >> Background

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On the topic of whether to also change the default, I'm not sure what > is best and will defer to others. On the topic of whether to whack > around the file naming scheme, -1 from me. This patch was posted > three months ago and nobody sugge

Re: [HACKERS] [COMMITTERS] pgsql: Sync pg_dump and pg_dumpall output

2017-03-22 Thread Andrew Dunstan
On 03/22/2017 12:10 PM, Stephen Frost wrote: >> Still I agree that we should have tests for both cases. > Perhaps, though if I recall correctly, we've basically had zero calls > for fsync() until this. If we don't feel like we need to test that in > the backend then it seems a bit silly to feel

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 1:31 AM, Michael Paquier wrote: > Okay, switched as ready for committer. One note for the committer > though: keeping the calls of pgstat_bestart() out of > BackgroundWorkerInitializeConnection() and > BackgroundWorkerInitializeConnectionByOid() keeps users the > possibilit

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Simon Riggs
On 22 March 2017 at 08:53, Craig Ringer wrote: > I'm splitting up the rest of the decoding on standby patch set with > the goal of getting minimal functionality for creating and managing > slots on standbys in, so we can maintain slots on standbys and use > them when the standby is promoted to ma

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Robert Haas
On Wed, Mar 22, 2017 at 8:46 AM, Stephen Frost wrote: >> I was definitely initially in favor of >> raising the value, but I got cold feet, a bit, when Alvaro pointed out >> that going to 64MB would require a substantial increase in >> min_wal_size. > > The performance concern of having 3 segments

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Kuntal Ghosh
On Wed, Mar 22, 2017 at 3:14 PM, Beena Emerson wrote: > PFA an updated patch which fixes a minor bug I found. It only increases the > string size in pretty_wal_size function. > The 01-add-XLogSegmentOffset-macro.patch has also been rebased. Thanks for the updated versions. Here is a partial review

  1   2   >