Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-06 Thread Michael Paquier
On Wed, Jul 5, 2017 at 4:57 PM, Michael Paquier wrote: > Why not refactoring a bit do_pg_stop_backup() so as the wait phase > happens even if a backup is started in recovery? Now wait_for_archive > is ignored because no wait is happening and the stop point is directly > returned back to the caller

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-07-06 Thread Michael Paquier
On Tue, Jul 4, 2017 at 1:34 PM, Noah Misch wrote: > Bundling code cleanup into commits that also do something else is strictly > worse than bundling whitespace cleanup, which is itself bad: > https://postgr.es/m/flat/20160113144826.gb3379...@tornado.leadboat.com FWIW, I agree with that. I favor a

[HACKERS] Fix header comment of streamutil.c

2017-07-06 Thread Masahiko Sawada
Hi, While reading source code, I found that the header comment of streamutil.c is not correct. I guess pg_receivelog is a mistake of pg_receivexlog and it's an oversight when changing xlog to wal. Attached patch fixes this. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread AP
On Thu, Jul 06, 2017 at 05:19:59PM +0530, Amit Kapila wrote: > I think if you are under development, it is always advisable to create > indexes after initial bulk load. That way it will be faster and will > take lesser space atleast in case of hash index. This is a bit of a pickle, actually: * if

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Amit Langote
On 2017/07/07 4:55, Dean Rasheed wrote: > On 5 July 2017 at 18:07, Dean Rasheed wrote: >> So if we were to go for maximum flexibility and compatibility with >> Oracle, then perhaps what we would do is more like the original idea >> of UNBOUNDED ABOVE/BELOW, except call them MINVALUE and MAXVALUE,

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Michael Paquier
On Thu, Jul 6, 2017 at 8:51 PM, Dilip Kumar wrote: > On Thu, Jul 6, 2017 at 4:18 PM, Kuntal Ghosh > wrote: >> But, I'm little concerned/doubt regarding the following part of the code. >> +/* >> + * Converts an int64 from network byte order to native format. >> + */ >> +static int64 >> +pg_recvin

Re: [HACKERS] New partitioning - some feedback

2017-07-06 Thread Amit Langote
Hi Mark, On 2017/07/07 9:02, Mark Kirkwood wrote: > I've been trying out the new partitioning in version 10. Firstly, I must > say this is excellent - so much nicer than the old inheritance based method! Thanks. :) > My only niggle is the display of partitioned tables via \d etc. e.g: > > part=#

Re: [HACKERS] Out of date comment in predicate.c

2017-07-06 Thread Thomas Munro
On Sat, Jul 1, 2017 at 6:38 AM, Peter Eisentraut wrote: > On 6/27/17 01:21, Thomas Munro wrote: >> Commit ea9df812d8502fff74e7bc37d61bdc7d66d77a7f got rid of >> FirstPredicateLockMgrLock, but it's still referred to in a comment in >> predicate.c where the locking protocol is documented. I think i

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Amit Langote
On 2017/07/06 18:30, Dean Rasheed wrote: > On 5 July 2017 at 10:43, Amit Langote wrote: >> 0001 is your patch to tidy up check_new_partition_bound() (must be >> applied before 0002) >> > > I pushed this first patch, simplifying check_new_partition_bound() for > range partitions, since it seemed

[HACKERS] New partitioning - some feedback

2017-07-06 Thread Mark Kirkwood
I've been trying out the new partitioning in version 10. Firstly, I must say this is excellent - so much nicer than the old inheritance based method! My only niggle is the display of partitioned tables via \d etc. e.g: part=# \d List of relations Schema | Name |

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Joe Conway
On 07/06/2017 01:24 PM, Dean Rasheed wrote: > On 6 July 2017 at 21:04, Tom Lane wrote: >> Dean Rasheed writes: >>> However, this is also an incompatible syntax change, and any attempt >>> to support both the old and new syntaxes is likely to be messy, so we >>> really need to get consensus on whe

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-06 Thread Tom Lane
Kuntal Ghosh writes: > On Thu, Jul 6, 2017 at 3:45 AM, Tom Lane wrote: > + * In the first bin (i==1), add a fudge factor that ensures > + * that histfrac is at least eq_selec. We do this because we > + * know that the first histogram entry does satisfy the > + * inequality (if !isgt) or not sati

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Dean Rasheed
On 6 July 2017 at 21:04, Tom Lane wrote: > Dean Rasheed writes: >> However, this is also an incompatible syntax change, and any attempt >> to support both the old and new syntaxes is likely to be messy, so we >> really need to get consensus on whether this is the right thing to do, >> and whether

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Tom Lane
Dean Rasheed writes: > However, this is also an incompatible syntax change, and any attempt > to support both the old and new syntaxes is likely to be messy, so we > really need to get consensus on whether this is the right thing to do, > and whether it *can* be done now for PG10. FWIW, I'd much

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Dean Rasheed
On 5 July 2017 at 18:07, Dean Rasheed wrote: > So if we were to go for maximum flexibility and compatibility with > Oracle, then perhaps what we would do is more like the original idea > of UNBOUNDED ABOVE/BELOW, except call them MINVALUE and MAXVALUE, > which conveniently are already unreserved k

Re: [HACKERS] [COMMITTERS] pgsql: pg_test_timing: Add NLS

2017-07-06 Thread Alvaro Herrera
Peter Eisentraut wrote: > pg_test_timing: Add NLS > > Also straighten out use of time unit abbreviations a bit. We (well, Carlos Chapi, who's doing the translation work now) just noticed that this has a bug in this line + printf("%6s %10s %10s\n", _("< us"), _("% of total"), _("count")); _(

Re: [HACKERS] Rust bindings to pgtypes lib

2017-07-06 Thread Andres Freund
Hi, On 2017-07-06 20:26:29 +0200, Michael Meskes wrote: > > But is this possible, or feasible? I see that the makefile refers to > > Possible yes, but in general I'm not a big fan of duplicating code. I > spend too much time to keep those copies in sync. Indeed. I'm quite strongly against expos

Re: [HACKERS] Rust bindings to pgtypes lib

2017-07-06 Thread Michael Meskes
> But is this possible, or feasible? I see that the makefile refers to Possible yes, but in general I'm not a big fan of duplicating code. I spend too much time to keep those copies in sync. > Makefile.global, and also includes stuff from ecpg, aat least. I don't want > to Yes, but these shou

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-07-06 Thread Greg Stark
On 6 July 2017 at 15:29, Jim Finnerty wrote: > > Feel free to knock down this 'straw man' and propose something better! I think the pattern in this design that we don't want is that it imposes extra complexity on every user of every page even when the page doesn't have the problem and even when t

Re: [HACKERS] More race conditions in logical replication

2017-07-06 Thread Petr Jelinek
On 06/07/17 17:33, Petr Jelinek wrote: > On 03/07/17 01:54, Tom Lane wrote: >> I noticed a recent failure that looked suspiciously like a race condition: >> >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2017-07-02%2018%3A02%3A07 >> >> The critical bit in the log file is >> >>

[HACKERS] paths in partitions of a dummy partitioned table

2017-07-06 Thread Ashutosh Bapat
If a partitioned table is proven dummy, set_rel_pathlist() doesn't mark the partition relations dummy and thus doesn't set any (dummy) paths in the partition relations. The lack of paths in the partitions means that we can not use partition-wise join while joining this table with some other similar

Re: [HACKERS] More race conditions in logical replication

2017-07-06 Thread Petr Jelinek
On 03/07/17 01:54, Tom Lane wrote: > I noticed a recent failure that looked suspiciously like a race condition: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2017-07-02%2018%3A02%3A07 > > The critical bit in the log file is > > error running SQL: 'psql::1: ERROR: could n

Re: [HACKERS] Bug in Physical Replication Slots (at least 9.5)?

2017-07-06 Thread Ryan Murphy
Poking this. Looking back through the discussion, this issue has been reproduced by multiple people. The patch still applies to HEAD without issues. I have no experience with PostgreSQL replication, so I'm not qualified to really review this. From what I can see with the patch, it's just a s

[HACKERS] Rust bindings to pgtypes lib

2017-07-06 Thread Kaare Rasmussen
Hi For learning purposes (and because I probably need it) , I've started to make Rust bindings to the pgtypes Library. One problem I'm thinking about right now is how to know if, and where, the library and the include files are installed. One way to avoid that problem is to include the source

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread AP
On Wed, Jul 05, 2017 at 07:31:39PM +1000, AP wrote: > On Tue, Jul 04, 2017 at 08:23:20PM -0700, Jeff Janes wrote: > > On Tue, Jul 4, 2017 at 3:57 AM, AP wrote: > > > The data being indexed is BYTEA, (quasi)random and 64 bytes in size. > > > The table has over 2 billion entries. The data is not uni

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread AP
On Thu, Jul 06, 2017 at 12:38:38PM +0530, Amit Kapila wrote: > On Thu, Jul 6, 2017 at 9:32 AM, AP wrote: > > On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: > >> On Thu, Jul 6, 2017 at 2:40 AM, AP wrote: > >> > On Wed, Jul 05, 2017 at 05:52:32PM +0530, Amit Kapila wrote: > >> >> >> >

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2017-07-06 Thread Heikki Linnakangas
On 07/03/2017 06:30 PM, Chapman Flack wrote: On 07/03/2017 09:39 AM, Heikki Linnakangas wrote: Hmm. That's not the problem, though. Imagine that instead of the loop above, you do just: WALInsertLockUpdateInsertingAt(CurrPos); AdvanceXLInsertBuffer(EndPos, false); AdvanceXLInsertBuffer() will

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread AP
On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: > On Thu, Jul 6, 2017 at 2:40 AM, AP wrote: > > On Wed, Jul 05, 2017 at 05:52:32PM +0530, Amit Kapila wrote: > >> >> > version | bucket_pages | overflow_pages | bitmap_pages | > >> >> > unused_pages | live_items | dead_items | free_p

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-07-06 Thread Mithun Cy
On Thu, Jul 6, 2017 at 10:52 AM, Amit Kapila wrote: > I am not able to understand what you want to say. Unlogged tables > should be empty in case of crash recovery. Also, we never flush > unlogged buffers except for shutdown checkpoint, refer BufferAlloc and > in particular below comment: -- Sor

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Dilip Kumar
On Thu, Jul 6, 2017 at 4:18 PM, Kuntal Ghosh wrote: > On Wed, Jul 5, 2017 at 9:35 AM, Michael Paquier > wrote: >> On Tue, Jul 4, 2017 at 4:41 PM, Kuntal Ghosh >> wrote: >>> I've not yet started the patch and it may take some time for me to >>> understand and write >>> the patch in a correct way

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 5:04 PM, AP wrote: > On Thu, Jul 06, 2017 at 12:38:38PM +0530, Amit Kapila wrote: >> On Thu, Jul 6, 2017 at 9:32 AM, AP wrote: >> > On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: >> >> On Thu, Jul 6, 2017 at 2:40 AM, AP wrote: >> >> > On Wed, Jul 05, 2017 at

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 3:24 PM, Kyotaro HORIGUCHI wrote: > Hello, > > At Tue, 4 Jul 2017 14:49:26 +0530, Amit Kapila > wrote in >> On Tue, Jul 4, 2017 at 1:23 PM, Michael Paquier >> wrote: >> > On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila >> > wrote: >> > >> > It seems to me that this qualifi

Re: [HACKERS] Add support for tuple routing to foreign partitions

2017-07-06 Thread Etsuro Fujita
On 2017/07/05 9:13, Amit Langote wrote: On 2017/06/29 20:20, Etsuro Fujita wrote: In relation to this, I also allowed expand_inherited_rtentry() to build an RTE and AppendRelInfo for each partition of a partitioned table that is an INSERT target, as mentioned by Amit in [1], by modifying trans

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Kuntal Ghosh
On Wed, Jul 5, 2017 at 9:35 AM, Michael Paquier wrote: > On Tue, Jul 4, 2017 at 4:41 PM, Kuntal Ghosh > wrote: >> I've not yet started the patch and it may take some time for me to >> understand and write >> the patch in a correct way. Since, you've almost written the patch, >> IMHO, please go a

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Kyotaro HORIGUCHI
FWIW.. At Thu, 06 Jul 2017 18:54:47 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170706.185447.256482539.horiguchi.kyot...@lab.ntt.co.jp> > > > + /* > > > +* Force the on-disk state of init forks to always be in sync > > > with the > > > +* state in shared b

Re: [HACKERS] increasing the default WAL segment size

2017-07-06 Thread Beena Emerson
On Thu, Jul 6, 2017 at 3:21 PM, tushar wrote: > On 07/06/2017 12:04 PM, Beena Emerson wrote: >> >> The 04-initdb-walsegsize_v2.patch has the following improvements: >> - Rebased over new 03 patch >> - Pass the wal-segsize intidb option as command-line option rathern >> than in an environment varia

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Kyotaro HORIGUCHI
Hello, At Tue, 4 Jul 2017 14:49:26 +0530, Amit Kapila wrote in > On Tue, Jul 4, 2017 at 1:23 PM, Michael Paquier > wrote: > > On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila wrote: > > > > It seems to me that this qualifies as an open item for 10. WAL-logging > > is new for hash tables. Amit, cou

Re: [HACKERS] increasing the default WAL segment size

2017-07-06 Thread tushar
On 07/06/2017 12:04 PM, Beena Emerson wrote: The 04-initdb-walsegsize_v2.patch has the following improvements: - Rebased over new 03 patch - Pass the wal-segsize intidb option as command-line option rathern than in an environment variable. - Since new function check_wal_size had only had two chec

Re: [HACKERS] Multi column range partition table

2017-07-06 Thread Dean Rasheed
On 5 July 2017 at 10:43, Amit Langote wrote: > 0001 is your patch to tidy up check_new_partition_bound() (must be > applied before 0002) > I pushed this first patch, simplifying check_new_partition_bound() for range partitions, since it seemed like a good simplification, but note that I don't th

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-06 Thread Kuntal Ghosh
On Thu, Jul 6, 2017 at 3:45 AM, Tom Lane wrote: > I wrote: >> (Pokes at it some more...) Oh, interesting: it behaves that way except >> when p is exactly the lowest histogram entry. > + /* + * In the first bin (i==1), add a fudge factor that ensures + * that histfrac is at least eq_selec. We do t

[HACKERS] Another comment typo in execMain.c

2017-07-06 Thread Etsuro Fujita
Here is a comment in ExecFindPartition() in execMain.c: /* * First check the root table's partition constraint, if any. No point in * routing the tuple it if it doesn't belong in the root table itself. */ I think that in the second sentence "it" just before "if" is a typo.

[HACKERS] Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-06 Thread Etsuro Fujita
Here is an example: postgres=# create table col_desc (a int, b int) partition by list (a); postgres=# create table col_desc_1 partition of col_desc for values in (1); postgres=# alter table col_desc_1 add check (b > 0); postgres=# create role col_desc_user; postgres=# grant insert on col_desc to

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 9:32 AM, AP wrote: > On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: >> On Thu, Jul 6, 2017 at 2:40 AM, AP wrote: >> > On Wed, Jul 05, 2017 at 05:52:32PM +0530, Amit Kapila wrote: >> >> >> > version | bucket_pages | overflow_pages | bitmap_pages | >> >> >> >

Re: [HACKERS] Update comment in ExecPartitionCheck

2017-07-06 Thread Etsuro Fujita
On 2017/07/04 18:15, Amit Langote wrote: On 2017/07/04 17:55, Etsuro Fujita wrote: This comment in an error handling in ExecPartitionCheck(): if (!ExecCheck(resultRelInfo->ri_PartitionCheckExpr, econtext)) { char *val_desc; Relationorig_rel = rel;