Expression based grouping equality implementation

2017-11-29 Thread Andres Freund
Hi, Similar to [1] (Expression based aggregate transition / combine function invocation), this patch provides small-medium performance benefits in order to later enable larger performance benefits with JIT compilation. What this basically does is to move execGrouping.c functions that compare tupl

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

2017-11-29 Thread Amit Kapila
On Tue, Nov 28, 2017 at 9:42 PM, Robert Haas wrote: > On Tue, Nov 28, 2017 at 2:23 AM, Amit Kapila wrote: >> That is wrong and I think you have hit a bug. It should be 2974 * 5 = >> 14870 as you have seen in other cases. The problem is that during >> rescan, we generally reinitialize the requir

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

2017-11-29 Thread Amit Kapila
On Tue, Nov 28, 2017 at 9:37 PM, Robert Haas wrote: > On Mon, Nov 27, 2017 at 6:54 AM, Amit Kapila wrote: >>> Anything "below" "Gather"? >>> >> I think it is "actual_time * 1" for anything below Gather. > > The actual time amounts below Gather show total elapsed time divided > by loop count, just

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-29 Thread Fabien COELHO
Hello, This patch enables building pgbench to use ppoll() instead of select() to allow for more than (FD_SETSIZE - 10) connections. As implemented, when using ppoll(), the only connection limitation is system resources. I'm fine with allowing more clients through ppoll, as large multi/many/

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2017-11-29 Thread amul sul
On Wed, Nov 29, 2017 at 7:51 AM, Amit Kapila wrote: > On Tue, Nov 28, 2017 at 5:58 PM, amul sul wrote: >> On Sat, Nov 25, 2017 at 11:39 AM, Amit Kapila >> wrote: >>> On Thu, Nov 23, 2017 at 5:18 PM, amul sul wrote: On Sat, Nov 11, 2017 at 1:05 AM, Robert Haas wrote: > On Wed, Sep 27,

Re: [HACKERS] postgres_fdw super user checks

2017-11-29 Thread Ashutosh Bapat
On Wed, Nov 29, 2017 at 4:56 AM, Stephen Frost wrote: > > Just to make it clear, I continue to agree with (3) and agree with Tom > that we shouldn't be behaving differently depending on who is calling > the view. I also would vote for 3. That looks consistent with the way we handle accesses base

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-11-29 Thread Tomas Vondra
On 11/29/2017 06:13 AM, Michael Paquier wrote: > On Tue, Nov 21, 2017 at 7:07 AM, Alexander Korotkov > wrote: >> On Mon, Nov 20, 2017 at 1:59 PM, Alexander Korotkov >> wrote: >>> >>> On Mon, Nov 20, 2017 at 4:09 AM, Tomas Vondra >>> wrote: Seems fine to me, although perhaps it should

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-11-29 Thread Alexander Korotkov
On Wed, Nov 29, 2017 at 1:30 PM, Tomas Vondra wrote: > On 11/29/2017 06:13 AM, Michael Paquier wrote: > > On Tue, Nov 21, 2017 at 7:07 AM, Alexander Korotkov > > wrote: > >> On Mon, Nov 20, 2017 at 1:59 PM, Alexander Korotkov > >> wrote: > >>> > >>> On Mon, Nov 20, 2017 at 4:09 AM, Tomas Vondra

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-11-29 Thread Andrey Borodin
> 29 нояб. 2017 г., в 15:59, Alexander Korotkov > написал(а): > > > Sure, patch got some review. I've no objection against moving this to the > next commitfest though. > Since, these patches include bug fix, it's possible that someone will commit > it before next commitfest. Hi! I've took

Re: [HACKERS] Runtime Partition Pruning

2017-11-29 Thread Beena Emerson
Hello Rajkumar, On Tue, Nov 14, 2017 at 2:22 PM, Rajkumar Raghuwanshi wrote: > On Tue, Nov 14, 2017 at 11:46 AM, Beena Emerson > wrote: >> >> PFA the updated patches. > > > Hi, > > I have started testing this along with fast pruning. It is crashing for sql > with subqueries. > one to test case

Re: [HACKERS] Runtime Partition Pruning

2017-11-29 Thread Beena Emerson
Hello, PFA the new version of the patch which can be applied over v11 patches of Amit Langote [1]. The patch has been completely modified and the 0001 patch of previous series is no longer required. As mentioned above, I have used the PartitionDispatchInfo and an array to which holds the actual su

Re: [HACKERS] Runtime Partition Pruning

2017-11-29 Thread Beena Emerson
On Tue, Nov 14, 2017 at 6:27 PM, David Rowley wrote: > On 14 November 2017 at 19:16, Beena Emerson wrote: >> PFA the updated patches. > > Hi Beena, > > Thanks for working on this. I've had a look at the patch to try to > understand how it is working. I found it a bit surprising that the > code as

Re: [HACKERS] Runtime Partition Pruning

2017-11-29 Thread Beena Emerson
Hello, On Wed, Nov 15, 2017 at 4:43 AM, David Rowley wrote: > On 15 November 2017 at 01:57, David Rowley > wrote: >> I think to do this you're going to have to store some sort of array >> that maps the partition index to the subpath in the Append node so you >> can correctly identify the subpat

Re: Skip index cleanup if autovacuum did not do any work

2017-11-29 Thread Feike Steenbergen
On 28 November 2017 at 23:17, Peter Geoghegan wrote: > BTW, a good short term solution for you might be to change the vacuum > cost delay settings. They're pretty conservative by default. > > There is a good chance that your indexes are mostly in memory even on > large tables, and B-Tree indexes a

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-11-29 Thread Alexander Korotkov
Hi! On Wed, Nov 29, 2017 at 2:32 PM, Andrey Borodin wrote: > 29 нояб. 2017 г., в 15:59, Alexander Korotkov > написал(а): > > > Sure, patch got some review. I've no objection against moving this to the > next commitfest though. > Since, these patches include bug fix, it's possible that someone

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-11-29 Thread David Steele
On 11/29/17 12:46 AM, Michael Paquier wrote:> On Wed, Nov 1, 2017 at 5:58 PM, Chris Travers wrote: > > Please note that I am still -1 for using a methodology different than > what is used for base backups with an inclusive method, and would much > prefer an exclusive method by reusing the existing

es_query_dsa is broken

2017-11-29 Thread Thomas Munro
Hi hackers, While reviewing commit c6755e23 I realised that es_query_dsa is broken. It might have made some kind of sense as a name and a concept in an earlier version of the proposal to add a DSA area for parallel query's use, when the DSA area was going to be independent of parallel query DSM s

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-11-29 Thread Michael Paquier
On Wed, Nov 29, 2017 at 7:59 PM, Alexander Korotkov wrote: > Sure, patch got some review. I've no objection against moving this to the > next commitfest though. Please note that as this is qualified as a bug fix, I was not going to mark it as returned with feedback or such. We want to keep track

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 3:40 AM, Fabien COELHO wrote: > ever is best. Not sure that "pfds" is the right name. If the two variables > means the same thing, they should have the same name, although possibly > different types. Although I agree with a good bit of what you say here, I don't agree with

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-29 Thread Fabien COELHO
Hello Robert, ever is best. Not sure that "pfds" is the right name. If the two variables means the same thing, they should have the same name, although possibly different types. Although I agree with a good bit of what you say here, I don't agree with that. If the member used by ppoll() (or

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-11-29 Thread Simon Riggs
On 25 September 2017 at 22:34, Kyotaro HORIGUCHI wrote: >> > Here is a small patch that skips scanning btree index if no pending >> > deleted pages exists. >> > It detects this situation by comparing pages_deleted with pages_free. > > It seems to work to prevent needless cleanup scans. So this l

Re: [HACKERS] postgres_fdw super user checks

2017-11-29 Thread Stephen Frost
Ashutosh, * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > On Wed, Nov 29, 2017 at 4:56 AM, Stephen Frost wrote: > > The "global rethink" being contemplated seems to be more about > > authentication forwarding than it is about this specific change. If > > there's some 'global rethink'

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 8:10 AM, Fabien COELHO wrote: > My point is consistent with my other advice which is to hide the stuff in > functions with identical (or compatible) signatures, so that the only place > where it would differ would be in the functions, where greping would work. > > #ifdef U

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Peter Eisentraut
On 11/29/17 00:35, Michael Paquier wrote: > On Fri, Sep 1, 2017 at 8:28 AM, Robert Haas wrote: >> On Thu, Aug 31, 2017 at 6:37 PM, Tom Lane wrote: >>> Meh. We support ancient versions of C for backwards compatibility >>> reasons, but considering that compiling backend code with C++ isn't >>> off

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2017-11-29 Thread Fabien COELHO
[...] Yeah, that sort of style would be OK with me. But I wouldn't like: struct blah { #ifdef FOO int doohicky; #else char *doohicky; }; Indeed. Me neither. -- Fabien.

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 9:26 AM, Peter Eisentraut wrote: > I'd still like a review of this patch. I don't think there's much to review apart from this one issue. Neither Tom nor I seem to be convinced about: +/* not worth providing a workaround */ I suggested that it was worth providing a worka

Re: new function for tsquery creartion

2017-11-29 Thread Victor Drobny
On 2017-11-28 17:57, Aleksander Alekseev wrote: Hi Aleksander, Thank you for review. I have tried to fix all of your comments. However i want to mention that the absence of comments for functions in to_tsany.c is justified by the absence of comments for other similar functions. Hi Victor, I lik

Re: new function for tsquery creartion

2017-11-29 Thread Victor Drobny
On 2017-11-29 17:56, Victor Drobny wrote: Sorry, forgot to attach new version of the patch. On 2017-11-28 17:57, Aleksander Alekseev wrote: Hi Aleksander, Thank you for review. I have tried to fix all of your comments. However i want to mention that the absence of comments for functions in to_t

Re: es_query_dsa is broken

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 7:30 AM, Thomas Munro wrote: > While reviewing commit c6755e23 I realised that es_query_dsa is > broken. It might have made some kind of sense as a name and a concept > in an earlier version of the proposal to add a DSA area for parallel > query's use, when the DSA area wa

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 29, 2017 at 9:26 AM, Peter Eisentraut > wrote: >> I'd still like a review of this patch. > I don't think there's much to review apart from this one issue. > Neither Tom nor I seem to be convinced about: > +/* not worth providing a workaround */ > I suggested tha

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Robert Haas
On Tue, Nov 28, 2017 at 1:59 PM, Sergei Kornilov wrote: > I write patch to speed up ALTER TABLE SET NOT NULL by check existed check > constraints or indexes. Huge phase 3 with verify table data will be skipped > if table has valid check constraint cover "alteredfield IS NOT NULL" > condition or

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-11-29 Thread Masahiko Sawada
On Wed, Nov 29, 2017 at 11:05 PM, Simon Riggs wrote: > On 25 September 2017 at 22:34, Kyotaro HORIGUCHI > wrote: > >>> > Here is a small patch that skips scanning btree index if no pending >>> > deleted pages exists. >>> > It detects this situation by comparing pages_deleted with pages_free. >> >

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Nov 28, 2017 at 1:59 PM, Sergei Kornilov wrote: > > I write patch to speed up ALTER TABLE SET NOT NULL by check existed check > > constraints or indexes. Huge phase 3 with verify table data will be skipped > > if table has valid che

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Petr Jelinek
Hi, (sorry for not being active here, I am still catching up after being away for some family issues) On 16/11/17 21:12, Robert Haas wrote: > On Thu, Nov 16, 2017 at 2:41 PM, Andres Freund wrote: >>> To me, it seems like SnapBuildWaitSnapshot() is fundamentally >>> misdesigned >> >> Maybe I'm co

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Tom Lane
Stephen Frost writes: > Isn't the first concern addressed by using SPI..? I did not look at the patch yet, but TBH if it uses SPI for sub-operations of ALTER TABLE I think that is sufficient reason to reject it out of hand. Doing things that way would create way too much of a vulnerability surfac

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Sergei Kornilov
Thanks all for reply! Robert, > Doing it based on an index scan doesn't necessarily seem like a good idea. We > have > no guarantee at all that the index scan will be faster than scanning > the table would have been I agree this. Thinking a little about idea of index scan i can not give reason

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Isn't the first concern addressed by using SPI..? > > I did not look at the patch yet, but TBH if it uses SPI for sub-operations > of ALTER TABLE I think that is sufficient reason to reject it out of hand. You mean like wha

Re: [HACKERS] Walsender timeouts and large transactions

2017-11-29 Thread Petr Jelinek
Hi, On 17/11/17 08:35, Kyotaro HORIGUCHI wrote: > > Moving around the code allow us to place ps_is_send_pending() in > the while condition, which seems to be more proper place to do > that. I haven't added test for this particular case. > > I tested this that > > - cleanly applies on the curren

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 10:52 AM, Sergei Kornilov wrote: > I agree this. Thinking a little about idea of index scan i can not give > reasonable usecase which required index. My target problem of adding NOT NULL > to big relation without long downtime can be done with ADD CONSTRAINT NOT > VALID,

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I did not look at the patch yet, but TBH if it uses SPI for sub-operations >> of ALTER TABLE I think that is sufficient reason to reject it out of hand. > You mean like what ALTER TABLE ... ADD FOREIGN KEY does? Yeah, and if you l

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Nov 29, 2017 at 10:52 AM, Sergei Kornilov wrote: > > I agree this. Thinking a little about idea of index scan i can not give > > reasonable usecase which required index. My target problem of adding NOT > > NULL to big relation witho

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Sergei Kornilov
Here is new patch with check only existed valid constraints and without SPI at all. Thanksdiff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d979ce2..7ab7580 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -370,6 +370,8 @

Re: using index or check in ALTER TABLE SET NOT NULL

2017-11-29 Thread Andres Freund
On November 29, 2017 8:50:31 AM PST, Stephen Frost wrote: >As for conflicting snapshots, isn't the lock we're taking already >AccessExclusive..? Doesn't help if e.g. the current xact is repeatable read or if your own xact deleted things (other xacts with snapshots could still see null rows, de

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-11-29 Thread Shubham Barai
On 27 November 2017 at 13:17, Alexander Korotkov wrote: > Hi, Shubham! > > On Wed, Nov 1, 2017 at 12:10 AM, Shubham Barai > wrote: > >> On 9 October 2017 at 18:57, Alexander Korotkov > > wrote: >> >>> Now, ITSM that predicate locks and conflict checks are placed right for >>> now. >>> However, i

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Andres Freund
On 2017-11-29 09:41:15 -0500, Robert Haas wrote: > On Wed, Nov 29, 2017 at 9:26 AM, Peter Eisentraut > wrote: > > I'd still like a review of this patch. > > I don't think there's much to review apart from this one issue. > Neither Tom nor I seem to be convinced about: > > +/* not worth providing

Re: simplehash: tb->sizemask = 0

2017-11-29 Thread Andres Freund
On 2017-11-27 22:53:41 -0500, Tom Lane wrote: > Tomas Vondra writes: > > I'm a bit puzzled by this code in SH_COMPUTE_PARAMETERS: > > > if (tb->size == SH_MAX_SIZE) > > tb->sizemask = 0; > > else > > tb->sizemask = tb->size - 1; > > > Doesn't that mean that with SH_MAX_SI

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Stas Kelvich
> On 29 Nov 2017, at 18:46, Petr Jelinek wrote: > > What I don't understand is how it leads to crash (and I could not > reproduce it using the pgbench file attached in this thread either) and > moreover how it leads to 0 xid being logged. The only explanation I can > come up is that some kind of

Re: [HACKERS] SQL procedures

2017-11-29 Thread Andrew Dunstan
On 11/28/2017 10:03 AM, Peter Eisentraut wrote: > Here is a new patch that addresses the previous review comments. > > If there are no new comments, I think this might be ready to go. > Looks good to me. Marking ready for committer. cheers andrew -- Andrew Dunstanhttps://www

Unclear regression test for postgres_fdw

2017-11-29 Thread Antonin Houska
The following test -- Input relation to aggregate push down hook is not safe to pushdown and thus -- the aggregate cannot be pushed down to foreign server. explain (verbose, costs off) select count(t1.c3) from ft1 t1, ft1 t2 where t1.c1 = postgres_fdw_abs(t1.c2); produces the following plan

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Robert Haas
On Wed, Nov 22, 2017 at 3:59 AM, Amit Langote wrote: > It seems I wrote an Assert in the code to support hash partitioning that > wasn't based on a valid assumption. I was wrongly assuming that all hash > partitions for a given modulus (largest modulus) must exist at any given > time, but that is

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Tom Lane
Andres Freund writes: > On 2017-11-29 09:41:15 -0500, Robert Haas wrote: >> +/* not worth providing a workaround */ > FWIW, I think that's a perfectly reasonable choice. Adding complications > in making static assertions work for random archaic compilers when > compiling with c++ just doesn't see

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Andres Freund
On 2017-11-29 16:39:14 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2017-11-29 09:41:15 -0500, Robert Haas wrote: > >> +/* not worth providing a workaround */ > > > FWIW, I think that's a perfectly reasonable choice. Adding complications > > in making static assertions work for random ar

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Robert Haas
On Wed, Nov 29, 2017 at 3:28 PM, Robert Haas wrote: > On Wed, Nov 22, 2017 at 3:59 AM, Amit Langote > wrote: >> It seems I wrote an Assert in the code to support hash partitioning that >> wasn't based on a valid assumption. I was wrongly assuming that all hash >> partitions for a given modulus (

Re: [HACKERS] postgres_fdw bug in 9.6

2017-11-29 Thread Tom Lane
Ashutosh Bapat writes: > On Tue, Nov 28, 2017 at 11:05 PM, Robert Haas wrote: >> On Tue, Nov 28, 2017 at 11:21 AM, Tom Lane wrote: >>> In short, we should get rid of all of this expensive and broken logic and >>> just make EPQ recheck on a foreign join be a no-op, just as it is for a >>> foreign

Re: [HACKERS] static assertions in C++

2017-11-29 Thread Tom Lane
Andres Freund writes: > On 2017-11-29 16:39:14 -0500, Tom Lane wrote: >> Andres Freund writes: >>> FWIW, I think that's a perfectly reasonable choice. Adding complications >>> in making static assertions work for random archaic compilers when >>> compiling with c++ just doesn't seem worth more th

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

2017-11-29 Thread Daniel Gustafsson
> On 29 Nov 2017, at 04:29, Michael Paquier wrote: > > On Tue, Nov 28, 2017 at 10:07 AM, Michael Paquier > wrote: >> I was just looking at the tsearch code which uses pg_strcmpcase, and >> those are defined with makeDefElem() so you should switch to strcmp in >> this case as well, no? If I patch

Would a BGW need shmem_access or database_connection to enumerate databases?

2017-11-29 Thread Chapman Flack
I'm thinking of writing a background worker that will enumerate the databases present, and spin off, for each one, another BGW that will establish a connection and do stuff. For the "master" one, what capabilities will it need to simply enumerate the current names of known databases? I suppose I c

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

2017-11-29 Thread Michael Paquier
On Thu, Nov 30, 2017 at 7:40 AM, Daniel Gustafsson wrote: >> On 29 Nov 2017, at 04:29, Michael Paquier wrote: >> >> On Tue, Nov 28, 2017 at 10:07 AM, Michael Paquier >> wrote: >>> I was just looking at the tsearch code which uses pg_strcmpcase, and >>> those are defined with makeDefElem() so you

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

2017-11-29 Thread Michael Paquier
On Thu, Nov 30, 2017 at 7:48 AM, Chapman Flack wrote: > For the "master" one, what capabilities will it need to simply > enumerate the current names of known databases? I suppose I could > have it connect to the null dbname and query pg_database. Would > that be the civilized way to do it, or am I

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

2017-11-29 Thread Chapman Flack
On 11/29/2017 05:54 PM, Michael Paquier wrote: > Yes. That's actually what the autovacuum launcher does. It connects > using InitPostgres(NULL, InvalidOid, NULL, NULL), and then scans > pg_database to fetch a list (see get_database_list). Thanks! It looks like if get_database_list were not static

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Petr Jelinek
On 29/11/17 20:11, Stas Kelvich wrote: > >> On 29 Nov 2017, at 18:46, Petr Jelinek wrote: >> >> What I don't understand is how it leads to crash (and I could not >> reproduce it using the pgbench file attached in this thread either) and >> moreover how it leads to 0 xid being logged. The only exp

Re: Would a BGW need shmem_access or database_connection to enumerate databases?

2017-11-29 Thread Andres Freund
On 2017-11-29 18:23:40 -0500, Chapman Flack wrote: > On 11/29/2017 05:54 PM, Michael Paquier wrote: > > > Yes. That's actually what the autovacuum launcher does. It connects > > using InitPostgres(NULL, InvalidOid, NULL, NULL), and then scans > > pg_database to fetch a list (see get_database_list)

Re: [HACKERS] logical decoding of two-phase transactions

2017-11-29 Thread Petr Jelinek
Hi, On 24/11/17 07:41, Craig Ringer wrote: > On 24 November 2017 at 13:44, Nikhil Sontakke   > > > How practical is adding a lock class? > > Am open to suggestions. This looks like it could work decently. > > > It looks amazingly simple from here. Which probably means there's more > t

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Andres Freund
On 2017-11-30 00:25:58 +0100, Petr Jelinek wrote: > Yes that helps thanks. Now that I reproduced it I understand, I was > confused by the backtrace that said xid was 0 on the input to > XactLockTableWait() but that's not the case, it's what xid is changed to > in the inner loop. > So what happens

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Petr Jelinek
On 30/11/17 00:40, Andres Freund wrote: > On 2017-11-30 00:25:58 +0100, Petr Jelinek wrote: >> Yes that helps thanks. Now that I reproduced it I understand, I was >> confused by the backtrace that said xid was 0 on the input to >> XactLockTableWait() but that's not the case, it's what xid is change

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Andres Freund
On 2017-11-30 00:45:44 +0100, Petr Jelinek wrote: > I don't understand. I mean sure the SnapBuildWaitSnapshot() can live > with it, but the problematic logic happens inside the > XactLockTableInsert() and SnapBuildWaitSnapshot() has no way of > detecting the situation short of reimplementing the >

Use of uninitialized variables in ExecFindPartition() for parent partition without leaves (HEAD only)

2017-11-29 Thread Michael Paquier
Hi all, Since commit 4e5fe9ad (committer Robert Haas and author Amit Langote), coverity has been complaining that the new code of ExecFindPartition() may use a set of values and isnull values which never get initialized. This is a state which can be easily reached with the following SQLs of a pare

Re: [HACKERS] Issues with logical replication

2017-11-29 Thread Petr Jelinek
On 30/11/17 00:47, Andres Freund wrote: > On 2017-11-30 00:45:44 +0100, Petr Jelinek wrote: >> I don't understand. I mean sure the SnapBuildWaitSnapshot() can live >> with it, but the problematic logic happens inside the >> XactLockTableInsert() and SnapBuildWaitSnapshot() has no way of >> detectin

Re: [HACKERS] postgres_fdw: Add support for INSERT OVERRIDING clause

2017-11-29 Thread Michael Paquier
On Wed, Nov 29, 2017 at 1:53 PM, Michael Paquier wrote: > On Wed, Nov 29, 2017 at 8:12 AM, Tom Lane wrote: >> IIRC, this issue was debated at great length back when we first put >> in foreign tables, because early drafts of postgres_fdw did what you >> propose here, and we ran into very nasty pro

Re: [HACKERS] JIT compiling expressions/deform + inlining prototype v2.0

2017-11-29 Thread Michael Paquier
On Thu, Sep 21, 2017 at 2:52 AM, Andres Freund wrote: > On 2017-09-19 12:57:33 +0300, Konstantin Knizhnik wrote: >> >> >> On 04.09.2017 23:52, Andres Freund wrote: >> > >> > Hi. That piece of code isn't particularly clear (and has a bug in the >> > submitted version), I'm revising it. >> >> ... >>

Re: Use of uninitialized variables in ExecFindPartition() for parent partition without leaves (HEAD only)

2017-11-29 Thread Amit Langote
Hi Michael. On 2017/11/30 9:07, Michael Paquier wrote: > Hi all, > > Since commit 4e5fe9ad (committer Robert Haas and author Amit Langote), > coverity has been complaining that the new code of ExecFindPartition() > may use a set of values and isnull values which never get initialized. > This is a

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-11-29 Thread Michael Paquier
On Fri, Aug 25, 2017 at 6:25 PM, David Rowley wrote: > I just had a quick glance over this and wondered about 2 things. > > 1. Why a GUC and not a new per user option so it can be configured > differently for different users? Something like ALTER USER ... WORKER > LIMIT ; perhaps. I mentioned abou

Re: Use of uninitialized variables in ExecFindPartition() for parent partition without leaves (HEAD only)

2017-11-29 Thread Amit Langote
On 2017/11/30 10:11, Amit Langote wrote: > in the attached updated version. Oops, I messed up taking the diff and mistakenly added noise to the patch. Fixed in the attached. Thanks, Amit diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 2fc411a9b5..8

Re: [HACKERS] Parallel Hash take II

2017-11-29 Thread Thomas Munro
On Mon, Nov 27, 2017 at 10:25 PM, Thomas Munro wrote: > On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro > wrote: >> Here's a new patch set with responses to the last batch of review comments. > > Rebased on top of the recent SGML->XML change. Andres asked me off-list how I tested the barrier.c ca

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-11-29 Thread Michael Paquier
On Wed, Jun 28, 2017 at 9:58 AM, Thomas Munro wrote: > Rebased for the recent re-indent and shm_toc API change; no functional > changes in this version. > > (I have a new patch set in the pipeline adding the skew optimisation > and some other things, more on that soon.) This patch does not apply.

Re: Protect syscache from bloating with negative cache entries

2017-11-29 Thread Michael Paquier
On Tue, Oct 31, 2017 at 6:46 PM, Kyotaro HORIGUCHI wrote: > This is a rebased version of the patch. As far as I can see, the patch still applies, compiles, and got no reviews. So moved to next CF. -- Michael

Re: [HACKERS] Parallel Append implementation

2017-11-29 Thread Michael Paquier
On Tue, Nov 28, 2017 at 8:02 PM, amul sul wrote: > Apart from this I have added few assert to keep eye on node->as_whichplan > value in the attached patch, thanks. This is still hot, moved to next CF. -- Michael

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-11-29 Thread Thomas Munro
On Thu, Nov 30, 2017 at 2:20 PM, Michael Paquier wrote: > This patch does not apply. And the thread has stalled for three months > now but I cannot see a review for what has been submitted. I am moving > it to next CF with waiting on author. Please provide a rebased > version. If there are other t

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

2017-11-29 Thread Michael Paquier
On Tue, Sep 12, 2017 at 9:48 PM, Tom Lane wrote: > Jim Nasby writes: >> I've verified that the patch still applies and make check-world is clean. > > Not any more :-(. Here's a v3 rebased over HEAD. No substantive > change from v2. Patch applies and compiles, and it got no reviews. Moved to CF

Re: [HACKERS] SERIALIZABLE with parallel query

2017-11-29 Thread Michael Paquier
On Fri, Nov 24, 2017 at 1:06 PM, Haribabu Kommi wrote: > The latest patch is good. It lacks a test that verifies the serialize > support with actual parallel workers, so in case if it broken, it is > difficult to know. Could this question be answered? The patch still applies so I am moving it to

Re: [HACKERS] [PATCH] Incremental sort (was: PoC: Partial sort)

2017-11-29 Thread Michael Paquier
On Mon, Mar 20, 2017 at 6:33 PM, Alexander Korotkov wrote: > Thank you for the report. > Please, find rebased patch in the attachment. This patch cannot be applied. Please provide a rebased version. I am moving it to next CF with waiting on author as status. -- Michael

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Amit Langote
On 2017/11/30 5:28, Robert Haas wrote: > On Wed, Nov 22, 2017 at 3:59 AM, Amit Langote > wrote: >> It seems I wrote an Assert in the code to support hash partitioning that >> wasn't based on a valid assumption. I was wrongly assuming that all hash >> partitions for a given modulus (largest modulu

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Amit Langote
On 2017/11/30 7:15, Robert Haas wrote: > On Wed, Nov 29, 2017 at 3:28 PM, Robert Haas wrote: >> On Wed, Nov 22, 2017 at 3:59 AM, Amit Langote >> wrote: >>> It seems I wrote an Assert in the code to support hash partitioning that >>> wasn't based on a valid assumption. I was wrongly assuming that

Re: [HACKERS] SERIALIZABLE with parallel query

2017-11-29 Thread Thomas Munro
On Thu, Nov 30, 2017 at 2:32 PM, Michael Paquier wrote: > On Fri, Nov 24, 2017 at 1:06 PM, Haribabu Kommi > wrote: >> The latest patch is good. It lacks a test that verifies the serialize >> support with actual parallel workers, so in case if it broken, it is >> difficult to know. > > Could this

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-11-29 Thread Michael Paquier
On Wed, Nov 29, 2017 at 5:33 AM, Robert Haas wrote: > On Sun, Nov 26, 2017 at 9:33 PM, Masahiko Sawada > wrote: >> Attached latest patch incorporated all comments so far. Please review it. > > I think you only need RelExtLockReleaseAllI() where we currently have > LockReleaseAll(DEFAULT_LOCKMETH

Re: [HACKERS] PoC: full merge join on comparison clause

2017-11-29 Thread Michael Paquier
On Mon, Oct 30, 2017 at 9:25 PM, Alexander Kuzmenkov wrote: > I am attaching the updated patch, rebased to 820c03. (Please avoid top-posting) This patch has rotten and conflicts with recent changes in joinrels.c. This did not get any reviews, so I am moving it to next CF with "waiting on author"

Re: [HACKERS] Cached plans and statement generalization

2017-11-29 Thread Michael Paquier
On Wed, Sep 13, 2017 at 2:11 AM, Konstantin Knizhnik wrote: > One more patch passing all regression tests with autoprepare_threshold=1. > I still do not think that it should be switch on by default... This patch does not apply, and did not get any reviews. So I am moving it to next CF with waitin

Re: [HACKERS] Surjective functional indexes

2017-11-29 Thread Michael Paquier
On Wed, Sep 27, 2017 at 4:07 PM, Simon Riggs wrote: > On 15 September 2017 at 16:34, Konstantin Knizhnik > wrote: > >> Attached please find yet another version of the patch. > > Thanks. I'm reviewing it. Two months later, this patch is still waiting for a review (you are listed as well as a revi

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-29 Thread Michael Paquier
On Sun, Nov 26, 2017 at 5:15 PM, Amit Kapila wrote: > Yeah and I think something like that can happen after your patch > because now the memory for tuples returned via TupleQueueReaderNext > will be allocated in ExecutorState and that can last for long. I > think it is better to free memory, but

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-11-29 Thread Masahiko Sawada
On Thu, Nov 30, 2017 at 10:52 AM, Michael Paquier wrote: > On Wed, Nov 29, 2017 at 5:33 AM, Robert Haas wrote: >> On Sun, Nov 26, 2017 at 9:33 PM, Masahiko Sawada >> wrote: >>> Attached latest patch incorporated all comments so far. Please review it. >> >> I think you only need RelExtLockReleas

Re: [HACKERS] <> join selectivity estimate question

2017-11-29 Thread Michael Paquier
On Thu, Sep 14, 2017 at 2:23 PM, Ashutosh Bapat wrote: > Are you referring to rounding errors? We should probably add some fuzz > factor to cover the rounding errors and cause a diff when difference > in expected and reported plan rows is beyond that fuzz factor. As far as I can see the patch pro

RE: [HACKERS] Cached plans and statement generalization

2017-11-29 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:michael.paqu...@gmail.com] > This patch does not apply, and did not get any reviews. So I am moving it > to next CF with waiting on author as status. Please provide a rebased version. > Tsunakawa-san, you are listed as a reviewer of this patch. If you are not > plannin

Re: [HACKERS] Block level parallel vacuum WIP

2017-11-29 Thread Michael Paquier
On Tue, Oct 24, 2017 at 5:54 AM, Masahiko Sawada wrote: > Yeah, I was thinking the commit is relevant with this issue but as > Amit mentioned this error is emitted by DROP SCHEMA CASCASE. > I don't find out the cause of this issue yet. With the previous > version patch, autovacuum workers were wok

Re: [HACKERS] Cached plans and statement generalization

2017-11-29 Thread Michael Paquier
On Thu, Nov 30, 2017 at 11:07 AM, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:michael.paqu...@gmail.com] >> This patch does not apply, and did not get any reviews. So I am moving it >> to next CF with waiting on author as status. Please provide a rebased >> version. >> Tsunakawa-sa

Re: [HACKERS] Partition-wise aggregation/grouping

2017-11-29 Thread Michael Paquier
On Tue, Nov 28, 2017 at 5:50 PM, Jeevan Chalke wrote: > [snip] This is still a hot topic so I am moving it to next CF. -- Michael

Re: [HACKERS] Secondary index access optimizations

2017-11-29 Thread Michael Paquier
On Mon, Nov 6, 2017 at 10:13 PM, Konstantin Knizhnik wrote: > Concerning broken partition_join test: it is "expected" failure: my patch > removes from the plans redundant checks. > So the only required action is to update expected file with results. > Attached please find updated patch. The last

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Michael Paquier
On Thu, Nov 30, 2017 at 10:43 AM, Amit Langote wrote: > I'm working on a revised version of these patches to address recent > comments by Horiguchi-san. I will also consider the points above before > sending the new version. Ok, this is fresh news, so I am moving this entry to next CF with waiti

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-11-29 Thread Michael Paquier
On Tue, Aug 22, 2017 at 9:17 PM, Sokolov Yura wrote: > Simplified a bit and more commented patch version is in attach. > > Algorithm were switched to linear probing, it makes code simpler and > clearer. > Flag usages were toggled: now it indicates that hash table were built, > it also makes code c

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2017-11-29 Thread Michael Paquier
On Thu, Nov 9, 2017 at 12:33 PM, Ashutosh Bapat wrote: > Looking at order_qual_clauses(), we can say that a set of quals q1 > qn are ordered the same irrespective of the set of clauses they > are subset of. E.g. if {q1 .. qn} is subset of Q (ordered as Qo) and > also Q' (ordered as Q'o) the o

Re: [HACKERS] path toward faster partition pruning

2017-11-29 Thread Amit Langote
On 2017/11/30 11:18, Michael Paquier wrote: > On Thu, Nov 30, 2017 at 10:43 AM, Amit Langote > wrote: >> I'm working on a revised version of these patches to address recent >> comments by Horiguchi-san. I will also consider the points above before >> sending the new version. > > Ok, this is fres

  1   2   >