Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-05 Thread Amit Kapila
oceed with it as this is not a problem of this patch. After this patch gets committed, we always need to revert it locally to narrow down the problem due to commit 6150a1b0. [1] - http://www.postgresql.org/message-id/caa4ek1+zeb8pmwwktf+3brs0pt4ux6rs6aom0uip8c6shjw...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-05 Thread Amit Kapila
On Tue, Apr 5, 2016 at 3:15 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > > On Tue, Apr 5, 2016 at 4:31 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Mon, Apr 4, 2016 at 1:58 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > >> > >>

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-05 Thread Amit Kapila
change in comment? /so frees it once parsing parameter/so frees it after parsing the parameter With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Choosing parallel_degree

2016-04-04 Thread Amit Kapila
On Mon, Apr 4, 2016 at 11:39 PM, Julien Rouhaud <julien.rouh...@dalibo.com> wrote: > > On 04/04/2016 17:03, Julien Rouhaud wrote: > > On 04/04/2016 08:55, Amit Kapila wrote: > > > > Thanks for the review! > > > >> Few comments: > >> 1. &

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-04 Thread Amit Kapila
significant gain shown in your tests is that on the m/c where you are testing the contention due to CLOGControlLock is not high enough that any reduction on the same will help. To me, it is visible in some of the high-end machines like which have 4 or more sockets. So, I think these results should be taken as an indication that there is no regression in the tests performed by you. Thanks for doing all the tests for these patches. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-04 Thread Amit Kapila
On Mon, Apr 4, 2016 at 4:31 PM, Magnus Hagander <mag...@hagander.net> wrote: > On Fri, Apr 1, 2016 at 6:47 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: > > >> Also, I think below part of documentation for pg_start_backup() needs to >> be modified: >

Re: [HACKERS] pgbench more operators & functions

2016-04-04 Thread Amit Kapila
cusing on that, while we're working on > getting 9.6 ready. > > +1. Extremely positive and encouraging way of involving other people. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Choosing parallel_degree

2016-04-04 Thread Amit Kapila
StdRdOptions *) (relation)->rd_options)->parallel_degree : (defaultmpd)) + There are minor in-consistencies in the above macro definition. a. RelationGetMaxParallelDegree - This should be RelationGetParallelDegree. b. defaultmpd - it is better to name it as defaultpd > > > The feature f

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-03 Thread Amit Kapila
f the questions asked by Andres upthread[1]? [1] - http://www.postgresql.org/message-id/20160401083518.ge9...@awork2.anarazel.de With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-02 Thread Amit Kapila
On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-31 15:07:22 +0530, Amit Kapila wrote: > > On Thu, Mar 31, 2016 at 4:39 AM, Andres Freund <and...@anarazel.de> wrote: > > > > > > On 2016-03-28 22:50:49 +0530, Amit

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-04-01 Thread Amit Kapila
sult || pqWait(TRUE, FALSE, conn) || pqReadData(conn) < 0) { /* * conn->errorMessage has been set by pqWait or pqReadData. We * want to append it to any already-received error message. */ pqSaveErrorResult(conn); conn->asyncStatus = PGASYNC_IDLE; return pqPrepareAsyncResult(conn); } .. } With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-03-31 Thread Amit Kapila
internal AS 'pg_start_backup'; One thing, that might be slightly inconvenient for user is if he or she wants to use this API for non-exclusive backups then, they need to pass the value of second parameter as well which doesn't seem to be a big issue. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Parallel Queries and PostGIS

2016-03-31 Thread Amit Kapila
s.html > > This seems non-optimal. No amount of cranking up the underlying > function COST seems to change this, perhaps because the join cost is > entirely based on the number of expected tuples in the join relation? > Is the function cost not being considered when given as join clause or you wanted to point in general for any parallel plan it is not considered? I think it should be considered when given as a clause for single table scan. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-03-31 Thread Amit Kapila
re :) > > +1 for doing that way for now. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-03-31 Thread Amit Kapila
ad_u32(>state) & BM_PERMANENT)) Is there a reason that you have kept macro's to read refcount and usagecount, but not for flags? Apart from this, I have verified that patch compiles on Windows and passed regressions (make check)! Nice work! With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-31 Thread Amit Kapila
On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-31 15:07:22 +0530, Amit Kapila wrote: > > On Thu, Mar 31, 2016 at 4:39 AM, Andres Freund <and...@anarazel.de> wrote: > > > > > > On 2016-03-28 22:50:49 +0530, Amit

Re: [HACKERS] Relation extension scalability

2016-03-31 Thread Amit Kapila
On Thu, Mar 31, 2016 at 4:28 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Mar 31, 2016 at 12:59 AM, Dilip Kumar <dilipbal...@gmail.com> > wrote: > > On Tue, Mar 29, 2016 at 10:08 AM, Amit Kapila <amit.kapil...@gmail.com> > > wrote: > &g

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-31 Thread Amit Kapila
On Thu, Mar 31, 2016 at 4:39 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-28 22:50:49 +0530, Amit Kapila wrote: > > On Fri, Sep 11, 2015 at 8:01 PM, Amit Kapila <amit.kapil...@gmail.com> > > wrote: > > > > > Amit, could you

Re: [HACKERS] Relation extension scalability

2016-03-31 Thread Amit Kapila
On Thu, Mar 31, 2016 at 10:29 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > > On Tue, Mar 29, 2016 at 10:08 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: > >> Yes, that makes sense. One more point is that if the reason for v13 >> giving better p

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-03-30 Thread Amit Kapila
o you want to change that? If yes, I think that will make this patch more invasive with respect to handling of failure modes. Also as David points out, I also feel that it will raise the bar for usage of this API. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-29 Thread Amit Kapila
ww.postgresql.org/message-id/566ef84f.1030...@iki.fi With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-28 Thread Amit Kapila
asing lockWaiters * 20 limit to may be lockWaiters * 25 or lockWaiters * 30. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-28 Thread Amit Kapila
On Fri, Sep 11, 2015 at 8:01 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Thu, Sep 3, 2015 at 5:11 PM, Andres Freund <and...@anarazel.de> wrote: > > > > Updated comments and the patch (increate_clog_bufs_v2.patch) > containing the same is attached. >

Re: [HACKERS] Relation extension scalability

2016-03-27 Thread Amit Kapila
ght, then climb up to the parent. Stop when we reach a node with enough free space (as we must, since the root has enough space). So shouldn't it be able to find the new FSM page where the bulk extend rolls over? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-25 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund <and...@anarazel.de> wrote: > > > > Even after changing to scale 500, the performance benefits on this, > > older 2 socket, machin

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-24 Thread Amit Kapila
[1] - http://www.postgresql.org/message-id/capphfdsrot1jmsnrnccqpnzeu9vut7tx6b-n1wyouwwfhd6...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-24 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund <and...@anarazel.de> wrote: > > > > Have you, in your evaluation of the performance of this patch, done > > profiles over time? I

Re: [HACKERS] Relation extension scalability

2016-03-24 Thread Amit Kapila
On Thu, Mar 24, 2016 at 1:48 PM, Petr Jelinek <p...@2ndquadrant.com> wrote: > > On 24/03/16 07:04, Dilip Kumar wrote: >> >> >> On Thu, Mar 24, 2016 at 10:44 AM, Robert Haas <robertmh...@gmail.com >> <mailto:robertmh...@gmail.com>> wrote: >&g

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund <and...@anarazel.de> wrote: > > > > Even after changing to scale 500, the performance benefits on this, > > older 2 socket, machin

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
suspicion is that a better approach for now would be to take Simon's > patch, but add a (per-page?) 'ClogModificationLock'; to avoid the need > of doing something fancier in TransactionIdSetStatusBit(). > I think we can try that as well and if you see better results by that Approach, then we can use that instead of current patch. [1] - http://www.postgresql.org/message-id/cad__oujrdwqdjdovhahqldg-6ivu6ibci9ij1qpu6atuqpl...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-23 Thread Amit Kapila
whatever you have said in another mail that we should introduce a new API to do a more targeted search for such cases. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Tue, Mar 22, 2016 at 12:33 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > > > On Thu, Mar 17, 2016 at 11:39 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > I have reviewed the patch.. here are some review comments, I will continue to review.. > > 1.

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Wed, Mar 23, 2016 at 12:26 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund <and...@anarazel.de> wrote: > > > > > > I think it's worthwhile to create a benchmark that does something like > > BE

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund <and...@anarazel.de> wrote: > > Hi, > > On 2016-03-15 10:47:12 +0530, Amit Kapila wrote: > > @@ -248,12 +256,67 @@ set_status_by_pages(int nsubxids, TransactionId *subxids, > > * Record the final state of transac

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-22 Thread Amit Kapila
On Tue, Mar 22, 2016 at 6:29 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-22 18:19:48 +0530, Amit Kapila wrote: > > > I'm actually rather unconvinced that it's all that common that all > > > subtransactions are on one page. If you have concurr

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-22 Thread Amit Kapila
On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund <and...@anarazel.de> wrote: > > Hi, > > On 2016-03-15 10:47:12 +0530, Amit Kapila wrote: > > @@ -248,12 +256,67 @@ set_status_by_pages(int nsubxids, TransactionId *subxids, > > * Record the final state of transac

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-21 Thread Amit Kapila
On Tue, Mar 22, 2016 at 9:46 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > > On Mon, Mar 21, 2016 at 10:13 PM, Robert Haas <robertmh...@gmail.com> wrote: > >> It is very difficult to believe that this is a

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-21 Thread Amit Kapila
On Tue, Mar 22, 2016 at 9:13 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Tue, Mar 22, 2016 at 2:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Mon, Mar 21, 2016 at 6:16 PM, Haribabu Kommi < kommi.harib...@gmail.com> > > wrote: >

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-21 Thread Amit Kapila
ussion about this previously as well [1]. [1] - http://www.postgresql.org/message-id/CAB7nPqT6gKj6iS9VTPth_h6Sz5Jo-177s6QJN_jrW66wyCjJ=w...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-21 Thread Amit Kapila
On Mon, Mar 21, 2016 at 6:16 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Mon, Mar 14, 2016 at 4:51 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> Operating system - windows 7 > >> Binary - PostgreSQL 9.5 (This doesn't matter, 9.4+ can p

Re: [HACKERS] Relation extension scalability

2016-03-21 Thread Amit Kapila
on after backend extends the relation for multiple blocks, what is the need of same? 12. I think it is good to once test pgbench read-write tests to ensure that this doesn't introduce any new regression. [1] - http://www.postgresql.org/message-id/caa4ek1lonxz4qa_dquqbanspxsctjxrkexjii8h3gnd9z8u..

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 10:26 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund <and...@anarazel.de> wrote: >> >> >> >> On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila < amit.kapil...@gmail.

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund <and...@anarazel.de> wrote: > > > On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila < > amit.kapil...@gmail.com> wrote: > > >The article pointed by you justifies that the way ResetEvent is done by > >patch

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Sun, Mar 20, 2016 at 7:13 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-19 15:43:27 +0530, Amit Kapila wrote: > > On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund <and...@anarazel.de> wrote: > > > > > > On March 18, 2016 11:52:08 PM

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 5:12 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Sun, Mar 20, 2016 at 3:01 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Sat, Mar 19, 2016 at 7:02 PM, Robert Haas <robertmh...@gmail.com> wrote: > >> On Sat

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-20 Thread Amit Kapila
On Sat, Mar 19, 2016 at 7:02 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Sat, Mar 19, 2016 at 12:28 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Won't in theory, without patch as well nentries can overflow after running > > for very long time? I thi

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 7:50 PM, Constantin S. Pan <kva...@gmail.com> wrote: > > On Wed, 16 Mar 2016 18:08:38 +0530 > Amit Kapila <amit.kapil...@gmail.com> wrote: > > > > > Why backend just waits, why can't it does the same work as any worker > > does?

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-03-19 Thread Amit Kapila
24576K NUMA node0 CPU(s): 0,65-71,96-103 NUMA node1 CPU(s): 72-79,104-111 NUMA node2 CPU(s): 80-87,112-119 NUMA node3 CPU(s): 88-95,120-127 NUMA node4 CPU(s): 1-8,33-40 NUMA node5 CPU(s): 9-16,41-48 NUMA node6 CPU(s): 17-24,49-56 NUMA node7 CPU(s): 25-32,57-64 With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-19 Thread Amit Kapila
pickup in next release (or for this release, if everybody feels strongly to have it in this release) if you don't see any patch for the same. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 6:41 PM, David Rowley <david.row...@2ndquadrant.com> wrote: > > On 18 March 2016 at 01:22, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Thu, Mar 17, 2016 at 10:35 AM, David Rowley > > <david.row...@2ndquadrant.com> wrote: &

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
think adding the reason for same in comments above function would be better. 7. tlist.c +} \ No newline at end of file There should be a new line at end of file. [1] - http://www.postgresql.org/message-id/CAA4eK1Jk8hm-2j-CKjvdd0CZTsdPX=edk_qhzc4689hq0xt...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 2:55 PM, Constantin S. Pan <kva...@gmail.com> wrote: > > On Wed, 16 Mar 2016 12:14:51 +0530 > Amit Kapila <amit.kapil...@gmail.com> wrote: > > > On Wed, Mar 16, 2016 at 5:41 AM, Constantin S. Pan <kva...@gmail.com> > > wrote: &g

Re: Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-19 Thread Amit Kapila
is possible that subpath and subplan correspond to different nodes when gating Result node is added on to top of scan plan by create_scan_plan(). I think it might be better to explain in comments, why it is safe to rely on projection capability of subpath to attach tlist to subplan.

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
stead of individually defining these functions under #ifndef WIN32, isn't it better to combine them all as they are at end of file. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
. Will look into it tomorrow once again and share if I find anything wrong with it, but feel to proceed if you want. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund <and...@anarazel.de> wrote: > > On March 18, 2016 11:52:08 PM PDT, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> >Won't the new code needs to ensure that ResetEvent(latchevent) > >should > >> >

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 10:35 AM, David Rowley <david.row...@2ndquadrant.com> wrote: > > On 17 March 2016 at 01:19, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Few assorted comments: > > > > 2. > > AggPath * > > create_agg_path(PlannerInfo *

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 11:57 PM, Jesper Pedersen < jesper.peder...@redhat.com> wrote: > > On 03/15/2016 01:17 AM, Amit Kapila wrote: >> >> I have updated the comments and changed the name of one of a variable from >> "all_trans_same_page" to &qu

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
I can help you verifying the windows implementation. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 2:56 PM, Constantin S. Pan <kva...@gmail.com> wrote: > > On Thu, 17 Mar 2016 13:21:32 +0530 > Amit Kapila <amit.kapil...@gmail.com> wrote: > > > On Wed, Mar 16, 2016 at 7:50 PM, Constantin S. Pan <kva...@gmail.com> > > wrote: >

Re: Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 7:10 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > > While reading above code changes, it looks like it is assuming that subpath > > and subplan will always be same (as it is verifying projec

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:14 PM, Andres Freund <and...@anarazel.de> wrote: > > > > On March 18, 2016 11:32:53 PM PDT, Amit Kapila <amit.kapil...@gmail.com> wrote: > >On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund <and...@anarazel.de> > >wrote: > >

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-18 20:14:07 +0530, Amit Kapila wrote: > > > I have done some > > tests on Windows with 0003 patch which includes running the regressions > > (vcregress check) and it passes.

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-18 Thread Amit Kapila
> To me, your idea sounds much better than current code in terms of understanding the free list concept as well. So, +1 for changing the code in this way. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-16 Thread Amit Kapila
-> Parallel Seq Scan on public.t1 (cost=0.00..45063.68 rows=18 width=4) Output: (c1 + 2) Filter: (t1.c1 < 10) (6 rows) In the above plans, you can notice that target list expression (c1 + 2) is pushed beneath Gather node after patch. Thoughts? With Regards, Amit

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-16 Thread Amit Kapila
in_parallel_workers' on a 4-CPU > machine. Seems 'gin_shared_mem' has no visible effect. > > wnum mem(MB) time(s) >0 16 247 >1 16 256 > It seems from you data that with 1 worker, you are always seeing slowdown, have you investigated the reason of same? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-15 Thread Amit Kapila
On Tue, Mar 15, 2016 at 7:54 PM, David Steele <da...@pgmasters.net> wrote: > > On 3/15/16 1:17 AM, Amit Kapila wrote: > > > On Tue, Mar 15, 2016 at 12:00 AM, David Steele <da...@pgmasters.net > > > >> This patch no longer applies cleanly: > >> >

Re: [HACKERS] Background Processes and reporting

2016-03-15 Thread Amit Kapila
On Tue, Mar 15, 2016 at 1:32 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-03-12 16:29:11 +0530, Amit Kapila wrote: > > On Sat, Mar 12, 2016 at 3:10 AM, Andres Freund <and...@anarazel.de> wrote: > > > > > > > > > > Similar

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 5:22 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Sat, Mar 12, 2016 at 1:58 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Yeah, that makes the addition of test for this functionality difficult. > > Robert, do you have any ide

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 12:00 AM, David Steele <da...@pgmasters.net> wrote: > > On 2/26/16 11:37 PM, Amit Kapila wrote: > >> On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila <amit.kapil...@gmail.com >> >> Here, we can see that there is a gain of ~15% t

Re: [HACKERS] Prepared Statement support for Parallel query

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 12:21 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Mon, Mar 14, 2016 at 9:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Fri, Feb 26, 2016 at 4:37 PM, Robert Haas <robertmh...@gmail.com> wrote: > > > > >

Re: [HACKERS] Prepared Statement support for Parallel query

2016-03-14 Thread Amit Kapila
orce during execution that statements for non read-only operations should not enter into parallel mode similar to what we are doing for non-zero tuple count case. Attached patch fixes the problem. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com prepared_stmt_parallel_quer

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-13 Thread Amit Kapila
On Fri, Mar 11, 2016 at 5:21 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Fri, Mar 11, 2016 at 12:00 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > >> I am not able to find the reason for this error. This error is occurring > >>

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-12 Thread Amit Kapila
On Sat, Mar 12, 2016 at 7:11 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > > > > On Sat, Mar 12, 2016 at 2:32 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >With force_parallel_mode=on, I could see many other failures as well. I think it is be

Re: [HACKERS] Background Processes and reporting

2016-03-12 Thread Amit Kapila
an I conclude that we reject detailed wait monitoring by design? > I don't think so. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Background Processes and reporting

2016-03-12 Thread Amit Kapila
mation. Another way could be to have new view like pg_stat_background_activity with only relevant fields or try expose via individual views like pg_stat_bgwriter. Do you intend to get this done for 9.6 considering an add-on patch for wait event information displayed in pg_stat_activity? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-12 Thread Amit Kapila
On Sat, Mar 12, 2016 at 2:02 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > > > > On Sat, Mar 12, 2016 at 12:28 PM, Amit Kapila <amit.kapil...@gmail.com> wrote > >I don't see how this test will fail with force_parallel_mode=regress and max_parallel_degree >

Re: [HACKERS] [COMMITTERS] pgsql: Provide much better wait information in pg_stat_activity.

2016-03-11 Thread Amit Kapila
ated views > that mimicked the catalog for a specific Postgres version. > That makes sense to me if other people agree to it, but I think there will be some maintenance overhead for it, but I see that as worth the effort in terms of user convenience. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-11 Thread Amit Kapila
y difficult. Robert, do you have any idea what kind of test would have caught this issue? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-11 Thread Amit Kapila
mbers for 1, 5, 10, 15, 25 or whatever other multiplier you have tried, so that it is clear that 20 is best? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-11 Thread Amit Kapila
On Fri, Mar 11, 2016 at 5:21 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Fri, Mar 11, 2016 at 12:00 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > > Okay, so one probable theory for such an error could be that when there is > > alr

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-10 Thread Amit Kapila
ion at end. What do you think? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com fix_typo_lwlock_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [COMMITTERS] pgsql: Provide much better wait information in pg_stat_activity.

2016-03-10 Thread Amit Kapila
e, but still wait_event and wait_event_type contain some wait information and I think that will look odd to anybody new looking at the view. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-10 Thread Amit Kapila
On Wed, Mar 9, 2016 at 5:46 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > On Wed, Mar 9, 2016 at 10:06 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Wed, Mar 9, 2016 at 11:46 AM, Haribabu Kommi < kommi.harib...@gmail.com> > > wrote: > >&

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-10 Thread Amit Kapila
In general, the patch looks good to me and solves the problem mentioned. I have ran the regression tests with force_parallel_mode and doesn't see any problem. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com Analyze_select_into_disable_parallel_scan_v2.patch Description: Binary d

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-03-10 Thread Amit Kapila
makes sense to check the performance by increasing CLOG buffers (patch for same is posted in Speed up Clog thread [1]) as that also relieves contention on CLOG as per the tests I have done? [1] - http://www.postgresql.org/message-id/caa4ek1lmmgnq439bum0lcs3p0sb8s9kc-cugu_thnqmwa8_...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-09 Thread Amit Kapila
On Wed, Mar 9, 2016 at 7:17 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Mar 9, 2016 at 8:31 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Thanks for the suggestion. I have updated the patch to include wait_event_type information in the wait_event tabl

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-09 Thread Amit Kapila
eral the performance data doesn't indicate any regression. [1] - http://www.postgresql.org/message-id/caa4ek1+zeb8pmwwktf+3brs0pt4ux6rs6aom0uip8c6shjw...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com extend_pg_stat_activity_v13.patch Description: Binary data -

Re: [HACKERS] WIP: Upper planner pathification

2016-03-09 Thread Amit Kapila
cially for cases when target list contain costly expressions, so I am slightly inclined to follow that even though that looks more work. Thoughts? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com apply_tlist_partial_path_v1.patch Description: Binary data -- Sent via pgs

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-09 Thread Amit Kapila
hould return the existing handle with an error ERROR_ALREADY_EXISTS. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-08 Thread Amit Kapila
On Tue, Mar 8, 2016 at 7:23 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Tue, Mar 8, 2016 at 4:27 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> Hmm. Can we drive this off of the heavyweight lock manager's idea of > >> how big the relation e

Re: [HACKERS] Relation extension scalability

2016-03-08 Thread Amit Kapila
On Mon, Mar 7, 2016 at 8:34 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Fri, Mar 4, 2016 at 11:49 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > I think one thing which needs more thoughts about this approach is that we > > need to maintain some number

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Amit Kapila
rg/message-id/cam3swzrs1mtvrkkasy1xbshgzxkd6-hnxx3gq7x-p-dz0zt...@mail.gmail.com > > In summary, I think it's surprising that a max_parallel_degree of 1 > doesn't disable parallel workers entirely. > I have responded on the thread where you have raised that point with my tho

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Amit Kapila
then we should change it. [1] - http://infocenter.sybase.com/archive/index.jsp?topic=/com.sybase.help.ase_15.0.sag1/html/sag1/sag1234.htm With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] WIP: Upper planner pathification

2016-03-06 Thread Amit Kapila
On Mon, Mar 7, 2016 at 9:13 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > >>>> Is there some reason why hash and nestloop are safe but merge isn't? > > > To make hash and nestloop work in parallel queries,

Re: [HACKERS] WIP: Upper planner pathification

2016-03-06 Thread Amit Kapila
On Sun, Mar 6, 2016 at 9:02 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > > On Sat, Mar 5, 2016 at 10:11 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Is there some reason why hash and nestloop are safe but merg

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
On Sat, Mar 5, 2016 at 10:11 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > > On Fri, Mar 4, 2016 at 11:31 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> (BTW, I found what seemed to be a couple of pre-

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
On Sat, Mar 5, 2016 at 4:51 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Fri, Mar 4, 2016 at 11:31 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > > > OK, here is a version that I think addresses all of the recent comments: > > > > * Fixed handling

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
_mergejoin_path. + * It's only needed atop a node that doesn't support projection "needed atop a node", seems unclear to me, typo? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-04 Thread Amit Kapila
de it for multiple relations, I think you need to advocate the reloid for relation in each proc and then get the relation descriptor for relation extension lock. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

<    8   9   10   11   12   13   14   15   16   17   >