Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-11-12 Thread Dilip Kumar
ment as well: > > I made an adjustment that I hope will address your concern here, made > a few other adjustments, and committed this. > Thanks, Robert. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] path toward faster partition pruning

2017-10-30 Thread Dilip Kumar
omething cleaner because if we want to access the ExprContext inside partkey_datum_from_expr then we may need to pass it to "get_partitions_from_clauses" which is a common function for optimizer and executor. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgs

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
30%. I understand that the this is the worst case for PWA where FinalizeAggregate is getting all the tuple. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
on t2 (cost=0.00..1885.00 rows=10 width=8) (actual time=1.407..11.689 rows=10 loops=1) Planning time: 0.146 ms Execution time: 263.678 ms (11 rows) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Dilip Kumar
On Thu, Oct 12, 2017 at 6:37 PM, Tomas Vondra wrote: > > > On 10/12/2017 02:40 PM, Dilip Kumar wrote: >> On Thu, Oct 12, 2017 at 4:31 PM, Tomas Vondra >> wrote: >>> Hi, >>> >>> It seems that Q19 from TPC-H is consistently failing with segfaul

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Dilip Kumar
m by itself and isshared is set for BitmapOr. Attached patch fixing the issue for me. I will thoroughly test this patch with other scenario as well. Thanks for reporting. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com diff --git a/src/backend/optimizer/plan/createplan.c b/src/

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-11 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 9:21 PM, Dilip Kumar wrote: > On Fri, Oct 6, 2017 at 7:24 PM, Dilip Kumar wrote: >> On Fri, Oct 6, 2017 at 6:08 PM, Alexander Kuzmenkov >> wrote: >>> >>>> Analysis: The estimated value of the lossy_pages is way higher than >>

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 7:24 PM, Dilip Kumar wrote: > On Fri, Oct 6, 2017 at 6:08 PM, Alexander Kuzmenkov > wrote: >> >>> Analysis: The estimated value of the lossy_pages is way higher than >>> its actual value and reason is that the total_pages calculated by the &

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 7:04 PM, Dilip Kumar wrote: > On Fri, Oct 6, 2017 at 6:36 PM, Robert Haas wrote: >> On Fri, Oct 6, 2017 at 2:12 AM, Dilip Kumar wrote: >>>> The performance results look good, but that's a slightly different >>>> thing

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
227229 > 8197114179172 > 10269227190192 > 14 110108106105 > Thanks for the testing number looks good to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 6:36 PM, Robert Haas wrote: > On Fri, Oct 6, 2017 at 2:12 AM, Dilip Kumar wrote: >>> The performance results look good, but that's a slightly different >>> thing from whether the estimate is accurate. >>> >>> +nbuckets = tbm_

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-05 Thread Dilip Kumar
On Thu, Oct 5, 2017 at 8:15 PM, Robert Haas wrote: > On Sun, Sep 17, 2017 at 7:04 AM, Dilip Kumar wrote: >> I used lossy_pages = max(0, total_pages - maxentries / 2). as >> suggesed by Alexander. > > Does that formula accurately estimate the number of lossy pages

Re: [HACKERS] path toward faster partition pruning

2017-09-28 Thread Dilip Kumar
On Thu, Sep 28, 2017 at 1:44 PM, Amit Langote wrote: > On 2017/09/28 13:58, Dilip Kumar wrote: >> I think the above logic is common between this patch and the runtime >> pruning. I think we can make >> a reusable function. Here we are preparing minkey and maxkey of Da

Re: [HACKERS] path toward faster partition pruning

2017-09-27 Thread Dilip Kumar
h i.e converting the Expr array to CONST array? [1] https://www.postgresql.org/message-id/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A%40mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] path toward faster partition pruning

2017-09-26 Thread Dilip Kumar
On Tue, Sep 26, 2017 at 2:45 PM, Amit Langote wrote: > On 2017/09/25 20:21, Dilip Kumar wrote: > I see. So, in the run-time pruning case, only the work of extracting > bounding values is deferred to execution time. Matching clauses with the > partition key still occurs during p

Re: [HACKERS] path toward faster partition pruning

2017-09-25 Thread Dilip Kumar
partitions" is doing. But, at optimizer time for PARAM we will not have Datum values for rightop, so we will keep track of the PARAM itself. And, finally at runtime when we get the PARAM value we can prepare minkey and maxkey and call get_partitions_for_keys function. -- Regards, Dilip

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-21 Thread Dilip Kumar
On Thu, Sep 21, 2017 at 4:50 PM, Rafia Sabih wrote: > On Sun, Sep 17, 2017 at 9:10 PM, Dilip Kumar wrote: >> On Wed, Sep 6, 2017 at 4:14 PM, Rafia Sabih >> wrote: >> > > Please find the attached file for the revised version. Thanks for the updated patch, I have s

Re: [HACKERS] path toward faster partition pruning

2017-09-19 Thread Dilip Kumar
On Sat, Sep 16, 2017 at 3:13 PM, Dilip Kumar wrote: > On Fri, Sep 15, 2017 at 2:20 PM, Amit Langote > wrote: >> On 2017/09/15 11:16, Amit Langote wrote: > > Thanks for the updated patch. I was going through the logic of > get_rel_partitions in 0002 as almost similar

Re: [HACKERS] UPDATE of partition key

2017-09-19 Thread Dilip Kumar
On Tue, Sep 19, 2017 at 1:15 PM, Amit Khandekar wrote: > On 18 September 2017 at 20:45, Dilip Kumar wrote: >> Please find few more comments. >> >> + * in which they appear in the PartitionDesc. Also, extract the >> + * partition key columns of the root part

Re: [HACKERS] UPDATE of partition key

2017-09-18 Thread Dilip Kumar
On Mon, Sep 18, 2017 at 11:29 AM, Dilip Kumar wrote: > On Fri, Sep 15, 2017 at 4:55 PM, Amit Khandekar > wrote: >> On 12 September 2017 at 12:39, Amit Khandekar wrote: >>> On 12 September 2017 at 11:57, Dilip Kumar wrote: >>>> On Tue, Sep 12, 2017 at 11:1

Re: [HACKERS] UPDATE of partition key

2017-09-17 Thread Dilip Kumar
On Fri, Sep 15, 2017 at 4:55 PM, Amit Khandekar wrote: > On 12 September 2017 at 12:39, Amit Khandekar wrote: >> On 12 September 2017 at 11:57, Dilip Kumar wrote: >>> On Tue, Sep 12, 2017 at 11:15 AM, Amit Khandekar >>> wrote: >>> > I found out that, i

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-17 Thread Dilip Kumar
On Sun, Sep 17, 2017 at 4:34 PM, Dilip Kumar wrote: >> >> I have repeated one of the tests after fixing the problems pointed by >> you but this time results are not that impressive. Seems like below >> check was the problem in the previous patch >> >>if

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-17 Thread Dilip Kumar
*/ one blank line between two functions. - + bool detached; + + detached = false; a better way is bool detached = false; - Compilation warning shm_mq.c: In function ‘write_in_local_queue’: shm_mq.c:1489:32: warning: variable ‘tuple_size’ set but not used [-Wunused-bu

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-17 Thread Dilip Kumar
On Mon, Sep 4, 2017 at 11:18 AM, Dilip Kumar wrote: > On Thu, Aug 31, 2017 at 11:27 PM, Robert Haas wrote: > > I have repeated one of the tests after fixing the problems pointed by > you but this time results are not that impressive. Seems like below > check was the problem

Re: [HACKERS] path toward faster partition pruning

2017-09-16 Thread Dilip Kumar
rectly fetching the "rightop->constvalue" whereas, for runtime pruning, we need to store rightop itself and calculate the value at runtime by param evaluation, I haven't yet thought how can we make this last part generic. -- Regards, Dilip Kumar EnterpriseDB: http://www.e

Re: [HACKERS] path toward faster partition pruning

2017-09-14 Thread Dilip Kumar
QUERY PLAN Append (cost=0.00..4.49 rows=2 width=8) -> Seq Scan on t1 (cost=0.00..2.24 rows=1 width=8) Filter: (2 > a) -> Seq Scan on t2 (cost=0.00..2.25 rows=1 width=8) Filter: (2 > a) (5 rows) -

Re: [HACKERS] UPDATE of partition key

2017-09-11 Thread Dilip Kumar
a "statement level delete trigger" with transition table still we are making the entry in transition table of the delete trigger and that will never be used. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] UPDATE of partition key

2017-09-11 Thread Dilip Kumar
On Thu, Sep 7, 2017 at 11:41 AM, Amit Khandekar wrote: > On 6 September 2017 at 21:47, Dilip Kumar wrote: > Actually, since transition tables came in, the functions like > ExecARUpdateTriggers() or ExecARInsertTriggers() have this additional > purpose of capturing transition table r

Re: [HACKERS] UPDATE of partition key

2017-09-06 Thread Dilip Kumar
e->operation == CMD_UPDATE && mtstate->mt_transition_capture) + ExecARUpdateTriggers(estate, resultRelInfo, InvalidOid, + NULL, + tuple, + NULL, + mtstate->mt_transition_capture); -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-03 Thread Dilip Kumar
206824 15 995417 798817 20 1654016 1588498 Still, we see a good reduction in lossy pages count. I will perform the test at different work_mem and for different values of TBM_FILFACTOR and share the number soon. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com --

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-09-01 Thread Dilip Kumar
n 1073 LWLock | XidGenLock 869 IPC | ClogGroupUpdate -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-08-30 Thread Dilip Kumar
3387 (offset 29 lines). Hunk #12 succeeded at 3424 (offset 29 lines). 3 out of 12 hunks FAILED -- saving rejects to file src/backend/executor/execMain.c.rej patching file src/backend/executor/execReplication.c patching file src/backend/executor/nodeModifyTable.c -- Regards, Dilip Kumar Ent

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-30 Thread Dilip Kumar
that. > > Note that a value higher than 0.9375 wouldn't be sane without some > additional safety precautions because maxentries could be as low as > 16. Thanks for the feedback. I will work on it. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > Th

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-28 Thread Dilip Kumar
On Wed, Aug 23, 2017 at 9:45 AM, Dilip Kumar wrote: >> >> ... >> if (tbm->nentries <= tbm->maxentries / 2) >> { >> /* >> * We have made enough room. >> ... >> I think we could try hi

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-22 Thread Dilip Kumar
rk the index scan per se with the 0.9 fill factor, but the > reduction of lossy pages was significant. I will try this and produce some performance number. Thanks for the input. > > [1] > https://www.postgresql.org/message-id/flat/251401bb-6f53-b957-4128-578ac22e8acf%40p

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-18 Thread Dilip Kumar
On Fri, 18 Aug 2017 at 4:48 PM, Amit Kapila wrote: > On Thu, Aug 17, 2017 at 2:45 PM, Dilip Kumar > wrote: > > On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar > wrote: > >> > >> Either we can pass "num_gene" to merge_clump or we can store num_gene >

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-17 Thread Dilip Kumar
On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar wrote: > > Either we can pass "num_gene" to merge_clump or we can store num_gene > in the root. And inside merge_clump we can check. Do you see some more > complexity? > After putting some more thought I see one more problem bu

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-17 Thread Dilip Kumar
op level scan/join node, I don't think that > will be sufficient, we have some special way to push target list below > Gather node in apply_projection_to_path, we need to move that part as > well in generate_gather_paths. > -- Regards, Dilip Kumar EnterpriseDB: http://www.enterpr

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-16 Thread Dilip Kumar
On Wed, Jul 26, 2017 at 10:35 PM, Robert Haas wrote: > On Thu, Jun 8, 2017 at 10:44 AM, Dilip Kumar wrote: >> On Thu, May 18, 2017 at 8:07 PM, Robert Haas wrote: >> Thanks for the feedback. I haven't yet worked on optimizer side feedback but I have done some work for im

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-25 Thread Dilip Kumar
Filter: (l_commitdate < l_receiptdate) Rows Removed by Filter: 1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-12 Thread Dilip Kumar
On Wed, Jul 12, 2017 at 10:55 AM, Amit Kapila wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes wrote: >> On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar wrote: >>> >>> So because of this high projection cost the seqpath and parallel path >>> both have fuzz

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-10 Thread Dilip Kumar
how to start looking into that. > > Also, how do you debug stuff like this? Are there some gdb tricks to make > this easier to introspect into the plans? > > Cheers, > > Jeff -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Dilip Kumar
ay if byte order is changed or we can use something like "be64toh" > > > -- > Thanks & Regards, > Kuntal Ghosh > EnterpriseDB: http://www.enterprisedb.com > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To ma

Re: [HACKERS] [POC] hash partitioning

2017-07-05 Thread Dilip Kumar
r to check ndatum first. 4. + * next larger modulus. For example, if you have a bunch + * of partitions that all have modulus 5, you can add a + * new new partition with modulus 10 or a new partition Typo, "new new partition" -> "new partition" -- Regards, Dilip Kumar Enterp

Re: [HACKERS] Default Partition for Range

2017-07-03 Thread Dilip Kumar
On Fri, Jun 30, 2017 at 11:56 AM, Beena Emerson wrote: > Hello Dilip, > > On Wed, Jun 21, 2017 at 6:27 PM, Dilip Kumar wrote: >> On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar wrote: >>> This is basically crashing in RelationBuildPartitionDesc, so I think > >

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-22 Thread Dilip Kumar
lot->len, seems like it will always be <= buf_len, and if not then we can do min(buf_len, slot->len) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
arly exiting wherever we are looping the content array of the DEFAULT bound. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
re using the DSA we should not directly access the DSM which is under DSA. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
mem->av_dsa_handle)) > { > AutoVacuumDSA = dsa_attach(AutoVacuumShmem->av_dsa_handle); > dsa_pin_mapping(AutoVacuumDSA); > > Thoughts? IMHO, It's not a good idea to use DSM call to verify the DSA handle. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterpris

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar wrote: > This is basically crashing in RelationBuildPartitionDesc, so I think > we don't have any test case for testing DEFAULT range partition where > partition key has more than one attribute. So I suggest we can add > such test

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-06-20 Thread Dilip Kumar
tems. Mine is Linux. CentOS Linux release 7.2.1511 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Default Partition for Range

2017-06-20 Thread Dilip Kumar
lly crashing in RelationBuildPartitionDesc, so I think we don't have any test case for testing DEFAULT range partition where partition key has more than one attribute. So I suggest we can add such test case. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-06-20 Thread Dilip Kumar
39.485 IST [9738] LOG: LinkedList Node Data read is ==> Hello world19 2017-06-20 16:49:39.485 IST [9738] STATEMENT: select test_dsa_data_access(1); test_dsa_data_access -- 0 I don't see any segmentation fault. Is there some other step which I have m

Re: [HACKERS] [BUGS] Postgresql bug report - unexpected behavior of suppress_redundant_updates_trigger

2017-06-19 Thread Dilip Kumar
SET field = rpad('', 2001, 'a') WHERE id = 1; > UPDATE 1 <--- BUG: expected 0, as we ran the same update twice Seems like in "suppress_redundant_updates_trigger" we are comparing toasted tuple with the new tuple and that is the cause of the bug. -- Regards, D

Re: [HACKERS] psql's \d and \dt are sending their complaints to different output files

2017-06-19 Thread Dilip Kumar
whereas \dt says "No matching relations found". -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-06-08 Thread Dilip Kumar
effective_cache size. But, actually, multiple tuples can fall into the same page. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
lt;= a < 10 before starting > the tuple-routing. > > I wonder if we should... Since we don't allow BR triggers on partitioned > tables, there should not be any harm in doing it just before calling > ExecFindPartition(). Perhaps, topic for a new thread. Yeah, correct.

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
r the child but illegal for the parent? Isn't it a good idea that if a user is inserting in the top level relation he should know for which partition exactly the constraint got violated? > > Updated patch attached. Thanks. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterpris

Re: [HACKERS] Default Partition for Range

2017-06-05 Thread Dilip Kumar
== content2[i]) + return 0; + else if (content1[i] == RANGE_DATUM_DEFAULT) + return -1; I don't see any comments why default partition should be considered smallest in the index comparison. For negative infinity, it's pretty obvious by the enum name itself. ----- -- Re

Re: [HACKERS] [POC] hash partitioning

2017-06-03 Thread Dilip Kumar
's only failing during ExecInsert while checking the partition constraint. Other than that, patch looks fine to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] <> join selectivity estimate question

2017-06-01 Thread Dilip Kumar
On Thu, Jun 1, 2017 at 8:24 PM, Robert Haas wrote: > On Wed, May 31, 2017 at 1:18 PM, Dilip Kumar wrote: >> + if (jointype = JOIN_SEMI) >> + { >> + sjinfo->jointype = JOIN_INNER; >> + } > > That is pretty obviously half-baked a

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-06-01 Thread Dilip Kumar
subscription in normal mode and then ALTER OWNER in single user mode but it makes little sense to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com subscription_error_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-05-31 Thread Dilip Kumar
On Wed, May 31, 2017 at 2:20 PM, Michael Paquier wrote: > Yeah, see 0e0f43d6 for example. A simple fix is to look at > IsUnderPostmaster when creating, altering or dropping a subscription > in subscriptioncmds.c. Yeah, below patch fixes that. -- Regards, Dilip Kumar Enterpris

Re: [HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-05-31 Thread Dilip Kumar
tial, because Citus hit this... IMHO, For copy_to I don't see any problem in parallelizing it. I just tested by changing the cursorOption and it's working in parallel. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com copy_to_parallel.patch Description: Binary data --

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Dilip Kumar
e choice. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] <> join selectivity estimate question

2017-05-31 Thread Dilip Kumar
jinfo->jointype = JOIN_INNER; + } /* * We want 1 - eqjoinsel() where the equality operator is the one * associated with this != operator, that is, its negator. We may need something similar for anti-join as well. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterpr

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Dilip Kumar
responsible for copying these static members to the shared location so that other processes can access the SH_TYPE. And, the dynamic part (the actual hash entries) can be allocated using DSA by registering SH_ALLOCATE function. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com --

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-05-31 Thread Dilip Kumar
le > descriptor IMHO, In single user mode, it can not support replication (it can not have background WALReciver task). However, I believe there should be a proper error if the above statement is correct. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-h

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-29 Thread Dilip Kumar
llocator function to it. So in the allocator function, you can allocate memory from DSA. After it reaches some threshold it expands the size (double) and it will again call the allocator function to allocate the bigger memory. You can refer pagetable_allocate in tidbitmap.c. -- Regards, Dilip K

Re: [HACKERS] HINT message for "ALTER SUBSCRIPTION.. WITH" need to change with SET keyword

2017-05-19 Thread Dilip Kumar
(slot_name = NONE) to > disassociate the subscription from the slot." Seems like syntax got changed in b807f59828fbc02fea612e1cbc0066c6dfa3be9b commit but missed to change the hint. Attached patch fixes that. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com fix_subscripti

Re: [HACKERS] [POC] hash partitioning

2017-05-18 Thread Dilip Kumar
On Wed, May 17, 2017 at 2:07 PM, amul sul wrote: >> I would suggest "non-zero positive", since that's what we are using in >> the documentation. >> > > Understood, Fixed in the attached version. Why non-zero positive? We do support zero for the rem

[HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-05-17 Thread Dilip Kumar
112817) compared to the actual unique heap pages fetched (293314). Therefore, while costing bitmap scan using 1112817 pages and 4MB of work_mem, we predicted that even after we lossify all the pages it can not fit into work_mem, hence bitmap scan was not selected. -- Regards, Dilip

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Dilip Kumar
putting some more thought I realised that only for "Before Update" or the "Before Insert" trigger row can be changed. Correct me if I am assuming something wrong? So now again option3 will make more sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprised

Re: [HACKERS] UPDATE of partition key

2017-05-16 Thread Dilip Kumar
he can not rely only upon update level triggers. Earlier I thought that option1 is better but later I think that this can complicate the situation as we are firing first BR update then BR delete and can change the row multiple time and defining such behaviour can be complicated. -- Regards

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Dilip Kumar
ulus 2, remainder 1); You can detach foo1, create two new partitions with modulus 4 and remainders 0 and 2, and move the data over from the old partition I think it will be good information for a user to have? or it's already documented and I missed it? -- Regards, Dilip Kumar Ente

Re: [HACKERS] [POC] hash partitioning

2017-05-15 Thread Dilip Kumar
On Mon, May 15, 2017 at 9:06 PM, Dilip Kumar wrote: > Test2: > postgres=# insert into x2 values(100); -- it should violates > partition constraint > INSERT 0 1 > > Seems like a bug or am I missing something completely? Sorry, my bad. It's modulus on the hashvalue, not th

Re: [HACKERS] [POC] hash partitioning

2017-05-15 Thread Dilip Kumar
ould violates partition constraint INSERT 0 1 Seems like a bug or am I missing something completely? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgr

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-15 Thread Dilip Kumar
e query as planned. + */ You might want to do similar changes for gather_merge_readnext. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-15 Thread Dilip Kumar
#x27;t we call rewriteTargetListUD for all the inheritors if the inheritor is a foreign table which will intern call the postgresAddForeignUpdateTargets? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-15 Thread Dilip Kumar
ables. Oh, right! -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-14 Thread Dilip Kumar
On Sun, May 14, 2017 at 9:54 PM, Dilip Kumar wrote: > After your fix, now tupleid is invalid which is expected, but seems > like we need to do something more. As per the comments seems like it > is expected to get the oldtuple from planSlot. But I don't see any > code for h

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-14 Thread Dilip Kumar
o get the oldtuple from planSlot. But I don't see any code for handling that part. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-05-12 Thread Dilip Kumar
ot a good idea to mix pgindent changes with your patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-05-11 Thread Dilip Kumar
int index; +} PartitionHashBound; It will good to add some comments to explain the structure members -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-20 Thread Dilip Kumar
licity of memory usage +analysys we spin up different context. Typo /analysys/analysis -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] AGG_HASHED cost estimate

2017-04-20 Thread Dilip Kumar
actually grouping aggregate with GatherMerge was faster during actual execution time and he suspected problems with costing of hashAggregate. Maybe this is one of those? [1]https://www.postgresql.org/message-id/CAGPqQf2164iV6k-_M75qEZWiCfRarA_SKSmHjc0Uh1rEf5RJrA%40mail.gmail.com -- Regards

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-05 Thread Dilip Kumar
On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar wrote: > Sure I can do that, In attached patch, I only fixed the problem of not > executing the bitmap test. Now, I will add few cases to cover other > parts especially rescan and prefetching logic. I have added two test cases to cov

[HACKERS] Parallel Bitmap Heap Scan - Prefetch pages are not updated properly

2017-04-04 Thread Dilip Kumar
While analyzing the coverage for the prefetching part, I found an issue that prefetch_pages were not updated properly while executing in parallel mode. Attached patch fixes the same. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com parallel_bitmap_prefetch_fix.patch

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-03 Thread Dilip Kumar
sts to > cover the prefetching logic - it's quite bad that it's currently not > tested at all :( Sure I can do that, In attached patch, I only fixed the problem of not executing the bitmap test. Now, I will add few cases to cover other parts especially rescan and prefetching

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-01 Thread Dilip Kumar
On Sat, Apr 1, 2017 at 12:16 AM, Andres Freund wrote: > Hi, > > The parallel code-path isn't actually exercised in the tests added in > [1], as evidenced by [2] (they just explain). That imo needs to be > fixed. Thanks for reporting. Attached patch fixes that. --

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Dilip Kumar
bc-2.17.so[.] __memcpy_ssse3_back +4.08% 4.06% postgres libc-2.17.so[.] get_next_seq +3.92% 0.00% postgres [unknown] [.] 0x6161616161616161 +3.07% 3.05% postgres postgres[.] heap_check_warm_chain Thanks to Amit for helping in disc

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-30 Thread Dilip Kumar
/work/pg_codes/pbms_final/postgresql/src' make: *** [all-src-recurse] Error 2 I tried doing maintainer-clean, deleting postgres.bki but still the same error. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Parallel bitmap heap scan

2017-03-27 Thread Dilip Kumar
alled from SH_CREATE, which isn't prepared for a NULL return > anyway. > > Am I all wet? Yes you are right that we are not passing DSA_ALLOC_NO_OOM, so dsa_allocate_extended will return error in case if allocation failed. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb

Re: [HACKERS] Parallel bitmap heap scan

2017-03-27 Thread Dilip Kumar
he NULL pointer return from dsa_get_address. Please verify with the attached patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com bitmap_hugepage_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-26 Thread Dilip Kumar
on't loose track of this issue. > > The result is correct with this patch. I ran make installcheck then > the same steps as above and the query result was correct after > creating the index. Thanks for confirming. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Dilip Kumar
arameter to ExecutorRun as execute_once so !fcache->returnsSet || !es->lazyEval is the correct one and future proof. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-21 Thread Dilip Kumar
=0.00..12.26 rows=17 width=0) (actual time=0.016..0.016 rows=17 loops=1) Index Cond: (i <> '192.168.1.0/24'::inet) Planning time: 0.113 ms Execution time: 5.691 ms (13 rows) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sen

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-21 Thread Dilip Kumar
.1.0/24 | 192.168.1.226/24 192.168.1.0/24 | 192.168.1.255/24 192.168.1.0/24 | 192.168.1.0/25 192.168.1.0/24 | 192.168.1.255/25 192.168.1.0/26 | 192.168.1.226 :::1.2.3.4/128 | ::4.3.2.1/24 10:23::f1/128 | 10:23::f1/64 10:23::8000/113| 10:23

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Dilip Kumar
is a >problem if the plan is a parallel plan. And you also need to test this case what Robert have mentioned up thread. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-16 Thread Dilip Kumar
On Thu, Mar 16, 2017 at 8:26 PM, Emre Hasegeli wrote: >> Hopefully, this time I got it correct. Since I am unable to reproduce >> the issue so I will again need your help in verifying the fix. > > It is not crashing with the new patch. Thank you. Thanks for verifying.

  1   2   3   4   5   >