Re: [HACKERS] [sqlsmith] Failed assertion in _hash_splitbucket_guts

2016-12-02 Thread Amit Kapila
On Sat, Dec 3, 2016 at 6:58 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sat, Dec 3, 2016 at 2:06 AM, Andreas Seltenreich <seltenre...@gmx.de> > wrote: >> Hi, >> >> the new hash index code on 11003eb failed an assertion yesterday: >> >>

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_splitbucket_guts

2016-12-02 Thread Amit Kapila
tion method to trust in pg_hba.conf), it says the user doesn't exist. Can you create a user in the database which I can use? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-12-01 Thread Amit Kapila
On Thu, Dec 1, 2016 at 8:56 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Dec 1, 2016 at 12:43 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Dec 1, 2016 at 2:18 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Wed, Nov 23, 2016

Re: [HACKERS] Parallel execution and prepared statements

2016-12-01 Thread Amit Kapila
On Thu, Dec 1, 2016 at 9:40 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Dec 1, 2016 at 7:57 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> I have tried to restrict all the non-readonly operation modes or modes >> where parallelism might n

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-12-01 Thread Amit Kapila
On Thu, Dec 1, 2016 at 9:44 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Dec 1, 2016 at 1:03 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Wed, Nov 9, 2016 at 7:40 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> On Tue, Nov 8, 2

Re: [HACKERS] Parallel execution and prepared statements

2016-12-01 Thread Amit Kapila
On Wed, Nov 30, 2016 at 11:57 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Nov 29, 2016 at 6:24 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> Robert, do you have any better ideas for this problem? > > Not really. I think your prep

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-11-30 Thread Amit Kapila
On Wed, Nov 9, 2016 at 7:40 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Nov 8, 2016 at 10:56 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> >> Unless we want to wait until that work is committed before doing more review >> and testing on this. >&

Re: [HACKERS] Hash Indexes

2016-11-30 Thread Amit Kapila
On Thu, Dec 1, 2016 at 2:18 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Nov 23, 2016 at 8:50 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> [ new patch ] > > Committed with some further cosmetic changes. > Thank you very much. > I think it

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-30 Thread Amit Kapila
On Mon, Nov 21, 2016 at 11:08 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Nov 15, 2016 at 9:59 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Tue, Nov 15, 2016 at 9:27 AM, Kyotaro HORIGUCHI >> <horiguchi.kyot...@lab.ntt.co.jp> wrote: &g

Re: [HACKERS] XactLockTableWait doesn't set wait_event correctly

2016-11-29 Thread Amit Kapila
ng on tuple lock displays log message as below: LOG: process 648 still waiting for ExclusiveLock on tuple (0,1) of relation 137344 of database 12245 after 1045.220 ms DETAIL: Process holding the lock: 6460. Wait queue: 648. STATEMENT: update t1 set c1=4 where c1=1; -- With Regards, Amit Kap

Re: [HACKERS] Parallel execution and prepared statements

2016-11-29 Thread Amit Kapila
On Tue, Nov 29, 2016 at 4:40 PM, Albe Laurenz <laurenz.a...@wien.gv.at> wrote: > Amit Kapila wrote: >>> But with Tobias' complete patch "make installcheck-world" succeeds. >> >> Okay. I have looked into patch proposed and found that it will >> res

Re: [HACKERS] make default TABLESPACE belong to target table.

2016-11-29 Thread Amit Kapila
On Mon, Nov 28, 2016 at 6:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sat, Nov 26, 2016 at 9:46 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> >> If we just did points 1 and 2 then a bool GUC would suffice. I'm >> not sure how to handle all thre

Re: [HACKERS] Parallel execution and prepared statements

2016-11-29 Thread Amit Kapila
On Fri, Nov 18, 2016 at 9:01 PM, Albe Laurenz <laurenz.a...@wien.gv.at> wrote: > Amit Kapila wrote: >> Can you once test by just passing CURSOR_OPT_PARALLEL_OK in >> PrepareQuery() and run the tests by having forc_parallel_mode=regress? >> It seems to me that the cod

Re: [HACKERS] UNDO and in-place update

2016-11-28 Thread Amit Kapila
On Mon, Nov 28, 2016 at 11:01 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Nov 27, 2016 at 10:44 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Mon, Nov 28, 2016 at 4:50 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> Well, my ori

Re: [HACKERS] make default TABLESPACE belong to target table.

2016-11-28 Thread Amit Kapila
On Sat, Nov 26, 2016 at 9:46 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> What will such a storage parameter (default_tablespace) mean at table >> level and how it will different from existing default_tablespace? I >&

Re: [HACKERS] UNDO and in-place update

2016-11-27 Thread Amit Kapila
r that TID. Each > time, the visibility information in UNDO will cause us to return the > correct tuple, but we've erred by returning it twice (once per index > entry) rather than just once. > Why will scan traverse the UNDO chain if it starts after step-3 commit? -- With Regards,

Re: [HACKERS] Parallel bitmap heap scan

2016-11-27 Thread Amit Kapila
HASE_INIT). > Do you think that using barrier's will simplify the patch as compared to using condition variables because in that case, it will make sense to use barriers? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] UNDO and in-place update

2016-11-26 Thread Amit Kapila
If this a problem, then I think we need some form of delete marking system for the index, probably transaction visibility information as well. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] make default TABLESPACE belong to target table.

2016-11-26 Thread Amit Kapila
sage asked by Amos is quite genuine, but not sure if introducing default_tablespace as a storage level parameter is the best way to address it. Another way could be that we allow the user to specify something like tablespace_name / or something like that. -- With Regards, Amit Kapila. EnterpriseDB: http://

Re: [HACKERS] Parallel Index Scans

2016-11-26 Thread Amit Kapila
On Sat, Oct 22, 2016 at 9:07 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Oct 21, 2016 at 10:55 PM, Robert Haas <robertmh...@gmail.com> wrote: I have rebased the patch (parallel_index_scan_v2) based on latest commit e8ac886c (condition variables). I have re

Re: [HACKERS] UNDO and in-place update

2016-11-25 Thread Amit Kapila
On Fri, Nov 25, 2016 at 8:03 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> > > Another way to do is to write UNDO log for split operation (with exact > record locations on pages that are split) so that you don't need to > perform this expensive search operation.

Re: [HACKERS] UNDO and in-place update

2016-11-25 Thread Amit Kapila
all possible cases if so, then it is better to do that in the first phase and then later we can implement UNDO for indexes as well. As of now, nobody has reported any such problem, so maybe we can stick with the only-heap-based-undo design. -- With Regards, Amit Kapila. EnterpriseDB: http://www.en

Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker in ExecInitSubPlan

2016-11-25 Thread Amit Kapila
On Fri, Nov 25, 2016 at 10:33 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Fri, Nov 25, 2016 at 3:26 AM, Andreas Seltenreich <seltenre...@gmx.de> >> wrote: >>> just caught another InitPlan below Gat

Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker in ExecInitSubPlan

2016-11-24 Thread Amit Kapila
e6ec43cd4d10 where we have allowed subqueries to be pushed to parallel workers. I think we should consider rel (where rtekind is RTE_SUBQUERY) to be parallel safe if the subquery is also parallel safe. Attached patch does that and fixes the reported problem for me. -- With Regards, Amit Kapil

Re: [HACKERS] Re: [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

2016-11-23 Thread Amit Kapila
On Thu, Nov 24, 2016 at 10:29 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila >> Thanks for the clarification, I could reproduce the issue and c

Re: [HACKERS] Re: [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

2016-11-23 Thread Amit Kapila
So I have modified the comment to explain the situation and reason of check, see if you find that as okay? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com postgresql_Thu_Patch.log Description: Binary data postgresql_Thu_Head.log Description: Binary data cascading

Re: [HACKERS] UNDO and in-place update

2016-11-22 Thread Amit Kapila
es a better job in terms of bloat management (by keeping it separate from main Heap) which is why in many cases they are preferred over PostgreSQL. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UNDO and in-place update

2016-11-22 Thread Amit Kapila
ansaction reading the page, you need to do something to read the old row/rows present on that page. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker in ExecInitSubPlan

2016-11-21 Thread Amit Kapila
On Mon, Nov 21, 2016 at 9:30 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Mon, Nov 21, 2016 at 6:10 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Here instead of checking whether top_plan has initPlan,

Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker in ExecInitSubPlan

2016-11-21 Thread Amit Kapila
On Mon, Nov 21, 2016 at 6:10 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sun, Nov 20, 2016 at 10:43 PM, Andreas Seltenreich > <seltenre...@gmx.de> wrote: >> Hi, >> >> the query below triggers a parallel worker assertion for me when run on

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-21 Thread Amit Kapila
On Mon, Nov 21, 2016 at 7:46 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila >> > shared_buffers tps >> > 256MB 990 >> >

Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker in ExecInitSubPlan

2016-11-21 Thread Amit Kapila
itPlan, another possibility is to traverse the whole tree to see if initPlan is present. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-20 Thread Amit Kapila
s the largest effective size" seems to be a superstition, although I > don't know the reason for the drop at 512MB. > It is difficult to say why the performance drops at 512MB, it could be run-to-run variation. How long have you run each test? -- With Regards, Amit Kapila. EnterpriseD

Re: [HACKERS] Re: [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

2016-11-18 Thread Amit Kapila
, all required WAL segments have been archived 20079/20079 kB (100%), 1/1 tablespace Let me know, if some parameters need to be tweaked to reproduce the issue? It seems that the patch proposed is good, but it is better if somebody other than you can reproduce the issue and verify if the

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-18 Thread Amit Kapila
On Mon, Nov 14, 2016 at 9:33 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Nov 14, 2016 at 12:49 PM, Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote: >> At Sat, 12 Nov 2016 10:28:56 +0530, Amit Kapila <ami

Re: [HACKERS] Hash Indexes

2016-11-18 Thread Amit Kapila
On Fri, Nov 18, 2016 at 12:11 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Nov 17, 2016 at 10:54 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Thu, Nov 17, 2016 at 12:05 PM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >> >>

Re: [HACKERS] Hash Indexes

2016-11-17 Thread Amit Kapila
On Thu, Nov 17, 2016 at 10:54 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Nov 17, 2016 at 12:05 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >>> I think this comment is saying that we'll release the pin on the >>> primary bucket page

Re: [HACKERS] Hash Indexes

2016-11-17 Thread Amit Kapila
On Thu, Nov 17, 2016 at 3:08 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Nov 12, 2016 at 12:49 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> You are right and I have changed the code as per your suggestion. > > So... > > +/* > +

Re: [HACKERS] Parallel execution and prepared statements

2016-11-16 Thread Amit Kapila
On Wed, Nov 16, 2016 at 7:10 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Nov 16, 2016 at 2:27 AM, Tobias Bussmann <t.bussm...@gmx.net> wrote: >> As the patch in [1] targeting the execution of the plan in ExecutePlan >> depending on the destination wa

Re: [HACKERS] Parallel execution and prepared statements

2016-11-16 Thread Amit Kapila
T_PARALLEL_OK in PrepareQuery() and run the tests by having forc_parallel_mode=regress? It seems to me that the code in the planner is changed for setting a parallelModeNeeded flag, so it might work as it is. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pg

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

2016-11-15 Thread Amit Kapila
On Tue, Nov 15, 2016 at 7:51 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila >> It looks like the code in 9.3 or later version uses the recptr as

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-15 Thread Amit Kapila
On Tue, Nov 15, 2016 at 9:27 AM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Mon, 14 Nov 2016 16:53:35 +0530, Amit Kapila <amit.kapil...@gmail.com> > wrote in >> > >> >> The progress parameter is used not only for c

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-15 Thread Amit Kapila
On Tue, Nov 15, 2016 at 7:14 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila >> Okay, not a problem. However, I am not sure the r

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-14 Thread Amit Kapila
On Mon, Nov 14, 2016 at 9:33 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Nov 14, 2016 at 12:49 PM, Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote: >> At Sat, 12 Nov 2016 10:28:56 +0530, Amit Kapila <amit.ka

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-14 Thread Amit Kapila
On Sun, Nov 13, 2016 at 1:23 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Mon, Nov 7, 2016 at 7:03 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Tue, Nov 8, 2016 at 5:12 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> > On Mon, Nov 7

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-14 Thread Amit Kapila
On Sat, Nov 12, 2016 at 11:00 PM, Magnus Hagander <mag...@hagander.net> wrote: > On Sat, Nov 12, 2016 at 4:03 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Fri, Nov 11, 2016 at 3:01 PM, Magnus Hagander <mag...@hagander.net> >> wrote: >&g

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

2016-11-12 Thread Amit Kapila
also not complete. It looks like the code in 9.3 or later version uses the recptr as the target segment location (targetSegmentPtr) whereas 9.2 uses recptr as beginning of segment (readOff = 0;). If above understanding is right then it will set different values for latestPagePtr in 9.2 and 9.3 onw

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-11-11 Thread Amit Kapila
AccessExclusiveLocks? This function is called not only in LogStandbySnapshot(), but during DDL operations as well where lockmode >= AccessExclusiveLock. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-11 Thread Amit Kapila
as the test is done for a short duration so that degradation could be just a run to run to run variation, that's why I suggested doing few more tests. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-11-09 Thread Amit Kapila
appropriate. As far as I can see, the > current style is just obfuscating the code. > Okay, we can do some study and try to change it in the way you are suggesting. It seems partially this has been derived from btree code where we have function _bt_relbuf(). I am sure that we don't need _h

Re: [HACKERS] Hash Indexes

2016-11-09 Thread Amit Kapila
On Wed, Nov 9, 2016 at 9:10 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Nov 9, 2016 at 9:04 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > I think we can give a brief explanation right in the code comment. I > referred to "decreasing the TIDs&q

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-11-09 Thread Amit Kapila
On Tue, Nov 8, 2016 at 10:56 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sat, Sep 24, 2016 at 10:00 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Fri, Sep 23, 2016 at 5:34 PM, Amit Kapila <amit.kapil...@gmail.com> >> wrote:

Re: [HACKERS] Hash Indexes

2016-11-09 Thread Amit Kapila
On Wed, Nov 9, 2016 at 1:23 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Nov 7, 2016 at 9:51 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> [ new patches ] > > Attached is yet another incremental patch with some suggested changes. > > + * This exp

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-11-08 Thread Amit Kapila
looks good to me. I am > marking it as ready for committer. Thanks for the review. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-11-07 Thread Amit Kapila
of a module, but not sure if it is worth to add a target. I think there is no harm in adding such a target, but lets take an opinion from committer or others as well before adding this target. What do you say? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com test_pg_stat

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-07 Thread Amit Kapila
On Tue, Nov 8, 2016 at 5:12 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Mon, Nov 7, 2016 at 5:55 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Tue, Sep 20, 2016 at 8:15 AM, Tsunakawa, Takayuki >> <tsunakawa.ta...@jp.fujitsu.com> wrote: &g

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

2016-11-07 Thread Amit Kapila
gt; shared_buffers tps > 256MB 1138 > 512MB 1187 > 1GB 1571 > 2GB 1650 > 4GB 1598 > Isn't it somewhat strange that writes are showing big win whereas reads doesn't show much win? Can you once do some longer read-write tests to see if we get consistent data and take the med

Re: [HACKERS] Gather Merge

2016-11-07 Thread Amit Kapila
t; or something? > Hmm. We have discussed upthread to name it as form_tuple_array. Now, you feel that is also not good, I think it is basically matter of perspective, so why not leave it as it is for now and we will come back to naming it towards end of patch review or may be we can leave it for committe

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Amit Kapila
On Fri, Nov 4, 2016 at 9:37 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Nov 1, 2016 at 9:09 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >>> [ new patches

Re: [HACKERS] Hash Indexes

2016-11-04 Thread Amit Kapila
On Fri, Nov 4, 2016 at 6:37 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Nov 3, 2016 at 6:25 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> +nblkno = _hash_get_newblk(rel, pageopaque); >>> >>> I think this is not a great name f

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-11-04 Thread Amit Kapila
to split the OS writes and Flush requests in separate locks) to reduce it. It is difficult to speculate results at this stage. I think after spending some more time (probably few weeks), we will be in position to share our findings. -- With Regards, Amit Kapila. EnterpriseDB: http://www.ente

Re: [HACKERS] Hash Indexes

2016-11-03 Thread Amit Kapila
On Wed, Nov 2, 2016 at 6:39 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> [ new patches ] > > I looked over parts of this today, mostly the hashinsert.c changes. > > +/* >

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-11-02 Thread Amit Kapila
ew log message or performance will be improved, OTOH if we see same behaviour, then I think we can probably assume it due to scheduler activity and move on. Also one point to note here is that even when the performance is down in that curve, it is equal to or better than HEA

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-31 Thread Amit Kapila
On Mon, Oct 31, 2016 at 7:58 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > On 10/31/2016 02:51 PM, Amit Kapila wrote: > And moreover, this setup (single device for the whole cluster) is very > common, we can't just neglect it. > > But my main point here really

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-31 Thread Amit Kapila
g else. > I have also seen previously that increasing clog buffers to 256 can impact performance negatively. So, probably here the gains due to group_update patch is negated due to the impact of increasing clog buffers. I am not sure if it is good idea to see the impact of increasing cl

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-31 Thread Amit Kapila
On Mon, Oct 31, 2016 at 12:02 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > Hi, > > On 10/27/2016 01:44 PM, Amit Kapila wrote: > > I've read that analysis, but I'm not sure I see how it explains the "zig > zag" behavior. I do understand that shi

Re: [HACKERS] Microvacuum support for Hash Index

2016-10-28 Thread Amit Kapila
no deletable items on page. You can take the metapage lock before decrementing the count. 3. Assert(offnum <= maxoff); + Spurious space. There are some other similar spurious white space changes in patch, remove them as well. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpr

Re: [HACKERS] Proposal: scan key push down to heap [WIP]

2016-10-28 Thread Amit Kapila
On Fri, Oct 28, 2016 at 12:16 PM, Andres Freund <and...@anarazel.de> wrote: > On 2016-10-28 11:23:22 +0530, Amit Kapila wrote: > >> I think if we decide to form the scan key from a qual only when qual >> refers to fixed length column and that column is before any varlen

Re: [HACKERS] Proposal : For Auto-Prewarm.

2016-10-28 Thread Amit Kapila
y. One point that seems to be worth discussing is when should the buffer information be dumped to a file? Shall we dump at each checkpoint or at regular intervals via auto prewarm worker or at some other time? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent v

Re: [HACKERS] Proposal: scan key push down to heap [WIP]

2016-10-27 Thread Amit Kapila
some caching mechanism to cache the tuple offset , however it might not be as good as slot_getattr(). I think if we decide to form the scan key from a qual only when qual refers to fixed length column and that column is before any varlen column, the increased cost will be alleviated. Do you have an

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-27 Thread Amit Kapila
ot;Used by the background writer when it wants to create a restartpoint." and lastCheckPointEndPtr gets used when we create a restartpoint. I think there is no harm in further improving the comment, but I see nothing wrong as it is. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-10-27 Thread Amit Kapila
On Fri, Oct 28, 2016 at 2:52 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Oct 24, 2016 at 10:30 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Amit, can you please split the buffer manager changes in this patch >>> into a separate

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-27 Thread Amit Kapila
On Thu, Oct 27, 2016 at 4:15 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > On 10/25/2016 06:10 AM, Amit Kapila wrote: >> >> On Mon, Oct 24, 2016 at 2:48 PM, Dilip Kumar <dilipbal...@gmail.com> >> wrote: >>> >>> On Fri, Oct 21,

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-27 Thread Amit Kapila
t record spans across multiple segments, because you are updating minRecoveryPoint to start of checkpoint record. We need to update it to end+1 of checkpoint record. Please find attached patch which takes care of same. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Rename max_parallel_degree?

2016-10-26 Thread Amit Kapila
On Wed, Oct 26, 2016 at 5:54 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Oct 24, 2016 at 3:48 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Oct 13, 2016 at 5:28 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Wed, Oct 1

Re: [HACKERS] 9.6, background worker processes, and PL/Java

2016-10-26 Thread Amit Kapila
onsidered > an unfortunate mistake by the goofball who declared the first > function safe? > No, we don't detect that explicitly before initiating parallelism, however there are checks in code which will report error if you do something unsafe in worker, example perform any write operation in worker.

Re: [HACKERS] 9.6, background worker processes, and PL/Java

2016-10-26 Thread Amit Kapila
ate processes, this should be safe. Any imagined > speed advantage of the parallel query is likely to evaporate while > the several processes load their own JVMs, but nothing should > outright break. > > That leads me to: > > Are BGWs for parallel queries born fresh f

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Kapila
On Wed, Oct 26, 2016 at 3:04 PM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2016/10/26 17:57, Amit Kapila wrote: >> @@ -123,6 +123,9 @@ typedef struct RelationData >> { >> .. >> MemoryContext rd_partkeycxt; /* private memory cxt for the belo

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-26 Thread Amit Kapila
On Wed, Oct 26, 2016 at 12:39 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Oct 26, 2016 at 1:40 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> If you are inclined towards this solution, then I think what we need >> to do is to change the

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Kapila
e of the cases like tuple routing. Isn't it feasible to get it in some other way, may be by using relpartbound from pg_class tuple? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-25 Thread Amit Kapila
On Mon, Oct 24, 2016 at 12:25 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> I think what you are saying is not completely right, because we do >> update minRecovery

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-25 Thread Amit Kapila
On Tue, Oct 25, 2016 at 10:43 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Oct 24, 2016 at 12:26 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> You are right that it will include additional WAL than strictly >> necessary, but that

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
On Wed, Oct 26, 2016 at 8:27 AM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2016/10/26 11:41, Amit Kapila wrote: >> On Wed, Oct 26, 2016 at 6:36 AM, Amit Langote >> <langote_amit...@lab.ntt.co.jp> wrote: >>>> 1. >>>> @@

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
"list partition" works here as a verb, as in "to list partition". > I am not an expert of this matter, so probably some one having better grip can comment. Are we using something similar in any other error message? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
itionedRelationId, /* PARTEDRELID */ Here PARTEDRELID sounds inconvenient, how about PARTRELID? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Implement failover on libpq connect level.

2016-10-25 Thread Amit Kapila
e of differing number of elements and > neither host nor port is a singleton, as an error. The suggestion to > ignore some parts seems too error-prone. > +1. I also think returning error is better option in above case. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
n in your case it should never fire for parent table, which means we could disallow statement level triggers as well on parent tables? Some of the other things that we might want to consider disallowing on parent table could be: a. Policy on table_name b. Alter table has many clauses, are all of t

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-24 Thread Amit Kapila
ql.org/message-id/CAFiTN-uQ%2BJbd31cXvRbj48Ba6TqDUDpLKSPnsUCCYRju0Y0U8Q%40mail.gmail.com [6] - http://tvondra.bitbucket.org/#pgbench-300-unlogged-sync-skip [7] - http://tvondra.bitbucket.org/#pgbench-3000-unlogged-sync-skip [8] - https://www.postgresql.org/message-id/CAA4eK1J9VxJUnpOiQDf0O%

Re: [HACKERS] Hash Indexes

2016-10-24 Thread Amit Kapila
On Mon, Oct 24, 2016 at 8:00 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Sep 29, 2016 at 6:04 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Wed, Sep 28, 2016 at 3:04 PM, Robert Haas <robertmh...@gmail.com> wrote: > > > Thanks for the val

Re: [HACKERS] Rename max_parallel_degree?

2016-10-24 Thread Amit Kapila
etup, like a pointer to the >> resource-limiting GUC variable. > > I don't think this can work, per the above. > I see the value in Peter's point, but could not think of a way to implement the same. [1] - https://www.postgresql.org/message-id/CA%2BTgmoZS7DFFGz7kKWLoTAsNnXRKUiQFDt5yHgf4L73z52dgAQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-23 Thread Amit Kapila
;> minimum recovery point which precedes the point at which they will >> start recovery is no better than one which is equal to the point at >> which they will start recovery. > > This is a candidate that I thought of. But I avoided to change > the behavior of minRecoveryPoint

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-10-23 Thread Amit Kapila
I suspect somebody could figure out a solution to this > problem, though. > With this approach, don't we need something similar for API's pg_stop_backup()/pg_stop_backup_v2()? > If we decide we don't want to aim for one of these tighter solutions > and just adopt the previously-discussed patch, then at the very least > it needs better comments. > +1. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Index Scans

2016-10-21 Thread Amit Kapila
On Fri, Oct 21, 2016 at 10:55 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Oct 21, 2016 at 9:27 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I think the parallel_workers reloption should override the degree of >>> parallelism for any sort of pa

Re: [HACKERS] Parallel Index Scans

2016-10-21 Thread Amit Kapila
On Thu, Oct 20, 2016 at 10:33 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Oct 19, 2016 at 11:07 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Ideally, the parallel_workers storage parameter will rarely be >>> necessary because the optimizer

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-21 Thread Amit Kapila
On Fri, Oct 21, 2016 at 1:07 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > On 10/21/2016 08:13 AM, Amit Kapila wrote: >> >> On Fri, Oct 21, 2016 at 6:31 AM, Robert Haas <robertmh...@gmail.com> >> wrote: >>> >>> On Thu, O

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-10-21 Thread Amit Kapila
On Fri, Oct 21, 2016 at 4:25 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Oct 18, 2016 at 3:48 AM, Peter Geoghegan <p...@heroku.com> wrote: >> On Mon, Oct 17, 2016 at 5:36 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> I read the fol

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-10-21 Thread Amit Kapila
On Tue, Oct 18, 2016 at 3:48 AM, Peter Geoghegan <p...@heroku.com> wrote: > On Mon, Oct 17, 2016 at 5:36 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > I read the following paragraph from the Volcano paper just now: > > """ > During implementa

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-10-21 Thread Amit Kapila
also there will be cases where your patch can win. In particular, the Gather Merge can win where workers needs to perform sort mostly in-memory. I am not sure if it's easy to get best of both the worlds. Your patch needs rebase and I noticed one warning. sort\logtape.c(1422): warning C4

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-10-21 Thread Amit Kapila
85917 | 109472 | 99609 pgbench-3000-unlogged-sync-skip | 72 | LWLockTranche | clog |4623 |25692 | 10422 |11755 -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Gather Merge

2016-10-20 Thread Amit Kapila
On Tue, Oct 18, 2016 at 5:29 PM, Rushabh Lathia <rushabh.lat...@gmail.com> wrote: > On Mon, Oct 17, 2016 at 2:26 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> There is lot of common code between ExecGatherMerge and ExecGather. >> Do you think it m

<    3   4   5   6   7   8   9   10   11   12   >