Re: [HACKERS] Parallel Index Scans

2017-01-19 Thread Amit Kapila
On Tue, Jan 17, 2017 at 11:27 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Jan 16, 2017 at 7:11 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Fixed. > > Thanks for the update. Some more comments: > > It shouldn't be necessary for M

Re: [HACKERS] Parallel Index Scans

2017-01-18 Thread Amit Kapila
On Wed, Jan 18, 2017 at 6:25 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Mon, Jan 16, 2017 at 11:11 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> > > + * index_beginscan_parallel - join parallel index scan > > The name and the des

Re: [HACKERS] Parallel Index Scans

2017-01-18 Thread Amit Kapila
On Tue, Jan 17, 2017 at 11:27 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Jan 16, 2017 at 7:11 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > WAIT_EVENT_PARALLEL_INDEX_SCAN is in fact btree-specific. There's no > guarantee that any other AMs the imp

Re: [HACKERS] Gather Merge

2017-01-17 Thread Amit Kapila
t is ensured. With every query execution - server is > stopped and also OS caches were dropped. > - The reported values of execution time (in ms) is median of 3 executions. > - power2 machine with 512GB of RAM > - With default postgres.conf > You haven't mentioned scale factor used i

Re: [HACKERS] pageinspect: Hash index support

2017-01-17 Thread Amit Kapila
memory in this function? Also, why are you 5 as a multiplier in both the above pallocs, shouldn't it be 4? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-01-17 Thread Amit Kapila
other reviewers I can move it to >> "Ready For Committer"? > > Seeing no objections, I have moved it to Ready For Committer. > Thanks for the review. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] pageinspect: Hash index support

2017-01-17 Thread Amit Kapila
nsertion of 3000 records, I think at > first split we can see bucket page flag is set with LH_BITMAP_PAGE. > I think your calculation is not right. 66 indicates LH_BUCKET_PAGE | LH_BUCKET_NEEDS_SPLIT_CLEANUP which is a valid state after the split. This flag will be cleared either during ne

Re: [HACKERS] Supporting huge pages on Windows

2017-01-16 Thread Amit Kapila
w...@postgresql.org] On Behalf Of Amit Kapila >> PGSharedMemoryReAttach is called after the startup of new process whereas >> pgwin32_ReserveSharedMemoryRegion is called before the new process could >> actually start. Basically, pgwin32_ReserveSharedMemoryRegion is used to >&

Re: [HACKERS] Cache Hash Index meta page.

2017-01-16 Thread Amit Kapila
On Fri, Jan 13, 2017 at 9:58 AM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Fri, Jan 6, 2017 at 11:43 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: Below are review comments on latest version of patch. 1. /* - * Read the metapage to fetch original bucket and tuple co

Re: [HACKERS] parallelize queries containing subplans

2017-01-16 Thread Amit Kapila
On Thu, Jan 12, 2017 at 7:56 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Jan 12, 2017 at 8:51 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Wed, Jan 11, 2017 at 9:58 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> The other alterna

Re: [HACKERS] Parallel Index Scans

2017-01-16 Thread Amit Kapila
put them all in the same place. > Agreed and changed accordingly. > Why do all of these fields except for the last one have a ps_ prefix, > but the last one doesn't? > No specific reason, so Changed as per suggestion. > I assume "ps" stands for "parallel scan&quo

Re: [HACKERS] Parallel Index Scans

2017-01-14 Thread Amit Kapila
On Fri, Jan 13, 2017 at 7:58 PM, Anastasia Lubennikova <a.lubennik...@postgrespro.ru> wrote: > 27.12.2016 17:33, Amit Kapila: > > > The similar problem has occurred while testing "parallel index only > scan" patch and Rafia has included the fix in her patch [1] whi

Re: [HACKERS] parallelize queries containing subplans

2017-01-14 Thread Amit Kapila
u have reported. > I've not looked at the patches, but just seeing this error message, > this looks like somebody's fat-fingered the correspondence between > outfuncs.c and readfuncs.c processing. > I think what we need is catversion bump as Param is part of stored rules. -- With Rega

Re: [HACKERS] UNDO and in-place update

2017-01-13 Thread Amit Kapila
On Thu, Jan 12, 2017 at 8:56 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jan 11, 2017 at 5:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Moving further, I have thought about consistent reads and different >> formats for storing undo with p

Re: [HACKERS] parallelize queries containing subplans

2017-01-12 Thread Amit Kapila
On Thu, Jan 12, 2017 at 8:51 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jan 11, 2017 at 9:58 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> The other alternative is to remember this information in SubPlan. We >> can retrieve parallel_safe information

Re: [HACKERS] parallelize queries containing subplans

2017-01-11 Thread Amit Kapila
On Tue, Jan 10, 2017 at 11:55 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Dec 28, 2016 at 11:47 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Attached patch implements the above idea. This will enable >> parallelism for queries containing un-corr

Re: [HACKERS] parallelize queries containing subplans

2017-01-11 Thread Amit Kapila
On Tue, Jan 10, 2017 at 10:55 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Dec 28, 2016 at 1:17 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Currently, queries that have references to SubPlans or >> AlternativeSubPlans are considered parallel-restrict

Re: [HACKERS] UNDO and in-place update

2017-01-11 Thread Amit Kapila
On Tue, Jan 10, 2017 at 7:28 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Jan 9, 2017 at 11:47 PM, Robert Haas <robertmh...@gmail.com> wrote: >> >> Yes, something like this can be done. You don't really need any new >> page-level header data,

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-01-10 Thread Amit Kapila
o consider such background worker connections different from backend connections. However, I think we should document it either in parallel query or in background worker or in Create User .. Connection section. [1] - https://www.postgresql.org/message-id/20161222111345.25620.8603%40wrigleys

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Amit Kapila
et both features as the indirect index can be used in future if somebody implements clustered index. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UNDO and in-place update

2017-01-10 Thread Amit Kapila
On Mon, Jan 9, 2017 at 11:47 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Jan 9, 2017 at 7:50 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> One idea could be that we have some fixed number of >> slots (i think we can make it variable as well, but for s

Re: [HACKERS] Block level parallel vacuum WIP

2017-01-09 Thread Amit Kapila
On Mon, Jan 9, 2017 at 2:18 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Sat, Jan 7, 2017 at 2:47 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Fri, Jan 6, 2017 at 11:08 PM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>> On Mon

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-09 Thread Amit Kapila
On Mon, Jan 9, 2017 at 11:59 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jan 6, 2017 at 11:32 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sat, Jan 7, 2017 at 2:33 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> It looks to to me lik

Re: [HACKERS] UNDO and in-place update

2017-01-09 Thread Amit Kapila
On Fri, Jan 6, 2017 at 7:41 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jan 6, 2017 at 6:28 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Also, I'm thinking the bit could be stored in the line pointer rather >>> than the tuple, becau

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

2017-01-06 Thread Amit Kapila
On Sat, Jan 7, 2017 at 11:27 AM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Thu, Jan 5, 2017 at 6:15 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> Your test and results look good, what kind of m/c you have used to >> test this. Let me see i

Re: [HACKERS] Block level parallel vacuum WIP

2017-01-06 Thread Amit Kapila
s former idea, which makes the > group locking conflict between parallel workers when parallel worker > tries to acquire extension lock on same page. > How are planning to ensure the same in deadlock detector? Currently, deadlock detector considers members from same lock group as non-bl

Re: [HACKERS] _hash_addovflpage has a bug

2017-01-06 Thread Amit Kapila
e the lock on the tail page, check if we still need an overflow page, if so, then proceed, else return the already added overflow page. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UNDO and in-place update

2017-01-06 Thread Amit Kapila
On Thu, Jan 5, 2017 at 9:25 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jan 4, 2017 at 6:05 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Okay, so this optimization can work only after all the active >> transactions operating on a page are finished.

Re: [HACKERS] Group clear xid can leak semaphore count

2017-01-05 Thread Amit Kapila
On Fri, Jan 6, 2017 at 1:13 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Dec 31, 2016 at 12:44 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> During the review of Group update Clog patch [1], Dilip noticed an >> issue with the patch where it can leak

Re: [HACKERS] Cache Hash Index meta page.

2017-01-05 Thread Amit Kapila
(*bufp == so->hashso_bucket_buf || *bufp == so->hashso_split_bucket_buf)) just above this code will suffice the need? If so, then you can check it once and use it in both places. 5. The reader and insertion algorithm needs to be updated in README. -- With Regards, Amit Kapila. EnterpriseDB: http

Re: [HACKERS] Supporting huge pages on Windows

2017-01-05 Thread Amit Kapila
egeValue."))); + return FALSE; + } The order of closing handle and ereport is different here than other places in the same function. If there is no specific reason for doing so, then keep the order consistent. [1] - https://msdn.microsoft.com/en-us/library/windows/desktop/ff961911(v=vs.85).aspx -- Wit

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

2017-01-05 Thread Amit Kapila
On Wed, Jan 4, 2017 at 11:45 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > > On Tue, Jan 3, 2017 at 9:33 PM, Robert Haas <robertmh...@gmail.com> wrote: >> >> On Mon, Jan 2, 2017 at 1:36 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrot

Re: [HACKERS] UNDO and in-place update

2017-01-05 Thread Amit Kapila
On Wed, Jan 4, 2017 at 8:35 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Jan 4, 2017 at 4:35 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> In this new system, I >> think we can't remove undo entries of heap page till we clear >> correspondi

Re: [HACKERS] Odd behavior with PG_TRY

2017-01-05 Thread Amit Kapila
On Thu, Jan 5, 2017 at 4:18 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: > On 1/3/17 9:20 PM, Amit Kapila wrote: >> >> >> Good to know that it worked, but what is the theory? From your >> experiment, it appears that in some cases accessing local pointer >&g

Re: [HACKERS] UNDO and in-place update

2017-01-04 Thread Amit Kapila
On Tue, Dec 6, 2016 at 9:35 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 5, 2016 at 4:49 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I can very well understand the reason for not doing so (IIUC, it is >> about complexity and time to get it right

Re: [HACKERS] Proposal : Parallel Merge Join

2017-01-03 Thread Amit Kapila
otal->parallel_safe) + try_partial_mergejoin_path(root, + joinrel, + outerpath, + inner_cheapest_total, + merge_pathkeys, + mergeclauses, + NIL, + innersortkeys, + jointype, + extra); In above code indentation is broken, similarly, there is another code in a patch where it is broken, try usin

Re: [HACKERS] Odd behavior with PG_TRY

2017-01-03 Thread Amit Kapila
periment, it appears that in some cases accessing local pointer variables is okay and in other cases, it is not okay. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] increasing the default WAL segment size

2017-01-03 Thread Amit Kapila
fect performance as WAL segment size increases. > Sorry, but I am not able to understand why this is a problem? The bigger the size of WAL segment, lesser the number of files. So IIUC, then it can only impact if zero-ing two 16MB files is cheaper than zero-ing one 32MB file. Is that y

Re: [HACKERS] Supporting huge pages on Windows

2017-01-03 Thread Amit Kapila
r version of the patch looks better than the previous one. Don't you need to consider MEM_LARGE_PAGES in VirtualAllocEx call (refer pgwin32_ReserveSharedMemoryRegion)? At least that is what is mentioned in MSDN [1]. Another point worth considering is that currently for VirtualAllocEx() we use PA

Re: [HACKERS] parallelize queries containing subplans

2017-01-03 Thread Amit Kapila
On Wed, Dec 28, 2016 at 11:47 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Now, we can further extend this to parallelize queries containing > correlated subplans like below: > > explain select * from t1 where t1.i in (select t2.i from t2 where t2.i=t1.i); >

Re: [HACKERS] Odd behavior with PG_TRY

2017-01-02 Thread Amit Kapila
On Mon, Jan 2, 2017 at 10:43 PM, Jim Nasby <jim.na...@bluetreble.com> wrote: > On 1/2/17 1:31 AM, Amit Kapila wrote: >> >> On Mon, Jan 2, 2017 at 11:14 AM, Jim Nasby <jim.na...@bluetreble.com> >> wrote: >> >> Looks strange, what is the value of 'i

Re: [HACKERS] Odd behavior with PG_TRY

2017-01-01 Thread Amit Kapila
n side effects : Couldn't dematerialize a > result variable: couldn't read its memory > Looks strange, what is the value of 'i'? Did you get the same result if you try to print args->in.r.atts[0] inside PG_TRY? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com --

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

2017-01-01 Thread Amit Kapila
On Mon, Jan 2, 2017 at 10:59 AM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Mon, Jan 2, 2017 at 10:17 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Mon, Jan 2, 2017 at 9:28 AM, Pavan Deolasee <pavan.deola...@gmail.com> >> wrot

Re: [HACKERS] Cache Hash Index meta page.

2017-01-01 Thread Amit Kapila
On Tue, Dec 27, 2016 at 1:36 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Thu, Dec 22, 2016 at 12:17 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Wed, Dec 21, 2016 at 9:26 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Tue, Dec 20

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

2017-01-01 Thread Amit Kapila
On Mon, Jan 2, 2017 at 9:28 AM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > > On Mon, Jan 2, 2017 at 8:52 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> >> I think there is some chance that such a change could induce >> regression fo

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

2017-01-01 Thread Amit Kapila
mention about indirect indexes in above comment considering indirect indexes are still not part of core code? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Group clear xid can leak semaphore count

2016-12-30 Thread Amit Kapila
in the committed patch, it got introduced in commit 4aec49899e5782247e134f94ce1c6ee926f88e1c. Patch to fix the issue is attached. [1] - https://www.postgresql.org/message-id/CAA4eK1J%2B67edo_Wnrfx8oJ%2BrWM_BAr%2Bv6JqvQjKPdLOxR%3D0d5g%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB

Re: [HACKERS] [sqlsmith] Short reads in hash indexes

2016-12-30 Thread Amit Kapila
On Fri, Dec 30, 2016 at 3:45 AM, Andreas Seltenreich <seltenre...@gmx.de> wrote: > Amit Kapila writes: > >> Can you please try with the patch posted on hash index thread [1] to >> see if you can reproduce any of these problems? >> >> [1] - >

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-30 Thread Amit Kapila
ear in such case we will > not make any call to PGSemaphoreUnlock. > Good catch! I have fixed it by initialising extraWaits to 0. This same issue exists from Group clear xid for which I will send a patch separately. Apart from above, the patch needs to be adjusted for commit be7b2848 whi

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Amit Kapila
r execution time). > Hmm, have you checked what output patch gives for above command, in above usage, it will print both planning and execution time. IIUC, then patch doesn't do what you have in mind. As far as I understand the proposed usage for the summary parameter was only for

Re: [HACKERS] Duplicate node tag assignments

2016-12-28 Thread Amit Kapila
're working in gdb, say, > it's easy enough to convert: > > (gdb) p (int) T_CreateReplicationSlotCmd > $8 = 950 > (gdb) p (enum NodeTag) 949 > $9 = T_BaseBackupCmd > > So I'm leaning to the second, more drastic, solution. > Sounds sensible. -- With Regards, Amit K

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Amit Kapila
On Wed, Dec 28, 2016 at 10:55 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Jim Nasby <jim.na...@bluetreble.com> writes: >> On 12/28/16 7:10 AM, Amit Kapila wrote: >>> Can we think of introducing new guc trace_system_waits or something >>> like that which will

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Amit Kapila
nk of introducing new guc trace_system_waits or something like that which will indicate that the sessions will report the value of wait_start in pg_stat_activity? The default value of such a parameter can be false which means wait_start will be shown as NULL in pg_stat_activity and when it is en

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Amit Kapila
mmary && es->analyze) Do you really need es->summary in above check? We should update documentation of Explain command, but maybe that can wait till we finalize the specs. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (p

[HACKERS] parallelize queries containing initplans

2016-12-28 Thread Amit Kapila
rallelized, then use the second approach (one-time evaluation by master backend and share the result with workers), otherwise use the first approach of pushing down the initplan to workers. Thoughts? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-

Re: [HACKERS] Parallel Index-only scan

2016-12-27 Thread Amit Kapila
PCV4y06o_v%3DZdZ1UThE%2BW9JhthX4B8uifnA%40mail.gmail.com > The link for the latest version is wrong, the correct link is: https://www.postgresql.org/message-id/CAA4eK1KthrAvNjmB2cWuUHz%2Bp3ZTTtbD7o2KUw49PC8HK4r1Wg%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com --

[HACKERS] parallelize queries containing subplans

2016-12-27 Thread Amit Kapila
n use parallelism. I think this will be a separate project in itself if we want to do this and based on my study of TPC-H and TPC-DS queries, I am confident that this will be helpful in certain queries at higher scale factors. Thoughts? -- With Regards, Amit Kapila. EnterpriseDB: http:/

Re: [HACKERS] Parallel Index Scans

2016-12-27 Thread Amit Kapila
dicates that the block number returned is either valid including > P_NONE > + * and scan is continued or block number is invalid and scan has just > + * begun. > I think the modification (including P_NONE and scan is continued) suggested by you can confuse the reader, because if the return

Re: [HACKERS] Parallel Index Scans

2016-12-27 Thread Amit Kapila
On Fri, Dec 23, 2016 at 6:42 PM, Anastasia Lubennikova <a.lubennik...@postgrespro.ru> wrote: > 22.12.2016 07:19, Amit Kapila: >> >> On Wed, Dec 21, 2016 at 8:46 PM, Anastasia Lubennikova >> <lubennikov...@gmail.com> wrote: >>> >>> The foll

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-22 Thread Amit Kapila
at point as a note to committer and let him take a final call. From the code perspective already Robert and Andres have taken one pass of review and I have addressed all their comments, so surely more review of code can help, but I think that is not a big deal considering patch size is relatively smal

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-12-22 Thread Amit Kapila
On Thu, Dec 22, 2016 at 9:56 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 5, 2016 at 2:46 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I'll review after that, since I have other things to review meanwhile. >> >> Attached, please find th

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Amit Kapila
istent. I don't >> mind changing the name, but not to consistency LSN. > > Well, progress could just as well be replay. Or the actual insertion > point. Or up to where we've written out. Or synced out. Or > replicated > > Open to other suggestions - I'm not really happy wi

Re: [HACKERS] Proposal for CSN based snapshots

2016-12-22 Thread Amit Kapila
z1Kk1DQgmy0isC7%3DOgX%2B3JtfGk9g%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Cache Hash Index meta page.

2016-12-21 Thread Amit Kapila
s will make metpage cache access somewhat similar to what we have in btree where we use cache to access rootpage. Will something like that address your concern? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Parallel Index Scans

2016-12-21 Thread Amit Kapila
On Thu, Dec 22, 2016 at 9:49 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Dec 21, 2016 at 8:46 PM, Anastasia Lubennikova > <lubennikov...@gmail.com> wrote: >> The following review has been posted through the commitfest application: >> make insta

Re: [HACKERS] Parallel Index Scans

2016-12-21 Thread Amit Kapila
te the state of parallel scan. > * Below flags are used TO indicate the state of parallel scan. > > * On success, release lock and pin on buffer on success. > * On success release lock and pin on buffer. > Will fix. > 8. I didn't find a description of the feature in documentation. &

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-21 Thread Amit Kapila
On Wed, Dec 21, 2016 at 9:46 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Dec 20, 2016 at 11:32 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Ashutosh Sharma has helped to test that pldebugger issue is fixed with >> attached version. > > Commi

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 8:04 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Dec 17, 2016 at 5:46 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Yeah, but we are planning to add a generic flag in WaitEvent structure >> which can be used for similar pur

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 7:44 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Dec 20, 2016 at 9:01 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Tue, Dec 20, 2016 at 7:11 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Tue, Dec 20, 2016

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 7:11 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Dec 20, 2016 at 4:51 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> We have mainly four actions for squeeze operation, add tuples to the >> write page, empty overflow page,

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Mon, Dec 19, 2016 at 11:05 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Dec 18, 2016 at 8:54 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I committed remove-hash-wrtbuf and fix_dirty_marking_v1 but I've got >>> some reservations

Re: [HACKERS] Hash Indexes

2016-12-18 Thread Amit Kapila
On Fri, Dec 16, 2016 at 9:57 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Dec 15, 2016 at 11:33 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Attached are the two patches on top of remove-hash-wrtbuf. Patch >> fix_dirty_marking_v1.patch allows to m

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-17 Thread Amit Kapila
So, for me i think Amit's fix is > absolutely fine and is restricted to Windows. > It is fine as per current usage of WaitEventSet API's, however, Robert's point is valid that user is not obliged to call ModifyWaitEvent before WaitEventSetWait. Imagine a case where some new us

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-17 Thread Amit Kapila
On Sat, Dec 17, 2016 at 9:34 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Dec 16, 2016 at 10:34 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I think it should be the responsibility of >>> WaitEventSetWaitBlock() to reset the

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-16 Thread Amit Kapila
On Fri, Dec 16, 2016 at 9:03 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Dec 15, 2016 at 4:17 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Dec 15, 2016 at 10:04 AM, Andres Freund <and...@anarazel.de> wrote: >>> Hi, >>> >

Re: [HACKERS] Hash Indexes

2016-12-15 Thread Amit Kapila
On Wed, Dec 14, 2016 at 10:47 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Dec 14, 2016 at 4:27 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> Yeah, it will fix the problem in hashbucketcleanup, but there are two >> other problems that need

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-15 Thread Amit Kapila
event due to which we are seeing this behavior. That seems to be required as per msdn as well, as pointed by Robert upthread. Find attached patch to implement the resetting of event only for the required case. This fixes the reported problem. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterp

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-14 Thread Amit Kapila
nsensus until PostgreSQL 10 is actually released. Thought? > > +1 on that. > +1. That is the safest option to proceed. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-14 Thread Amit Kapila
et *after* > the while loop in WaitEventSetWait(). > As mentioned above, the current patch doesn't do it correctly. > I could understand something > happening inside the loop if WSAEnumNetworkEvents() updates things on > the fly though... > No, nothing wrong in that, it

Re: [HACKERS] Hash Indexes

2016-12-14 Thread Amit Kapila
On Tue, Dec 13, 2016 at 11:30 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 12, 2016 at 9:21 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> The reason is to make the operations consistent in master and standby. >> In WAL patch, for clearing the S

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-13 Thread Amit Kapila
Ashutosh will post a patch on those lines later today. Let us know if you have something else in mind. [1] - https://msdn.microsoft.com/en-us/library/windows/desktop/ms741576(v=vs.85).aspx -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-12 Thread Amit Kapila
On Mon, Dec 12, 2016 at 9:54 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Mon, Dec 12, 2016 at 9:52 PM, Fujii Masao <masao.fu...@gmail.com> wrote: >> On Mon, Dec 12, 2016 at 9:31 PM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>> On

Re: [HACKERS] Hash Indexes

2016-12-12 Thread Amit Kapila
On Tue, Dec 13, 2016 at 2:51 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Dec 11, 2016 at 1:24 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> With above fixes, the test ran successfully for more than a day. > > Instead of doing this: > >

Re: [HACKERS] Hash Indexes

2016-12-11 Thread Amit Kapila
On Mon, Dec 12, 2016 at 10:25 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sun, Dec 11, 2016 at 8:37 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Sun, Dec 11, 2016 at 11:54 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >&

Re: [HACKERS] Hash Indexes

2016-12-11 Thread Amit Kapila
On Tue, Dec 6, 2016 at 1:29 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Thu, Dec 1, 2016 at 10:54 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: > > With the latest HASH WAL patch applied, I get different but apparently > related errors > > 41993 UPD

Re: [HACKERS] Hash Indexes

2016-12-11 Thread Amit Kapila
On Sun, Dec 11, 2016 at 11:54 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Dec 7, 2016 at 2:02 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > > With above fixes, the test ran successfully for more than a day. > There was a small typo in the previous patch wh

Re: [HACKERS] [sqlsmith] Short reads in hash indexes

2016-12-10 Thread Amit Kapila
On Thu, Dec 8, 2016 at 2:38 AM, Andreas Seltenreich <seltenre...@gmx.de> wrote: > Andreas Seltenreich writes: > >> Amit Kapila writes: >> >>> On Sat, Dec 3, 2016 at 3:44 PM, Andreas Seltenreich <seltenre...@gmx.de> >>> wrote: >

Re: [HACKERS] Hash Indexes

2016-12-10 Thread Amit Kapila
On Wed, Dec 7, 2016 at 2:02 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Tue, Dec 6, 2016 at 4:00 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Tue, Dec 6, 2016 at 1:29 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> > >> > >

Re: [HACKERS] Parallel bitmap heap scan

2016-12-10 Thread Amit Kapila
is missing at multiple places in above line. It should be written as below: dsa_entry = (void *)(((char *) dsa_entry) + sizeof(dsa_pointer)); Similar stuff needs to be taken care at other places in the patch as well. I think it will be better if you run pgindent on your patch. -- With Regards, Ami

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-10 Thread Amit Kapila
* member_names contains nmembers consecutive nul-terminated C strings */ char member_names[FLEXIBLE_ARRAY_MEMBER]; } SyncRepConfigData; Can't we use 1 or 2 bytes to store sync_method information? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] Short reads in hash indexes

2016-12-07 Thread Amit Kapila
On Thu, Dec 8, 2016 at 2:38 AM, Andreas Seltenreich <seltenre...@gmx.de> wrote: > Andreas Seltenreich writes: > >> Amit Kapila writes: >> >>> On Sat, Dec 3, 2016 at 3:44 PM, Andreas Seltenreich <seltenre...@gmx.de> >>> wrote: >

Re: [HACKERS] Parallel execution and prepared statements

2016-12-07 Thread Amit Kapila
rom a parallel worker? In any case, the current wording >> of the comment is a complete fail at explaining this. > > Oops. You're right. [ uh, why exactly? ] -> no database changes > whatsoever are allowed while in parallel mode. (This restriction > might be lif

Re: [HACKERS] Hash Indexes

2016-12-06 Thread Amit Kapila
reproduce the issue, but in the meantime, if by anychance, you have a callstack, then please share the same. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UNDO and in-place update

2016-12-05 Thread Amit Kapila
On Thu, Dec 1, 2016 at 8:55 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Nov 29, 2016 at 12:21 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > I see what you're going for, but I'm not sure it's worth it. I mean, > say you just have one bit per index tuple

Re: [HACKERS] commitfest 2016-11 status summary

2016-12-05 Thread Amit Kapila
s done a good job in wrapping up this commitfest. However, we might want to consider sending one consolidated mail for the patches "moved to next CF" or "Returned with Feedback" or "Rejected" rather than sending independent mails. That might save some traffic on the l

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-04 Thread Amit Kapila
On Mon, Dec 5, 2016 at 6:00 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Fri, Nov 4, 2016 at 8:20 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Thu, Nov 3, 2016 at 8:38 PM, Robert Haas <robertmh...@gmail.com> wrote: >> &g

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_splitbucket_guts

2016-12-03 Thread Amit Kapila
On Sat, Dec 3, 2016 at 3:44 PM, Andreas Seltenreich <seltenre...@gmx.de> wrote: > Amit Kapila writes: > >> How should I connect to this database? If I use the user fdw >> mentioned in pg_hba.conf (changed authentication method to trust in >> pg_hba.conf), it says th

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-03 Thread Amit Kapila
On Fri, Dec 2, 2016 at 9:50 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Nov 30, 2016 at 7:53 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> + * Switch segment only when WAL has done some progress since the >> + * > last time a segment

Re: [HACKERS] Hash Indexes

2016-12-02 Thread Amit Kapila
On Sat, Dec 3, 2016 at 12:13 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Dec 2, 2016 at 1:54 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I want to split when the average bucket >>> contains 10 pages worth of tuples. >> >> oh,

Re: [HACKERS] Parallel execution and prepared statements

2016-12-02 Thread Amit Kapila
with me. > >> Am 02.12.2016 um 07:22 schrieb Amit Kapila <amit.kapil...@gmail.com>: >> Done that way in attached patch. > > Did a quick review: > Thanks for the review. > You should however include a sentence in the documentation on that parallel > plan w/

<    2   3   4   5   6   7   8   9   10   11   >