Re: [HACKERS] Reviewing freeze map code

2016-07-23 Thread Amit Kapila
On Mon, Jul 18, 2016 at 2:03 PM, Andres Freund <and...@anarazel.de> wrote: > On 2016-07-18 10:02:52 +0530, Amit Kapila wrote: >> > >> >> Consider the below scenario. >> >> Vacuum >> a. acquires a cleanup lock for page - 10 >> b. busy in checkin

Re: [HACKERS] freeze map open item

2016-07-22 Thread Amit Kapila
critical but worthy of being fixed. I recommend we open > a new open issue for that, or at least fix it. > +1. I think, sometime needs to be spend to confirm if that theoretical risk can practically cause any problem. I am planning to spend some time on it in next week or so and will rep

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-07-21 Thread Amit Kapila
s, then he has to artificially add where clause which seems slightly inconvenient, but may be such cases are less. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-20 Thread Amit Kapila
ange the status of patch accordingly. [1] - https://www.postgresql.org/message-id/CAB7nPqTv5gmKQcNDoFGTGqoqXz2xLz4RRw247oqOJzZTVy6-7Q%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-20 Thread Amit Kapila
On Wed, Jul 20, 2016 at 5:12 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Jul 19, 2016 at 9:08 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Tue, Jul 19, 2016 at 10:43 AM, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-19 Thread Amit Kapila
On Wed, Jul 20, 2016 at 7:57 AM, Andres Freund <and...@anarazel.de> wrote: > > > On July 19, 2016 7:14:42 PM PDT, Amit Kapila <amit.kapil...@gmail.com> wrote: >>On Wed, Jul 20, 2016 at 5:02 AM, Andres Freund <and...@anarazel.de> >>wrote: >>> On

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-19 Thread Amit Kapila
old. Do you have something else in mind? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-19 Thread Amit Kapila
On Tue, Jul 19, 2016 at 10:43 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sat, Jul 16, 2016 at 9:20 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Wed, Jul 13, 2016 at 8:56 AM, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >>

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Amit Kapila
On Mon, Jul 18, 2016 at 9:13 AM, Andres Freund <and...@anarazel.de> wrote: > On 2016-07-18 09:07:19 +0530, Amit Kapila wrote: >> + /* >> + * Before locking the buffer, pin the visibility map page if it may be >> + * necessary. >> + */ >> >>

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Amit Kapila
On Mon, Jul 18, 2016 at 8:18 AM, Andres Freund <and...@anarazel.de> wrote: > On 2016-07-16 10:45:26 -0700, Andres Freund wrote: >> >> >> On July 16, 2016 8:49:06 AM PDT, Tom Lane <t...@sss.pgh.pa.us> wrote: >> >Amit Kapila <amit.kapil...@gmail.com&g

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-16 Thread Amit Kapila
On Wed, Jul 13, 2016 at 8:56 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Jul 12, 2016 at 12:22 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I think updating minRecoveryPoint unconditionally can change it's >> purpose in some cases. R

Re: [HACKERS] Reviewing freeze map code

2016-07-16 Thread Amit Kapila
case) to have been updated. > > I think we have two choices how to deal with that: First, we can add a > new flags variable to xl_heap_lock similar to > xl_heap_insert/update/... and bump page magic, > +1 for going in this way. This will keep us consistent with how clear the visibi

Re: [HACKERS] Unable to test parallel aggregate/joins in Postgres beta 2

2016-07-14 Thread Amit Kapila
cases it does not choose a parallel plan. > > Is there any other considerations apart from the mentioned ones on why pg > would not choose a parallel plan ? > You can try by setting parallel_setup_cost=0 and parallel_tuple_cost=0, though changing that way is not advisable. If that doe

Re: [HACKERS] Reviewing freeze map code

2016-07-14 Thread Amit Kapila
YMAP_ALL_FROZEN); >> + } >> + > > FWIW, I don't think it's worth introducing visibilitymap_clear_extended. > As this is a 9.6 only patch, i think it's better to change > visibilitymap_clear's API. > > Unless somebody protests I'm planning to commit with those adjustments >

Re: [HACKERS] Hash Indexes

2016-07-14 Thread Amit Kapila
On Wed, Jun 22, 2016 at 8:48 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jun 22, 2016 at 5:14 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> We can do it in the way as you are suggesting, but there is another thing >> which we need to consider here.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-13 Thread Amit Kapila
add a check for HeapTupleSatisfiesToast in TestForOldSnapshot()? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_basebackup wish list

2016-07-13 Thread Amit Kapila
nother possibility is to enhance -P option as -P sec, such that it will display progress after ever 'sec' seconds. Something like we have for pgbench. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Simplifying the interface of UpdateMinRecoveryPoint

2016-07-13 Thread Amit Kapila
if it got introduced due to (c), then your patch does the right thing by removing it. Personally, I feel overloading the parameter for multiple purposes makes code less maintainable, so retaining as it is in HEAD has some merits. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.co

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-11 Thread Amit Kapila
merits, but I think doing it unconditionally might lead to change in behaviour in some cases. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-11 Thread Amit Kapila
st of open items, adding Magnus in CC whose >> commit for non-exclusive backups is at the origin of this defect. > > > I agree this looks correct. > > But isn't this also a pre-existing bug in 9.5? Or did we change something > else that suddenly made it visible? > I think th

Re: [HACKERS] INSERT .. SET syntax

2016-07-10 Thread Amit Kapila
in the documentation part of the patch. I noticed that you have used transformUpdateTargetList() to generate expression list for this case, but that function raises some internal errors which indicates that error is from Update. I think that might be misleading to users, if they ever got raised. -

Re: [HACKERS] [BUGS] BUG #14230: Wrong timeline returned by pg_stop_backup on a standby

2016-07-09 Thread Amit Kapila
ackup_label file. In > short, +1 for this fix. > +1, the fix looks right to me as well. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-07-08 Thread Amit Kapila
ROZEN(relation, block, )) + { + visibilitymap_clear_extended(relation, block, vmbuffer, + VISIBILITYMAP_ALL_FROZEN); + } + + if (RelationNeedsWAL(relation)) + { .. + XLogBeginInsert(); + XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); + + xlrec.offnum = ItemPointerGetOffsetNumber(_self);

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Amit Kapila
> "Parallelism". > Another option could be to name it as Parallel Mode. We are using that in the description of "Parallel" in "Create Function" documentation. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] copyParamList

2016-07-08 Thread Amit Kapila
k we can just assign from->paramMask to retval->paramMask to make sure that even if it gets used in future, the code works as expected. Alternatively, one might think of adding an Assert there, but that doesn't seem to be future-proof. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpr

Re: [HACKERS] EXPLAIN ANALYZE for parallel query doesn't report the SortMethod information.

2016-07-07 Thread Amit Kapila
On Thu, Jul 7, 2016 at 7:37 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Thu, Jul 7, 2016 at 1:23 PM, Fujii Masao <masao.fu...@gmail.com> wrote: >>> I found $SUBJECT while trying to test parallel qu

Re: [HACKERS] EXPLAIN ANALYZE for parallel query doesn't report the SortMethod information.

2016-07-07 Thread Amit Kapila
me=0.021..181.079 rows=100 > loops=1) > I think this can never happen for force_parallel_mode TO off, because we don't generate a gather on top of sort node. The reason why we are able to push Sort below gather, because it is marked as parallel_safe (create_sort_path). I think we sho

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-06 Thread Amit Kapila
> possibly use that here too, once we have it. > makes sense. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-07-06 Thread Amit Kapila
>> +#define XLHL_ALL_VISIBLE_CLEARED 0x20 > > Hm. We can't easily do that in the back-patched version; because a > standby won't know to check for the flag . That's kinda ok, since we > don't yet need to clear all-visible yet at that point of > heap_update. But that better means

Re: [HACKERS] Hash Indexes

2016-07-06 Thread Amit Kapila
On Wed, Jun 22, 2016 at 8:44 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jun 22, 2016 at 5:10 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> > Insertion will happen by scanning the appropriate bucket and needs to >>> > retain pin on primary

Re: [HACKERS] to_date_valid()

2016-07-04 Thread Amit Kapila
Won't the additional function calls make to_date much costlier than its current implementation? I don't know if there is a better way, but I think it is worth to consider, if we can find a cheaper way to detect validity of date. Note - Your patch is small (~13KB) enough that it doesn't nee

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

2016-07-04 Thread Amit Kapila
nts in nearby code. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-07-04 Thread Amit Kapila
On Mon, Jul 4, 2016 at 2:31 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Sat, Jul 2, 2016 at 12:17 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sat, Jul 2, 2016 at 12:53 AM, Andres Freund <and...@anarazel.de> wrote: >>> On 2016-07-

Re: [HACKERS] Re: [ADMIN] problems using pg_start_backup/pg_stop_backup and pg_basebackup at same time

2016-07-02 Thread Amit Kapila
ne dependency of archiver (2) to finish which is that corresponding .ready file should be generated. Can it be possible, that due to some reason the .ready file is not generated corresponding to stoppoint determined by pg_stop_backup()? Is it possible to verify if the required .ready files are p

Re: [HACKERS] fixing subplan/subquery confusion

2016-07-01 Thread Amit Kapila
der_parallel = false; + What exactly makes Append plan to not able to run some of the child nodes is other process? [1] - https://www.postgresql.org/message-id/CAA4eK1Jg_GvaTEjJSaV5vZY6acDmi-B3iXWvdiXa%2BpUFbnkyTg%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 7:52 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Fri, Jul 1, 2016 at 11:12 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> Why do you think IndexOnlyScan will return wrong result? If the >> server crash in the way

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Amit Kapila
ther we need to have a new API or add a new parameter to visibilitymap_clear() to indicate the same. If we want to go that route, isn't it better to have PD_ALL_FROZEN as well? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 9:30 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jul 1, 2016 at 10:09 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>>> Ah, I see. So your suggestion is to do this job in lazy_scan_heap() >>>> when scanning each block

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Amit Kapila
r > all_visible_according_to_vm at the beginning of that so that we don't > add more visibility map checks for pages where we already know the VM > bit can't possibly be set. > > Other opinions on the concept or the patch? > +1 on the idea. + PageClearAllVisible(page); + Mark

Re: [HACKERS] [sqlsmith] crashes in RestoreSnapshot on hot standby

2016-06-30 Thread Amit Kapila
On Fri, Jul 1, 2016 at 9:38 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Fri, Jul 1, 2016 at 3:25 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Fri, Jul 1, 2016 at 8:48 AM, Thomas Munro <thomas.mu...@enterprisedb.com> >> wrote: >

Re: [HACKERS] [sqlsmith] crashes in RestoreSnapshot on hot standby

2016-06-30 Thread Amit Kapila
reason of crash. You can see how CopySnapshot calculates the subxipoff, may be writing code that way will be more consistent. In case of recovery, I think serialized_snapshot->xcnt will always be zero as we fill everything in subxip array (refer below code in GetSnapshotData). GetSnapshotDa

Re: [HACKERS] Reviewing freeze map code

2016-06-30 Thread Amit Kapila
On Thu, Jun 30, 2016 at 8:10 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Thu, Jun 30, 2016 at 3:12 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Jun 30, 2016 at 9:13 AM, Andres Freund <and...@anarazel.de> wrote: >>> On 2016-06-

Re: [HACKERS] fixing subplan/subquery confusion

2016-06-30 Thread Amit Kapila
On Tue, Jun 28, 2016 at 5:22 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Jun 28, 2016 at 8:25 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> In the appendrel case, I tend to agree that the easiest solution is to >> scan all the children of the appendrel

Re: [HACKERS] Reviewing freeze map code

2016-06-30 Thread Amit Kapila
On Thu, Jun 30, 2016 at 9:13 AM, Andres Freund <and...@anarazel.de> wrote: > On 2016-06-30 08:59:16 +0530, Amit Kapila wrote: >> On Wed, Jun 29, 2016 at 10:30 PM, Andres Freund <and...@anarazel.de> wrote: >> > On 2016-06-29 19:04:31 +0530, Amit Kapila wrote: >>

Re: [HACKERS] Reviewing freeze map code

2016-06-29 Thread Amit Kapila
On Wed, Jun 29, 2016 at 10:30 PM, Andres Freund <and...@anarazel.de> wrote: > On 2016-06-29 19:04:31 +0530, Amit Kapila wrote: >> There is nothing in this record which recorded the information about >> visibility clear flag. > > I think we can actually defer the clearin

Re: [HACKERS] Rename max_parallel_degree?

2016-06-29 Thread Amit Kapila
On Wed, Jun 29, 2016 at 11:54 AM, Julien Rouhaud <julien.rouh...@dalibo.com> wrote: > On 29/06/2016 06:29, Amit Kapila wrote: >> On Wed, Jun 29, 2016 at 2:57 AM, Julien Rouhaud >> <julien.rouh...@dalibo.com> wrote: >>> >>> Thanks a lot for the help! &

Re: [HACKERS] Reviewing freeze map code

2016-06-29 Thread Amit Kapila
On Wed, Jun 29, 2016 at 11:14 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Fri, Jun 24, 2016 at 11:04 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Fri, Jun 24, 2016 at 4:33 AM, Andres Freund <and...@anarazel.de> wrote: >>> On 2016-06-23

Re: [HACKERS] Rename max_parallel_degree?

2016-06-29 Thread Amit Kapila
mething else? If earlier, then I am not sure if it is good idea, because it can cause some confusion to the user about usage of both the parameters together. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] Rename max_parallel_degree?

2016-06-28 Thread Amit Kapila
the description of max_parallel_worker, user can set its value more than max_wroker_processes which we don't want. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] fixing subplan/subquery confusion

2016-06-28 Thread Amit Kapila
On Tue, Jun 28, 2016 at 8:25 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> I had couple of questions [1] related to that patch. See if you find >> those as relevant? > > I do not think those cases are direct

Re: [HACKERS] Rename max_parallel_degree?

2016-06-27 Thread Amit Kapila
On Mon, Jun 27, 2016 at 10:35 PM, Julien Rouhaud <julien.rouh...@dalibo.com> wrote: > On 27/06/2016 07:18, Amit Kapila wrote: >> On Mon, Jun 27, 2016 at 10:21 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >>> >>> I think as the parallel_t

Re: [HACKERS] fixing subplan/subquery confusion

2016-06-27 Thread Amit Kapila
couple of questions [1] related to that patch. See if you find those as relevant? [1] - https://www.postgresql.org/message-id/CAA4eK1%2ByGs-onuJDy%2BTTqnrnT0hty_QQPC1GipS%2Bce-W3872QQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rename max_parallel_degree?

2016-06-26 Thread Amit Kapila
On Mon, Jun 27, 2016 at 10:21 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sun, Jun 26, 2016 at 3:57 PM, Julien Rouhaud > <julien.rouh...@dalibo.com> wrote: >> On 26/06/2016 08:37, Amit Kapila wrote: >>> >>> @@ -370,6 +379,8 @@ ForgetBackgroundWo

Re: [HACKERS] Rename max_parallel_degree?

2016-06-26 Thread Amit Kapila
On Sun, Jun 26, 2016 at 3:57 PM, Julien Rouhaud <julien.rouh...@dalibo.com> wrote: > On 26/06/2016 08:37, Amit Kapila wrote: >> >> @@ -370,6 +379,8 @@ ForgetBackgroundWorker(slist_mutable_iter *cur) >> Assert(rw->rw_shmem_slot < >> max_worker_processe

Re: [HACKERS] Rename max_parallel_degree?

2016-06-26 Thread Amit Kapila
On Sat, Jun 25, 2016 at 2:27 PM, Julien Rouhaud <julien.rouh...@dalibo.com> wrote: > On 25/06/2016 09:33, Amit Kapila wrote: >> On Wed, Jun 15, 2016 at 11:43 PM, Julien Rouhaud >> <julien.rouh...@dalibo.com> wrote: >>> >>> Attached v4 implements the

Re: [HACKERS] Rename max_parallel_degree?

2016-06-25 Thread Amit Kapila
. I think it is better to change above in documentation to indicate that "pool of processes established by guc-max-parallel-workers". -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] Hash Indexes

2016-06-24 Thread Amit Kapila
On Fri, Jun 24, 2016 at 2:38 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Thu, Jun 16, 2016 at 12:58 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: > > I have a question regarding code changes in _hash_first. > > +/* > + * Conditional

Re: [HACKERS] Hash Indexes

2016-06-23 Thread Amit Kapila
On Thu, Jun 23, 2016 at 10:56 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jun 22, 2016 at 10:13 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> A scan that has seen the flag won't look at the >>> tuple in any case. >> >> Why so?

Re: [HACKERS] Reviewing freeze map code

2016-06-23 Thread Amit Kapila
gt; there in the back branches. > You are right, I think we can try such an optimization in Head and that too if we see a performance hit with adding this new WAL in heap_update. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Indexes

2016-06-22 Thread Amit Kapila
On Wed, Jun 22, 2016 at 8:48 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jun 22, 2016 at 5:14 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> We can do it in the way as you are suggesting, but there is another thing >> which we need to consider here.

Re: [HACKERS] Hash Indexes

2016-06-22 Thread Amit Kapila
On Wed, Jun 22, 2016 at 8:44 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jun 22, 2016 at 5:10 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> >>> I think this is basically correct, although I don't find it to be as >>> clear as I think

Re: [HACKERS] Hash Indexes

2016-06-22 Thread Amit Kapila
On Tue, Jun 21, 2016 at 9:33 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Thu, Jun 16, 2016 at 3:28 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> Incomplete splits can be completed either by vacuum or insert as both > >> needs exclusive lo

Re: [HACKERS] Hash Indexes

2016-06-22 Thread Amit Kapila
On Tue, Jun 21, 2016 at 9:26 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Tue, May 10, 2016 at 8:09 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > Once the split operation has set the split-in-progress flag, it will begin scanning bucket (N+1)/2.

Re: [HACKERS] Reviewing freeze map code

2016-06-20 Thread Amit Kapila
On Tue, Jun 21, 2016 at 9:21 AM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-06-21 08:59:13 +0530, Amit Kapila wrote: > > Can we consider to use some strategy to avoid deadlocks without releasing > > the lock on old page? Consider if w

Re: [HACKERS] Reviewing freeze map code

2016-06-20 Thread Amit Kapila
On Tue, Jun 21, 2016 at 9:16 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > > On Tue, Jun 21, 2016 at 3:29 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Some others ways could be: > > > > Before releasing the lock on buffer containing old tu

Re: [HACKERS] Reviewing freeze map code

2016-06-20 Thread Amit Kapila
On Tue, Jun 21, 2016 at 9:08 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > > On Tue, Jun 21, 2016 at 3:29 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Tue, Jun 21, 2016 at 1:03 AM, Andres Freund <and...@anarazel.de> wrote: > >> Well, I

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-20 Thread Amit Kapila
sential today. > > > And while I don't know what this mode should be called, I'm pretty sure > > that neither "single_copy" nor "pipeline" are useful descriptions. > > Maybe we should make this an enum rather than a boolean, since there > seem to be more than two useful behaviors. > How about calling it as control_parallel/control_parallelism or override_parallel/override_parallelism? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Reviewing freeze map code

2016-06-20 Thread Amit Kapila
uple header before releasing lock on buffer and teach tqual.c to honor the flag. I think tqual.c should consider HEAP_XMAX_UNLOGGED as an indication of aborted transaction unless it is currently in-progress. Also, I think we need to clear this flag before WAL logging in heap_update. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-20 Thread Amit Kapila
On Thu, Jun 16, 2016 at 8:20 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Jun 15, 2016 at 10:48 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > exec_stmt_execsql() is used to execute SQL statements insider plpgsql which > > includes dml statements a

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-06-20 Thread Amit Kapila
On Mon, Jun 20, 2016 at 12:47 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > > On 2016/06/20 15:42, Amit Kapila wrote: > > > >> > >> After spent time to investigate this behaviour, ISTM that the problem > >> is nloops of Parallel Seq S

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-06-20 Thread Amit Kapila
[1] useful. Refer Explain Analyze section. [1] - https://www.postgresql.org/docs/9.6/static/using-explain.html With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-06-20 Thread Amit Kapila
but nloops is incremented to 3. > nloops here indicates, that it is done for 2 workers and a master backend. > So its "actual rows" is calculated 333(1000 / 3) at explain.c:L1223. > Thats how it should be considered. You might want to compare the behaviour with other cases where value of nloops is used. > Is it a bug? > I don't think so. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-19 Thread Amit Kapila
no? > > Would this not result in unstable test output depending on whether the > code executes in the leader or a worker? > Before doing that test, we set force_parallel_mode=1, so it should always execute in worker which will ensure a stable output. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-17 Thread Amit Kapila
On Fri, Jun 17, 2016 at 8:18 AM, Amit Kapila <amit.kap...@enterprisedb.com> wrote: > > On Fri, Jun 17, 2016 at 3:20 AM, Robert Haas <robertmh...@gmail.com> wrote: > > > > > Something like what you have there might work if you use > > create_projection_p

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-17 Thread Amit Kapila
On Fri, Jun 17, 2016 at 11:39 AM, Andreas Joseph Krogh <andr...@visena.com> wrote: > På torsdag 16. juni 2016 kl. 20:19:44, skrev Tom Lane <t...@sss.pgh.pa.us>: > > Amit Kapila <amit.kap...@enterprisedb.com> writes: > > On Mon, Jun 13, 2016 at 10:36 PM, To

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-16 Thread Amit Kapila
On Fri, Jun 17, 2016 at 3:20 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Thu, Jun 16, 2016 at 12:16 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Thu, Jun 16, 2016 at 8:00 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >>> 1. The case

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-16 Thread Amit Kapila
On Thu, Jun 16, 2016 at 11:49 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kap...@enterprisedb.com> writes: > > On Mon, Jun 13, 2016 at 10:36 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> min_parallel_relation_size, or min_parallelizabl

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-16 Thread Amit Kapila
On Thu, Jun 16, 2016 at 9:26 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Tue, Jun 14, 2016 at 12:18 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > >> On Mon, Jun 13, 2016 at 9:37 PM, Tom Lane <t

Re: [HACKERS] Hash Indexes

2016-06-16 Thread Amit Kapila
On Tue, May 10, 2016 at 5:39 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > Incomplete Splits > -- > Incomplete splits can be completed either by vacuum or insert as both > needs exclusive lock on bucket. If vacuum finds split-in-pro

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-15 Thread Amit Kapila
On Thu, Jun 16, 2016 at 12:46 AM, Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Jun 15, 2016 at 5:23 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> > Considering above analysis is correct, we have below options: > >> > a. Modify the tes

Re: [HACKERS] Reviewing freeze map code

2016-06-15 Thread Amit Kapila
On Wed, Jun 15, 2016 at 7:13 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Jun 15, 2016 at 9:43 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Wed, Jun 15, 2016 at 6:26 PM, Robert Haas <robertmh...@gmail.com> wrote: > >> On Wed

Re: [HACKERS] Reviewing freeze map code

2016-06-15 Thread Amit Kapila
ing the all-visible flag and > then releases the lock on the heap page to go do some more work that > might even ERROR out. > Can't we clear the all-visible flag before releasing the lock? We can use logic of already_marked as it is currently used in code to clear it just once. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-15 Thread Amit Kapila
On Tue, Jun 14, 2016 at 4:48 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Jun 13, 2016 at 8:11 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > > I do > > not share your confidence that using apply_projection_to_path within > > create_gro

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-15 Thread Amit Kapila
On Wed, Jun 15, 2016 at 12:07 PM, Noah Misch <n...@leadboat.com> wrote: > > On Wed, Jun 15, 2016 at 11:50:33AM +0530, Amit Kapila wrote: > > In short, this test doesn't serve it's purpose which is to generate an > > error from worker. > > That's bad. Thanks for figu

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-15 Thread Amit Kapila
On Wed, Jun 15, 2016 at 12:07 PM, Noah Misch <n...@leadboat.com> wrote: > > On Wed, Jun 15, 2016 at 11:50:33AM +0530, Amit Kapila wrote: > > do $$begin > > Perform stringu1::int2 from tenk1 where unique1 = 1; > > end$$; > > > > ERROR: invalid inpu

Re: [HACKERS] parallel.c is not marked as test covered

2016-06-15 Thread Amit Kapila
at it actually generates an error and to hide the context, we can use force_parallel_mode = regress; c. Remove this test and then think of a better way to exercise error path in worker. Thoughts? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-14 Thread Amit Kapila
w, I have changed it to INT_MAX/3 so as to be consistent with guc.c. I am not sure if it is advisable to use PG_INT32_MAX in guc.c as other similar parameters use INT_MAX. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com min_parallel_relation_size_v1.patch Description: Binar

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-14 Thread Amit Kapila
in target of childrel. Basically, I am wondering about the same problem as we discussed here [1]. [1] - https://www.postgresql.org/message-id/CAA4eK1Jz5tG2D2PCNYqYob2cb4dKowKYwVJ7OmP5vwyRyCMx4g%40mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-14 Thread Amit Kapila
On Mon, Jun 13, 2016 at 8:11 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kap...@enterprisedb.com> writes: > > On Mon, Jun 13, 2016 at 7:51 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> I think the real question here is why the c

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-13 Thread Amit Kapila
On Mon, Jun 13, 2016 at 9:37 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > I wrote: > > Amit Kapila <amit.kap...@enterprisedb.com> writes: > >> It is slightly tricky to write a reproducible parallel-query test, but > >> point taken and I think we sh

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-13 Thread Amit Kapila
On Mon, Jun 13, 2016 at 7:51 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Robert Haas <robertmh...@gmail.com> writes: > > On Mon, Jun 13, 2016 at 3:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> In create_grouping_paths(), we are building part

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-13 Thread Amit Kapila
On Mon, Jun 13, 2016 at 7:17 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Mon, Jun 13, 2016 at 3:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > In create_grouping_paths(), we are building partial_grouping_path and same > > is used for gather

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

2016-06-13 Thread Amit Kapila
path for partial pathlist? Attached patch just does that and I don't see error with patch. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com apply_partial_pathtarget_partial_pathlist_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Reviewing freeze map code

2016-06-12 Thread Amit Kapila
ill still > be marked all-frozen, which is bad. > To clarify, are you talking about a case where insertion has aborted? Won't in such a case all_visible flag be set to false due to return value from HeapTupleSatisfiesVacuum() and if so, later code shouldn't mark it as all_frozen? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] [COMMITTERS] pgsql: Don't generate parallel paths for rels with parallel-restricted

2016-06-11 Thread Amit Kapila
On Sat, Jun 11, 2016 at 2:07 PM, Andreas Seltenreich <seltenre...@gmx.de> wrote: > > Amit Kapila writes: > > > I have moved it to CLOSE_WAIT state because we have derived our > > queries to reproduce the problem based on original report[1]. If next > > run of

Re: [HACKERS] [COMMITTERS] pgsql: Don't generate parallel paths for rels with parallel-restricted

2016-06-10 Thread Amit Kapila
On Fri, Jun 10, 2016 at 8:20 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > > Amit Kapila <amit.kapil...@gmail.com> writes: > > On Thu, Jun 9, 2016 at 11:38 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Robert Haas <robertmh...@gmail.com> writes: >

Re: [HACKERS] Reviewing freeze map code

2016-06-10 Thread Amit Kapila
On Fri, Jun 10, 2016 at 12:09 PM, Andres Freund <and...@anarazel.de> wrote: > > On 2016-06-10 11:58:26 +0530, Amit Kapila wrote: > > > > While looking at code in this area, I observed that during replay of > > records (heap_xlog_delete), we first clear the vm, then up

Re: [HACKERS] [COMMITTERS] pgsql: Don't generate parallel paths for rels with parallel-restricted

2016-06-10 Thread Amit Kapila
aths for such relations. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Reviewing freeze map code

2016-06-10 Thread Amit Kapila
e want the logic similar to lazy_scan_heap(), then I think we should also consider applying snapshot old threshold limit to oldestxmin. We currently do that in vacuum_set_xid_limits() for Vacuum. Is there a reason for not considering it for visibility check function? With Regards, Amit Ka

Re: [HACKERS] Reviewing freeze map code

2016-06-10 Thread Amit Kapila
On Fri, Jun 10, 2016 at 8:27 AM, Andres Freund <and...@anarazel.de> wrote: > > > On June 9, 2016 7:46:06 PM PDT, Amit Kapila <amit.kapil...@gmail.com> > wrote: > >On Fri, Jun 10, 2016 at 8:08 AM, Andres Freund <and...@anarazel.de> > >wrote: > >

<    6   7   8   9   10   11   12   13   14   15   >