On Mon, Nov 23, 2015 at 10:39 PM, Robert Haas wrote:
>
> On Mon, Nov 23, 2015 at 7:45 AM, Amit Kapila
wrote:
> > Without this patch, that 0.5 (or 50% of leaders effort) is considered
for
> > Gather node irrespective of the number of workers or other factors, but
> > I think with Patch that is no
On Mon, Nov 23, 2015 at 7:45 AM, Amit Kapila wrote:
> Without this patch, that 0.5 (or 50% of leaders effort) is considered for
> Gather node irrespective of the number of workers or other factors, but
> I think with Patch that is no longer true and that's what I am worrying
> about.
Nope, that p
On Fri, Nov 20, 2015 at 7:06 PM, Robert Haas wrote:
>
> On Fri, Nov 20, 2015 at 12:45 AM, Amit Kapila
wrote:
> > Okay, but I think that's not what I am talking about. I am talking
about
> > below code in cost_seqscan:
> >
> > - if (nworkers > 0)
> >
> > - run_cost = run_cost / (nworkers + 0.5);
On Fri, Nov 20, 2015 at 12:45 AM, Amit Kapila wrote:
> Okay, but I think that's not what I am talking about. I am talking about
> below code in cost_seqscan:
>
> - if (nworkers > 0)
>
> - run_cost = run_cost / (nworkers + 0.5);
>
> + if (path->parallel_degree > 0)
>
> + run_cost = run_cost / (pat
On Fri, Nov 20, 2015 at 1:25 AM, Robert Haas wrote:
>
> On Thu, Nov 19, 2015 at 2:59 AM, Amit Kapila
wrote:
> > Won't it be useful to consider parameterized paths for below kind of
> > plans where we can push the jointree to worker and each worker can
> > scan the complete outer relation A and th
On Thu, Nov 19, 2015 at 2:59 AM, Amit Kapila wrote:
> On Thu, Nov 19, 2015 at 12:27 AM, Robert Haas wrote:
>>
>> On Wed, Nov 18, 2015 at 7:25 AM, Amit Kapila
>> wrote:
>> > Don't we need the startup cost incase we need to build partial paths for
>> > joinpaths like mergepath?
>> > Also, I think
On Thu, Nov 19, 2015 at 1:29 PM, Amit Kapila
wrote:
>
> On Thu, Nov 19, 2015 at 12:27 AM, Robert Haas
wrote:
> -
> Is the main reason to have add_partial_path() is that it has some
> less checks or is it that current add_path will give wrong answers
> in any case?
>
> If there is no case where ad
On Thu, Nov 19, 2015 at 12:27 AM, Robert Haas wrote:
>
> On Wed, Nov 18, 2015 at 7:25 AM, Amit Kapila
wrote:
> > Don't we need the startup cost incase we need to build partial paths for
> > joinpaths like mergepath?
> > Also, I think there are other cases for single relation scan where
startup
>
On Wed, Nov 18, 2015 at 7:25 AM, Amit Kapila wrote:
> Don't we need the startup cost incase we need to build partial paths for
> joinpaths like mergepath?
> Also, I think there are other cases for single relation scan where startup
> cost can matter like when there are psuedoconstants in qualifica
On Tue, Nov 17, 2015 at 4:59 PM, Thom Brown wrote:
> On 17 November 2015 at 20:08, Robert Haas wrote:
>> On Tue, Nov 17, 2015 at 4:26 AM, Thom Brown wrote:
>>
>>> However, the first parallel seq scan shows it getting 170314 rows.
>>> Another run shows it getting 194165 rows. The final result is
On Sat, Nov 14, 2015 at 3:39 AM, Robert Haas wrote:
>
> On Thu, Nov 12, 2015 at 12:09 AM, Kouhei Kaigai
wrote:
> > I'm now designing the parallel feature of Append...
> >
> > Here is one challenge. How do we determine whether each sub-plan
> > allows execution in the background worker context?
>
On 17 November 2015 at 20:08, Robert Haas wrote:
> On Tue, Nov 17, 2015 at 4:26 AM, Thom Brown wrote:
>
>> However, the first parallel seq scan shows it getting 170314 rows.
>> Another run shows it getting 194165 rows. The final result is
>> correct, but as you can see from the rows on the Appen
On Tue, Nov 17, 2015 at 4:26 AM, Thom Brown wrote:
> Okay, I've tried this patch.
Thanks!
> Yes, it's working!
Woohoo.
> However, the first parallel seq scan shows it getting 170314 rows.
> Another run shows it getting 194165 rows. The final result is
> correct, but as you can see from the ro
On 13 November 2015 at 22:09, Robert Haas wrote:
> On Thu, Nov 12, 2015 at 12:09 AM, Kouhei Kaigai wrote:
>> I'm now designing the parallel feature of Append...
>>
>> Here is one challenge. How do we determine whether each sub-plan
>> allows execution in the background worker context?
>
> I've be
On Mon, Nov 16, 2015 at 10:10 AM, Kouhei Kaigai wrote:
> This idea will solve my concern gracefully.
> The new partial_pathlist keeps candidate of path-nodes to be gathered
> in this level or upper. Unlike path-nodes in the pathlist already, we
> don't need to rip off GatherPath later.
Cool, yes.
> On Thu, Nov 12, 2015 at 12:09 AM, Kouhei Kaigai wrote:
> > I'm now designing the parallel feature of Append...
> >
> > Here is one challenge. How do we determine whether each sub-plan
> > allows execution in the background worker context?
>
> I've been thinking about these questions for a bit n
On Thu, Nov 12, 2015 at 12:09 AM, Kouhei Kaigai wrote:
> I'm now designing the parallel feature of Append...
>
> Here is one challenge. How do we determine whether each sub-plan
> allows execution in the background worker context?
I've been thinking about these questions for a bit now, and I thin
> On 2015/11/12 14:09, Kouhei Kaigai wrote:
> > I'm now designing the parallel feature of Append...
> >
> > Here is one challenge. How do we determine whether each sub-plan
> > allows execution in the background worker context?
> >
> > The commit f0661c4e8c44c0ec7acd4ea7c82e85b265447398 added
> > '
On 2015/11/12 14:09, Kouhei Kaigai wrote:
> I'm now designing the parallel feature of Append...
>
> Here is one challenge. How do we determine whether each sub-plan
> allows execution in the background worker context?
>
> The commit f0661c4e8c44c0ec7acd4ea7c82e85b265447398 added
> 'parallel_aware
s
> Cc: pgsql-hackers@postgresql.org; Amit Kapila; Kyotaro HORIGUCHI
> Subject: Re: [HACKERS] [DESIGN] ParallelAppend
>
> > On Wed, Oct 28, 2015 at 3:55 PM, Kouhei Kaigai wrote:
> > > At PGconf.EU, I could have a talk with Robert about this topic,
> > > then it be
> On Wed, Oct 28, 2015 at 3:55 PM, Kouhei Kaigai wrote:
> > At PGconf.EU, I could have a talk with Robert about this topic,
> > then it became clear we have same idea.
> >
> >> ++
> >> |sub-plan | * Sub-Plan 1 ... Index Scan on p1
> >> |index on *-> * Sub-Plan 2 ... PartialSeqSca
On Wed, Oct 28, 2015 at 3:55 PM, Kouhei Kaigai wrote:
> At PGconf.EU, I could have a talk with Robert about this topic,
> then it became clear we have same idea.
>
>> ++
>> |sub-plan | * Sub-Plan 1 ... Index Scan on p1
>> |index on *-> * Sub-Plan 2 ... PartialSeqScan on p2
>> |sh
org
> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
> Sent: Tuesday, October 27, 2015 6:46 AM
> To: Robert Haas
> Cc: pgsql-hackers@postgresql.org; Amit Kapila; Kyotaro HORIGUCHI
> Subject: Re: [HACKERS] [DESIGN] ParallelAppend
>
> > -Original Mes
UCHI
> Subject: Re: [HACKERS] [DESIGN] ParallelAppend
>
> On Sun, Oct 25, 2015 at 9:23 PM, Kouhei Kaigai wrote:
> > I entirely agree with your suggestion.
> >
> > We may be able to use an analogy between PartialSeqScan and the
> > parallel- aware Append node.
> &
On Sun, Oct 25, 2015 at 9:23 PM, Kouhei Kaigai wrote:
> I entirely agree with your suggestion.
>
> We may be able to use an analogy between PartialSeqScan and the
> parallel- aware Append node.
> PartialSeqScan fetches blocks pointed by the index on shared memory
> segment, thus multiple workers e
> On Sat, Jul 25, 2015 at 11:13 PM, Kouhei Kaigai wrote:
> > I'm recently working/investigating on ParallelAppend feature
> > towards the next commit fest. Below is my design proposal.
> >
> > 1. Concept
> > --
> > Its concept is quite simple anybody might consider more than once.
> > Para
On Sat, Jul 25, 2015 at 11:13 PM, Kouhei Kaigai wrote:
> I'm recently working/investigating on ParallelAppend feature
> towards the next commit fest. Below is my design proposal.
>
> 1. Concept
> --
> Its concept is quite simple anybody might consider more than once.
> ParallelAppend node
On Tue, Aug 25, 2015 at 6:19 AM, Kouhei Kaigai wrote:
>
> > On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas
wrote:
> >
> > It could be possible, but let me summarize what I thought would be
required
> > for above use case. For Parallel Append, we need to push multiple
> > planned statements in cont
> On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas wrote:
> >
> > On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila
> > wrote:
> > >> Here is one other issue I found. Existing code assumes a TOC segment has
> > >> only one contents per node type, so it uses pre-defined key (like
> > >> PARALLEL_KEY_SCAN
On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas wrote:
>
> On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila
wrote:
> >> Here is one other issue I found. Existing code assumes a TOC segment
has
> >> only one contents per node type, so it uses pre-defined key (like
> >> PARALLEL_KEY_SCAN) per node type,
On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila wrote:
>> Here is one other issue I found. Existing code assumes a TOC segment has
>> only one contents per node type, so it uses pre-defined key (like
>> PARALLEL_KEY_SCAN) per node type, however, it is problematic if we put
>> multiple PlannedStmt or
On Thu, Aug 13, 2015 at 5:26 PM, Kouhei Kaigai wrote:
> > On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai
> wrote:
> > >
> >
> > Sure, that is what we should do, however the tricky part would be when
> > the path for doing local scan is extremely cheaper than path for parallel
> > scan for one of
> On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai wrote:
> >
> > > On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai
> > > wrote:
> > > >
> >
> > > > > > If we pull Funnel here, I think the plan shall be as follows:
> > > > > > Funnel
> > > > > >--> SeqScan on rel1
> > > > > >--> PartialSeqS
On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai wrote:
>
> > On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai
wrote:
> > >
>
> > > > > If we pull Funnel here, I think the plan shall be as follows:
> > > > > Funnel
> > > > >--> SeqScan on rel1
> > > > >--> PartialSeqScan on rel2
> > > > >
> On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai wrote:
> >
> > > On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai
> > > wrote:
> > >
> > > I am not sure, but what problem do you see in putting Funnel node
> > > for one of the relation scans and not for the others.
> > >
> > At this moment, I'm not
On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai wrote:
>
> > On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai
wrote:
> >
> > I am not sure, but what problem do you see in putting Funnel node
> > for one of the relation scans and not for the others.
> >
> At this moment, I'm not certain whether backgr
> On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai wrote:
> >
> > > On Tue, Jul 28, 2015 at 7:59 AM, Kouhei Kaigai
> > > wrote:
> > > >
> > > > > -Original Message-
> > > > > From: pgsql-hackers-ow...@postgresql.org
> > > > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei
On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai wrote:
>
> > On Tue, Jul 28, 2015 at 7:59 AM, Kouhei Kaigai
wrote:
> > >
> > > > -Original Message-
> > > > From: pgsql-hackers-ow...@postgresql.org
> > > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei
Kaigai
> > > > Sent:
On 2015-07-29 AM 11:02, Kouhei Kaigai wrote:
>>
>> ...
>> synchronous Append path vs. parallel asynchronous Append with Funnel
>> (below/above?) it. I guess the asynchronous version would always be
>> cheaper. So, even if we end up with non-parallel sub-plans do we still add
>> a Funnel to make App
> On 2015-07-28 PM 09:58, Kouhei Kaigai wrote:
> >>
> >> From my understanding of parallel seqscan patch, each worker's
> >> PartialSeqScan asks for a block to scan using a shared parallel heap scan
> >> descriptor that effectively keeps track of division of work among
> >> PartialSeqScans in terms
KaiGai-san,
On 2015-07-28 PM 09:58, Kouhei Kaigai wrote:
>>
>> From my understanding of parallel seqscan patch, each worker's
>> PartialSeqScan asks for a block to scan using a shared parallel heap scan
>> descriptor that effectively keeps track of division of work among
>> PartialSeqScans in ter
> On 27 July 2015 at 21:09, Kyotaro HORIGUCHI
> wrote:
>
>
> Hello, can I ask some questions?
>
> I suppose we can take this as the analog of ParalleSeqScan. I
> can see not so distinction between Append(ParalleSeqScan) and
> ParallelAppend(SeqScan). What difference is
> KaiGai-san,
>
> On 2015-07-27 PM 11:07, Kouhei Kaigai wrote:
> >
> > Append
> >--> Funnel
> > --> PartialSeqScan on rel1 (num_workers = 4)
> >--> Funnel
> > --> PartialSeqScan on rel2 (num_workers = 8)
> >--> SeqScan on rel3
> >
> > shall be rewritten to
> > Funn
> On Tue, Jul 28, 2015 at 7:59 AM, Kouhei Kaigai wrote:
> >
> > > -Original Message-
> > > From: pgsql-hackers-ow...@postgresql.org
> > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
> > > Sent: Monday, July 27, 2015 11:07 PM
> > > To: Amit Kapila
> > > >
> > > >
On Tue, Jul 28, 2015 at 12:59 PM, David Rowley wrote:
>
> On 27 July 2015 at 21:09, Kyotaro HORIGUCHI <
> horiguchi.kyot...@lab.ntt.co.jp> wrote:
>
>> Hello, can I ask some questions?
>>
>> I suppose we can take this as the analog of ParalleSeqScan. I
>> can see not so distinction between Append
On 27 July 2015 at 21:09, Kyotaro HORIGUCHI wrote:
> Hello, can I ask some questions?
>
> I suppose we can take this as the analog of ParalleSeqScan. I
> can see not so distinction between Append(ParalleSeqScan) and
> ParallelAppend(SeqScan). What difference is there between them?
>
> If other n
KaiGai-san,
On 2015-07-27 PM 11:07, Kouhei Kaigai wrote:
>
> Append
>--> Funnel
> --> PartialSeqScan on rel1 (num_workers = 4)
>--> Funnel
> --> PartialSeqScan on rel2 (num_workers = 8)
>--> SeqScan on rel3
>
> shall be rewritten to
> Funnel
> --> PartialSeq
On Tue, Jul 28, 2015 at 7:59 AM, Kouhei Kaigai wrote:
>
> > -Original Message-
> > From: pgsql-hackers-ow...@postgresql.org
> > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
> > Sent: Monday, July 27, 2015 11:07 PM
> > To: Amit Kapila
> > >
> > > Is there a real ne
Subject: Re: [HACKERS] [DESIGN] ParallelAppend
>
> > On Sun, Jul 26, 2015 at 8:43 AM, Kouhei Kaigai wrote:
> > >
> > > Hello,
> > >
> > > I'm recently working/investigating on ParallelAppend feature
> > > towards the next commit fest. Below is
> On Sun, Jul 26, 2015 at 8:43 AM, Kouhei Kaigai wrote:
> >
> > Hello,
> >
> > I'm recently working/investigating on ParallelAppend feature
> > towards the next commit fest. Below is my design proposal.
> >
> > 1. Concept
> > --
> > Its concept is quite simple anybody might consider more t
> Hello, can I ask some questions?
>
> I suppose we can take this as the analog of ParalleSeqScan. I
> can see not so distinction between Append(ParalleSeqScan) and
> ParallelAppend(SeqScan). What difference is there between them?
>
Append does not start to execute the second or later node until
f
On Sun, Jul 26, 2015 at 8:43 AM, Kouhei Kaigai wrote:
>
> Hello,
>
> I'm recently working/investigating on ParallelAppend feature
> towards the next commit fest. Below is my design proposal.
>
> 1. Concept
> --
> Its concept is quite simple anybody might consider more than once.
> Parallel
Hello, can I ask some questions?
I suppose we can take this as the analog of ParalleSeqScan. I
can see not so distinction between Append(ParalleSeqScan) and
ParallelAppend(SeqScan). What difference is there between them?
If other nodes will have the same functionality as you mention at
the last
Hello,
I'm recently working/investigating on ParallelAppend feature
towards the next commit fest. Below is my design proposal.
1. Concept
--
Its concept is quite simple anybody might consider more than once.
ParallelAppend node kicks background worker process to execute
child nodes in par
54 matches
Mail list logo
