On Mon, Jun 7, 2021 at 6:36 PM Etsuro Fujita wrote:
> I created a patch for that, which I'm attaching. I'm planning
> to commit the patch.
Done.
Best regards,
Etsuro Fujita
On Fri, Jun 4, 2021 at 12:33 AM Andrey Lepikhov
wrote:
> Good, this text is clear for me.
Cool! I created a patch for that, which I'm attaching. I'm planning
to commit the patch.
Thanks for reviewing!
Best regards,
Etsuro Fujita
note-about-sync-vs-async.patch
Description: Binary data
On Fri, Jun 4, 2021 at 7:26 PM Etsuro Fujita wrote:
> On Tue, Jun 1, 2021 at 6:30 PM Etsuro Fujita wrote:
> > On Fri, May 28, 2021 at 10:53 PM Etsuro Fujita
> > wrote:
> > > On Fri, May 28, 2021 at 5:29 PM Kyotaro Horiguchi
> > > wrote:
> > > > The patch drops some "= NULL" (initial)
On Tue, Jun 1, 2021 at 6:30 PM Etsuro Fujita wrote:
> On Fri, May 28, 2021 at 10:53 PM Etsuro Fujita
> wrote:
> > On Fri, May 28, 2021 at 5:29 PM Kyotaro Horiguchi
> > wrote:
> > > The patch drops some "= NULL" (initial) initializations when
> > > nasyncplans == 0. AFAICS makeNode() fills the
On 3/6/21 14:49, Etsuro Fujita wrote:
On Tue, May 11, 2021 at 6:55 PM Etsuro Fujita wrote:
On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov
wrote:
On 11/5/21 12:24, Etsuro Fujita wrote:
-> Append (actual rows=3000 loops=1)
-> Async Foreign Scan on f1 (actual rows=0
On Tue, May 11, 2021 at 6:55 PM Etsuro Fujita wrote:
> On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov
> wrote:
> > On 11/5/21 12:24, Etsuro Fujita wrote:
>
> > >> -> Append (actual rows=3000 loops=1)
> > >>-> Async Foreign Scan on f1 (actual rows=0 loops=1)
> > >>
On Fri, May 28, 2021 at 10:53 PM Etsuro Fujita wrote:
> On Fri, May 28, 2021 at 5:29 PM Kyotaro Horiguchi
> wrote:
> > The patch drops some "= NULL" (initial) initializations when
> > nasyncplans == 0. AFAICS makeNode() fills the returned memory with
> > zeroes but I'm not sure it is our
Horiguchi-san,
On Fri, May 28, 2021 at 5:29 PM Kyotaro Horiguchi
wrote:
> At Fri, 28 May 2021 16:30:29 +0900, Etsuro Fujita
> wrote in
> > The root cause would
> > be that we call classify_matching_subplans() to re-determine
> > sync/async subplans when called from the first ExecAppend() after
At Fri, 28 May 2021 16:30:29 +0900, Etsuro Fujita
wrote in
> On Wed, Mar 31, 2021 at 6:55 PM Etsuro Fujita wrote:
> > On Tue, Mar 30, 2021 at 8:40 PM Etsuro Fujita
> > wrote:
> > > I'm happy with the patch, so I'll commit it if there are no objections.
> >
> > Pushed.
>
> I noticed that
On Wed, Mar 31, 2021 at 6:55 PM Etsuro Fujita wrote:
> On Tue, Mar 30, 2021 at 8:40 PM Etsuro Fujita wrote:
> > I'm happy with the patch, so I'll commit it if there are no objections.
>
> Pushed.
I noticed that rescan of async Appends is broken when
do_exec_prune=false, leading to incorrect
On Sun, May 16, 2021 at 11:39 PM Etsuro Fujita wrote:
> Attached is a new version.
I have committed the patch.
Best regards,
Etsuro Fujita
I'm resending this because I failed to reply to all.
On Sat, May 8, 2021 at 12:55 AM Etsuro Fujita wrote:
> On Fri, May 7, 2021 at 2:12 AM Stephen Frost wrote:
> > In order to ensure that the data being returned from a foreign server
> > is consistent, postgres_fdw will only open one connection
On Tue, May 11, 2021 at 6:55 PM Etsuro Fujita wrote:
> On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov
> wrote:
> > On 11/5/21 12:24, Etsuro Fujita wrote:
>
> > >> -> Append (actual rows=3000 loops=1)
> > >>-> Async Foreign Scan on f1 (actual rows=0 loops=1)
> > >>
On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov
wrote:
> On 11/5/21 12:24, Etsuro Fujita wrote:
> >> -> Append (actual rows=3000 loops=1)
> >>-> Async Foreign Scan on f1 (actual rows=0 loops=1)
> >>-> Async Foreign Scan on f2 (actual rows=0 loops=1)
> >>
On 11/5/21 12:24, Etsuro Fujita wrote:
On Tue, May 11, 2021 at 11:58 AM Andrey Lepikhov
The extra tuple, which is from f1 or f2, would have been kept in the
Append node's as_asyncresults, not returned from the Append node to
the Limit node. The async Foreign Scan nodes would fetch tuples
before
On Tue, May 11, 2021 at 11:58 AM Andrey Lepikhov
wrote:
> Your patch fixes the problem. But I found two more problems:
>
> EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
> SELECT * FROM (
> (SELECT * FROM f1)
> UNION ALL
> (SELECT * FROM f2)
>
On 10/5/21 08:03, Etsuro Fujita wrote:
On Fri, May 7, 2021 at 7:32 PM Andrey Lepikhov
I think a simple fix for this would be just remove the check whether
the instr->running flag is set or not in InstrUpdateTupleCount().
Attached is an updated patch, in which I also updated a comment in
On Fri, May 7, 2021 at 7:32 PM Andrey Lepikhov
wrote:
> Ok, I agree with the approach, but the next test case failed:
>
> EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
> SELECT * FROM (
> (SELECT * FROM f1) UNION ALL (SELECT * FROM f2)
> ) q1 LIMIT 100;
> ERROR:
On Fri, May 7, 2021 at 7:35 PM Andrey Lepikhov
wrote:
> On 6/5/21 14:11, Etsuro Fujita wrote:
> > On Tue, Apr 27, 2021 at 3:57 PM Andrey V. Lepikhov
> > wrote:
> >> One more question. Append choose async plans at the stage of the Append
> >> plan creation.
> >> Later, the planner performs some
On Fri, May 7, 2021 at 2:12 AM Stephen Frost wrote:
> I'd suggest the language point out that it's not actually possible to do
> otherwise, since they all need to be part of the same transaction.
>
> Without that, it looks like we're just missing a trick somewhere and
> someone might think that
On 6/5/21 14:11, Etsuro Fujita wrote:
On Tue, Apr 27, 2021 at 3:57 PM Andrey V. Lepikhov
wrote:
One more question. Append choose async plans at the stage of the Append
plan creation.
Later, the planner performs some optimizations, such as eliminating
trivial Subquery nodes. So, AsyncAppend is
On 6/5/21 11:45, Etsuro Fujita wrote:
On Tue, Apr 27, 2021 at 9:31 PM Etsuro Fujita wrote:
The patch fixes the issue, but I don’t think it’s the right way to go,
because it requires an extra ExecProcNode() call, which wouldn’t be
efficient. Also, the patch wouldn’t address another issue I
On 6/5/21 22:12, Stephen Frost wrote:
* Etsuro Fujita (etsuro.fuj...@gmail.com) wrote:
I think the user should be careful about this. How about adding a
note about it to the “Asynchronous Execution Options” section in
postgres-fdw.sgml, like the attached?
+1
... then again, it'd really be
Greetings,
* Etsuro Fujita (etsuro.fuj...@gmail.com) wrote:
> On Thu, Mar 4, 2021 at 1:00 PM Etsuro Fujita wrote:
> > Another thing I'm concerned about in the postgres_fdw part is the case
> > where all/many postgres_fdw ForeignScans of an Append use the same
> > connection, because in that case
On Tue, Apr 27, 2021 at 3:57 PM Andrey V. Lepikhov
wrote:
> One more question. Append choose async plans at the stage of the Append
> plan creation.
> Later, the planner performs some optimizations, such as eliminating
> trivial Subquery nodes. So, AsyncAppend is impossible in some
> situations,
On Tue, Apr 27, 2021 at 9:31 PM Etsuro Fujita wrote:
> On Mon, Apr 26, 2021 at 7:35 PM Andrey V. Lepikhov
> wrote:
> > Small mistake i found. If no tuple was received from a foreign
> > partition, explain shows that we never executed node.
> > The patch in the attachment fixes this.
>
> Will
On Thu, Mar 4, 2021 at 1:00 PM Etsuro Fujita wrote:
> Another thing I'm concerned about in the postgres_fdw part is the case
> where all/many postgres_fdw ForeignScans of an Append use the same
> connection, because in that case those ForeignScans are executed one
> by one, not in parallel, and
On Mon, Apr 26, 2021 at 7:35 PM Andrey V. Lepikhov
wrote:
> Small mistake i found. If no tuple was received from a foreign
> partition, explain shows that we never executed node. For example,
> if we have 0 tuples in f1 and 100 tuples in f2:
>
> Query:
> EXPLAIN (ANALYZE, VERBOSE, TIMING OFF,
On Mon, Apr 26, 2021 at 3:01 PM Andrey V. Lepikhov
wrote:
> While studying the capabilities of AsyncAppend, I noticed an
> inconsistency with the cost model of the optimizer:
> Here I see two problems:
> 1. Cost of an AsyncAppend is the same as cost of an Append. But
> execution time of the
On 4/23/21 8:12 AM, Etsuro Fujita wrote:
On Thu, Apr 22, 2021 at 12:30 PM Etsuro Fujita wrote:
I have committed the patch.
One more question. Append choose async plans at the stage of the Append
plan creation.
Later, the planner performs some optimizations, such as eliminating
trivial
On 4/23/21 8:12 AM, Etsuro Fujita wrote:
I have committed the patch.
Small mistake i found. If no tuple was received from a foreign
partition, explain shows that we never executed node. For example,
if we have 0 tuples in f1 and 100 tuples in f2:
Query:
EXPLAIN (ANALYZE, VERBOSE, TIMING OFF,
On 4/23/21 8:12 AM, Etsuro Fujita wrote:
I have committed the patch.
While studying the capabilities of AsyncAppend, I noticed an
inconsistency with the cost model of the optimizer:
async_capable = off:
Append (cost=100.00..695.00 ...)
-> Foreign Scan on f1 part_1
On Thu, Apr 22, 2021 at 12:30 PM Etsuro Fujita wrote:
> > > +* We'll prefer to consider this join async-capable if
> > > any table from
> > > +* either side of the join is considered async-capable.
> > > +*/
> > > +
On Wed, Mar 31, 2021 at 2:12 PM Etsuro Fujita wrote:
> On Wed, Mar 31, 2021 at 10:11 AM Kyotaro Horiguchi
> wrote:
> > +* We'll prefer to consider this join async-capable if any
> > table from
> > +* either side of the join is considered async-capable.
> > +
On Tue, Apr 6, 2021 at 5:45 PM Etsuro Fujita wrote:
> Also, we should improve the code to avoid
> the consistency mentioned above,
Sorry, s/consistency/inconsistency/.
> I'll apply the patch.
Done. Let's see if this works.
Best regards,
Etsuro Fujita
On Tue, Apr 6, 2021 at 12:01 PM Kyotaro Horiguchi
wrote:
> At Mon, 5 Apr 2021 17:15:47 +0900, Etsuro Fujita
> wrote in
> > On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote:
> > > The buildfarm points out that this fails under valgrind.
> > > I easily reproduced it here:
> > >
> > >
Thanks for the patch.
At Mon, 5 Apr 2021 17:15:47 +0900, Etsuro Fujita
wrote in
> On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote:
> > The buildfarm points out that this fails under valgrind.
> > I easily reproduced it here:
> >
> > ==00:00:03:42.115 3410499== Syscall param epoll_wait(events)
On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote:
> The buildfarm points out that this fails under valgrind.
> I easily reproduced it here:
>
> ==00:00:03:42.115 3410499== Syscall param epoll_wait(events) points to
> unaddressable byte(s)
> ==00:00:03:42.115 3410499==at 0x58E926B: epoll_wait
On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote:
> The buildfarm points out that this fails under valgrind.
> I easily reproduced it here:
> Sorta looks like something is relying on a pointer into the relcache
> to be valid for longer than it can safely rely on that. The
> CLOBBER_CACHE_ALWAYS
Etsuro Fujita writes:
> Pushed.
The buildfarm points out that this fails under valgrind.
I easily reproduced it here:
==00:00:03:42.115 3410499== Syscall param epoll_wait(events) points to
unaddressable byte(s)
==00:00:03:42.115 3410499==at 0x58E926B: epoll_wait (epoll_wait.c:30)
On Tue, Mar 30, 2021 at 8:40 PM Etsuro Fujita wrote:
> I'm happy with the patch, so I'll commit it if there are no objections.
Pushed.
Best regards,
Etsuro Fujita
On Wed, Mar 31, 2021 at 10:11 AM Kyotaro Horiguchi
wrote:
> + async_capable
> +
> +
> + This option controls whether postgres_fdw allows
> + foreign tables to be scanned concurrently for asynchronous execution.
> + It can be specified for a foreign table or a
At Tue, 30 Mar 2021 20:40:35 +0900, Etsuro Fujita
wrote in
> On Mon, Mar 29, 2021 at 6:50 PM Etsuro Fujita wrote:
> > I think the patch would be committable.
>
> Here is a new version of the patch.
>
> * Rebased the patch against HEAD.
> * Tweaked docs/comments a bit further.
> * Added the
On Fri, Mar 19, 2021 at 9:57 PM Justin Pryzby wrote:
> is_async_capable_path() should probably have a "break" for case T_ForeignPath.
Good catch! Will fix.
> little typos:
> aready
> sigle
> givne
> a event
Lots of typos. :-( Will fix.
Thank you for the review!
Best regards,
Etsuro Fujita
is_async_capable_path() should probably have a "break" for case T_ForeignPath.
little typos:
aready
sigle
givne
a event: an event
--
Justin
On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote:
> * I haven't yet added some planner/resowner changes from Horiguchi-san's
> patch.
The patch in [1] allocates, populates, frees a wait event set every
time when doing ExecAppendAsyncEventWait(), so it wouldn’t leak wait
event sets.
On Thu, Mar 4, 2021 at 1:00 PM Etsuro Fujita wrote:
> To avoid that, how about 1) adding the
> table/server options to postgres_fdw that allow/disallow async
> execution, and 2) setting them to false by default?
There seems to be no objections, so I went ahead and added the
table/server option
On Mon, Mar 1, 2021 at 5:56 PM Etsuro Fujita wrote:
> Here is an updated version of the patch.
Another thing I'm concerned about in the postgres_fdw part is the case
where all/many postgres_fdw ForeignScans of an Append use the same
connection, because in that case those ForeignScans are
On Sat, Feb 20, 2021 at 3:35 PM Etsuro Fujita wrote:
> So I’ll update the patch using your patch without the
> postgresIterateForeignScan() change.
Here is an updated version of the patch. Based on your idea of
completing an in-progress command (if any) before sending a new
command to the
On Thu, Feb 18, 2021 at 3:16 PM Kyotaro Horiguchi
wrote:
> At Thu, 18 Feb 2021 11:51:59 +0900, Etsuro Fujita
> wrote in
> > I noticed that this doesn’t work for cases where ForeignScans are
> > executed inside functions, and I don’t have any simple solution for
>
> Ah, concurrent fetches in
Sorry that I haven't been able to respond.
At Thu, 18 Feb 2021 11:51:59 +0900, Etsuro Fujita
wrote in
> On Wed, Feb 10, 2021 at 9:31 PM Etsuro Fujita wrote:
> > Please find attached an updated patch.
>
> I noticed that this doesn’t work for cases where ForeignScans are
> executed inside
On Wed, Feb 10, 2021 at 9:31 PM Etsuro Fujita wrote:
> Please find attached an updated patch.
I noticed that this doesn’t work for cases where ForeignScans are
executed inside functions, and I don’t have any simple solution for
that. So I’m getting back to what Horiguchi-san proposed for
On Fri, Feb 12, 2021 at 5:30 PM Kyotaro Horiguchi
wrote:
> It seems too specific to async Append so I look it as a PoC of the
> mechanism.
Are you saying that the patch only reconsiders async ForeignScans?
> It creates a hash table that keyed by connection umid to record
> planids run on the
At Wed, 10 Feb 2021 21:31:15 +0900, Etsuro Fujita
wrote in
> On Wed, Feb 10, 2021 at 7:31 PM Etsuro Fujita wrote:
> > Attached is an updated version of the patch. Sorry for the delay.
>
> I noticed that I forgot to add new files. :-(. Please find attached
> an updated patch.
Thanks for
On Wed, Feb 10, 2021 at 7:31 PM Etsuro Fujita wrote:
> Attached is an updated version of the patch. Sorry for the delay.
I noticed that I forgot to add new files. :-(. Please find attached
an updated patch.
Best regards,
Etsuro Fujita
async-wip-2021-02-10-v2.patch
Description: Binary data
On Thu, Feb 4, 2021 at 7:21 PM Etsuro Fujita wrote:
> On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita wrote:
> > Rather than doing so, I'd like to propose to allow
> > FDWs to disable async execution of them in problematic cases by
> > themselves during executor startup in the first cut. What I
On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita wrote:
> On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote:
> > * I haven't yet done anything about the issue on postgres_fdw's
> > handling of concurrent data fetches by multiple ForeignScan nodes
> > (below *different* Append nodes in the query)
On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote:
> * I haven't yet done anything about the issue on postgres_fdw's
> handling of concurrent data fetches by multiple ForeignScan nodes
> (below *different* Append nodes in the query) using the same
> connection discussed in [2]. I modified the
On Fri, Jan 15, 2021 at 4:54 PM Kyotaro Horiguchi
wrote:
> Mmm. I meant that the function explicitly calls
> ExecAppendAsyncRequest(), which finally calls fetch_more_data_begin()
> (if needed). Conversely if the function dosn't call
> ExecAppendAsyncRequsest, the next request to remote doesn't
>
At Sat, 19 Dec 2020 17:55:22 +0900, Etsuro Fujita
wrote in
> On Mon, Dec 14, 2020 at 4:01 PM Kyotaro Horiguchi
> wrote:
> > At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita
> > wrote in
> > > On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi
> > > wrote:
> > > > The reason for
> > > > the
On Sun, Dec 20, 2020 at 5:15 PM Etsuro Fujita wrote:
> On Thu, Nov 26, 2020 at 10:28 AM movead...@highgo.ca
> wrote:
> > Issue one:
> > Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in
> > ExecAppendAsyncRequest() function when I use more than two foreign table
> > on
On Thu, Dec 31, 2020 at 7:15 PM Etsuro Fujita wrote:
> * I tweaked comments a bit to address your comments.
I forgot to update some comments. :-( Attached is a new version of
the patch updating comments further. I did a bit of cleanup for the
postgres_fdw part as well.
Best regards,
Etsuro
On Sat, Dec 19, 2020 at 5:55 PM Etsuro Fujita wrote:
> On Mon, Dec 14, 2020 at 4:01 PM Kyotaro Horiguchi
> wrote:
> > At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita
> > wrote in
> > > On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi
> > > wrote:
> > > > The reason for
> > > > the early
On Thu, Dec 10, 2020 at 3:38 PM Andrey V. Lepikhov
wrote:
> On 11/17/20 2:56 PM, Etsuro Fujita wrote:
> > On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita
> > wrote:
> > Comments welcome! The attached is still WIP and maybe I'm missing
> > something, though.
> I reviewed your patch and used it in
On Thu, Nov 26, 2020 at 10:28 AM movead...@highgo.ca
wrote:
> I test the patch and occur several issues as blow:
Thank you for the review!
> Issue one:
> Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in
> ExecAppendAsyncRequest() function when I use more than two
On Mon, Dec 14, 2020 at 5:56 PM Kyotaro Horiguchi
wrote:
> At Sat, 12 Dec 2020 19:06:51 +0900, Etsuro Fujita
> wrote in
> > On Fri, Nov 20, 2020 at 8:16 PM Kyotaro Horiguchi
> > wrote:
> > > + /* wait or poll async events */
> > > + if
On Mon, Dec 14, 2020 at 4:01 PM Kyotaro Horiguchi
wrote:
> At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita
> wrote in
> > On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi
> > wrote:
> > > The reason for
> > > the early fetching is letting fdw send the next request as early as
> > >
At Sat, 12 Dec 2020 19:06:51 +0900, Etsuro Fujita
wrote in
> On Fri, Nov 20, 2020 at 8:16 PM Kyotaro Horiguchi
> wrote:
> > I looked through the nodeAppend.c and postgres_fdw.c part and those
> > are I think the core of this patch.
>
> Thanks again for the review!
>
> > -*
At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita
wrote in
> On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi
> wrote:
> > At Tue, 17 Nov 2020 18:56:02 +0900, Etsuro Fujita
> > wrote in
> > > * In Robert's patch [1] (and Horiguchi-san's, which was created based
> > > on Robert's),
On Fri, Nov 20, 2020 at 8:16 PM Kyotaro Horiguchi
wrote:
> I looked through the nodeAppend.c and postgres_fdw.c part and those
> are I think the core of this patch.
Thanks again for the review!
> -* figure out which subplan we are currently processing
> +* try to
On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi
wrote:
> At Tue, 17 Nov 2020 18:56:02 +0900, Etsuro Fujita
> wrote in
> > * In Robert's patch [1] (and Horiguchi-san's, which was created based
> > on Robert's), ExecAppend() was modified to retrieve tuples from
> > async-aware children *before*
On 11/17/20 2:56 PM, Etsuro Fujita wrote:
On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita wrote:
Comments welcome! The attached is still WIP and maybe I'm missing
something, though.
I reviewed your patch and used it in my TPC-H benchmarks. It is still
WIP. Will you improve this patch?
I also
"On Thu, Nov 26, 2020 at 9:28 AM movead...@highgo.ca
wrote:
>
>
> I test the patch and occur several issues as blow:
>
> Issue one:
> Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in
> ExecAppendAsyncRequest() function when I use more than two foreign table
> on
I test the patch and occur several issues as blow:
Issue one:
Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in
ExecAppendAsyncRequest() function when I use more than two foreign table
on different foreign server.
I research the code and do such change then the Assert
At Fri, 20 Nov 2020 20:16:42 +0900 (JST), Kyotaro Horiguchi
wrote in
me> + /* If this was the second part of an async request, we must
fetch until NULL. */
me> + if (fsstate->async_aware)
me> + {
me> + /* call once and raise error if not NULL as
Hello.
I looked through the nodeAppend.c and postgres_fdw.c part and those
are I think the core of this patch.
-* figure out which subplan we are currently processing
+* try to get a tuple from async subplans
+*/
+ if
Thanks you for the new version.
At Tue, 17 Nov 2020 18:56:02 +0900, Etsuro Fujita
wrote in
> On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita wrote:
> > Yes, if there are no objections from you or Thomas or Robert or anyone
> > else, I'll update Robert's patch as such.
>
> Here is a new version
On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita wrote:
> Yes, if there are no objections from you or Thomas or Robert or anyone
> else, I'll update Robert's patch as such.
Here is a new version of the patch (as promised in the developer
unconference in PostgresConf.CN & PGConf.Asia 2020):
* In
On Thu, Oct 8, 2020 at 8:40 PM Andrey Lepikhov
wrote:
> I want to suggest one more improvement. Currently the
> is_async_capable_path() routine allow only ForeignPath nodes as async
> capable path. But in some cases we can allow SubqueryScanPath as async
> capable too.
> The patch in attachment
On Thu, Oct 8, 2020 at 6:39 PM Andrey V. Lepikhov
wrote:
> I found a small problem. If we have a mix of async and sync subplans
> when we catch an assertion on a busy connection. Just for example:
>
> PLAN
>
> Nested Loop (cost=100.00..174316.95 rows=975 width=8) (actual
> time=5.191..9.262
Hi,
I want to suggest one more improvement. Currently the
is_async_capable_path() routine allow only ForeignPath nodes as async
capable path. But in some cases we can allow SubqueryScanPath as async
capable too.
For example:
SELECT * FROM ((SELECT * FROM foreign_1)
UNION ALL
(SELECT a FROM
On 10/5/20 11:35 AM, Etsuro Fujita wrote:
Hi,
I found a small problem. If we have a mix of async and sync subplans
when we catch an assertion on a busy connection. Just for example:
PLAN
Nested Loop (cost=100.00..174316.95 rows=975 width=8) (actual
time=5.191..9.262 rows=9 loops=1)
On Mon, Oct 5, 2020 at 1:30 PM Kyotaro Horiguchi
wrote:
> At Sun, 4 Oct 2020 18:36:05 +0900, Etsuro Fujita
> wrote in
> > It’s the contract of the ExecProcNode() API: if the result is NULL or
> > an empty slot, there is nothing more to do. You changed it to
> > something like this: “even if
At Sun, 4 Oct 2020 18:36:05 +0900, Etsuro Fujita
wrote in
> On Fri, Oct 2, 2020 at 3:39 PM Kyotaro Horiguchi
> wrote:
> > At Fri, 2 Oct 2020 09:00:53 +0900, Etsuro Fujita
> > wrote in
> > > I think we should avoid changing the ExecProcNode() API.
>
> > Could you explain about what the
On Fri, Oct 2, 2020 at 3:39 PM Kyotaro Horiguchi
wrote:
> At Fri, 2 Oct 2020 09:00:53 +0900, Etsuro Fujita
> wrote in
> > I think we should avoid changing the ExecProcNode() API.
> Could you explain about what the "change" you are mentioning is?
It’s the contract of the ExecProcNode() API: if
At Fri, 2 Oct 2020 09:00:53 +0900, Etsuro Fujita
wrote in
> On Tue, Sep 29, 2020 at 4:45 AM Etsuro Fujita wrote:
> > BTW: I noticed that you changed the ExecProcNode() API so that an
> > Append calling FDWs can know wether they return tuples immediately or
> > not:
>
> > That is, 1) in
On Tue, Sep 29, 2020 at 4:45 AM Etsuro Fujita wrote:
> BTW: I noticed that you changed the ExecProcNode() API so that an
> Append calling FDWs can know wether they return tuples immediately or
> not:
> That is, 1) in postgresIterateForeignScan() postgres_fdw sets the new
> PlanState’s flag
At Thu, 1 Oct 2020 12:56:02 +0900, Michael Paquier wrote
in
> On Thu, Oct 01, 2020 at 11:16:53AM +0900, Kyotaro Horiguchi wrote:
> > Thanks. Since it starts all remote nodes before local ones, the
> > startup gain would be the shorter of the startup time of the fastest
> > remote and the time
On Thu, Oct 01, 2020 at 11:16:53AM +0900, Kyotaro Horiguchi wrote:
> Thanks. Since it starts all remote nodes before local ones, the
> startup gain would be the shorter of the startup time of the fastest
> remote and the time required for all local nodes. Plus remote
> transfer gets asynchronous
At Wed, 30 Sep 2020 16:30:41 +0900, Etsuro Fujita
wrote in
> On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi
> wrote:
> > At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita
> > wrote in
> > > Your patch (and the original patch by Robert [1]) modified
> > > ExecAppend() so that it can
On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi
wrote:
> At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita
> wrote in
> > Your patch (and the original patch by Robert [1]) modified
> > ExecAppend() so that it can process local plan nodes while waiting for
> > the results from remote queries,
On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi
wrote:
> At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita
> wrote in
> > Here are some review comments on “ExecAsync stuff” (the 0002 patch):
> >
> > @@ -192,10 +196,20 @@ ExecInitAppend(Append *node, EState *estate, int
> > eflags)
> >
> >
Thanks for reviewing.
At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita
wrote in
> > Come to think of "complex", ExecAsync stuff in this patch might be
> > too-much for a short-term solution until executor overhaul, if it
> > comes shortly. (the patch of mine here as a whole is like that,
> >
On Thu, Aug 20, 2020 at 4:36 PM Kyotaro Horiguchi
wrote:
> This is rebased version.
Thanks for the rebased version!
> Come to think of "complex", ExecAsync stuff in this patch might be
> too-much for a short-term solution until executor overhaul, if it
> comes shortly. (the patch of mine here
Your AsyncAppend doesn't switch to another source if the data in current
leader is available:
/*
* The request for the next node cannot be sent before the leader
* responds. Finish the current leader if possible.
*/
if (PQisBusy(leader_state->s.conn))
{
int rc = WaitLatchOrSocket(NULL,
On Tue, Sep 22, 2020 at 9:52 PM Konstantin Knizhnik
wrote:
> On 20.08.2020 10:36, Kyotaro Horiguchi wrote:
> > Come to think of "complex", ExecAsync stuff in this patch might be
> > too-much for a short-term solution until executor overhaul, if it
> > comes shortly. (the patch of mine here as a
On 22.09.2020 16:40, Konstantin Knizhnik wrote:
On 22.09.2020 15:52, Konstantin Knizhnik wrote:
On 20.08.2020 10:36, Kyotaro Horiguchi wrote:
At Wed, 19 Aug 2020 23:25:36 -0500, Justin Pryzby
wrote in
On Thu, Jul 02, 2020 at 11:14:48AM +0900, Kyotaro Horiguchi wrote:
As the result of
On 22.09.2020 15:52, Konstantin Knizhnik wrote:
On 20.08.2020 10:36, Kyotaro Horiguchi wrote:
At Wed, 19 Aug 2020 23:25:36 -0500, Justin Pryzby
wrote in
On Thu, Jul 02, 2020 at 11:14:48AM +0900, Kyotaro Horiguchi wrote:
As the result of a discussion with Fujita-san off-list, I'm going to
On 20.08.2020 10:36, Kyotaro Horiguchi wrote:
At Wed, 19 Aug 2020 23:25:36 -0500, Justin Pryzby wrote
in
On Thu, Jul 02, 2020 at 11:14:48AM +0900, Kyotaro Horiguchi wrote:
As the result of a discussion with Fujita-san off-list, I'm going to
hold off development until he decides whether
At Wed, 19 Aug 2020 23:25:36 -0500, Justin Pryzby wrote
in
> On Thu, Jul 02, 2020 at 11:14:48AM +0900, Kyotaro Horiguchi wrote:
> > As the result of a discussion with Fujita-san off-list, I'm going to
> > hold off development until he decides whether mine or Thomas' is
> > better.
>
> The
1 - 100 of 126 matches
Mail list logo
