Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-15 Thread Etsuro Fujita
On 2016/01/12 18:00, Etsuro Fujita wrote: On 2016/01/12 2:36, Alvaro Herrera wrote: I wonder, --- 2166,2213 } /* ! * If rel is a base relation, detect whether any system columns are ! * requested from the rel. (If rel is a join relation, rel->re

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-14 Thread Etsuro Fujita
On 2016/01/12 20:31, Rushabh Lathia wrote: On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/06 18:58, Rushabh Lathia wrote: .) What the need of following change ? @@ -

Re: [HACKERS] FDW join pushdown and scanclauses

2016-01-13 Thread Etsuro Fujita
s in case a system column appears in the joinclauses it will not be considered. IIUC, we assume that such system columns are assumed to be contained in fdw_scan_tlist in the joinrel case. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-12 Thread Etsuro Fujita
e a good idea. Will update the patch. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-12 Thread Etsuro Fujita
On 2016/01/07 21:45, Etsuro Fujita wrote: On 2016/01/06 18:58, Rushabh Lathia wrote: .) Documentation for the new API is missing (fdw-callbacks). Will add the docs. I added docs for new FDW APIs. Other changes: * Rename relation_has_row_level_triggers to relation_has_row_triggers

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-12 Thread Etsuro Fujita
On 2016/01/12 20:36, Thom Brown wrote: On 8 January 2016 at 05:08, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/01/06 20:37, Thom Brown wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remo

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-07 Thread Etsuro Fujita
On 2016/01/06 20:37, Thom Brown wrote: On 25 December 2015 at 10:00, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached is an updated version of the patch, which is still WIP, but I'd be happy if I could get any feedback. I've run into an issue: *# UPDATE master_customers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-07 Thread Etsuro Fujita
, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server. I think this depends on the capabilities of the FDW. .) Documentation for the new API is missing (fdw-callbacks). Will add the docs. Best regards, Etsuro Fujita -- Sent via pgsql-hackers

Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-07 Thread Etsuro Fujita
On 2016/01/07 21:50, Etsuro Fujita wrote: On 2016/01/06 20:37, Thom Brown wrote: On 25 December 2015 at 10:00, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached is an updated version of the patch, which is still WIP, but I'd be happy if I could get any feedback. I'

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2015-12-27 Thread Etsuro Fujita
t regards, Etsuro Fujita *** a/src/backend/optimizer/plan/createplan.c --- b/src/backend/optimizer/plan/createplan.c *** *** 2097,2106 create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path, RelOptInfo *rel = best_path->path.parent; Index scan_relid = rel-&g

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-12-25 Thread Etsuro Fujita
RT .. RETURNING .., but I'd like to leave that for future work. I'll add this to the next CF. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 816,822 deparseTargetList(StringInfo buf, * * If params is not N

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-12-21 Thread Etsuro Fujita
On 2015/11/26 18:00, Etsuro Fujita wrote: On 2015/11/25 20:36, Thom Brown wrote: On 13 May 2015 at 04:10, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2015/05/13 0:55, Stephen Frost wrote: While the EXPLAIN output changed, the structure hasn't really changed from what was dis

Re: [HACKERS] Remaining 9.5 open items

2015-12-11 Thread Etsuro Fujita
On 2015/12/11 1:18, Robert Haas wrote: On Wed, Dec 9, 2015 at 2:52 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Thank you for committing the patch! Sorry, I overlooked a typo in docs: s/more that one/more than one/ Please find attached a patch. Committed, thanks. Thanks!

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-11 Thread Etsuro Fujita
On 2015/12/11 14:16, Ashutosh Bapat wrote: On Thu, Dec 10, 2015 at 11:20 PM, Robert Haas <robertmh...@gmail.com <mailto:robertmh...@gmail.com>> wrote: On Tue, Dec 8, 2015 at 6:40 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>&

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-09 Thread Etsuro Fujita
On 2015/12/09 13:26, Kouhei Kaigai wrote: On Tue, Dec 8, 2015 at 10:00 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I think the actual regression test outputs are fine, and that your desire to suppress part of the plan tree from showing up in the EXPLAIN output is misguided.

[HACKERS] Minor comment update in setrefs.c

2015-12-08 Thread Etsuro Fujita
can. Best regards, Etsuro Fujita *** a/src/backend/optimizer/plan/setrefs.c --- b/src/backend/optimizer/plan/setrefs.c *** *** 1108,1114 set_foreignscan_references(PlannerInfo *root, if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0) { ! /* Adjust tlist,

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-08 Thread Etsuro Fujita
On 2015/12/08 17:27, Ashutosh Bapat wrote: On Thu, Dec 3, 2015 at 12:36 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: Generating paths A join between two foreign relations is considered

Re: [HACKERS] Remaining 9.5 open items

2015-12-08 Thread Etsuro Fujita
atch. Best regards, Etsuro Fujita diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index 0090e24..dc2d890 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml/fdwhandler.sgml @@ -793,7 +793,7 @@ RecheckForeignScan (ForeignScanState *node, TupleTableSlot *slot);

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-08 Thread Etsuro Fujita
On 2015/12/09 1:13, Robert Haas wrote: On Tue, Dec 8, 2015 at 5:49 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I'd like to discuss the next thing about his patch. As I mentioned in [1], the following change in the patch will break the EXPLAIN output. @@ -205,6 +

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-06 Thread Etsuro Fujita
On 2015/12/05 5:15, Robert Haas wrote: On Tue, Dec 1, 2015 at 10:20 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: One thing I can think of is that we can keep both the structure of a ForeignPath node and the API of create_foreignscan_path as-is. The latter is a good thing f

Re: [HACKERS] Remaining 9.5 open items

2015-12-03 Thread Etsuro Fujita
ersion of the patch for this is in good shape, but that would need some changes as proposed on that thread. So, if there are no objections, I'll update the patch. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-02 Thread Etsuro Fujita
otherclauses" (rinfo->is_pushed_down=true) and that 2) were pushed down to the remote server, not scan quals relevant to all the base tables invoved in the foreign-join. So in this definition, I think fdw_recheck_quals for a foreign-join will be workable, regardless of the join type.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-02 Thread Etsuro Fujita
On 2015/12/02 14:54, Kouhei Kaigai wrote: On 2015/12/02 1:41, Robert Haas wrote: On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: The attached patch adds: Path *fdw_outerpath field to ForeignPath node. FDW driver can set arbitrary but one path-nod

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-02 Thread Etsuro Fujita
or foreign joins. Corresponding changes to postgres_fdw will need to be added once those changes get committed. Yeah, we would need those changes including helper functions to create a local join execution plan for that support. I'd like to add those changes to your updated patch if

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-02 Thread Etsuro Fujita
On 2015/12/02 1:53, Robert Haas wrote: On Fri, Nov 27, 2015 at 1:33 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Plan *plan = >scan.plan; @@ -3755,7 +3763,7 @@ make_foreignscan(List *qptlist, /* cost will be filled in by create_foreignscan_plan */

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-12-01 Thread Etsuro Fujita
On 2015/12/02 1:41, Robert Haas wrote: On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: The attached patch adds: Path *fdw_outerpath field to ForeignPath node. FDW driver can set arbitrary but one path-node here. After that, this path-node shall be trans

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-26 Thread Etsuro Fujita
orry, I don't understand this. In my understanding, fdw_recheck_quals can be defined for a foreign join, regardless of the join type, and when the fdw_recheck_quals are defined, the RecheckForeignScan callback routine doesn't need to evaluate the fdw_recheck_quals by itself. No? Best regards, Etsuro Fujit

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-26 Thread Etsuro Fujita
should be mirror images of each other, but I think that that break would be harmless. Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/55def5f0@lab.ntt.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-11-26 Thread Etsuro Fujita
Hi Thom, Thank you for paying attention to this! On 2015/11/25 20:36, Thom Brown wrote: On 13 May 2015 at 04:10, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2015/05/13 0:55, Stephen Frost wrote: While the EXPLAIN output changed, the structure hasn't really changed fro

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-26 Thread Etsuro Fujita
es_fdw to run the alternative sub-plan on EPQ recheck. Will do. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-23 Thread Etsuro Fujita
egards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-23 Thread Etsuro Fujita
and evaluate the otherclauses as part of the local join execution plan, so that fdw_scan_tlist won't end up being longer, as in the patch [1]. (Note that in that patch, remote_exprs==NIL when calling make_foreignscan during postgresGetForeignPlan in case of foreign joins.) Best regards, Etsuro Fuj

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-23 Thread Etsuro Fujita
On 2015/11/09 9:26, Kouhei Kaigai wrote: The attached patch is an adjusted version of the previous one. There seems to be no changes to make_foreignscan. Is that OK? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-20 Thread Etsuro Fujita
missing something, but what use cases are you thinking? Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/55af3c08.1070...@lab.ntt.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-19 Thread Etsuro Fujita
On 2015/11/20 6:57, Robert Haas wrote: On Wed, Nov 18, 2015 at 10:54 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Noted, but let's do it that way and move on. It would be a shame if we didn't end up with a working FDW join pushdown system in 9.6 because of a disagr

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-18 Thread Etsuro Fujita
On 2015/11/19 12:34, Robert Haas wrote: On Tue, Nov 17, 2015 at 9:30 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I suppose you (and KaiGai-san) are probably right, but I really fail to see it actually doing that. Noted, but let's do it that way and move on. It would be a

Re: [HACKERS] Minor comment improvement to create_foreignscan_plan

2015-11-18 Thread Etsuro Fujita
On 2015/11/19 5:29, Robert Haas wrote: On Tue, Nov 17, 2015 at 9:29 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2015/11/18 2:57, Robert Haas wrote: On Sun, Nov 15, 2015 at 9:25 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Oops, I've found another one.

Re: [HACKERS] Minor comment improvement to create_foreignscan_plan

2015-11-17 Thread Etsuro Fujita
On 2015/11/18 2:57, Robert Haas wrote: On Sun, Nov 15, 2015 at 9:25 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Oops, I've found another one. I think we should update a comment in postgresGetForeignPlan, too; add remote filtering expressions to the list of information

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-17 Thread Etsuro Fujita
On 2015/11/18 3:19, Robert Haas wrote: On Thu, Nov 12, 2015 at 12:54 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Really? I think there would be not a little burden on an FDW author; when postgres_fdw delegates to the subplan to the remote server, for example, it would need to

Re: [HACKERS] Minor comment improvement to create_foreignscan_plan

2015-11-15 Thread Etsuro Fujita
On 2015/11/12 19:02, Etsuro Fujita wrote: On 2015/11/10 3:53, Robert Haas wrote: On Mon, Nov 9, 2015 at 5:34 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Here is a small patch to update an comment in create_foreignscan_plan; add fdw_recheck_quals to the list of expressions tha

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-15 Thread Etsuro Fujita
. The "complicated" is a subjectively-based term. At least, we don't share your experience, so it is hard to understand the how complexity. I don't mean to object that idea. I'm unfamiliar with that idea, so I just wanted to know the reason, or use cases. Best regards, Etsuro Fujita

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-15 Thread Etsuro Fujita
, but I also think it's important not to burden FDW authors. Maybe I'm missing something, though. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-12 Thread Etsuro Fujita
Robert and Kaigai-san, Sorry, I sent in an unfinished email. On 2015/11/12 15:30, Kouhei Kaigai wrote: On 2015/11/12 2:53, Robert Haas wrote: On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: To test this change, I think we should update the postgr

Re: [HACKERS] Minor comment improvement to create_foreignscan_plan

2015-11-12 Thread Etsuro Fujita
On 2015/11/10 3:53, Robert Haas wrote: On Mon, Nov 9, 2015 at 5:34 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Here is a small patch to update an comment in create_foreignscan_plan; add fdw_recheck_quals to the list of expressions that need the replace_nestloop_params processi

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-12 Thread Etsuro Fujita
my unfinished email. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-11 Thread Etsuro Fujita
On 2015/11/12 2:53, Robert Haas wrote: On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: To test this change, I think we should update the postgres_fdw patch so as to add the RecheckForeignScan. Having said that, as I said previously, I don't see much

[HACKERS] Minor comment improvement to create_foreignscan_plan

2015-11-09 Thread Etsuro Fujita
Hi, Here is a small patch to update an comment in create_foreignscan_plan; add fdw_recheck_quals to the list of expressions that need the replace_nestloop_params processing. I should have updated the comment when I proposed the patch for the fdw_recheck_quals. Best regards, Etsuro Fujita

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-09 Thread Etsuro Fujita
r and stored in the corresponding estate->es_epqTuple[], the routine wouldn't allow for what Robert proposed in [2]. To do such a thing, I think we would probably need to change the existing EPQ machinery more drastically and rethink the right place for calling the routine. Best regards, Etsuro F

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-08 Thread Etsuro Fujita
Node, as my patch does [1]. Maybe I'm missing something, though. Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/5624d583.10...@lab.ntt.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-05 Thread Etsuro Fujita
On 2015/11/04 18:50, Etsuro Fujita wrote: On 2015/11/04 17:10, Kouhei Kaigai wrote: On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) Wh

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-04 Thread Etsuro Fujita
those tables use different rowmark types such as ROW_MARK_COPY and ROW_MARK_EXCLUSIVE, as I pointed out upthread. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-04 Thread Etsuro Fujita
On 2015/11/04 17:10, Kouhei Kaigai wrote: On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) What I'm concerned about is the following: SELECT

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-04 Thread Etsuro Fujita
Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ATT_FOREIGN_TABLE and ATWrongRelkindError()

2015-11-03 Thread Etsuro Fujita
On 2015/10/28 20:10, Robert Haas wrote: On Fri, Oct 23, 2015 at 11:51 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: BTW, I found an incorrect error message in ATWrongRelkindError. Attached is a patch for fixing the message. Committed and back-patched to 9.3. Thanks! Best r

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-11-03 Thread Etsuro Fujita
On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) What I'm concerned about is the following: SELECT * FROM localtab JOIN (ft1 LEFT JOIN ft2 ON

Re: [HACKERS] ATT_FOREIGN_TABLE and ATWrongRelkindError()

2015-10-23 Thread Etsuro Fujita
t;\"%s\" is not a table, view, or foreign table"? BTW, I found an incorrect error message in ATWrongRelkindError. Attached is a patch for fixing the message. Best regards, Etsuro Fujita diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index a5bc508..6436d

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-22 Thread Etsuro Fujita
for the workload where EPQ rechecks are rarely invoked, because we just need to transfer ctids, not whole-row images. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-21 Thread Etsuro Fujita
On 2015/10/21 13:34, Kouhei Kaigai wrote: On 2015/10/20 13:11, Etsuro Fujita wrote: On 2015/10/20 5:34, Robert Haas wrote: No. You just need to populate fdw_recheck_quals correctly, same as for the scan case. As I said yesterday, that opinion of me is completely wrong. Sorry

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-20 Thread Etsuro Fujita
On Mon, Oct 19, 2015 at 3:45 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: As Tom mentioned, just recomputing the original join tuple is not good enough. We would need to rejoin the test tuples for the baserels even if ROW_MARK_COPY is in use. Consider: A=# BEGIN; A=# UPDATE

[HACKERS] Typos in plannodes.h

2015-10-20 Thread Etsuro Fujita
Hi, I found typos in plannodes.h: s/scan.plan.quals/scan.plan.qual/g Please find attached a patch. Best regards, Etsuro Fujita diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h index 92fd8e4..6b28c8e 100644 --- a/src/include/nodes/plannodes.h +++ b/src/include/nodes

Re: [HACKERS] Typos in plannodes.h

2015-10-20 Thread Etsuro Fujita
On 2015/10/21 0:13, Robert Haas wrote: On Tue, Oct 20, 2015 at 7:45 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I found typos in plannodes.h: s/scan.plan.quals/scan.plan.qual/g Please find attached a patch. Oops. Good catch. Committed. Thanks for picking this up!

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-20 Thread Etsuro Fujita
On 2015/10/20 13:11, Etsuro Fujita wrote: On 2015/10/20 5:34, Robert Haas wrote: On Mon, Oct 19, 2015 at 3:45 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: As Tom mentioned, just recomputing the original join tuple is not good enough. We would need to rejoin the test

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-19 Thread Etsuro Fujita
join looks at its EPQ slot, I think the plan can return at most one tuple. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-19 Thread Etsuro Fujita
On 2015/10/17 12:22, Robert Haas wrote: On Fri, Oct 16, 2015 at 9:51 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: Robert Haas <robertmh...@gmail.com> writes: Both you and Etsuro Fujita are proposing to fix this problem by somehow making it the FDW's problem to reconstruct the tupl

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-19 Thread Etsuro Fujita
I don't think it is a good interface design to enforce people to > follow a particular implementation manner. It should be discretion > of the extension. I think that if you think so, you should give at least one concrete example for that. Ideally accompanied by a demo of how that works well.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-19 Thread Etsuro Fujita
add a bit more comments Thanks for the comments! Best regards, Etsuro Fujita *** a/contrib/file_fdw/file_fdw.c --- b/contrib/file_fdw/file_fdw.c *** *** 525,530 fileGetForeignPaths(PlannerInfo *root, --- 525,531 total_cost, NIL, /* no pathkeys */

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-19 Thread Etsuro Fujita
On 2015/10/20 5:34, Robert Haas wrote: On Mon, Oct 19, 2015 at 3:45 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: As Tom mentioned, just recomputing the original join tuple is not good enough. We would need to rejoin the test tuples for the baserels even if ROW_MARK_COPY is

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-16 Thread Etsuro Fujita
On 2015/10/14 17:31, Etsuro Fujita wrote: As KaiGai-san also pointed out before, I think we should address this in each of the following cases: 1) remote qual (scanrelid>0) 2) remote join (scanrelid==0) As for #2, I updated the patch, which uses a local join execution p

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-16 Thread Etsuro Fujita
k routine to be kicked from > ForeignRecheck(). Honestly, I still don't think I would see the much value in doing so. As Robert mentioned in [1], I think that if we're inside EPQ, pushed-down quals and/or pushed-down joins should be locally rechecked in the same way as other cases such as Inde

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-15 Thread Etsuro Fujita
ed for that. If it's okay, I'd like to propose an updated version of the patch in a few days. > I cannot understand why Fujita-san never "try" this approach. Maybe my explanation was not correct, but I didn't say such a thing. What I rather objected against was to add a new FDW ca

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-15 Thread Etsuro Fujita
trying to vary. In subselect.c, the added break is in the added curly-braces but it would be better to place it after the closing brace, like the other cases. Changed that, and committed. Thanks, Robert and Horiguchi-san. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-14 Thread Etsuro Fujita
On 2015/10/14 12:07, Kouhei Kaigai wrote: >> On 2015/10/07 15:39, Etsuro Fujita wrote: >> I noticed that the approach using a column to populate the foreign >> scan's slot directly wouldn't work well in some cases. For example, >> consider: >> >> SELECT * FRO

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-14 Thread Etsuro Fujita
On 2015/10/10 10:17, Robert Haas wrote: On Thu, Oct 8, 2015 at 11:00 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: The best plan is presumably something like this as you said before: LockRows -> Nested Loop -> Seq Scan on verysmall v -> Foreign Scan on bi

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

2015-10-09 Thread Etsuro Fujita
On 2015/09/11 6:30, Robert Haas wrote: On Wed, Sep 9, 2015 at 2:30 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: But that path might have already been discarded on the basis of cost. I think Tom's idea is better: let the FDW consult some state cached for this purpose in the RelO

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-09 Thread Etsuro Fujita
On 2015/10/09 15:04, Kyotaro HORIGUCHI wrote: At Fri, 9 Oct 2015 12:00:30 +0900, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote in <56172dce.7080...@lab.ntt.co.jp> On 2015/10/08 19:55, Etsuro Fujita wrote: I noticed that the approach using a column to populate the foreign

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-09 Thread Etsuro Fujita
On 2015/09/12 1:38, Robert Haas wrote: On Thu, Sep 10, 2015 at 11:36 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I've proposed the following API changes: * I modified create_foreignscan_path, which is called from postgresGetForeignJoinPaths/postgresGetForeignPaths, so that

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-08 Thread Etsuro Fujita
On 2015/10/07 15:39, Etsuro Fujita wrote: On 2015/10/07 15:06, Kyotaro HORIGUCHI wrote: At Wed, 7 Oct 2015 00:24:57 -0400, Robert Haas <robertmh...@gmail.com> wrote I think it rather requires *replacing* two resjunk columns by one new one. The whole-row references for the individual f

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-08 Thread Etsuro Fujita
On 2015/10/08 19:55, Etsuro Fujita wrote: I noticed that the approach using a column to populate the foreign scan's slot directly wouldn't work well in some cases. For example, consider: SELECT * FROM verysmall v LEFT JOIN (bigft1 JOIN bigft2 ON bigft1.x = bigft2.x) ON v.q = bigft1.q AND v.r

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-07 Thread Etsuro Fujita
upport EvalPlanQual; if we instead have a column to populate the foreign scan's slot directly, then we can use that column for that purpose directly and there's no remaining use for the whole-row vars on the baserels. It is what I had in mind. OK I'll investigate this further. Best regards, Etsuro

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-06 Thread Etsuro Fujita
don't actually need to be able to fetch such a tuple from the remote side because we can just construct it. I think. I also was thinking whether we could replace one of the whole-row vars with a whole-row var that represents the scan slot of the ForeignScanState node. Best regards, Etsuro

Re: [HACKERS] Obsolete comment in tidpath.c

2015-10-06 Thread Etsuro Fujita
On 2015/10/07 7:01, Tom Lane wrote: > Robert Haas <robertmh...@gmail.com> writes: >> On Mon, Oct 5, 2015 at 3:05 AM, Etsuro Fujita >> <fujita.ets...@lab.ntt.co.jp> wrote: >>> I think "best_inner_indexscan()" in the following comment in tidpath.c

[HACKERS] Obsolete comment in tidpath.c

2015-10-05 Thread Etsuro Fujita
ther table, there * doesn't seem to be enough use-case to justify adding a lot of code * for that. How about s/best_inner_indexscan()/parameterized scans/? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-05 Thread Etsuro Fujita
On 2015/09/29 16:36, Etsuro Fujita wrote: For the foreign table case (scanrelid>0), I imagined an approach different than yours. In that case, I thought the issue would be probably addressed by just modifying the remote query performed in RefetchForeignRow, which would be of the form &quo

Re: [HACKERS] Confusing remark about UPSERT in fdwhandler.sgml

2015-10-04 Thread Etsuro Fujita
On 2015/10/03 5:57, Robert Haas wrote: On Fri, Oct 2, 2015 at 4:04 AM, Peter Geoghegan <p...@heroku.com> wrote: On Fri, Oct 2, 2015 at 1:00 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: ISTM that the sentence "as remote constraints are not locally known" is somew

[HACKERS] Confusing remark about UPSERT in fdwhandler.sgml

2015-10-02 Thread Etsuro Fujita
own"? Attached is a patch for that. Best regards, Etsuro Fujita *** a/doc/src/sgml/fdwhandler.sgml --- b/doc/src/sgml/fdwhandler.sgml *** *** 1196,1204 GetForeignServerByName(const char *name, bool missing_ok); INSERT with an ON CONFLICT clause does not !

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-01 Thread Etsuro Fujita
On 2015/10/01 19:02, Kyotaro HORIGUCHI wrote: At Thu, 1 Oct 2015 17:50:25 +0900, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote in <560cf3d1.9060...@lab.ntt.co.jp> From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas

[HACKERS] Typo in /src/backend/optimizer/README

2015-10-01 Thread Etsuro Fujita
forming the query to B leftjoin (A leftjoin (C innerjoin D) on Pa) on (Pbcd) which yields totally wrong answers. We prevent that by forcing the min LHS for the upper join to include B. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-01 Thread Etsuro Fujita
that that function should fetch the same version previously obtained for each such relation successfully. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Typo in /src/backend/optimizer/README

2015-10-01 Thread Etsuro Fujita
On 2015/10/01 22:31, Tom Lane wrote: > Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> writes: >> The following is a remark added to /src/backend/optimizer/README by >> commit 8703059c6b55c427100e00a09f66534b6ccbfaa1, and IIUC, I think "LHS" >> in the last sentenc

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-01 Thread Etsuro Fujita
On 2015/10/01 11:15, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas On Mon, Sep 28, 2015 at 11:15 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I thought the same thing [1]. While I t

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-01 Thread Etsuro Fujita
the applicability of late row locking to the foreign join case, allowing component foreign tables involved in a foreign join to have different rowmark methods other than ROW_MARK_COPY. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-30 Thread Etsuro Fujita
on previously obtained. But when RefetchForeignRow is called from ExecLockRows, the tuple should be just ignored as the above code, if the latest version on the remote side didn't satisfy the remote quals. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-30 Thread Etsuro Fujita
On 2015/09/30 6:55, Robert Haas wrote: On Mon, Sep 28, 2015 at 11:15 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I thought the same thing [1]. While I thought it was relatively easy to make changes to RefetchForeignRow that way for the foreign table case (scanrelid

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 13:55, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita On 2015/09/29 9:13, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 17:49, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita RefetchForeignRow() does not take ForeignScanState as its argument, so it is not obvious to access its private field, isn't it? ExecRowMark

Re: [HACKERS] Comment update to pathnode.c

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 20:51, Robert Haas wrote: On Tue, Sep 29, 2015 at 1:55 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Thanks for the comments! Attached is an updated version of the patch. Committed and back-patched to 9.5. Thanks! Best regards, Etsuro Fujita -- Sent via

Re: [HACKERS] Comment update to pathnode.c

2015-09-28 Thread Etsuro Fujita
On 2015/09/12 4:26, Robert Haas wrote: On Fri, Sep 11, 2015 at 6:22 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: The comments for create_foreignscan_path says as follows, but that it's now possible that the function is called by GetForeignJoinPaths, which was added in 9.5.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-28 Thread Etsuro Fujita
On 2015/09/29 9:13, Kouhei Kaigai wrote: -Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas Sent: Tuesday, September 29, 2015 5:46 AM To: Kaigai Kouhei(海外 浩平) Cc: Etsuro Fujita; PostgreSQL-development; 花田茂

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-11 Thread Etsuro Fujita
On 2015/09/11 6:02, Robert Haas wrote: On Thu, Sep 3, 2015 at 6:25 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I gave it another thought; the following changes to ExecInitNode would make the patch much simpler, ie, we would no longer need to call the new code in ExecInitForei

Re: Hooking at standard_join_search (Was: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual)

2015-09-11 Thread Etsuro Fujita
On 2015/09/11 6:30, Robert Haas wrote: On Wed, Sep 9, 2015 at 2:30 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: But that path might have already been discarded on the basis of cost. I think Tom's idea is better: let the FDW consult some state cached for this purpose in the RelO

<    1   2   3   4   5   6   7   8   9   10   >