Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-05-11 Thread Etsuro Fujita
On 2016/04/28 13:45, Michael Paquier wrote: On Wed, Apr 27, 2016 at 12:16 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/04/26 21:45, Etsuro Fujita wrote: While re-reviewing the fix, I noticed that since PQcancel we added to pgfdw_xact_callback to cancel a DML pushdown

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-05-10 Thread Etsuro Fujita
On 2016/05/02 22:06, Robert Haas wrote: On Thu, Apr 28, 2016 at 7:59 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/03/14 17:56, Ashutosh Bapat wrote: On Mon, Mar 14, 2016 at 1:29 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.c

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-04-28 Thread Etsuro Fujita
On 2016/03/14 17:56, Ashutosh Bapat wrote: On Mon, Mar 14, 2016 at 1:29 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: /* * Build the fdw_private list that will be available to the executor. * Items in t

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-26 Thread Etsuro Fujita
On 2016/04/26 21:45, Etsuro Fujita wrote: While re-reviewing the fix, I noticed that since PQcancel we added to pgfdw_xact_callback to cancel a DML pushdown query isn't followed by a ROLLBACK, the connection to the remote server will be discarded at the end of the while loop in that function

Re: [HACKERS] postgres_fdw : Not able to update foreign table referring to a local table's view when use_remote_estimate = true

2016-04-26 Thread Etsuro Fujita
a foreign table at a view is fine for SELECT, but that UPDATE and DELETE may not work, seems like enough. +1 for just updating the documentation. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgre

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-26 Thread Etsuro Fujita
performed in subtransactions. I think cancellation logic would also need to be added to pgfdw_subxact_callback. Comments are welcome! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-21 Thread Etsuro Fujita
On 2016/04/22 9:22, Michael Paquier wrote: On Thu, Apr 21, 2016 at 11:53 PM, Robert Haas <robertmh...@gmail.com> wrote: I have committed this patch. Thanks for working on this. Sorry for the delay. Thanks for the push. open_item--; Thank you, Robert and Michael. Best regards,

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-21 Thread Etsuro Fujita
On 2016/04/19 14:25, Etsuro Fujita wrote: On 2016/04/19 13:59, Michael Paquier wrote: But huge objection to that because this fragilizes the current logic postgres_fdw is based on: PQexec returns the last result to caller, I'd rather not break that logic for 9.6 stability's sake. IIUC, I

Re: [HACKERS] Description of ForeignPath

2016-04-20 Thread Etsuro Fujita
r foreign upper-relation processing"? I think we would probably need to update the comment in src/backend/optimizer/README (L358), too. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-18 Thread Etsuro Fujita
On 2016/04/19 13:59, Michael Paquier wrote: On Tue, Apr 19, 2016 at 1:14 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: What do you think about that? + /* Wait for the result */ + res = pgfdw_get_result(conn, query); + if (res == NULL) + pgfdw_report_error(ERROR

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-18 Thread Etsuro Fujita
On 2016/04/19 12:45, Etsuro Fujita wrote: On 2016/04/19 12:26, Michael Paquier wrote: Note for Robert: pgfdw_get_result copycats PQexec by discarding all PQresult received except the last one. I think that's fine for the purposes of postgres_fdw, but perhaps you have a different opinion

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-18 Thread Etsuro Fujita
query contains multiple SQL commands. So, +1 for the idea of simply encapsulating the while (PQisBusy(...)) loop into a new function pgfdw_get_result(). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-15 Thread Etsuro Fujita
On 2016/04/14 15:20, Ashutosh Bapat wrote: On Thu, Apr 14, 2016 at 8:42 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: As you mentioned, we could support FULL JOIN fully, by encapsulating a joining relation with

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-15 Thread Etsuro Fujita
On 2016/04/15 14:31, Michael Paquier wrote: On Thu, Apr 14, 2016 at 10:44 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/04/13 21:50, Michael Paquier wrote: On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmh...@gmail.com> wrote: On Tue, Apr 12, 2016 at 10:2

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-15 Thread Etsuro Fujita
On 2016/04/14 13:04, Robert Haas wrote: On Wed, Apr 13, 2016 at 11:21 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: 2. When a join is pushed down, deparse system columns using something like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID column, whi

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Etsuro Fujita
On 2016/04/14 12:04, Etsuro Fujita wrote: On 2016/04/14 4:57, Robert Haas wrote: 2. When a join is pushed down, deparse system columns using something like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID column, which gets deparsed with the table OID in

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-13 Thread Etsuro Fujita
that it is relatively easy to do that, by borrowing some ideas from Hanada-san's original join pushdown patch. If it's OK, I'll create a patch for that in a few days. Sorry for speaking up late. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Etsuro Fujita
outer joins. I think that that would cause useless data transfer for such culumns. Why not set values locally for such columns? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-13 Thread Etsuro Fujita
On 2016/04/13 21:50, Michael Paquier wrote: On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmh...@gmail.com> wrote: On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: How about we encapsulate the while (PQisBusy(...)) loop into a

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-12 Thread Etsuro Fujita
eems like a good idea. Will do. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-11 Thread Etsuro Fujita
n't it be better to issue a WARNING here if PQcancel does not succeed? Seems like a good idea. Attached is an updated version of the patch. Thanks for the review! Best regards, Etsuro Fujita diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c index 189f290..da1758b 100

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-10 Thread Etsuro Fujita
t to *false*. The PGresult argument is NULL there, so no need to release the PGresult. Attached is an updated patch. Best regards, Etsuro Fujita diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c index 189f290..8820597 100644 --- a/contrib/postgres_fdw/connec

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-08 Thread Etsuro Fujita
On 2016/04/08 13:42, Noah Misch wrote: On Tue, Apr 05, 2016 at 03:22:03PM +0900, Etsuro Fujita wrote: On 2016/04/04 20:35, Michael Paquier wrote: On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Here is a patch to fix this issue. As proposed by Mich

[HACKERS] Typo in src/interfaces/libpq/fe-connect.c

2016-04-05 Thread Etsuro Fujita
Hi, I ran into a typo in src/interfaces/libpq/fe-connect.c: s/PQcancelGet/PQgetCancel/ Attached is a patch to fis that typo. Best regards, Etsuro Fujita diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 5ad4755..9b2839b 100644 --- a/src/interfaces/libpq/fe

Re: [HACKERS] Odd oid-system-column handling in postgres_fdw

2016-04-05 Thread Etsuro Fujita
On 2016/03/16 16:25, Etsuro Fujita wrote: > PG9.5 allows us to add an oid system column to foreign tables, using > ALTER FOREIGN TABLE SET WITH OIDS, but currently, that column reads as > zeroes in postgres_fdw. That seems to me like a bug. So, I'd like to > propose to fix that, b

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-05 Thread Etsuro Fujita
On 2016/03/29 15:37, Etsuro Fujita wrote: I added two helper functions: GetFdwScanTupleExtraData and FillFdwScanTupleSysAttrs. The FDW author could use the former to get info about system attributes other than ctids and oids in fdw_scan_tlist during BeginForeignScan, and the latter to set

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-05 Thread Etsuro Fujita
On 2016/04/04 20:35, Michael Paquier wrote: On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Here is a patch to fix this issue. As proposed by Michael, I modified execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams. Any comments are w

[HACKERS] Incorrect comment in contrib/postgres_fdw/deparse.c

2016-04-04 Thread Etsuro Fujita
Hi, I found an incorrect comment in contrib/postgres_fdw/deparse.c: s/FOR SELECT or FOR SHARE/FOR UPDATE or FOR SHARE/ Attached is a patch to fix that comment. Best regards, Etsuro Fujita diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index bdc410d..d78ac79 100644

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-04 Thread Etsuro Fujita
On 2016/03/31 16:38, Etsuro Fujita wrote: On 2016/03/31 14:07, Noah Misch wrote: On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote: On 2016/03/24 11:14, Michael Paquier wrote: On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <t...@linux.com> wrote: I've noticed that you now

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-31 Thread Etsuro Fujita
On 2016/03/31 14:07, Noah Misch wrote: On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote: On 2016/03/24 11:14, Michael Paquier wrote: On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <t...@linux.com> wrote: I've noticed that you now can't cancel a query if there's DML pu

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-29 Thread Etsuro Fujita
On 2016/03/25 17:16, Etsuro Fujita wrote: The approach that we discussed would minimize the code for the FDW author to write, by providing the support functions you proposed. I'll post a patch for that early next week. I added two helper functions: GetFdwScanTupleExtraData

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-03-28 Thread Etsuro Fujita
default: /* Should not happen, we have just check this above */ elog(ERROR, "unsupported join type %d", jointype); } wrongly pulls up remote_conds from joining relations in the FULL JOIN case. I think we should not pull up such conditions in the FUL

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-25 Thread Etsuro Fujita
s the same issue as for handling system columns in a RETURNING clause in such pushed-down UPDATE/DELETE. So I'd like to propose that approach as a common functionality.) Sorry for bringing this solution late to the table. No problem. Best regards, Etsuro Fujita [1] http://www.postgresql.org/

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-23 Thread Etsuro Fujita
s call and loop on PQgetResult with CHECK_FOR_INTERRUPTS() in it. Will fix. Thanks for the report, Thom! Thanks for the advice, Michael! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgres

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-23 Thread Etsuro Fujita
another function that would fill system columns using these Lists when creating a scan tuple. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Etsuro Fujita
On 2016/03/22 21:10, Ashutosh Bapat wrote: On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/03/22 14:54, Ashutosh Bapat wrote: Earlier in this mail chain, I suggested that the core should

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Etsuro Fujita
On 2016/03/22 14:54, Ashutosh Bapat wrote: On Tue, Mar 22, 2016 at 8:03 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: OK, I'll modify the patch so that the join is pushed down even if any of xmins, xmaxs, cmins, and cmaxs are re

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-21 Thread Etsuro Fujita
On 2016/03/19 4:51, Robert Haas wrote: On Thu, Mar 17, 2016 at 7:00 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: So, I'd like to propose: (1) when tableoids are requested from the remote server, postgres_fdw sets valid values for them locally, instead (core should s

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-21 Thread Etsuro Fujita
On 2016/03/19 3:30, Robert Haas wrote: On Fri, Mar 18, 2016 at 5:15 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached is the updated version of the patch. Committed. Thank you. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-19 Thread Etsuro Fujita
the tests in postgresPlanDirectModify. That just becomes a nuisance to keep up to date as things change. Agreed. I updated the patch to address these comments. Attached is the updated version of the patch. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw

[HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-19 Thread Etsuro Fujita
in practice. Attached is a proposed patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 123,129 static void deparseTargetList(StringInfo buf, Bitmapset *attrs_used, bool qualify_col

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-18 Thread Etsuro Fujita
On 2016/03/17 22:15, Ashutosh Bapat wrote: On Thu, Mar 17, 2016 at 4:30 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: BUT: we don't make any effort to ensure that local and remote values match, so system columns other than

[HACKERS] Odd oid-system-column handling in postgres_fdw

2016-03-16 Thread Etsuro Fujita
requested. I'm attaching a proposed patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 287,298 foreign_expr_walker(Node *node, /* Var belongs to foreign table */ /* ! * System columns other

Re: [HACKERS] Missing conversion error handling in postgres_fdw

2016-03-15 Thread Etsuro Fujita
On 2016/03/16 5:55, Robert Haas wrote: On Tue, Mar 15, 2016 at 4:06 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached is a patch for that. Hmm, I'd say you are right. Committed. Thank you for taking care of this! Best regards, Etsuro Fujita -- Sent via pgsql-h

[HACKERS] Missing conversion error handling in postgres_fdw

2016-03-15 Thread Etsuro Fujita
umGetPointer(datum); } } I think errpos.cur_attno should be set for the ctid case as well. Attached is a patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 3755,3772 make_tuple_from_result

Re: [HACKERS] Obsolete comment in postgres_fdw.c

2016-03-14 Thread Etsuro Fujita
On 2016/03/14 16:42, Ashutosh Bapat wrote: On Mon, Mar 14, 2016 at 9:05 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: Here is the comments for foreign_join_ok in postgres_fdw.c: /* * Assess whether the join between inn

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-03-14 Thread Etsuro Fujita
egards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 65,72 enum FdwScanPrivateIndex FdwScanPrivateRetrievedAttrs, /* Integer representing the desired fetch_size */ FdwScanPrivateFetchSize, - /* Oid of user mapping to b

[HACKERS] Obsolete comment in postgres_fdw.c

2016-03-13 Thread Etsuro Fujita
some related comments in src/include/nodes/relation.h and src/backend/optimizer/path/joinpath.c. Attached is a patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 3345,3354

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-13 Thread Etsuro Fujita
On 2016/03/14 11:51, Tom Lane wrote: > Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> writes: >> On 2016/03/13 4:46, Andres Freund wrote: >>> ... The difference apears to be the >>> check that's now in build_simple_rel() - there was nothing hitting the >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-13 Thread Etsuro Fujita
levant, and don't attempt join pushdown unless it's the same for all relevant relations. Per a suggestion from Tom Lane. Shigeru Hanada and Ashutosh Bapat, reviewed by Etsuro Fujita and KaiGai Kohei, with some further changes by me. I noticed that this breaks some citus regression tests in a minor

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-10 Thread Etsuro Fujita
now have new ones. Personally, I think we should just commit the darned thing and you can rewrite it later if you want. If you want to rewrite it now instead, I can live with that, too. But let's figure out how we're going to move this forward. I agree with Robert. Best regards, Etsuro Fujita

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-09 Thread Etsuro Fujita
On 2016/03/08 2:35, Robert Haas wrote: On Mon, Mar 7, 2016 at 7:53 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Another option to avoid such a hazard would be to remove the two changes from ExecInitModifyTable and create ExecAuxRowMarks and junk filters even in the pushdown ca

Re: [HACKERS] Minor documentation tweak to GetForeignPlan documentation

2016-03-09 Thread Etsuro Fujita
On 2016/03/09 4:36, Robert Haas wrote: On Fri, Jan 15, 2016 at 6:20 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached patch makes minor modification to the GetForeignPlan documentation. This adds the description about outer_plan, the new parameter added in 9.5. Good

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-07 Thread Etsuro Fujita
egin the execution phase. + End the table update and release resources. It is normally not important And I think this one should say "Clean up following a bulk modification on the remote server". It's not actually ending the update; the iterate method already did that. OK, will

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-01 Thread Etsuro Fujita
of joined relations in RETURNING, other than ctid. I'll fix that ASAP. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 130,135 static void deparseReturningList(StringInfo buf, PlannerInfo *root, --- 130,143

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-22 Thread Etsuro Fujita
On 2016/02/22 20:13, Rushabh Lathia wrote: I did another round of review for the latest patch and well as performed the sanity test, and haven't found any functional issues. Found couple of issue, see in-line comments for the same. Thanks! On Thu, Feb 18, 2016 at 3:15 PM, Etsuro Fujita

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-18 Thread Etsuro Fujita
On 2016/02/16 16:40, Etsuro Fujita wrote: On 2016/02/16 16:02, Ashutosh Bapat wrote: On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/02/16 15:22, Ashutosh Bapat wrote: During j

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-18 Thread Etsuro Fujita
On 2016/02/12 21:19, Etsuro Fujita wrote: Comments on specific points follow. This seems to need minor rebasing in the wake of the join pushdown patch. Will do. Done. + /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */ + if (IsA(plan, ForeignScan

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
On 2016/02/16 16:02, Ashutosh Bapat wrote: On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/02/16 15:22, Ashutosh Bapat wrote: During join planning, the planner tries multiple

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
, but I don't think that that needs to be considered because we have this at the beginning of postgresGetGForeignJoinPaths: /* * Skip if this join combination has been considered already. */ if (joinrel->fdw_private) return; Best regards, Etsuro Fujita -- Sent via pg

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
level joins. Maybe I'm missing something, but I don't understand why such a change in those lists happens. Could you explain about that in more detail? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-15 Thread Etsuro Fujita
On 2016/02/15 15:20, Rushabh Lathia wrote: On Fri, Feb 12, 2016 at 5:40 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: As a result of our discussions, we reached a conclusion that the DML pushdown APIs should be provi

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-14 Thread Etsuro Fujita
me 'baz'); CREATE FOREIGN TABLE postgres=# select * from foo, bar, baz where foo.a = bar.a and bar.a = baz.a and foo.a < 10 and bar.a < 10 and baz.a < 10; Attached is a patch to avoid the breakage. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/cont

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-14 Thread Etsuro Fujita
On 2016/02/12 21:46, Robert Haas wrote: On Fri, Feb 12, 2016 at 7:19 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I think that displaying target lists would be confusing for users. Here is an example: EXPLAIN (verbose, costs off) DELETE FROM rem1; -- can be

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-12 Thread Etsuro Fujita
licating the docs. Also: "When the query doesn't has the clause, the FDW must also increment the row count for the ForeignScanState node in the EXPLAIN ANALYZE case." Should read "doesn't have" Will fix. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-12 Thread Etsuro Fujita
postgresBeginForeignScan and postgresBeginForeignModify can call. execute_dml_stmt() has some of the same disease. Will do. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Etsuro Fujita
Here is a patch to use %u not %d to print umid and userid. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/connection.c --- b/contrib/postgres_fdw/connection.c *** *** 159,165 GetConnection(UserMapping *user, bool will_prep_stmt) entry->have_error = false; en

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-08 Thread Etsuro Fujita
k executed should succeed.) The reason for that is that the targetlist of the local join plan is the same as for the ForeignScan, which outputs neither foo.a nor bar.a required as an argument of the function add(). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Etsuro Fujita
think the patch fails to adjust the targetlist of the top plan created that way, to output the fdw_scan_tlist, as discussed in [1] (ie, I think the attached patch is needed, which is created on top of your patch pg_fdw_join_v8.patch). Best regards, Etsuro Fujita [1] http://www.postgre

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Etsuro Fujita
xecuted, thus even if root->rowMarks is NIL, EPQ check can fire and we will need alternate local plan. Yeah, I think that is true, but if root->rowMarks==NIL, we won't have non-target foreign tables, and therefore postgresGetForeignJoinPaths() will never be called. No? Best regards, Etsur

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
On 2016/02/04 17:58, Etsuro Fujita wrote: On 2016/02/04 8:00, Robert Haas wrote: On Wed, Feb 3, 2016 at 5:56 PM, Robert Haas <robertmh...@gmail.com> wrote: On Wed, Feb 3, 2016 at 12:08 PM, Ashutosh Bapat <ashutosh.ba...@enterprisedb.com> wrote: PFA patches with naming convent

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-04 Thread Etsuro Fujita
of the patch. You don't need to apply the patch fdw-foreign-modify-rmh-v2.patch attached before. Thanks for the review! Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 1069,1074 deparseUpdateSql(StringInfo buf

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
On 2016/01/29 17:52, Ashutosh Bapat wrote: On Fri, Jan 29, 2016 at 2:05 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/29 1:26, Ashutosh Bapat wrote: Here is the summary of changes from the last set of patches

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
h; + } * IIUC, that function will be used by custom joins, so I think it would be better to put that function somewhere in the /optimizer directory (pathnode.c?). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-03 Thread Etsuro Fujita
On 2016/01/28 15:20, Rushabh Lathia wrote: On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/27 21:23, Rushabh Lathia wrote: If I understood correctly, above documentation means, that if

Re: [HACKERS] 2016-01 Commitfest

2016-02-03 Thread Etsuro Fujita
On 2016/02/04 12:04, Robert Haas wrote: On Wed, Feb 3, 2016 at 10:00 PM, Noah Misch <n...@leadboat.com> wrote: Thank you. +1. Thank you! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Copy-pasto in the ExecForeignDelete documentation

2016-02-03 Thread Etsuro Fujita
On 2016/02/04 0:13, Robert Haas wrote: On Mon, Feb 1, 2016 at 5:26 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I don't think the data is referenced by the AFTER ROW DELETE triggers. Why do you think that? And why would DELETE triggers be different from UPDATE triggers, wh

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-02-02 Thread Etsuro Fujita
On 2016/02/03 3:31, Alvaro Herrera wrote: Etsuro Fujita wrote: Done. Attached is an updated version of the patch. Pushed, thanks. Thank you! I kinda wonder why this struct member has a name that doesn't match the naming convention in the rest of the struct, and also why isn't

[HACKERS] Copy-pasto in the ExecForeignDelete documentation

2016-02-01 Thread Etsuro Fujita
columns, but the I don't think the data is referenced by the AFTER ROW DELETE triggers. Attached is a patch to fix that. The patch also avoids adding an unnecessary RETURNING clause to DELETE when deparsing a remote DELETE statement in postgres_fdw. I'll add this to the next CF. Best regards, Etsuro

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Etsuro Fujita
-foreign-join-path cases, though that would work well for the unparameterized-foreign-join-path cases. We don't support parameterized-foreign-join paths for 9.6? Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/5666b59f.6010...@lab.ntt.co.jp -- Sent via pgsql-hackers

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-28 Thread Etsuro Fujita
On 2016/01/28 12:58, Robert Haas wrote: On Thu, Jan 21, 2016 at 4:05 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: By the way, I'm not too sure I understand the need for the core changes that are part of this patch, and I think that point merits some discussion. Whenever you

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-28 Thread Etsuro Fujita
On 2016/01/28 18:15, Alvaro Herrera wrote: Etsuro Fujita wrote: On 2016/01/28 12:13, Robert Haas wrote: I don't think this is a good idea. Most of the time, no system columns will be present, and we'll just be scanning the Bitmapset twice rather than once. Sure, that doesn't take many

Re: [HACKERS] New committer

2016-01-28 Thread Etsuro Fujita
e PostgreSQL project. Congratulations, Dean. Well deserved. Congratulations, Dean! Congratulations! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
On 2016/01/28 15:20, Rushabh Lathia wrote: On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/27 21:23, Rushabh Lathia wrote: If I understood correctly, above documentation means, that if

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-27 Thread Etsuro Fujita
On 2016/01/28 12:13, Robert Haas wrote: On Thu, Jan 21, 2016 at 5:55 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/01/21 7:04, Alvaro Herrera wrote: Etsuro Fujita wrote: On second thought, I noticed that detecting whether we see a system column that way needs more

[HACKERS] Minor improvement to fdwhandler.sgml

2016-01-27 Thread Etsuro Fujita
Here is a small patch to do s/for/For/ to two section titles in fdwhandlers.sgml, for consistency. Best regards, Etsuro Fujita diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index dc2d890..9c8406c 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
On 2016/01/27 12:20, Etsuro Fujita wrote: On 2016/01/26 22:57, Rushabh Lathia wrote: On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/25 17:03, Rushabh Lathia wrote:

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-26 Thread Etsuro Fujita
lUpdatable discussed above reports not only the updatability but the insertability and deletability of a foreign table!). So, +1 for leaving that as-is. Apart from this perform sanity testing on the new patch and things working as expected. Thanks for the review! Best regards, Etsuro Fujita

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-26 Thread Etsuro Fujita
On 2016/01/26 22:57, Rushabh Lathia wrote: On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/25 17:03, Rushabh Lathia wrote: int IsForeignRelUpdatable (

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-21 Thread Etsuro Fujita
On 2016/01/21 7:04, Alvaro Herrera wrote: Etsuro Fujita wrote: On second thought, I noticed that detecting whether we see a system column that way needs more cycles in cases where the reltargetlist and the restriction clauses don't contain any system columns. ISTM that such cases are rather

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-21 Thread Etsuro Fujita
On 2016/01/21 5:06, Robert Haas wrote: On Wed, Jan 20, 2016 at 4:50 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: My concern about that is that would make the code in deparseTargetList() complicated. Essentially, I think your propossal needs a two-pass algorithm for deparseTarg

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-21 Thread Etsuro Fujita
a fix for a wrong copy-and-paste. Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/ca+tgmoz40j2uc5ac1nxu03oj4crvolks15xx+ptfp-1u-8z...@mail.gmail.com *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 823,829 deparseTargetList

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-21 Thread Etsuro Fujita
On 2016/01/19 19:04, Thom Brown wrote: On 12 January 2016 at 11:49, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/01/12 20:36, Thom Brown wrote: On 8 January 2016 at 05:08, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: On 2016/01/06 20:37, Thom Brown wrote

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-20 Thread Etsuro Fujita
On 2016/01/20 3:42, Robert Haas wrote: On Tue, Jan 19, 2016 at 1:59 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remote SQL command: UPDATE public.custome

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-18 Thread Etsuro Fujita
On 2016/01/08 14:08, Etsuro Fujita wrote: On 2016/01/07 21:50, Etsuro Fujita wrote: On 2016/01/06 20:37, Thom Brown wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remote SQL command: UPDATE public.customers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-18 Thread Etsuro Fujita
). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-18 Thread Etsuro Fujita
On 2016/01/15 19:00, Etsuro Fujita wrote: On 2016/01/12 18:00, Etsuro Fujita wrote: On 2016/01/12 2:36, Alvaro Herrera wrote: I wonder, --- 2166,2213 } /* ! * If rel is a base relation, detect whether any system columns are ! * requested from the rel. (If rel

[HACKERS] Minor documentation tweak to GetForeignPlan documentation

2016-01-15 Thread Etsuro Fujita
Attached patch makes minor modification to the GetForeignPlan documentation. This adds the description about outer_plan, the new parameter added in 9.5. Best regards, Etsuro Fujita *** a/doc/src/sgml/fdwhandler.sgml --- b/doc/src/sgml/fdwhandler.sgml *** *** 178,184

Re: [HACKERS] Minor comment update in setrefs.c

2016-01-15 Thread Etsuro Fujita
On 2015/12/11 2:21, Robert Haas wrote: On Tue, Dec 8, 2015 at 6:16 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: Attached is a small patch to adjust a comment in setrefs.c; in set_foreignscan_references, fdw_recheck_quals also gets adjusted to reference foreign scan tuple, i

<    1   2   3   4   5   6   7   8   9   10   >