Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-10-02 Thread Etsuro Fujita
(2018/10/02 16:45), Michael Paquier wrote: On Tue, Oct 02, 2018 at 04:11:54PM +0900, Amit Langote wrote: I tried to close it as being committed, but couldn't do so, because I can't find Fujita-san's name in the list of committers in the CF app's drop down box that lists all committers. Indeed,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-10-02 Thread Michael Paquier
On Tue, Oct 02, 2018 at 04:11:54PM +0900, Amit Langote wrote: > I tried to close it as being committed, but couldn't do so, because I > can't find Fujita-san's name in the list of committers in the CF app's > drop down box that lists all committers. Indeed, Fujita-san has been added to the list.

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-10-02 Thread Amit Langote
On 2018/08/31 21:40, Etsuro Fujita wrote: > (2018/08/31 21:30), Jonathan S. Katz wrote: >> Thank you for taking care of that and for committing the patch. I have >> now closed this issues on the open items list. > > Thanks! I noticed that the CF entry for this was not closed. As of this morning,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-31 Thread Etsuro Fujita
(2018/08/31 21:30), Jonathan S. Katz wrote: On 8/31/18 7:54 AM, Etsuro Fujita wrote: (2018/08/30 20:25), Etsuro Fujita wrote: (2018/08/29 18:40), Etsuro Fujita wrote: (2018/08/29 0:21), Jonathan S. Katz wrote: On Aug 24, 2018, at 8:38 AM, Etsuro Fujita wrote: (2018/08/24 11:47), Michael Paqu

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-31 Thread Jonathan S. Katz
On 8/31/18 7:54 AM, Etsuro Fujita wrote: > (2018/08/30 20:25), Etsuro Fujita wrote: >> (2018/08/29 18:40), Etsuro Fujita wrote: >>> (2018/08/29 0:21), Jonathan S. Katz wrote: > On Aug 24, 2018, at 8:38 AM, Etsuro > Fujita wrote: > (2018/08/24 11:47), Michael Paquier wrote: >> On Thu

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-31 Thread Etsuro Fujita
(2018/08/30 20:25), Etsuro Fujita wrote: (2018/08/29 18:40), Etsuro Fujita wrote: (2018/08/29 0:21), Jonathan S. Katz wrote: On Aug 24, 2018, at 8:38 AM, Etsuro Fujita wrote: (2018/08/24 11:47), Michael Paquier wrote: On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: I tried this

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-30 Thread Etsuro Fujita
(2018/08/29 18:40), Etsuro Fujita wrote: (2018/08/29 0:21), Jonathan S. Katz wrote: On Aug 24, 2018, at 8:38 AM, Etsuro Fujita wrote: (2018/08/24 11:47), Michael Paquier wrote: On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: I tried this today, but doing git behind the corporate

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-29 Thread Etsuro Fujita
(2018/08/29 0:21), Jonathan S. Katz wrote: On Aug 24, 2018, at 8:38 AM, Etsuro Fujita wrote: (2018/08/24 11:47), Michael Paquier wrote: On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: I tried this today, but doing git behind the corporate firewall doesn't work. I don't know the

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-28 Thread Jonathan S. Katz
> On Aug 24, 2018, at 8:38 AM, Etsuro Fujita > wrote: > > (2018/08/24 11:47), Michael Paquier wrote: >> On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: >>> I tried this today, but doing git behind the corporate firewall doesn't >>> work. I don't know the clear cause of that, so

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-24 Thread Etsuro Fujita
(2018/08/24 11:47), Michael Paquier wrote: On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: I tried this today, but doing git behind the corporate firewall doesn't work. I don't know the clear cause of that, so I'll investigate that tomorrow. You may be able to tweak that by usi

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-23 Thread Michael Paquier
On Thu, Aug 23, 2018 at 10:00:49PM +0900, Etsuro Fujita wrote: > I tried this today, but doing git behind the corporate firewall doesn't > work. I don't know the clear cause of that, so I'll investigate that > tomorrow. You may be able to tweak that by using https as origin point or proper git pr

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-23 Thread Etsuro Fujita
(2018/08/22 20:08), Etsuro Fujita wrote: (2018/08/16 12:00), Etsuro Fujita wrote: (2018/08/15 23:40), Robert Haas wrote: Given the comments from the RMT, and also on general principle, it seems like we really need to get on with committing something here. It's my understanding you plan to do th

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-22 Thread Etsuro Fujita
(2018/08/16 12:00), Etsuro Fujita wrote: (2018/08/15 23:40), Robert Haas wrote: Given the comments from the RMT, and also on general principle, it seems like we really need to get on with committing something here. It's my understanding you plan to do that, since it's your patch. When? I plan

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-15 Thread Etsuro Fujita
(2018/08/15 23:40), Robert Haas wrote: Given the comments from the RMT, and also on general principle, it seems like we really need to get on with committing something here. It's my understanding you plan to do that, since it's your patch. When? I plan to do that late next week as I'll go on le

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-15 Thread Robert Haas
On Wed, Aug 15, 2018 at 7:35 AM, Etsuro Fujita wrote: > Thanks for the comments, Robert! Given the comments from the RMT, and also on general principle, it seems like we really need to get on with committing something here. It's my understanding you plan to do that, since it's your patch. When?

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-15 Thread Etsuro Fujita
(2018/08/15 13:04), Amit Langote wrote: On 2018/08/15 12:25, Etsuro Fujita wrote: (2018/08/15 0:51), Robert Haas wrote: On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita wrote: One thing I noticed might be an improvement is to skip build_joinrel_partition_info if the given joinrel will be to

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-14 Thread Amit Langote
On 2018/08/15 12:25, Etsuro Fujita wrote: > (2018/08/15 0:51), Robert Haas wrote: >> On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita >>   wrote: >>> One thing I noticed might be an improvement is to skip >>> build_joinrel_partition_info if the given joinrel will be to have >>> consider_partitionwis

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-14 Thread Etsuro Fujita
(2018/08/15 0:51), Robert Haas wrote: On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita wrote: One thing I noticed might be an improvement is to skip build_joinrel_partition_info if the given joinrel will be to have consider_partitionwise_join=false; in the previous patch, that function created

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-14 Thread Jonathan S. Katz
> On Aug 14, 2018, at 11:51 AM, Robert Haas wrote: > > On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita > wrote: >> One thing I noticed might be an improvement is to skip >> build_joinrel_partition_info if the given joinrel will be to have >> consider_partitionwise_join=false; in the previous pa

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-14 Thread Robert Haas
On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita wrote: > One thing I noticed might be an improvement is to skip > build_joinrel_partition_info if the given joinrel will be to have > consider_partitionwise_join=false; in the previous patch, that function > created the joinrel's partition info such

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-13 Thread Etsuro Fujita
(2018/08/13 11:57), Robert Haas wrote: On Mon, Aug 6, 2018 at 8:30 AM, Etsuro Fujita wrote: In the above I used the test whether the relation's reloptkind is RELOPT_BASEREL or not, but I noticed that I had overlooked the case of a multi-level partitioned table. So I fixed that and added regre

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-12 Thread Robert Haas
On Mon, Aug 6, 2018 at 8:30 AM, Etsuro Fujita wrote: > In the above I used the test whether the relation's reloptkind is > RELOPT_BASEREL or not, but I noticed that I had overlooked the case of a > multi-level partitioned table. So I fixed that and added regression test > cases for that. I also

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-06 Thread Etsuro Fujita
(2018/08/03 22:28), Etsuro Fujita wrote: (2018/08/03 22:18), Etsuro Fujita wrote: Here is a patch for refusing to generate PWJ paths when WRVs are involved: * We no longer need to handle WRVs, so I've simplified build_joinrel_tlist() and setrefs.c to what they were before partition-wise join we

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Robert Haas
On Fri, Aug 3, 2018 at 10:36 AM, Tom Lane wrote: > Anyway, what I'm basically suggesting is that we just disable support for > PWJ in the problematic cases in v11. As long as PWJ isn't even on by > default, that's not much of a loss. Obviously we'll want to fix it in the > future, but the hour g

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 3, 2018 at 10:07 AM, Tom Lane wrote: >> As far as I can see from the example that started this thread, >> postgres_fdw injects WRVs into a PWJ whether or not the query involves >> FOR UPDATE; that's why this bug is reproducible in a query without FOR >> UPDATE.

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Robert Haas
On Fri, Aug 3, 2018 at 10:07 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Aug 2, 2018 at 4:25 PM, Tom Lane wrote: >>> Now, that's a bit of a problem for postgres_fdw, because it seems to >>> insist on injecting WRVs even when the query text does not require any. >>> Why is that, and can'

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 2, 2018 at 4:25 PM, Tom Lane wrote: >> Now, that's a bit of a problem for postgres_fdw, because it seems to >> insist on injecting WRVs even when the query text does not require any. >> Why is that, and can't we get rid of it? > I don't quite know what you mean

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Robert Haas
On Thu, Aug 2, 2018 at 4:25 PM, Tom Lane wrote: > I do not trust Ashutosh's patch because of the point you noted that it > will kick in on ConvertRowtypeExprs that are not related to partitioning. I don't remember making that point -- can you clarify? > It's also got a rather fundamental concept

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Etsuro Fujita
(2018/08/03 22:18), Etsuro Fujita wrote: Here is a patch for refusing to generate PWJ paths when WRVs are involved: * We no longer need to handle WRVs, so I've simplified build_joinrel_tlist() and setrefs.c to what they were before partition-wise join went in, as in the previous patch. * attr_n

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-03 Thread Etsuro Fujita
(2018/08/03 5:25), Tom Lane wrote: What I'm thinking might be a more appropriate thing, at least for getting v11 out the door, is to refuse to generate partitionwise joins when any whole-row vars are involved. Agreed. (Perhaps that's not enough to dodge all the problems, though?) Now, that's

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-02 Thread Tom Lane
Robert Haas writes: > Does anyone else want to weigh in on this? It seems to me that there > are several people who are quite willing to complain about the fact > that this hasn't been fixed, but not willing to express an opinion > about the shape of the fix. Either the RMT needs to take executi

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-02 Thread Andres Freund
Hi, On 2018-08-02 15:19:49 -0400, Robert Haas wrote: > Does anyone else want to weigh in on this? It seems to me that there > are several people who are quite willing to complain about the fact > that this hasn't been fixed, but not willing to express an opinion > about the shape of the fix. Eit

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-02 Thread Robert Haas
On Thu, Aug 2, 2018 at 7:20 AM, Etsuro Fujita wrote: > The new approach seems to me more localized than what Ashutosh had proposed. > One obvious advantage of the new approach is that we no longer need changes > to postgres_fdw for it to work for partitionwise joins, which would apply > for any ot

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-02 Thread Etsuro Fujita
(2018/08/02 4:30), Robert Haas wrote: On Wed, Aug 1, 2018 at 7:44 AM, Etsuro Fujita wrote: I updated the patch that way. Updated patch attached. I fixed a bug and did a bit of cleanups as well. Looking this over from a technical point of view, I think it's better than what you proposed bef

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-02 Thread Etsuro Fujita
(2018/08/02 2:44), Robert Haas wrote: Tom, Ashutosh, and I all seem to agree that we shouldn't try to re-jigger things at create-plan time. I think that a 3-1 consensus against your proposal is sufficient to say we shouldn't go that way. Agreed. Now, here you've got a new approach which avoi

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 7:44 AM, Etsuro Fujita wrote: > I updated the patch that way. Updated patch attached. I fixed a bug and > did a bit of cleanups as well. Looking this over from a technical point of view, I think it's better than what you proposed before but I still don't agree with the ap

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 7:46 AM, Etsuro Fujita wrote: > I posted the updated patch [1]. Etsuro-san: I really think we should just go with what Ashutosh had proposed. Tom, Ashutosh, and I all seem to agree that we shouldn't try to re-jigger things at create-plan time. I think that a 3-1 consensus

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Etsuro Fujita
(2018/08/01 5:31), Alvaro Herrera wrote: On 2018-Jul-31, Etsuro Fujita wrote: (2018/07/31 4:06), Andres Freund wrote: On 2018-07-20 08:38:09 -0400, Robert Haas wrote: I'm going to study this some more now, but I really think this is going in the wrong direction. We're going to have to get so

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Etsuro Fujita
(2018/07/26 21:11), Etsuro Fujita wrote: (2018/07/26 5:27), Robert Haas wrote: Well, I could have the wrong idea here, but I tend to think allowing for ConvertRowTypeExpr elsewhere won't be that bad. I still don't like that because in my opinion, changes needed for that would not be localized,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-31 Thread Alvaro Herrera
On 2018-Jul-31, Etsuro Fujita wrote: > (2018/07/31 4:06), Andres Freund wrote: > > On 2018-07-20 08:38:09 -0400, Robert Haas wrote: > > > I'm going to study this some more now, but I really think this is > > > going in the wrong direction. > > > > We're going to have to get somewhere on this topi

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-30 Thread Etsuro Fujita
(2018/07/31 4:06), Andres Freund wrote: On 2018-07-20 08:38:09 -0400, Robert Haas wrote: I'm going to study this some more now, but I really think this is going in the wrong direction. We're going to have to get somewhere on this topic soon. This thread has been open for nearly half a year, an

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-30 Thread Andres Freund
Hi, On 2018-07-20 08:38:09 -0400, Robert Haas wrote: > I'm going to study this some more now, but I really think this is > going in the wrong direction. We're going to have to get somewhere on this topic soon. This thread has been open for nearly half a year, and we're late in the beta phase now.

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-26 Thread Etsuro Fujita
(2018/07/26 5:27), Robert Haas wrote: On Tue, Jul 24, 2018 at 7:51 AM, Etsuro Fujita wrote: Isn't that assumption fundamental to your whole approach? I don't think so. What I mean here is: currently the subplan would be a scan/join node, but in future we might have eg, a Sort node atop the

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-25 Thread Ashutosh Bapat
On Thu, Jul 26, 2018 at 2:12 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jul 24, 2018 at 7:51 AM, Etsuro Fujita >> wrote: >>> I'm not sure that's a good idea, because I think we have a trade-off >>> relation; the more we make create_plan simple, the more we need to make >>> earlier stat

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-25 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 24, 2018 at 7:51 AM, Etsuro Fujita > wrote: >> I'm not sure that's a good idea, because I think we have a trade-off >> relation; the more we make create_plan simple, the more we need to make >> earlier states of the planner complicated. >> >> And it looks to me

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-25 Thread Robert Haas
On Tue, Jul 24, 2018 at 7:51 AM, Etsuro Fujita wrote: >> Isn't that assumption fundamental to your whole approach? > > I don't think so. What I mean here is: currently the subplan would be a > scan/join node, but in future we might have eg, a Sort node atop the > scan/join node, so it would be be

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-24 Thread Etsuro Fujita
(2018/07/24 3:09), Robert Haas wrote: On Mon, Jul 23, 2018 at 3:49 AM, Etsuro Fujita wrote: I have to admit that that is not a good idea. So, I'll update the patch so that we don't assume the projection capability of the subplan anymore, if we go this way. Isn't that assumption fundamental

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-23 Thread Robert Haas
On Mon, Jul 23, 2018 at 3:49 AM, Etsuro Fujita wrote: > I have to admit that that is not a good idea. So, I'll update the patch so > that we don't assume the projection capability of the subplan anymore, if we > go this way. Isn't that assumption fundamental to your whole approach? >> Also, eve

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-23 Thread Ashutosh Bapat
On Fri, Jul 20, 2018 at 8:56 PM, Robert Haas wrote: [ ... clipped portion ...] > > In short, plan creation time is just the wrong time to change the > plan. It's just a time to translate the plan from the form needed by > the planner to the form needed by the executor. It would be fine if > the

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-23 Thread Etsuro Fujita
(2018/07/21 0:26), Robert Haas wrote: On Fri, Jul 20, 2018 at 8:38 AM, Robert Haas wrote: This looks like a terrible design to me. If somebody in future invents a new plan type that is not projection-capable, then this could fail an assertion here and there won't be any simple fix. And if you

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-20 Thread Robert Haas
On Fri, Jul 20, 2018 at 8:38 AM, Robert Haas wrote: > This looks like a terrible design to me. If somebody in future > invents a new plan type that is not projection-capable, then this > could fail an assertion here and there won't be any simple fix. And > if you reach here and the child targetl

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-20 Thread Robert Haas
On Wed, Jul 18, 2018 at 8:00 AM, Etsuro Fujita wrote: > [ new patch ] + /* + * If the child plan is an Append or MergeAppend, the targetlists of its + * subplans would have already been adjusted before we get here, so need + * to work here. + */ + if (IsA(subplan, Append) || IsA(subplan, MergeApp

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-18 Thread Etsuro Fujita
(2018/07/13 23:05), Ashutosh Bapat wrote: On Thu, Jul 12, 2018 at 4:32 PM, Etsuro Fujita wrote: In this example, the value of the whole-row reference to the child table ptp1 for that record is ('foo',1), and that of the index expression for that record is (1,'foo'). Those have different colum

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-18 Thread Etsuro Fujita
(2018/07/04 11:00), Etsuro Fujita wrote: (2018/07/04 1:35), Andres Freund wrote: What's the plan forward here? I still think that this approach would be the right way to go, so I plan to polish the patch. The approach would not require extra cycles where partitioning is not involved as disc

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-13 Thread Ashutosh Bapat
On Thu, Jul 12, 2018 at 4:32 PM, Etsuro Fujita wrote: > > In this example, the value of the whole-row reference to the child table > ptp1 for that record is ('foo',1), and that of the index expression for that > record is (1,'foo'). Those have different column orders, but the latter > could be ma

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-12 Thread Etsuro Fujita
(2018/07/12 13:38), Ashutosh Bapat wrote: On Thu, Jul 12, 2018 at 9:02 AM, Etsuro Fujita wrote: (2018/07/11 20:02), Ashutosh Bapat wrote: On Wed, Jul 11, 2018 at 1:23 PM, Etsuro Fujita wrote: Actually, even if we could create such an index on the child table and the targetlist had the Con

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Ashutosh Bapat
On Thu, Jul 12, 2018 at 9:02 AM, Etsuro Fujita wrote: > (2018/07/11 20:02), Ashutosh Bapat wrote: >> >> On Wed, Jul 11, 2018 at 1:23 PM, Etsuro Fujita >> wrote: >>> >>> Actually, even if we could create such an index on the child table and >>> the >>> targetlist had the ConvertRowtypeExpr, the p

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Etsuro Fujita
(2018/07/11 20:02), Ashutosh Bapat wrote: On Wed, Jul 11, 2018 at 1:23 PM, Etsuro Fujita wrote: Actually, even if we could create such an index on the child table and the targetlist had the ConvertRowtypeExpr, the planner would still not be able to use an index-only scan with that index; becau

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Ashutosh Bapat
On Wed, Jul 11, 2018 at 1:23 PM, Etsuro Fujita wrote: > > > Actually, even if we could create such an index on the child table and the > targetlist had the ConvertRowtypeExpr, the planner would still not be able > to use an index-only scan with that index; because check_index_only would > not cons

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Etsuro Fujita
(2018/07/10 19:58), Ashutosh Bapat wrote: On Tue, Jul 10, 2018 at 9:08 AM, Etsuro Fujita wrote: (2018/07/09 20:43), Ashutosh Bapat wrote: At the cost of having targetlist being type inconsistent. I don't have any testcase either to show that that's a problem in practice. As I said before,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-10 Thread Ashutosh Bapat
On Tue, Jul 10, 2018 at 9:08 AM, Etsuro Fujita wrote: > (2018/07/09 20:43), Ashutosh Bapat wrote: >>> >>> >>> I don't have any numbers right now, so that is nothing but a concern. But >>> as >>> I said in a previous email, in the approach I proposed, we don't need to >>> spend extra cycles where p

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Etsuro Fujita
(2018/07/09 20:43), Ashutosh Bapat wrote: I don't have any numbers right now, so that is nothing but a concern. But as I said in a previous email, in the approach I proposed, we don't need to spend extra cycles where partitioning is not involved. I think that is a good thing in itself. No? A

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Ashutosh Bapat
> > I don't have any numbers right now, so that is nothing but a concern. But as > I said in a previous email, in the approach I proposed, we don't need to > spend extra cycles where partitioning is not involved. I think that is a > good thing in itself. No? At the cost of having targetlist bein

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Etsuro Fujita
(2018/07/09 20:06), Ashutosh Bapat wrote: On Mon, Jul 9, 2018 at 4:33 PM, Etsuro Fujita wrote: As I said, we do spend cycles in that function testing whether a node is Aggref or not even when the query doesn't have aggregates or grouping OR spend cycles in testing whether a node is a PlaceHold

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Ashutosh Bapat
On Mon, Jul 9, 2018 at 4:33 PM, Etsuro Fujita wrote: >> >> >> As I said, we do spend cycles in that function testing whether a node >> is Aggref or not even when the query doesn't have aggregates or >> grouping OR spend cycles in testing whether a node is a PlaceHolderVar >> when the query doesn't

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Etsuro Fujita
(2018/07/06 20:20), Ashutosh Bapat wrote: On Fri, Jul 6, 2018 at 4:29 PM, Etsuro Fujita wrote: (2018/07/04 21:37), Ashutosh Bapat wrote: On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita wrote: Let me explain about that: 1) my patch won't apply that function to a child if its top parent is

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-06 Thread Ashutosh Bapat
On Fri, Jul 6, 2018 at 4:29 PM, Etsuro Fujita wrote: > (2018/07/04 21:37), Ashutosh Bapat wrote: >> >> On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita >> wrote: >>> >>> (2018/07/04 19:04), Ashutosh Bapat wrote: On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita wrote: > > (20

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-06 Thread Etsuro Fujita
(2018/07/04 21:37), Ashutosh Bapat wrote: On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita wrote: (2018/07/04 19:04), Ashutosh Bapat wrote: On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita wrote: (2018/06/22 22:54), Ashutosh Bapat wrote: + if (enable_partitionwise_join&& rel->top_parent

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-05 Thread Etsuro Fujita
(2018/07/05 22:04), Ashutosh Bapat wrote: On Thu, Jul 5, 2018 at 4:45 PM, Etsuro Fujita wrote: Another thing I noticed is: actually, we don't produce an Append with child Gathers in apply_scanjoin_target_to_paths, which I thought we would do that in the case of scanjoin_target_parallel_safe=fa

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-05 Thread Ashutosh Bapat
On Thu, Jul 5, 2018 at 4:45 PM, Etsuro Fujita wrote: > > > In the case where scanjoin_target_parallel_safe=false, we actually will > consider such parallel paths using the existing partial paths for the parent > appendrel in the code path shown in a previous email (note: we would already > have do

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-05 Thread Etsuro Fujita
(2018/07/05 20:15), Etsuro Fujita wrote: (2018/07/04 19:11), Ashutosh Bapat wrote: On Wed, Jul 4, 2018 at 3:36 PM, Etsuro Fujita wrote: I don't produce a test case where the plan is an Append with Gather subplans, but I'm not sure that it's a good thing to allow us to consider such a plan bec

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-05 Thread Etsuro Fujita
(2018/07/04 19:11), Ashutosh Bapat wrote: On Wed, Jul 4, 2018 at 3:36 PM, Etsuro Fujita wrote: I don't produce a test case where the plan is an Append with Gather subplans, but I'm not sure that it's a good thing to allow us to consider such a plan because in that plan, each Gather would try

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita wrote: > (2018/07/04 19:04), Ashutosh Bapat wrote: >> >> On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita >> wrote: >>> >>> (2018/06/22 22:54), Ashutosh Bapat wrote: + if (enable_partitionwise_join&& rel->top_parent_is_partitioned)

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Etsuro Fujita
(2018/07/04 19:04), Ashutosh Bapat wrote: On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita wrote: (2018/06/22 22:54), Ashutosh Bapat wrote: + if (enable_partitionwise_join&& rel->top_parent_is_partitioned) + { + build_childrel_tlist(root, rel, childrel, 1,&appinfo); +

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
On Wed, Jul 4, 2018 at 3:36 PM, Etsuro Fujita wrote: > > I don't produce a test case where the plan is an Append with Gather > subplans, but I'm not sure that it's a good thing to allow us to consider > such a plan because in that plan, each Gather would try to grab its own pool > of workers. Am

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Etsuro Fujita
(2018/07/02 18:46), Etsuro Fujita wrote: (2018/06/22 23:58), Robert Haas wrote: And, in general, it seems to me that we want to produce the right outputs at the lowest possible level of the plan tree. For instance, suppose that one of the relations being scanned is not parallel-safe but the othe

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita wrote: > (2018/06/22 22:54), Ashutosh Bapat wrote: >> >> I have started reviewing the patch. > > > Thanks for the review! > >> + if (enable_partitionwise_join&& rel->top_parent_is_partitioned) >> + { >> + build_childrel_tlist(ro

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-03 Thread Etsuro Fujita
(2018/07/04 1:35), Andres Freund wrote: On 2018-06-22 10:58:28 -0400, Robert Haas wrote: On Tue, Jun 19, 2018 at 8:46 AM, Etsuro Fujita wrote: Here is a patch for that. I think this approach is going to run into trouble if the level at which we have to apply the ConvertRowTypeExpr happens

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-03 Thread Andres Freund
Hi Ashutosh, Etsuro, Robert, On 2018-06-22 10:58:28 -0400, Robert Haas wrote: > On Tue, Jun 19, 2018 at 8:46 AM, Etsuro Fujita > wrote: > > Here is a patch for that. > > > > * As I said upthread, the patch makes code much more simple; I removed all > > the changes to setrefs.c added by the partit

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-02 Thread Etsuro Fujita
(2018/06/22 23:58), Robert Haas wrote: I think this approach is going to run into trouble if the level at which we have to apply the ConvertRowTypeExpr happens not to be a projection-capable node. Actually, the level we have to do that would be a child rel of a partitioned table or a child joi

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-29 Thread Etsuro Fujita
(2018/06/22 22:54), Ashutosh Bapat wrote: I have started reviewing the patch. Thanks for the review! + if (enable_partitionwise_join&& rel->top_parent_is_partitioned) + { + build_childrel_tlist(root, rel, childrel, 1,&appinfo); + } Why do we need rel->top_parent_

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-22 Thread Robert Haas
On Tue, Jun 19, 2018 at 8:46 AM, Etsuro Fujita wrote: > Here is a patch for that. > > * As I said upthread, the patch makes code much more simple; I removed all > the changes to setrefs.c added by the partitionwise-join patch. I also > simplified the logic for building a tlist for a child-join re

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-22 Thread Ashutosh Bapat
On Tue, Jun 19, 2018 at 6:16 PM, Etsuro Fujita wrote: > > * As I said upthread, the patch makes code much more simple; I removed all > the changes to setrefs.c added by the partitionwise-join patch. I also > simplified the logic for building a tlist for a child-join rel. The original > PWJ comput

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-19 Thread Etsuro Fujita
(2018/06/15 20:56), Etsuro Fujita wrote: Actually, I've created a patch implementing that proposal. But I think that patch needs more work, so I'm planning to post it next week. Here is a patch for that. * As I said upthread, the patch makes code much more simple; I removed all the changes

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-15 Thread Etsuro Fujita
(2018/06/14 22:37), Ashutosh Bapat wrote: On Mon, Jun 11, 2018 at 4:05 PM, Etsuro Fujita wrote: (2018/06/07 19:42), Ashutosh Bapat wrote: On Wed, Jun 6, 2018 at 5:00 PM, Etsuro Fujita wrote: Yeah, but I mean we modify set_append_rel_size so that we only map a parent wholerow Var in the pa

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-14 Thread Ashutosh Bapat
On Mon, Jun 11, 2018 at 4:05 PM, Etsuro Fujita wrote: > (2018/06/07 19:42), Ashutosh Bapat wrote: >> >> On Wed, Jun 6, 2018 at 5:00 PM, Etsuro Fujita >> wrote: >>> >>> Since I'm not 100% sure that that is the right way to go, I've been >>> rethinking how to fix this issue. Yet another idea I ca

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-11 Thread Etsuro Fujita
(2018/06/07 19:42), Ashutosh Bapat wrote: On Wed, Jun 6, 2018 at 5:00 PM, Etsuro Fujita wrote: Since I'm not 100% sure that that is the right way to go, I've been rethinking how to fix this issue. Yet another idea I came up with to fix this is to redesign the handling of the tlists for childr

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-07 Thread Ashutosh Bapat
On Wed, Jun 6, 2018 at 5:00 PM, Etsuro Fujita wrote: > (2018/05/18 16:33), Etsuro Fujita wrote: >> >> Other than pull_var_clause things, >> the updated version looks good to me, so I'll mark this as Ready for >> Committer. > > > Since I'm not 100% sure that that is the right way to go, I've been >

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-06-06 Thread Etsuro Fujita
(2018/05/18 16:33), Etsuro Fujita wrote: Other than pull_var_clause things, the updated version looks good to me, so I'll mark this as Ready for Committer. Since I'm not 100% sure that that is the right way to go, I've been rethinking how to fix this issue. Yet another idea I came up with to

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-18 Thread Etsuro Fujita
(2018/05/17 23:19), Ashutosh Bapat wrote: On Thu, May 17, 2018 at 4:50 PM, Etsuro Fujita wrote: (2018/05/16 22:49), Ashutosh Bapat wrote: On Wed, May 16, 2018 at 4:01 PM, Etsuro Fujita wrote: However, considering that pull_var_clause is used in many places where partitioning is *not* invo

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-17 Thread Ashutosh Bapat
On Thu, May 17, 2018 at 4:50 PM, Etsuro Fujita wrote: > (2018/05/16 22:49), Ashutosh Bapat wrote: >> >> On Wed, May 16, 2018 at 4:01 PM, Etsuro Fujita >> wrote: >>> >>> However, considering that >>> pull_var_clause is used in many places where partitioning is *not* >>> involved, >>> I still thin

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-17 Thread Etsuro Fujita
(2018/05/16 22:49), Ashutosh Bapat wrote: On Wed, May 16, 2018 at 4:01 PM, Etsuro Fujita wrote: However, considering that pull_var_clause is used in many places where partitioning is *not* involved, I still think it's better to avoid spending extra cycles needed only for partitioning in that f

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-16 Thread Ashutosh Bapat
On Wed, May 16, 2018 at 4:01 PM, Etsuro Fujita wrote: > So we could really minimize the code change and avoid the additional overhead caused by the is_converted_whole_row_reference test added to pull_var_clause. I think the latter would be rather important, because P

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-16 Thread Etsuro Fujita
(2018/05/14 15:34), Ashutosh Bapat wrote: On Mon, May 14, 2018 at 10:21 AM, Ashutosh Bapat wrote: On Fri, May 11, 2018 at 6:31 PM, Etsuro Fujita wrote: Here's the query. explain verbose select * from fprt1 t1 join fprt2 t2 on t1.a = t2.b where t1 = row_as_is(row(t2.a, t2.b, t2.c)::ftprt2_p1

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-13 Thread Ashutosh Bapat
On Mon, May 14, 2018 at 10:21 AM, Ashutosh Bapat wrote: > On Fri, May 11, 2018 at 6:31 PM, Etsuro Fujita > wrote: >> >> I guess you forgot to show the query. > > Sorry. Here's the query. > > explain verbose select * from fprt1 t1 join fprt2 t2 on t1.a = t2.b > where t1 = row_as_is(row(t2.a, t2.b,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-13 Thread Ashutosh Bapat
On Fri, May 11, 2018 at 6:31 PM, Etsuro Fujita wrote: > > I guess you forgot to show the query. Sorry. Here's the query. explain verbose select * from fprt1 t1 join fprt2 t2 on t1.a = t2.b where t1 = row_as_is(row(t2.a, t2.b, t2.c)::ftprt2_p1)::fprt2; > > Yet yet another case where pull_var_cla

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-11 Thread Etsuro Fujita
(2018/05/11 16:17), Ashutosh Bapat wrote: On Thu, May 10, 2018 at 6:23 PM, Etsuro Fujita wrote: Yeah, but I think the IsA test would be sufficient to make things work correctly because we can assume that there aren't any other nodes than Var, PHV, and CRE translating a wholerow value of a chil

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-11 Thread Ashutosh Bapat
On Thu, May 10, 2018 at 6:23 PM, Etsuro Fujita wrote: > (2018/05/10 13:04), Ashutosh Bapat wrote: >> >> On Wed, May 9, 2018 at 5:20 PM, Etsuro Fujita >> wrote: >>> >>> (2018/04/25 18:51), Ashutosh Bapat wrote: Actually I noticed that ConvertRowtypeExpr are used to cast a child's w

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-10 Thread Etsuro Fujita
(2018/05/10 13:04), Ashutosh Bapat wrote: On Wed, May 9, 2018 at 5:20 PM, Etsuro Fujita wrote: (2018/04/25 18:51), Ashutosh Bapat wrote: Actually I noticed that ConvertRowtypeExpr are used to cast a child's whole row reference expression (not just a Var node) into that of its parent and not.

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-05-09 Thread Ashutosh Bapat
On Wed, May 9, 2018 at 5:20 PM, Etsuro Fujita wrote: > > (2018/04/25 18:51), Ashutosh Bapat wrote: >> Actually I noticed that ConvertRowtypeExpr are used to cast a child's >> whole row reference expression (not just a Var node) into that of its >> parent and not. For example a cast like NULL::chil

  1   2   >