Re: [DISCUSS] get RexExecutor from RexSimplify in method reduceExpressionsInternal
I’m a little worried about it the default RexExecutorImpl can handle all the downstream projects expressions, and very probably not, there would be some Janino compile exception if it can not translate the RexNodes correctly. So strictly to say, change the RexExecutor to a default implementation may break something. I think it’s better if we have a real case to illustrate that the modification is meaningful. In production, if an engine really wants to support constant reduction for their all kinds of expression, they should set up the RexExecutor explicitly. If they do not set up that, the constant reduction just not happens, it is better than supplying a default RexExecutor but does not really work for all expression. So I’m +0 for this. Best, Danny Chan 在 2020年3月17日 +0800 PM4:16，JiaTao Tao ，写道： > Hi Danny > > Thanks for your reply, I think Stamatis Zampetakis's opinion is summative, > and here the problem I think is a default RexExecutor is better than null, > especially, in this case, cuz `reduceExpressionsInternal` and > `reduceExpressions` is in the same path, thought the use of RexExecutor may > be different, but it still makes people confusing. > > IMHO, if "return RexUtil.EXECUTOR" >= "return null", we can do the modify. > > If you think so, I can open a JIRA and do this minor change. > > Hope to hear your voice. > > Regards! > > Aron Tao > > > JiaTao Tao 于2020年3月17日周二 下午4:02写道： > > > Hi Stamatis Zampetakis > > > > I agree with this completely: "The API of RexExecutor says the following > > "If an expression cannot be > > reduced, writes the original expression..." so we don't break anything by > > providing a default one." > > > > > > > > Regards! > > > > Aron Tao > > > > > > Stamatis Zampetakis 于2020年3月16日周一 下午9:52写道： > > > > > Interestingly, I was looking at this same piece of code not so long ago > > > and > > > I agree it is a bit confusing. > > > > > > Looking around the places that we obtain a RexExecutor, most often > > > (always?) we observe the following pattern: > > > > > > RexExecutor executor = > > > Util.first(query.getCluster().getPlanner().getExecutor(), > > > RexUtil.EXECUTOR); > > > > > > I think it is always useful to have an executor in the planner thus I am > > > tempted to change the API of RelOptPlanner#getExecutor to always return an > > > (default) executor if an explicit one is not set. > > > > > > The API of RexExecutor says the following "If an expression cannot be > > > reduced, writes the original expression..." so we don't break anything by > > > providing a default one. > > > > > > What do you think? > > > > > > Best, > > > Stamatis > > > > > > On Mon, Mar 16, 2020 at 11:11 AM Danny Chan wrote: > > > > > > > Thanks, the code is a little mess, here is how I understand it: > > > > > > > > The executor from `final RexExecutor executor > > > > = Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)` is > > > > mainly used to construct the RexSimplify, in the RexSimplify, the > > > > expression that we evaluate is what we can make sure RexUtil.EXECUTOR > > > can > > > > resolve(if you check the code, it only reduce the literals). > > > > > > > > But the expressions in the ReduceExpressionsRule may be more complex, > > > > somehow we must relay on the engine to plugin their RexExecutor to make > > > a > > > > constant reduction(some engine use code generation, some use Java > > > > reflection). > > > > > > > > So, in total, the executor in RexSimplify has a fallback is because it’s > > > > expression to reduce is simple enough. > > > > > > > > Best, > > > > Danny Chan > > > > 在 2020年3月16日 +0800 PM3:57，JiaTao Tao ，写道： > > > > > In method reduceExpressionsInternal, we get RexExecutor from cluster, > > > it > > > > can be null: > > > > > <> > > > > > > > > > > But in the outside(reduceExpressions), `final RexExecutor executor = > > > > Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)`, it > > > can't > > > > be null. > > > > > > > > > > And reduceExpressions is the only caller of reduceExpressionsInternal, > > > > so I think this is an inconsistent behavior. > > > > > > > > > > IMHO, we should create RexUtil.EXECUTOR if it is null > > > > in reduceExpressionsInternal, or just get RexExecutor from RexSimplify. > > > > > <> > > > > > > > > > > Regards! > > > > > Aron Tao > > > > > > > > >
Re: TableModify does not keep UPSERT keyword
I don't think there's a significant difference between the UPSERT and MERGE. The differences are in marketing (which keyword to use) and in technical details (e.g. concurrency semantics). Not worth splitting a core concept over. We spend a lot of effort keeping like-things-like. On Tue, Mar 17, 2020 at 1:11 AM Christian Beikov wrote: > > AFAIK MERGE has different concurrency semantics than what some DBMS call > UPSERT. PostgreSQL for example has a guaranteed insert or update > semantics whereas MERGE could end with constraint violation errors: > https://wiki.postgresql.org/wiki/UPSERT > > Maybe it's worth adding that to the relational model after all? > > Am 17.03.2020 um 07:17 schrieb Enrico Olivelli: > > Il Lun 16 Mar 2020, 23:55 Julian Hyde ha scritto: > > > >> Change the unparse operation for the dialect so that you generate UPSERT > >> rather than MERGE. > >> > >> IIRC we did this for another dialect - maybe Phoenix. > >> > > Julian, > > In my case (HerdDB) I need to see the presence of UPSERT in the RelNode > > (EnumerableTableModify oŕ LogicalTableModify) > > I saw the JIRA where you introduced UPSERT for Phoenix > > I will check deeper, thanks > > > > Enrico > > > > > > > >> Julian > >> > >>> On Mar 16, 2020, at 1:22 AM, Enrico Olivelli > >> wrote: > >>> Il Lun 16 Mar 2020, 09:06 Stamatis Zampetakis ha > >>> scritto: > >>> > Hi Enrico, > > I have the impression that UPSERT is currently supported only at the > >> parser > level [1] so it seems normal that you don't find something relevant in > LogicalTableModify etc. Note that the SQL standard equivalent is the > >> MERGE > statement [2] but this also seems to be supported only at the > parser/validator level [2]. > I guess it is not necessary to introduce more things in TableModify > >> since > UPSERT seems to be syntactic sugar. I think that most of the work can be > done in RelToSqlConverter [4] and possibly the rest of the code can be > >> left > intact. > > >>> I would like to sens a patch that introduces the ability to keep the > >>> SqlInsert 'keywords' and pass them to TableModify? > >>> Would it be a good approach? > >>> > >>> The alternative is to introduce a new Operation but it would be a more > >>> invasive change and I think it is not worth > >>> > >>> > >>> Enrico > >>> > >>> > Best, > Stamatis > > [1] https://issues.apache.org/jira/browse/CALCITE-492 > [2] https://en.wikipedia.org/wiki/Merge_(SQL) > [3] https://issues.apache.org/jira/browse/CALCITE-985 > [4] > > > >> https://github.com/apache/calcite/blob/d234626227954eefffe49f42abec65c649ffe3a6/core/src/test/java/org/apache/calcite/test/SqlToRelConverterTest.java#L2395 > > On Sun, Mar 15, 2020 at 6:53 PM Enrico Olivelli > > wrote: > > > > Hi, > > I am trying to use UPSERT but it seems to me that in TableModify (or > > best LogicalTableModify or EnumerableTableModify) there is no way to > > distinguish an INSERT from an UPSERT. > > > > Am I missing something? > > > > Regards > > Enrico > >
Calcite-Master - Build # 1650 - Still Failing
The Apache Jenkins build system has built Calcite-Master (build #1650) Status: Still Failing Check console output at https://builds.apache.org/job/Calcite-Master/1650/ to view the results.
[jira] [Created] (CALCITE-3863) Truncate return type inference through rel data type factory
Praveen Kumar created CALCITE-3863: -- Summary: Truncate return type inference through rel data type factory Key: CALCITE-3863 URL: https://issues.apache.org/jira/browse/CALCITE-3863 Project: Calcite Issue Type: Improvement Components: core Reporter: Praveen Kumar Allow truncate return type to be inferred through rel data type, so that consumers can override this behavior if required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] get RexExecutor from RexSimplify in method reduceExpressionsInternal
I don't know what others think but +1 from my side. Best, Stamatis On Tue, Mar 17, 2020 at 9:16 AM JiaTao Tao wrote: > Hi Danny > > Thanks for your reply, I think Stamatis Zampetakis's opinion is summative, > and here the problem I think is a default RexExecutor is better than null, > especially, in this case, cuz `reduceExpressionsInternal` and > `reduceExpressions` is in the same path, thought the use of RexExecutor may > be different, but it still makes people confusing. > > IMHO, if "return RexUtil.EXECUTOR" >= "return null", we can do the modify. > > If you think so, I can open a JIRA and do this minor change. > > Hope to hear your voice. > > Regards! > > Aron Tao > > > JiaTao Tao 于2020年3月17日周二 下午4:02写道： > > > Hi Stamatis Zampetakis > > > > I agree with this completely: "The API of RexExecutor says the following > > "If an expression cannot be > > reduced, writes the original expression..." so we don't break anything by > > providing a default one." > > > > > > > > Regards! > > > > Aron Tao > > > > > > Stamatis Zampetakis 于2020年3月16日周一 下午9:52写道： > > > >> Interestingly, I was looking at this same piece of code not so long ago > >> and > >> I agree it is a bit confusing. > >> > >> Looking around the places that we obtain a RexExecutor, most often > >> (always?) we observe the following pattern: > >> > >> RexExecutor executor = > >> Util.first(query.getCluster().getPlanner().getExecutor(), > >> RexUtil.EXECUTOR); > >> > >> I think it is always useful to have an executor in the planner thus I am > >> tempted to change the API of RelOptPlanner#getExecutor to always return > an > >> (default) executor if an explicit one is not set. > >> > >> The API of RexExecutor says the following "If an expression cannot be > >> reduced, writes the original expression..." so we don't break anything > by > >> providing a default one. > >> > >> What do you think? > >> > >> Best, > >> Stamatis > >> > >> On Mon, Mar 16, 2020 at 11:11 AM Danny Chan > wrote: > >> > >> > Thanks, the code is a little mess, here is how I understand it: > >> > > >> > The executor from `final RexExecutor executor > >> > = Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)` is > >> > mainly used to construct the RexSimplify, in the RexSimplify, the > >> > expression that we evaluate is what we can make sure RexUtil.EXECUTOR > >> can > >> > resolve(if you check the code, it only reduce the literals). > >> > > >> > But the expressions in the ReduceExpressionsRule may be more complex, > >> > somehow we must relay on the engine to plugin their RexExecutor to > make > >> a > >> > constant reduction(some engine use code generation, some use Java > >> > reflection). > >> > > >> > So, in total, the executor in RexSimplify has a fallback is because > it’s > >> > expression to reduce is simple enough. > >> > > >> > Best, > >> > Danny Chan > >> > 在 2020年3月16日 +0800 PM3:57，JiaTao Tao ，写道： > >> > > In method reduceExpressionsInternal, we get RexExecutor from > cluster, > >> it > >> > can be null: > >> > > <> > >> > > > >> > > But in the outside(reduceExpressions), `final RexExecutor executor = > >> > Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)`, it > >> can't > >> > be null. > >> > > > >> > > And reduceExpressions is the only caller of > reduceExpressionsInternal, > >> > so I think this is an inconsistent behavior. > >> > > > >> > > IMHO, we should create RexUtil.EXECUTOR if it is null > >> > in reduceExpressionsInternal, or just get RexExecutor from > RexSimplify. > >> > > <> > >> > > > >> > > Regards! > >> > > Aron Tao > >> > > >> > > >
Re: [DISCUSS] get RexExecutor from RexSimplify in method reduceExpressionsInternal
Hi Danny Thanks for your reply, I think Stamatis Zampetakis's opinion is summative, and here the problem I think is a default RexExecutor is better than null, especially, in this case, cuz `reduceExpressionsInternal` and `reduceExpressions` is in the same path, thought the use of RexExecutor may be different, but it still makes people confusing. IMHO, if "return RexUtil.EXECUTOR" >= "return null", we can do the modify. If you think so, I can open a JIRA and do this minor change. Hope to hear your voice. Regards! Aron Tao JiaTao Tao 于2020年3月17日周二 下午4:02写道： > Hi Stamatis Zampetakis > > I agree with this completely: "The API of RexExecutor says the following > "If an expression cannot be > reduced, writes the original expression..." so we don't break anything by > providing a default one." > > > > Regards! > > Aron Tao > > > Stamatis Zampetakis 于2020年3月16日周一 下午9:52写道： > >> Interestingly, I was looking at this same piece of code not so long ago >> and >> I agree it is a bit confusing. >> >> Looking around the places that we obtain a RexExecutor, most often >> (always?) we observe the following pattern: >> >> RexExecutor executor = >> Util.first(query.getCluster().getPlanner().getExecutor(), >> RexUtil.EXECUTOR); >> >> I think it is always useful to have an executor in the planner thus I am >> tempted to change the API of RelOptPlanner#getExecutor to always return an >> (default) executor if an explicit one is not set. >> >> The API of RexExecutor says the following "If an expression cannot be >> reduced, writes the original expression..." so we don't break anything by >> providing a default one. >> >> What do you think? >> >> Best, >> Stamatis >> >> On Mon, Mar 16, 2020 at 11:11 AM Danny Chan wrote: >> >> > Thanks, the code is a little mess, here is how I understand it: >> > >> > The executor from `final RexExecutor executor >> > = Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)` is >> > mainly used to construct the RexSimplify, in the RexSimplify, the >> > expression that we evaluate is what we can make sure RexUtil.EXECUTOR >> can >> > resolve(if you check the code, it only reduce the literals). >> > >> > But the expressions in the ReduceExpressionsRule may be more complex, >> > somehow we must relay on the engine to plugin their RexExecutor to make >> a >> > constant reduction(some engine use code generation, some use Java >> > reflection). >> > >> > So, in total, the executor in RexSimplify has a fallback is because it’s >> > expression to reduce is simple enough. >> > >> > Best, >> > Danny Chan >> > 在 2020年3月16日 +0800 PM3:57，JiaTao Tao ，写道： >> > > In method reduceExpressionsInternal, we get RexExecutor from cluster, >> it >> > can be null: >> > > <> >> > > >> > > But in the outside(reduceExpressions), `final RexExecutor executor = >> > Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)`, it >> can't >> > be null. >> > > >> > > And reduceExpressions is the only caller of reduceExpressionsInternal, >> > so I think this is an inconsistent behavior. >> > > >> > > IMHO, we should create RexUtil.EXECUTOR if it is null >> > in reduceExpressionsInternal, or just get RexExecutor from RexSimplify. >> > > <> >> > > >> > > Regards! >> > > Aron Tao >> > >> >
Re: TableModify does not keep UPSERT keyword
AFAIK MERGE has different concurrency semantics than what some DBMS call UPSERT. PostgreSQL for example has a guaranteed insert or update semantics whereas MERGE could end with constraint violation errors: https://wiki.postgresql.org/wiki/UPSERT Maybe it's worth adding that to the relational model after all? Am 17.03.2020 um 07:17 schrieb Enrico Olivelli: Il Lun 16 Mar 2020, 23:55 Julian Hyde ha scritto: Change the unparse operation for the dialect so that you generate UPSERT rather than MERGE. IIRC we did this for another dialect - maybe Phoenix. Julian, In my case (HerdDB) I need to see the presence of UPSERT in the RelNode (EnumerableTableModify oŕ LogicalTableModify) I saw the JIRA where you introduced UPSERT for Phoenix I will check deeper, thanks Enrico Julian On Mar 16, 2020, at 1:22 AM, Enrico Olivelli wrote: Il Lun 16 Mar 2020, 09:06 Stamatis Zampetakis ha scritto: Hi Enrico, I have the impression that UPSERT is currently supported only at the parser level [1] so it seems normal that you don't find something relevant in LogicalTableModify etc. Note that the SQL standard equivalent is the MERGE statement [2] but this also seems to be supported only at the parser/validator level [2]. I guess it is not necessary to introduce more things in TableModify since UPSERT seems to be syntactic sugar. I think that most of the work can be done in RelToSqlConverter [4] and possibly the rest of the code can be left intact. I would like to sens a patch that introduces the ability to keep the SqlInsert 'keywords' and pass them to TableModify? Would it be a good approach? The alternative is to introduce a new Operation but it would be a more invasive change and I think it is not worth Enrico Best, Stamatis [1] https://issues.apache.org/jira/browse/CALCITE-492 [2] https://en.wikipedia.org/wiki/Merge_(SQL) [3] https://issues.apache.org/jira/browse/CALCITE-985 [4] https://github.com/apache/calcite/blob/d234626227954eefffe49f42abec65c649ffe3a6/core/src/test/java/org/apache/calcite/test/SqlToRelConverterTest.java#L2395 On Sun, Mar 15, 2020 at 6:53 PM Enrico Olivelli wrote: Hi, I am trying to use UPSERT but it seems to me that in TableModify (or best LogicalTableModify or EnumerableTableModify) there is no way to distinguish an INSERT from an UPSERT. Am I missing something? Regards Enrico
Re: [DISCUSS] get RexExecutor from RexSimplify in method reduceExpressionsInternal
Hi Stamatis Zampetakis I agree with this completely: "The API of RexExecutor says the following "If an expression cannot be reduced, writes the original expression..." so we don't break anything by providing a default one." Regards! Aron Tao Stamatis Zampetakis 于2020年3月16日周一 下午9:52写道： > Interestingly, I was looking at this same piece of code not so long ago and > I agree it is a bit confusing. > > Looking around the places that we obtain a RexExecutor, most often > (always?) we observe the following pattern: > > RexExecutor executor = > Util.first(query.getCluster().getPlanner().getExecutor(), > RexUtil.EXECUTOR); > > I think it is always useful to have an executor in the planner thus I am > tempted to change the API of RelOptPlanner#getExecutor to always return an > (default) executor if an explicit one is not set. > > The API of RexExecutor says the following "If an expression cannot be > reduced, writes the original expression..." so we don't break anything by > providing a default one. > > What do you think? > > Best, > Stamatis > > On Mon, Mar 16, 2020 at 11:11 AM Danny Chan wrote: > > > Thanks, the code is a little mess, here is how I understand it: > > > > The executor from `final RexExecutor executor > > = Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)` is > > mainly used to construct the RexSimplify, in the RexSimplify, the > > expression that we evaluate is what we can make sure RexUtil.EXECUTOR can > > resolve(if you check the code, it only reduce the literals). > > > > But the expressions in the ReduceExpressionsRule may be more complex, > > somehow we must relay on the engine to plugin their RexExecutor to make a > > constant reduction(some engine use code generation, some use Java > > reflection). > > > > So, in total, the executor in RexSimplify has a fallback is because it’s > > expression to reduce is simple enough. > > > > Best, > > Danny Chan > > 在 2020年3月16日 +0800 PM3:57，JiaTao Tao ，写道： > > > In method reduceExpressionsInternal, we get RexExecutor from cluster, > it > > can be null: > > > <> > > > > > > But in the outside(reduceExpressions), `final RexExecutor executor = > > Util.first(cluster.getPlanner().getExecutor(), RexUtil.EXECUTOR)`, it > can't > > be null. > > > > > > And reduceExpressions is the only caller of reduceExpressionsInternal, > > so I think this is an inconsistent behavior. > > > > > > IMHO, we should create RexUtil.EXECUTOR if it is null > > in reduceExpressionsInternal, or just get RexExecutor from RexSimplify. > > > <> > > > > > > Regards! > > > Aron Tao > > >
