Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-31 Thread Vladimir Sitnikov
Stamatis>This is a change that most likely will have impact on many projects I don't see how it will impact projects. Really. Are there projects that use up to date Calcite versions? Are they ready for adding a CI job to test with Calcite master branch? It is very disappointing to hear that it

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-31 Thread Stamatis Zampetakis
This is a change that most likely will have impact on many projects so I think it deserves a bit more discussion. Given that 1.22 should be out quite soon and at the same time this period many people are on holidays it would seem better to at least postpone the merge for 1.23. In terms of code, I

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Feng Zhu
--------- > > 发件人:Rui Wang > > 日 期:2019年12月31日 07:09:44 > > 收件人: > > 主 题:Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form > > > > I think the concern of breaking plan comparison has appeared more than > > once. Not only t

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Chunwei Lei
- > 发件人:Rui Wang > 日 期:2019年12月31日 07:09:44 > 收件人: > 主 题:Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form > > I think the concern of breaking plan comparison has appeared more than > once. Not only to this proposal but also to others (e.g. replace &q

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Haisheng Yuan
人:Rui Wang 日 期:2019年12月31日 07:09:44 收件人: 主 题:Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form I think the concern of breaking plan comparison has appeared more than once. Not only to this proposal but also to others (e.g. replace "$" with "_" in names). Fro

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Rui Wang
I think the concern of breaking plan comparison has appeared more than once. Not only to this proposal but also to others (e.g. replace "$" with "_" in names). >From a think of another perspective, the widely used practice of string comparison based plan evaluation also reduces the flexibility of

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Vladimir Sitnikov
The change improves slow tests from 80 min to 60, and the changes are minimal Vladimir

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Julian Hyde
I think this proposed change does more harm than good. I agree with the arguments that Danny and Stamatis have made. We should not do this. > On Dec 30, 2019, at 9:32 AM, Vladimir Sitnikov > wrote: > > Technically speaking, I would love to refrain from using toString for > equals/hashCode,

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Vladimir Sitnikov
Technically speaking, I would love to refrain from using toString for equals/hashCode, however, it looks like a much more invasive change. Yet another idea is to skip normalization when rendering a plan with SqlExplainLevel != DIGEST_ATTRIBUTES. In other words, the normalization is there, it is

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Stamatis Zampetakis
Normalization sounds useful but I would prefer to keep it optional and separate from the RexNode classes (another RexNode visitor?). There are various use cases where people rely on the structure of the plan to do things and I am afraid that if we always apply the normalization we will break many

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Vladimir Sitnikov
Danny>almost all of the plan change are meaningless What do you mean by meaningless? The purpose of the change is to improve planning time, and to improve plan stability. Danny>and the execution graph is very probably relevant with the state/storage, if we breaks them, the state also crashed

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Danny Chan
I agree for some cases, the change is useful. But the plan compatibility is more important for production, because many systems use the plan to generate an execution graph, and the execution graph is very probably relevant with the state/storage, if we breaks them, the state also crashed. We

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
isheng -- 发件人:Vladimir Sitnikov 日 期:2019年12月30日 15:04:25 收件人:Apache Calcite dev list 主 题:Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form Danny>How much cases are there in production ? This example itself seems very marginalized. I’m not against wit

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Vladimir Sitnikov
Danny>How much cases are there in production ? This example itself seems very marginalized. I’m not against with it, I’m suspicious about the value of the feature. It improves JdbcTest#testJoinManyWay 2 times or so. master. JdbcTest#testJoinManyWay: 5.8sec

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Danny Chan
> We do not normalize RexNodes, thus it results in excessive planning time, > especially when the planner is trying to reorder joins. > For instance, it thinks Join(A, B, $0=$1) and Join(A, B, $1=$0) are > different joins, however, they are equivalent. How much cases are there in production ?

Re: Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
come from left relation, if the two come from different relations. - Haisheng -- 发件人:Vladimir Sitnikov 日 期:2019年12月30日 04:33:11 收件人:Apache Calcite dev list 主 题:Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form J

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Vladimir Sitnikov
Just in case, my motivation of comparing by string length first is for the cases like below: =(CAST(PREV(UP.$0, 0)):INTEGER NOT NULL, 100) vs =(100, CAST(PREV(UP.$0, 0)):INTEGER NOT NULL) As for me, the second one is easier to understand, do the expression starts with simpler bits, and the

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Vladimir Sitnikov
Haisheng> variable always left, constant always right for applicable binary operators; Oh, I did not think of making different behavior for literals, variables. What do you think re "$n.field = 42" where $n.field is a dot operator I'm not fond of adding complicated checks there, however, I think

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Vladimir Sitnikov
It turned out "b" (sort operands in computeDigest) is easier to implement. I've filed a PR: https://github.com/apache/calcite/pull/1703 >($0, 2) vs <(2, $0) might be less trivial to implement, but I think it is worth doing at the same time. In any case, lots of expressions will need to be

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
for reversable binary operators. - Haisheng -- 发件人:Enrico Olivelli 日 期:2019年12月30日 03:28:38 收件人: 主 题:Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form Il dom 29 dic 2019, 20:09 Vladimir Sitnikov ha scritto: >

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Enrico Olivelli
Il dom 29 dic 2019, 20:09 Vladimir Sitnikov ha scritto: > Hi, > > We have a 1-year old issue with an idea to sort RexNode operands so they > are consistent. > > For instance, "x=5" and "5=x" have the same semantics, so it would make > sense to stick to a single implementation. > A discussion can

[DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Vladimir Sitnikov
Hi, We have a 1-year old issue with an idea to sort RexNode operands so they are consistent. For instance, "x=5" and "5=x" have the same semantics, so it would make sense to stick to a single implementation. A discussion can be found in https://issues.apache.org/jira/browse/CALCITE-2450 We do