Re: Question about Interpreter and Corelations
You may modify your sql and operator Collect should not reference correlate
variables.
XING JIN 于2019年11月9日周六 下午12:32写道：
> Currently RelDecorrelator doesn't support a decorrelateRel(Collect),
> that's why decorrelate failed
>
> Zoltan Farkas 于2019年11月9日周六 上午5:46写道：
>
>> Done: https://issues.apache.org/jira/browse/CALCITE-3488 <
>> https://issues.apache.org/jira/browse/CALCITE-3488>
>> let me know if you need any more info.
>>
>> thank you
>>
>> —Z
>>
>> > On Nov 8, 2019, at 2:17 PM, Julian Hyde wrote:
>> >
>> > As I said, I think you should log a bug.
>> >
>> > On Fri, Nov 8, 2019 at 10:23 AM Haisheng Yuan
>> wrote:
>> >>
>> >> I am afraid this query can't be easily decorrelated.
>> >>
>> >> - Haisheng
>> >>
>> >> --
>> >> 发件人：Zoltan Farkas
>> >> 日 期：2019年11月08日 22:46:53
>> >> 收件人：
>> >> 主 题：Re: Question about Interpreter and Corelations
>> >>
>> >> Thanks Julian,
>> >>
>> >> Any idea how could I de-corelate the query?
>> >> RelDecorrelator.decorrelateQuery does not seem to alter the plan.
>> >>
>> >> Will log a jira for the interpreter issue.
>> >>
>> >> thank you
>> >>
>> >> —Z
>> >>
>> >>
>> >>
>> >>> On Nov 7, 2019, at 5:08 PM, Julian Hyde
>> wrote:
>> >>>
>> >>> I don’t recall what the interpreter is currently capable of, but you
>> should log a bug.
>> >>>
>> >>> I wonder whether if you could get the query to work in the
>> interpreter if you decorrelated the query first.
>> >>>
>> >>> Julian
>> >>>
>> >>>> On Nov 7, 2019, at 11:16, Zoltan Farkas
>> wrote:
>> >>>>
>> >>>> for a test query with the following plan:
>> >>>>
>> >>>> LogicalProject(name=[$1], friends=[$4])
>> >>>> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
>> requiredColumns=[{0}])
>> >>>> LogicalTableScan(table=[[characters]])
>> >>>> Collect(field=[EXPR$0])
>> >>>>LogicalProject(name=[$2])
>> >>>> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
>> >>>>LogicalProject(characterId2=[$1])
>> >>>> LogicalFilter(condition=[=($0, $cor0.characterId)])
>> >>>> LogicalTableScan(table=[[friendships]])
>> >>>>LogicalProject(characterId=[$0], name=[$1])
>> >>>> LogicalTableScan(table=[[characters]])
>> >>>>
>> >>>> I get :
>> >>>>
>> >>>> java.lang.nsupportedOperationException
>> >>>> at
>> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
>> >>>> at
>> o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
>> >>>> at ^.translate(^:199)[^]
>> >>>> at ^.translate0(^:684)[^]
>> >>>> at ^.translate(^:199)[^]
>> >>>> at ^.translate(^:194)[^]
>> >>>> ...
>> >>>>
>> >>>> when trying to use the Interpreter.
>> >>>>
>> >>>> Is this a current limitation?
>> >>>>
>> >>>> the query is :
>> >>>>
>> >>>> select name,
>> >>>>ARRAY(select c2.name from friendships f, characters c2
>> >>>> where f.characterId1 = c.characterId and
>> f.characterId2 = c2.characterId) as friends
>> >>>> from characters c
>> >>>>
>> >>>> let me know
>> >>>>
>> >>>> thank you
>> >>>>
>> >>>> —Z
>>
>>
Re: Question about Interpreter and Corelations
Currently RelDecorrelator doesn't support a decorrelateRel(Collect), that's
why decorrelate failed
Zoltan Farkas 于2019年11月9日周六 上午5:46写道：
> Done: https://issues.apache.org/jira/browse/CALCITE-3488 <
> https://issues.apache.org/jira/browse/CALCITE-3488>
> let me know if you need any more info.
>
> thank you
>
> —Z
>
> > On Nov 8, 2019, at 2:17 PM, Julian Hyde wrote:
> >
> > As I said, I think you should log a bug.
> >
> > On Fri, Nov 8, 2019 at 10:23 AM Haisheng Yuan
> wrote:
> >>
> >> I am afraid this query can't be easily decorrelated.
> >>
> >> - Haisheng
> >>
> >> ----------
> >> 发件人：Zoltan Farkas
> >> 日 期：2019年11月08日 22:46:53
> >> 收件人：
> >> 主 题：Re: Question about Interpreter and Corelations
> >>
> >> Thanks Julian,
> >>
> >> Any idea how could I de-corelate the query?
> >> RelDecorrelator.decorrelateQuery does not seem to alter the plan.
> >>
> >> Will log a jira for the interpreter issue.
> >>
> >> thank you
> >>
> >> —Z
> >>
> >>
> >>
> >>> On Nov 7, 2019, at 5:08 PM, Julian Hyde
> wrote:
> >>>
> >>> I don’t recall what the interpreter is currently capable of, but you
> should log a bug.
> >>>
> >>> I wonder whether if you could get the query to work in the interpreter
> if you decorrelated the query first.
> >>>
> >>> Julian
> >>>
> >>>> On Nov 7, 2019, at 11:16, Zoltan Farkas
> wrote:
> >>>>
> >>>> for a test query with the following plan:
> >>>>
> >>>> LogicalProject(name=[$1], friends=[$4])
> >>>> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
> requiredColumns=[{0}])
> >>>> LogicalTableScan(table=[[characters]])
> >>>> Collect(field=[EXPR$0])
> >>>>LogicalProject(name=[$2])
> >>>> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
> >>>>LogicalProject(characterId2=[$1])
> >>>> LogicalFilter(condition=[=($0, $cor0.characterId)])
> >>>> LogicalTableScan(table=[[friendships]])
> >>>>LogicalProject(characterId=[$0], name=[$1])
> >>>> LogicalTableScan(table=[[characters]])
> >>>>
> >>>> I get :
> >>>>
> >>>> java.lang.nsupportedOperationException
> >>>> at
> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
> >>>> at
> o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
> >>>> at ^.translate(^:199)[^]
> >>>> at ^.translate0(^:684)[^]
> >>>> at ^.translate(^:199)[^]
> >>>> at ^.translate(^:194)[^]
> >>>> ...
> >>>>
> >>>> when trying to use the Interpreter.
> >>>>
> >>>> Is this a current limitation?
> >>>>
> >>>> the query is :
> >>>>
> >>>> select name,
> >>>>ARRAY(select c2.name from friendships f, characters c2
> >>>> where f.characterId1 = c.characterId and
> f.characterId2 = c2.characterId) as friends
> >>>> from characters c
> >>>>
> >>>> let me know
> >>>>
> >>>> thank you
> >>>>
> >>>> —Z
>
>
Re: Question about Interpreter and Corelations
Done: https://issues.apache.org/jira/browse/CALCITE-3488
<https://issues.apache.org/jira/browse/CALCITE-3488>
let me know if you need any more info.
thank you
—Z
> On Nov 8, 2019, at 2:17 PM, Julian Hyde wrote:
>
> As I said, I think you should log a bug.
>
> On Fri, Nov 8, 2019 at 10:23 AM Haisheng Yuan wrote:
>>
>> I am afraid this query can't be easily decorrelated.
>>
>> - Haisheng
>>
>> --
>> 发件人：Zoltan Farkas
>> 日 期：2019年11月08日 22:46:53
>> 收件人：
>> 主 题：Re: Question about Interpreter and Corelations
>>
>> Thanks Julian,
>>
>> Any idea how could I de-corelate the query?
>> RelDecorrelator.decorrelateQuery does not seem to alter the plan.
>>
>> Will log a jira for the interpreter issue.
>>
>> thank you
>>
>> —Z
>>
>>
>>
>>> On Nov 7, 2019, at 5:08 PM, Julian Hyde wrote:
>>>
>>> I don’t recall what the interpreter is currently capable of, but you should
>>> log a bug.
>>>
>>> I wonder whether if you could get the query to work in the interpreter if
>>> you decorrelated the query first.
>>>
>>> Julian
>>>
>>>> On Nov 7, 2019, at 11:16, Zoltan Farkas
>>>> wrote:
>>>>
>>>> for a test query with the following plan:
>>>>
>>>> LogicalProject(name=[$1], friends=[$4])
>>>> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
>>>> requiredColumns=[{0}])
>>>> LogicalTableScan(table=[[characters]])
>>>> Collect(field=[EXPR$0])
>>>>LogicalProject(name=[$2])
>>>> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
>>>>LogicalProject(characterId2=[$1])
>>>> LogicalFilter(condition=[=($0, $cor0.characterId)])
>>>> LogicalTableScan(table=[[friendships]])
>>>>LogicalProject(characterId=[$0], name=[$1])
>>>> LogicalTableScan(table=[[characters]])
>>>>
>>>> I get :
>>>>
>>>> java.lang.nsupportedOperationException
>>>> at
>>>> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
>>>> at o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
>>>> at ^.translate(^:199)[^]
>>>> at ^.translate0(^:684)[^]
>>>> at ^.translate(^:199)[^]
>>>> at ^.translate(^:194)[^]
>>>> ...
>>>>
>>>> when trying to use the Interpreter.
>>>>
>>>> Is this a current limitation?
>>>>
>>>> the query is :
>>>>
>>>> select name,
>>>>ARRAY(select c2.name from friendships f, characters c2
>>>> where f.characterId1 = c.characterId and
>>>> f.characterId2 = c2.characterId) as friends
>>>> from characters c
>>>>
>>>> let me know
>>>>
>>>> thank you
>>>>
>>>> —Z
Re: Re: Question about Interpreter and Corelations
As I said, I think you should log a bug.
On Fri, Nov 8, 2019 at 10:23 AM Haisheng Yuan wrote:
>
> I am afraid this query can't be easily decorrelated.
>
> - Haisheng
>
> --
> 发件人：Zoltan Farkas
> 日 期：2019年11月08日 22:46:53
> 收件人：
> 主 题：Re: Question about Interpreter and Corelations
>
> Thanks Julian,
>
> Any idea how could I de-corelate the query?
> RelDecorrelator.decorrelateQuery does not seem to alter the plan.
>
> Will log a jira for the interpreter issue.
>
> thank you
>
> —Z
>
>
>
> > On Nov 7, 2019, at 5:08 PM, Julian Hyde wrote:
> >
> > I don’t recall what the interpreter is currently capable of, but you should
> > log a bug.
> >
> > I wonder whether if you could get the query to work in the interpreter if
> > you decorrelated the query first.
> >
> > Julian
> >
> >> On Nov 7, 2019, at 11:16, Zoltan Farkas
> >> wrote:
> >>
> >> for a test query with the following plan:
> >>
> >> LogicalProject(name=[$1], friends=[$4])
> >> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
> >> requiredColumns=[{0}])
> >> LogicalTableScan(table=[[characters]])
> >> Collect(field=[EXPR$0])
> >> LogicalProject(name=[$2])
> >> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
> >> LogicalProject(characterId2=[$1])
> >> LogicalFilter(condition=[=($0, $cor0.characterId)])
> >>LogicalTableScan(table=[[friendships]])
> >> LogicalProject(characterId=[$0], name=[$1])
> >> LogicalTableScan(table=[[characters]])
> >>
> >> I get :
> >>
> >> java.lang.nsupportedOperationException
> >> at
> >> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
> >> at
> >> o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
> >> at ^.translate(^:199)[^]
> >> at ^.translate0(^:684)[^]
> >> at ^.translate(^:199)[^]
> >> at ^.translate(^:194)[^]
> >> ...
> >>
> >> when trying to use the Interpreter.
> >>
> >> Is this a current limitation?
> >>
> >> the query is :
> >>
> >> select name,
> >> ARRAY(select c2.name from friendships f, characters c2
> >>where f.characterId1 = c.characterId and
> >> f.characterId2 = c2.characterId) as friends
> >> from characters c
> >>
> >> let me know
> >>
> >> thank you
> >>
> >> —Z
Re: Re: Question about Interpreter and Corelations
I am afraid this query can't be easily decorrelated.
- Haisheng
--
发件人：Zoltan Farkas
日 期：2019年11月08日 22:46:53
收件人：
主 题：Re: Question about Interpreter and Corelations
Thanks Julian,
Any idea how could I de-corelate the query?
RelDecorrelator.decorrelateQuery does not seem to alter the plan.
Will log a jira for the interpreter issue.
thank you
—Z
> On Nov 7, 2019, at 5:08 PM, Julian Hyde wrote:
>
> I don’t recall what the interpreter is currently capable of, but you should
> log a bug.
>
> I wonder whether if you could get the query to work in the interpreter if you
> decorrelated the query first.
>
> Julian
>
>> On Nov 7, 2019, at 11:16, Zoltan Farkas wrote:
>>
>> for a test query with the following plan:
>>
>> LogicalProject(name=[$1], friends=[$4])
>> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
>> requiredColumns=[{0}])
>> LogicalTableScan(table=[[characters]])
>> Collect(field=[EXPR$0])
>> LogicalProject(name=[$2])
>> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
>> LogicalProject(characterId2=[$1])
>> LogicalFilter(condition=[=($0, $cor0.characterId)])
>>LogicalTableScan(table=[[friendships]])
>> LogicalProject(characterId=[$0], name=[$1])
>> LogicalTableScan(table=[[characters]])
>>
>> I get :
>>
>> java.lang.nsupportedOperationException
>> at
>> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
>> at o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
>> at ^.translate(^:199)[^]
>> at ^.translate0(^:684)[^]
>> at ^.translate(^:199)[^]
>> at ^.translate(^:194)[^]
>> ...
>>
>> when trying to use the Interpreter.
>>
>> Is this a current limitation?
>>
>> the query is :
>>
>> select name,
>> ARRAY(select c2.name from friendships f, characters c2
>>where f.characterId1 = c.characterId and
>> f.characterId2 = c2.characterId) as friends
>> from characters c
>>
>> let me know
>>
>> thank you
>>
>> —Z
Re: Question about Interpreter and Corelations
Thanks Julian,
Any idea how could I de-corelate the query?
RelDecorrelator.decorrelateQuery does not seem to alter the plan.
Will log a jira for the interpreter issue.
thank you
—Z
> On Nov 7, 2019, at 5:08 PM, Julian Hyde wrote:
>
> I don’t recall what the interpreter is currently capable of, but you should
> log a bug.
>
> I wonder whether if you could get the query to work in the interpreter if you
> decorrelated the query first.
>
> Julian
>
>> On Nov 7, 2019, at 11:16, Zoltan Farkas wrote:
>>
>> for a test query with the following plan:
>>
>> LogicalProject(name=[$1], friends=[$4])
>> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
>> requiredColumns=[{0}])
>> LogicalTableScan(table=[[characters]])
>> Collect(field=[EXPR$0])
>> LogicalProject(name=[$2])
>> LogicalJoin(condition=[=($0, $1)], joinType=[inner])
>> LogicalProject(characterId2=[$1])
>> LogicalFilter(condition=[=($0, $cor0.characterId)])
>>LogicalTableScan(table=[[friendships]])
>> LogicalProject(characterId=[$0], name=[$1])
>> LogicalTableScan(table=[[characters]])
>>
>> I get :
>>
>> java.lang.nsupportedOperationException
>> at
>> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
>> at o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
>> at ^.translate(^:199)[^]
>> at ^.translate0(^:684)[^]
>> at ^.translate(^:199)[^]
>> at ^.translate(^:194)[^]
>> ...
>>
>> when trying to use the Interpreter.
>>
>> Is this a current limitation?
>>
>> the query is :
>>
>> select name,
>> ARRAY(select c2.name from friendships f, characters c2
>>where f.characterId1 = c.characterId and
>> f.characterId2 = c2.characterId) as friends
>> from characters c
>>
>> let me know
>>
>> thank you
>>
>> —Z
Re: Question about Interpreter and Corelations
Hi Zoltan,
+1 on what Julian said.
Interpreter is not 100% accurate in this case since as you probably
realized this part of the code generates and compiles code.
Other than that I confirm that the Interpreter is not able to handle
correlations at the moment.
Best,
Stamatis
On Thu, Nov 7, 2019 at 11:08 PM Julian Hyde wrote:
> I don’t recall what the interpreter is currently capable of, but you
> should log a bug.
>
> I wonder whether if you could get the query to work in the interpreter if
> you decorrelated the query first.
>
> Julian
>
> > On Nov 7, 2019, at 11:16, Zoltan Farkas
> wrote:
> >
> > for a test query with the following plan:
> >
> > LogicalProject(name=[$1], friends=[$4])
> > LogicalCorrelate(correlation=[$cor0], joinType=[inner],
> requiredColumns=[{0}])
> >LogicalTableScan(table=[[characters]])
> >Collect(field=[EXPR$0])
> > LogicalProject(name=[$2])
> >LogicalJoin(condition=[=($0, $1)], joinType=[inner])
> > LogicalProject(characterId2=[$1])
> >LogicalFilter(condition=[=($0, $cor0.characterId)])
> > LogicalTableScan(table=[[friendships]])
> > LogicalProject(characterId=[$0], name=[$1])
> >LogicalTableScan(table=[[characters]])
> >
> > I get :
> >
> > java.lang.nsupportedOperationException
> >at
> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
> >at
> o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
> >at ^.translate(^:199)[^]
> >at ^.translate0(^:684)[^]
> >at ^.translate(^:199)[^]
> >at ^.translate(^:194)[^]
> >...
> >
> > when trying to use the Interpreter.
> >
> > Is this a current limitation?
> >
> > the query is :
> >
> > select name,
> > ARRAY(select c2.name from friendships f, characters c2
> > where f.characterId1 = c.characterId and
> f.characterId2 = c2.characterId) as friends
> > from characters c
> >
> > let me know
> >
> > thank you
> >
> > —Z
>
Re: Question about Interpreter and Corelations
I don’t recall what the interpreter is currently capable of, but you should log
a bug.
I wonder whether if you could get the query to work in the interpreter if you
decorrelated the query first.
Julian
> On Nov 7, 2019, at 11:16, Zoltan Farkas wrote:
>
> for a test query with the following plan:
>
> LogicalProject(name=[$1], friends=[$4])
> LogicalCorrelate(correlation=[$cor0], joinType=[inner],
> requiredColumns=[{0}])
>LogicalTableScan(table=[[characters]])
>Collect(field=[EXPR$0])
> LogicalProject(name=[$2])
>LogicalJoin(condition=[=($0, $1)], joinType=[inner])
> LogicalProject(characterId2=[$1])
>LogicalFilter(condition=[=($0, $cor0.characterId)])
> LogicalTableScan(table=[[friendships]])
> LogicalProject(characterId=[$0], name=[$1])
>LogicalTableScan(table=[[characters]])
>
> I get :
>
> java.lang.nsupportedOperationException
>at
> o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
>at o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
>at ^.translate(^:199)[^]
>at ^.translate0(^:684)[^]
>at ^.translate(^:199)[^]
>at ^.translate(^:194)[^]
>...
>
> when trying to use the Interpreter.
>
> Is this a current limitation?
>
> the query is :
>
> select name,
> ARRAY(select c2.name from friendships f, characters c2
> where f.characterId1 = c.characterId and
> f.characterId2 = c2.characterId) as friends
> from characters c
>
> let me know
>
> thank you
>
> —Z
Question about Interpreter and Corelations
for a test query with the following plan:
LogicalProject(name=[$1], friends=[$4])
LogicalCorrelate(correlation=[$cor0], joinType=[inner], requiredColumns=[{0}])
LogicalTableScan(table=[[characters]])
Collect(field=[EXPR$0])
LogicalProject(name=[$2])
LogicalJoin(condition=[=($0, $1)], joinType=[inner])
LogicalProject(characterId2=[$1])
LogicalFilter(condition=[=($0, $cor0.characterId)])
LogicalTableScan(table=[[friendships]])
LogicalProject(characterId=[$0], name=[$1])
LogicalTableScan(table=[[characters]])
I get :
java.lang.nsupportedOperationException
at
o.a.c.i.JaninoRexCompiler.lambda$compile$0(JaninoRexCompiler.java:94)[calcite-core-1.21.0.jar:1.21.0]
at
o.a.c.a.e.RexToLixTranslator.translate0(RexToLixTranslator.java:714)[^]
at ^.translate(^:199)[^]
at ^.translate0(^:684)[^]
at ^.translate(^:199)[^]
at ^.translate(^:194)[^]
...
when trying to use the Interpreter.
Is this a current limitation?
the query is :
select name,
ARRAY(select c2.name from friendships f, characters c2
where f.characterId1 = c.characterId and
f.characterId2 = c2.characterId) as friends
from characters c
let me know
thank you
—Z
