Re: Need a Jason output if physical tree

[Vladimir Ozerov]

Hi Liya,

This will not work AFAIK. Consider the query "SELECT a FROM t WHERE b>1".
The top-level operator has only one column:

Root[$0]
  Project[$0]
Filter[$1>1]
  Scan[$0=a, $1=b]

If you invoke RelMdColumnOrigins on Root[$0], you will get [t.a], but miss
[t.b].
To my knowledge, rules are the only way to reliably. constrain columns
returned from the scan.

Regards,
Vladimir.

вт, 22 дек. 2020 г. в 05:14, Fan Liya :

> Hi Bhavya,
>
> Sorry I am not sure if I fully understand your question.
>
> Let me try to answer it according to my understanding:
>
> 1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object, which
> includes a RelOptTable object.
> 2. The table scan also has a RelOptTable object, and all table scans of the
> plan can be found (e.g. through a visitor)
> 3. With the information of 1 and 2, given any output column, we can get to
> know it is derived from which columns from which table scans.
> 4. With the information of 3, given a table scan, we can get to know which
> column is never used in any output columns, and such columns can be pruned.
>
> Best,
> Liya Fan
>
>
> On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal 
> wrote:
>
> > Hi Liya,
> >
> > I had a look at the RelColumnMDOrigins and it is useful in determining
> > which columns are from which table but still I am not sure how can I get
> > the column information for TableScan without the rules. If you have any
> > specific example where we have used this approach will be really helpful
> to
> > me.
> >
> > Thanks and regards
> > Bhavya
> >
> > On Mon, Dec 21, 2020 at 5:53 PM Fan Liya  wrote:
> >
> > > Hi Bhavya,
> > >
> > > IMO, to solve the problem from a global view, the following steps needs
> > to
> > > be taken:
> > >
> > > 1. Generate a physical plan in the original way (without considering
> > column
> > > pruning in the table scan)
> > > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > > utility (the details have been described above)
> > > 3. Post process the plan with one of the following ways:
> > > a) a plan visitor that adjusts other operators in the tree.
> > > b) a light-weight planner (Hep or Volcano with limited rule sets)
> > >
> > > Run the query with the finally generated plan.
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal 
> > > wrote:
> > >
> > > > Hi Fan,
> > > >
> > > > I looked at the class RelColumnMDOrigins and it is giving me the
> origin
> > > of
> > > > the column, but even if I want to take it as a global decision I am
> not
> > > > sure how to proceed. Can you please elaborate on how to achieve this
> ?
> > I
> > > am
> > > > literally stuck as I do not want to use so many rules as in any case
> I
> > > have
> > > > to pass these to the TableScan, even if the user does a select * from
> > > > table, I need to add all those columns to the table scan.
> > > >
> > > > Regards
> > > > Bhavya
> > > >
> > > > --
> > > > Your feedback matters - At Knoldus we aim to be very professional in
> > our
> > > > quality of work, commitment to results, and proactive communication.
> If
> > > > you
> > > > feel otherwise please share your feedback
> > > >  and we would work on it.
> > > >
> > >
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. 
> > +91-9910483067
> > Canada - USA - India - Singapore
> >   >
> >  
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> >  and we would work on it.
> >
>

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

Sorry I am not sure if I fully understand your question.

Let me try to answer it according to my understanding:

1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object, which
includes a RelOptTable object.
2. The table scan also has a RelOptTable object, and all table scans of the
plan can be found (e.g. through a visitor)
3. With the information of 1 and 2, given any output column, we can get to
know it is derived from which columns from which table scans.
4. With the information of 3, given a table scan, we can get to know which
column is never used in any output columns, and such columns can be pruned.

Best,
Liya Fan


On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal  wrote:

> Hi Liya,
>
> I had a look at the RelColumnMDOrigins and it is useful in determining
> which columns are from which table but still I am not sure how can I get
> the column information for TableScan without the rules. If you have any
> specific example where we have used this approach will be really helpful to
> me.
>
> Thanks and regards
> Bhavya
>
> On Mon, Dec 21, 2020 at 5:53 PM Fan Liya  wrote:
>
> > Hi Bhavya,
> >
> > IMO, to solve the problem from a global view, the following steps needs
> to
> > be taken:
> >
> > 1. Generate a physical plan in the original way (without considering
> column
> > pruning in the table scan)
> > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > utility (the details have been described above)
> > 3. Post process the plan with one of the following ways:
> > a) a plan visitor that adjusts other operators in the tree.
> > b) a light-weight planner (Hep or Volcano with limited rule sets)
> >
> > Run the query with the finally generated plan.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Hi Fan,
> > >
> > > I looked at the class RelColumnMDOrigins and it is giving me the origin
> > of
> > > the column, but even if I want to take it as a global decision I am not
> > > sure how to proceed. Can you please elaborate on how to achieve this ?
> I
> > am
> > > literally stuck as I do not want to use so many rules as in any case I
> > have
> > > to pass these to the TableScan, even if the user does a select * from
> > > table, I need to add all those columns to the table scan.
> > >
> > > Regards
> > > Bhavya
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > >  and we would work on it.
> > >
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. 
> +91-9910483067
> Canada - USA - India - Singapore
>  
>  
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
>  and we would work on it.
>

Re: [ANNOUNCE] New Calcite PMC chair: Haisheng Yuan

[Haisheng Yuan]

Thanks everyone.
It is my great honor to be appointed to serve as the community's PMC chair, and 
thanks Stamatis for your hard work and contribution you have done for the 
Calcite community.

Regards,
Haisheng Yuan

On 2020/12/18 09:07:07, Ruben Q L  wrote: 
> Congratulations Haisheng!
> Thanks for your work Stamatis!
> 
> 
> On Fri, Dec 18, 2020 at 8:17 AM Alessandro Solimando <
> alessandro.solima...@gmail.com> wrote:
> 
> > Thanks Stamatis for your hard work and dedication, and congratulations to
> > Haisheng for this appointment!
> >
> > Best regards,
> > Alessandro
> >
> > On Fri, 18 Dec 2020 at 07:31, Xin Wang  wrote:
> >
> > > Congrats Haisheng! Thanks for your work, Stamatis!
> > >
> > >
> > > Fan Liya  于2020年12月18日周五 下午12:08写道：
> > >
> > > > Congratulations, Haisheng!
> > > > Looking forward to your great work in the coming year!
> > > >
> > > > Stamatis, thanks for your great work in the past year!
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > On Fri, Dec 18, 2020 at 11:11 AM Feng Zhu 
> > wrote:
> > > >
> > > > > Thanks for your work and effort, Stamatis!
> > > > > Congratulations, Haisheng!
> > > > >
> > > > > Stamatis Zampetakis  于2020年12月17日周四 下午9:49写道：
> > > > >
> > > > > > Calcite community members,
> > > > > >
> > > > > > I am pleased to announce that we have a new PMC chair and VP as per
> > > our
> > > > > > tradition of rotating the chair once a year. I have resigned, and
> > > > > > Haisheng was duly elected by the PMC and approved unanimously by
> > the
> > > > > Board.
> > > > > >
> > > > > > Please join me in congratulating Haisheng!
> > > > > >
> > > > > > Best,
> > > > > > Stamatis
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > Xin
> > >
> >
> 

Re: New contributor - ostumpf

[Francis Chuang]

Hey Ondrej,

I added you to the contributor role in jira.

Francis

On 21/12/2020 11:31 pm, Ondra Štumpf wrote:

Hi there,



kindly add me as a contributor in Jira to be able to assign https://issues.
apache.org/jira/browse/CALCITE-4443 to myself. The username is "ostumpf".




Thank you




Ondrej Stumpf

[jira] [Created] (CALCITE-4444) Elasticsearch adapter fails for ES 6 if query contains aggregations

[Vova Vysotskyi (Jira)]

Vova Vysotskyi created CALCITE-:
---

 Summary: Elasticsearch adapter fails for ES 6 if query contains 
aggregations
 Key: CALCITE-
 URL: https://issues.apache.org/jira/browse/CALCITE-
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.26.0
Reporter: Vova Vysotskyi


Queries like this one:
{code:sql}
select distinct _MAP['a'] from "elastic"."nested"
{code}

Fails with the following error for the case when using ElasticSearch 6:
{noformat}
Error: Error while executing SQL "select distinct _MAP['a'] from 
"elastic"."nested"": Field a not defined for nested (state=,code=0)
java.sql.SQLException: Error while executing SQL "select distinct _MAP['a'] 
from "elastic"."nested"": Field a not defined for employee
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at 
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:217)
at sqlline.Commands.executeSingleQuery(Commands.java:1054)
at sqlline.Commands.execute(Commands.java:1003)
at sqlline.Commands.sql(Commands.java:967)
at sqlline.SqlLine.dispatch(SqlLine.java:734)
at sqlline.SqlLine.begin(SqlLine.java:541)
at sqlline.SqlLine.start(SqlLine.java:267)
at sqlline.SqlLine.main(SqlLine.java:206)
Caused by: java.lang.IllegalArgumentException: Field a not defined for nested
at 
org.apache.calcite.adapter.elasticsearch.ElasticsearchMapping.missingValueFor(ElasticsearchMapping.java:85)
at 
org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.aggregate(ElasticsearchTable.java:217)
at 
org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.find(ElasticsearchTable.java:119)
at 
org.apache.calcite.adapter.elasticsearch.ElasticsearchTable.access$000(ElasticsearchTable.java:61)
at 
org.apache.calcite.adapter.elasticsearch.ElasticsearchTable$ElasticsearchQueryable.find(ElasticsearchTable.java:366)
at Baz.bind(Unknown Source)
at 
org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:363)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:320)
at 
org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:517)
at 
org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:508)
at 
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
at 
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
at 
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
at 
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:667)
at 
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:577)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 8 more

{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Need a Jason output if physical tree

[Bhavya Aggarwal]

Hi Liya,

I had a look at the RelColumnMDOrigins and it is useful in determining
which columns are from which table but still I am not sure how can I get
the column information for TableScan without the rules. If you have any
specific example where we have used this approach will be really helpful to
me.

Thanks and regards
Bhavya

On Mon, Dec 21, 2020 at 5:53 PM Fan Liya  wrote:

> Hi Bhavya,
>
> IMO, to solve the problem from a global view, the following steps needs to
> be taken:
>
> 1. Generate a physical plan in the original way (without considering column
> pruning in the table scan)
> 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> utility (the details have been described above)
> 3. Post process the plan with one of the following ways:
> a) a plan visitor that adjusts other operators in the tree.
> b) a light-weight planner (Hep or Volcano with limited rule sets)
>
> Run the query with the finally generated plan.
>
> Best,
> Liya Fan
>
>
> On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal 
> wrote:
>
> > Hi Fan,
> >
> > I looked at the class RelColumnMDOrigins and it is giving me the origin
> of
> > the column, but even if I want to take it as a global decision I am not
> > sure how to proceed. Can you please elaborate on how to achieve this ? I
> am
> > literally stuck as I do not want to use so many rules as in any case I
> have
> > to pass these to the TableScan, even if the user does a select * from
> > table, I need to add all those columns to the table scan.
> >
> > Regards
> > Bhavya
> >
> > --
> > Your feedback matters - At Knoldus we aim to be very professional in our
> > quality of work, commitment to results, and proactive communication. If
> > you
> > feel otherwise please share your feedback
> >  and we would work on it.
> >
>


-- 
*Bhavya Aggarwal*
CTO & Partner
Knoldus Inc. 
+91-9910483067
Canada - USA - India - Singapore
 
 

-- 
Your feedback matters - At Knoldus we aim to be very professional in our 
quality of work, commitment to results, and proactive communication. If you 
feel otherwise please share your feedback 
 and we would work on it. 

New contributor - ostumpf

[Ondra Štumpf]

Hi there,



kindly add me as a contributor in Jira to be able to assign https://issues.
apache.org/jira/browse/CALCITE-4443 to myself. The username is "ostumpf".




Thank you




Ondrej Stumpf

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

IMO, to solve the problem from a global view, the following steps needs to
be taken:

1. Generate a physical plan in the original way (without considering column
pruning in the table scan)
2. Modify all the table scans in the plan with the RelColumnMDOrigins
utility (the details have been described above)
3. Post process the plan with one of the following ways:
a) a plan visitor that adjusts other operators in the tree.
b) a light-weight planner (Hep or Volcano with limited rule sets)

Run the query with the finally generated plan.

Best,
Liya Fan


On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal  wrote:

> Hi Fan,
>
> I looked at the class RelColumnMDOrigins and it is giving me the origin of
> the column, but even if I want to take it as a global decision I am not
> sure how to proceed. Can you please elaborate on how to achieve this ? I am
> literally stuck as I do not want to use so many rules as in any case I have
> to pass these to the TableScan, even if the user does a select * from
> table, I need to add all those columns to the table scan.
>
> Regards
> Bhavya
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
>  and we would work on it.
>

[jira] [Created] (CALCITE-4443) Add support for the ILIKE operator to core

[Jira]

Ondřej Štumpf created CALCITE-4443:
--

 Summary: Add support for the ILIKE operator to core
 Key: CALCITE-4443
 URL: https://issues.apache.org/jira/browse/CALCITE-4443
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Ondřej Štumpf


Extend the list of supported relational operators in SqlStdOperatorTable with 
ILIKE. The new ILIKE keyword should be unreserved and behave exactly like LIKE, 
but without case-sensitivity.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)