[jira] [Created] (CALCITE-6569) RelToSqlConverter support IGNORE NULLS for window functions

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6569:
---

 Summary: RelToSqlConverter support IGNORE NULLS for window 
functions
 Key: CALCITE-6569
 URL: https://issues.apache.org/jira/browse/CALCITE-6569
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.20.0
Reporter: Jiajun Xie
Assignee: Jiajun Xie


CALCITE-883(https://issues.apache.org/jira/browse/CALCITE-883) Support  IGNORE 
NULLS option for LEAD, LAG, FIRST_VALUE, LAST_VALUE and NTH_VALUE window 
functions.

But RelToSqlConverter not support them, IGNORE NULLS will be lost.

 

For example

 
{code:java}
SELECT LAG("employee_id", 2) IGNORE NULLS OVER (ORDER BY "hire_date") 
FROM "employee"{code}
 

It lost `IGNORE NULLS `
{code:java}
SELECT LAG("employee_id", 2) OVER (ORDER BY "hire_date") 
FROM "employee"{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6318) Add some rules that WeTune discovers and verifies

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6318:
---

 Summary: Add some rules that WeTune discovers and verifies
 Key: CALCITE-6318
 URL: https://issues.apache.org/jira/browse/CALCITE-6318
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie


[WeTune|https://ipads.se.sjtu.edu.cn/_media/publications/wtune_extend.pdf] is a 
rule generator that can automatically discover new query rewrite rules for SQL 
query optimization.

 

The paper was published in 2022, and some optimization rules have recently been 
implemented in Calcite.

e.g. CALCITE-6214 is rule No.2 in Table.7 of the paper.

 

But there are still many rules that have not been implemented,
which are worth discussing.

Here are all the discovered rewrite 
[rules|https://ipads.se.sjtu.edu.cn:1312/opensource/wetune/-/blob/main/wtune_data/rules/rules.test.txt?ref_type=heads].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Sergey Nuyanzin joins Calcite PMC

[Jiajun Xie]

Congratulations, Sergey!

On Wed, 6 Mar 2024 at 09:43, Cancai Cai  wrote:

> Congratulations, Sergey!
>

Reviewing blog repository

[Jiajun Xie]

Hello all,

When I read blogs about Calcite,
I always want to replicate other people's examples on my own.

Constructing examples requires a lot of pre-work,
which can be time-consuming for newcomers.

So I built a repository that combines code and blogs.
Everyone can read blogs and debug code directly without any additional
preparation work.

Now I have the first chapter:
https://github.com/JiajunBernoulli/calcite-notes/tree/cbo
Welcome to comment on this PR:
https://github.com/JiajunBernoulli/calcite-notes/pull/1

[jira] [Created] (CALCITE-6240) Removed ROW_NUMBER if OVER PARTITION BY unique key

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6240:
---

 Summary: Removed ROW_NUMBER if OVER PARTITION BY unique key
 Key: CALCITE-6240
 URL: https://issues.apache.org/jira/browse/CALCITE-6240
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


The SQL Only have ROW_NUMBER function.
{code:java}
SELECT empno, ename FROM ( 
   SELECT ROW_NUMBER() OVER (PARTITION BY empno) rn, empno, ename
   FROM emp
) WHERE rn <= 5{code}
to 
{code:java}
SELECT empno, ename FROM emp{code}
 

If we know every partition count is less than filter count, we can optimize
{code:java}
SELECT deptno, ename FROM ( 
   SELECT ROW_NUMBER() OVER (PARTITION BY deptno) rn, deptno, ename
   FROM emp
) WHERE rn <= N 
-- Statistic let we know every department count is less than N{code}
to
{code:java}
SELECT empno, ename FROM emp {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6214) Remove `DISTINCT` in `COUNT` if field is unique

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6214:
---

 Summary: Remove `DISTINCT` in `COUNT` if field is unique
 Key: CALCITE-6214
 URL: https://issues.apache.org/jira/browse/CALCITE-6214
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


For the sql
{code:java}
select count(distinct x) cnt
from(
   select distinct sal x from emp
) t  {code}
The distinct that in count can be removed.
{code:java}
LogicalAggregate(group=[{}], CNT=[COUNT($0)])
  LogicalAggregate(group=[{0}])
LogicalProject(X=[$5])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}
But `CoreRules#AGGREGATE_REMOVE` not support it, so there are two DISTINCT.
{code:java}
LogicalAggregate(group=[{}], CNT=[COUNT(DISTINCT $0)])
  LogicalAggregate(group=[{0}])
LogicalProject(X=[$5])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6201) Merge Aggregate node if count distinct column has been deduplicated

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6201:
---

 Summary: Merge Aggregate node if count distinct column has been 
deduplicated
 Key: CALCITE-6201
 URL: https://issues.apache.org/jira/browse/CALCITE-6201
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


distinct column is redundant because count has distinct. 
{code:java}
select count(distinct x) cnt
from(
   select distinct sal x from emp
) t {code}
PlanBefore is
{noformat}
LogicalAggregate(group=[{}], CNT=[COUNT(DISTINCT $0)])
  LogicalAggregate(group=[{5}])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]){noformat}
PlanAfter should be 
{noformat}
LogicalAggregate(group=[{}], CNT=[COUNT(DISTINCT $5)])
LogicalTableScan(table=[[CATALOG, SALES, EMP]]){noformat}
But `CoreRules.AGGREGATE_MERGE` not support it, we can improve it.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6200) RelJson throw UnsupportedOperationException for RexDynamicParam

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6200:
---

 Summary: RelJson throw UnsupportedOperationException for 
RexDynamicParam
 Key: CALCITE-6200
 URL: https://issues.apache.org/jira/browse/CALCITE-6200
 Project: Calcite
  Issue Type: Bug
Reporter: Jiajun Xie
Assignee: Jiajun Xie


Dump plan for the sql `select * from emp limit ?`.

RelJson will throw the exception:
{code:java}
unknown rex ?0
java.lang.UnsupportedOperationException: unknown rex ?0
    at org.apache.calcite.rel.externalize.RelJson.toJson(RelJson.java:649)
    at org.apache.calcite.rel.externalize.RelJson.toJson(RelJson.java:455)
    at 
org.apache.calcite.rel.externalize.RelJsonWriter.put(RelJsonWriter.java:102)
    at 
org.apache.calcite.rel.externalize.RelJsonWriter.explain_(RelJsonWriter.java:85)
    at 
org.apache.calcite.rel.externalize.RelJsonWriter.done(RelJsonWriter.java:135)
    at org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:253)
    at org.apache.calcite.plan.RelOptUtil.dumpPlan(RelOptUtil.java:2109) {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6199) Trim unused fields for SNAPSHOT and SAMPLE if table has VIRTUAL column

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6199:
---

 Summary: Trim unused fields for SNAPSHOT and  SAMPLE if table has 
VIRTUAL column
 Key: CALCITE-6199
 URL: https://issues.apache.org/jira/browse/CALCITE-6199
 Project: Calcite
  Issue Type: Improvement
Reporter: Jiajun Xie
Assignee: Jiajun Xie


RelTri not support SNAPSHOT( So do Sample)
 * Here is a SQL example
{code:java}
select D, E from VIRTUALCOLUMNS.VC_T1 
for system_time as of TIMESTAMP '2011-01-02 00:00:00' {code}

 * Before Trimming
{code:java}
LogicalProject(D=[$3], E=[$4])
  LogicalSnapshot(period=[2011-01-02 00:00:00])
    LogicalProject(A=[$0], B=[$1], C=[$2], D=[$3], $f4=[+($0, 1)])
      LogicalTableScan(table=[[CATALOG, VIRTUALCOLUMNS, VC_T1]]){code}

 * After Trimming
{code:java}
LogicalSnapshot(period=[2011-01-02 00:00:00])
  LogicalProject(D=[$3], $f4=[+($0, 1)])
    LogicalTableScan(table=[[CATALOG, VIRTUALCOLUMNS, VC_T1]]){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [SERVEY] Talk at Apache Community Over Code Asia 2024

[Jiajun Xie]

Hi, Benchao

I have been focusing on Differential-Privacy work for the past six months.
I only rewritten SQL by using calcite to protect data security.
If the topic is also appropriate, I will try to share my practical
experience in Differential-Privacy .

BTW,  "Apache CoC Asia 2024" will be held in Hangzhou in July.
We have many contributors in Hangzhou.
This is a great opportunity for us to meet offline.

On Mon, 8 Jan 2024 at 13:05, Benchao Li  wrote:

> Hi all, especially members in Asia,
>
> We (ALC Beijing) is planning Apache Community Over Code 2024. Since
> Apache Calcite is extensively used and contributed in Asia, we had a
> discussion about whether opening a session about Calcite. It depends
> on if we'll have enough talks for this session, so I'm reaching out to
> collect some feedback if you'll be interested in giving a talk if
> there is a dedicated session at Apache CoC Asia 2024. This would be a
> great opportunity for us to meet each other in person.
>
> Please note that this is not a formal CFP (Call For Proposal), I'm
> just conducting a survey, so do not hesitate to reach to me if you
> have any ideas about this. You can reply to this email thread, email
> me (libenc...@apache.org), or even talk to me in WeChat (to add me as
> a friend, you can write to my person email).
>
> Since this is mostly targeted to contributors in Asia, I'm posting the
> corresponding Chinese version below:
>
> 各位成员们大家好，
>
> 我们（ALC Beijing）正在组织 Apache Community Over Code Asia 2024 会议。因为 Apache
> Calcite 在亚洲有非常广泛的使用和贡献，所以我们讨论了是否要为 Calcite
> 单独开一个专场。这取决于我们是否可以有足够数量的议题，所以我现在期望通过这封邮件收集一些反馈。这是一个我们可以线下见面交流的非常好的机会。
>
> 请注意，这封邮件并不是正式的议题征集，我只是在做一个小调查，如果会有任何想法，欢迎与我联系。你可以直接回复这封邮件，也可以单独给我（
> libenc...@apache.org）发邮件，也可以微信联系我（可以给我发邮件，来添加微信好友）。
>
> --
>
> Best,
> Benchao Li
>

[jira] [Created] (CALCITE-6191) Remove join if input is one row literal

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6191:
---

 Summary: Remove join if input is one row literal
 Key: CALCITE-6191
 URL: https://issues.apache.org/jira/browse/CALCITE-6191
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


Here are some Join Rules
{noformat}
at CoreRules.java | grep ' #JOIN_'
   * @see #JOIN_REDUCE_EXPRESSIONS
   * @see #JOIN_SUB_QUERY_TO_CORRELATE */
   * @see #JOIN_SUB_QUERY_TO_CORRELATE */
   * @see #JOIN_TO_SEMI_JOIN */
  /** As {@link #JOIN_COMMUTE} but swaps outer joins as well as inner joins. */
  /** As {@link #JOIN_PROJECT_BOTH_TRANSPOSE} but only the left input is
  /** As {@link #JOIN_PROJECT_BOTH_TRANSPOSE} but only the right input is
  /** As {@link #JOIN_PROJECT_BOTH_TRANSPOSE} but match outer as well as
  /** As {@link #JOIN_PROJECT_LEFT_TRANSPOSE} but match outer as well as
  /** As {@link #JOIN_PROJECT_RIGHT_TRANSPOSE} but match outer as well as
   * @see #JOIN_TO_MULTI_JOIN
{noformat}
I cannot optimize the SQL by using them.
{noformat}
SELECT EMPNO FROM emp, (SELECT 1 as c) temp WHERE EMPNO = temp.c{noformat}
Here is optimized RelNode that I want get.
{code:java}
LogicalProject(EMPNO=[$0])   
  LogicalFilter(condition=[=($0, 1)])     
     LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
But now it is
{noformat}
LogicalProject(EMPNO=[$0])
  LogicalFilter(condition=[=($0, $9)])
    LogicalJoin(condition=[true], joinType=[inner])
      LogicalTableScan(table=[[CATALOG, SALES, EMP]])
      LogicalValues(tuples=[[{ 1 }]]){noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Calcite PMC chair: Benchao Li

[Jiajun Xie]

Congratulations Benchao!
Thanks Stamatis for his great job!

On Fri, 22 Dec 2023 at 05:01, Julian Hyde  wrote:

> Congratulations, Benchao. And thank you - for your work over many
> years making our community kinder and healthier. I know you will be an
> excellent PMC chair.
>
> Thank you, Stamatis, for serving as chair not once but twice.
>
> Julian
>
> On Thu, Dec 21, 2023 at 6:40 AM Enrico Olivelli 
> wrote:
> >
> > Congratulations !
> >
> > Enrico
> >
> > Il giorno gio 21 dic 2023 alle ore 15:07 Tanner Clary
> >  ha scritto:
> > >
> > > Congrats, Benchao :)
> > >
> > > -Tanner
> > >
> > > On Thu, Dec 21, 2023 at 2:33 AM Alessandro Solimando <
> > > alessandro.solima...@gmail.com> wrote:
> > >
> > > > Congratulations Benchao, and thanks again to Stamatis for his great
> job!
> > > >
> > > > On Thu, 21 Dec 2023 at 11:18, Ruben Q L  wrote:
> > > >
> > > > > Congratulations Benchao!!
> > > > >
> > > > >
> > > > > On Thu, Dec 21, 2023 at 10:05 AM Sergey Nuyanzin <
> snuyan...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Congratulations, Benchao!
> > > > > >
> > > > > > On Thu, Dec 21, 2023 at 10:47 AM Francis Chuang <
> > > > > francischu...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Congratulations, Benchao!
> > > > > > >
> > > > > > > On 21/12/2023 8:24 pm, Guangdong Liu wrote:
> > > > > > > > Congratulations!
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Best Regards
> > > > > > > >
> > > > > > > > 
> > > > > > > >
> > > > > > > > Liugddx
> > > > > > > > liug...@gmail.com
> > > > > > > >
> > > > > > > >
> > > > > > > > Stamatis Zampetakis  于2023年12月21日周四
> 17:18写道：
> > > > > > > >
> > > > > > > >> Calcite community members,
> > > > > > > >>
> > > > > > > >> I am pleased to announce that we have a new PMC chair and
> VP as
> > > > per
> > > > > > > >> our tradition of rotating the chair once a year. I have
> resigned,
> > > > > and
> > > > > > > >> Benchao Li was duly elected by the PMC and approved
> unanimously by
> > > > > the
> > > > > > > >> Board.
> > > > > > > >>
> > > > > > > >> Please join me in congratulating Benchao!
> > > > > > > >>
> > > > > > > >> Best regards,
> > > > > > > >> Stamatis
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Sergey
> > > > > >
> > > > >
> > > >
>

Re: [DISCUSS] Using easy-fix label in JIRA

[Jiajun Xie]

Thanks Stamatis!

In fact, the label name is not important to me.
We haven't added such a tag for a long time, so I completely forgot about it

As you said, I will continue to use the newbie[1] label.

[1]
https://issues.apache.org/jira/browse/CALCITE-2901?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20newbie

On Mon, 11 Dec 2023 at 17:15, Stamatis Zampetakis  wrote:

> In our doc page we are already suggesting the use of the "newbie"
> label [1]. If someone has cycles to go over JIRA issues and label them
> accordingly that would be indeed very helpful for newcomers.
>
> A few months ago, I did a quick cross project JIRA search for similar
> labels and it appears that "newbie" is the most popular one; I would
> suggest continuing using that one.
>
> labels = newbie 7K
> labels = "newbie++" 292
> labels = "newbie," 4
> labels = newbiee  11
> labels = newbiew 1
> labels = newbe 7
> labels = noob 111
> labels = easy 158
> labels = easy-fix 233
> labels = easyfix 2.4K
> labels = easy-fix 233
> labels = Newcomer 15
> labels = beginner 1531
> labels = beginner-friendly 24
> labels = beginners 45
>
> Best,
> Stamatis
>
> [1]
> https://github.com/apache/calcite/blob/acc087a8a7c1d301ec713965dae782d14d969064/site/develop/index.md?plain=1#L354
>
> On Sun, Dec 10, 2023 at 9:41 AM LakeShen 
> wrote:
> >
> > I think that it is useful to help newcomers who want to contribute to
> > Calcite to find their first issue.
> >
> > Best,
> > LakeShen
> >
> > Jiajun Xie  于2023年12月10日周日 15:29写道：
> >
> > > Hi, community:
> > >
> > > Recently, more and more newcomers have started to contribute.
> > > Improving documentation and adding unit tests are very easy tasks.
> > > But some JIRA issues are also easy to fix.
> > >
> > > # For current contributors
> > > We can create or tag some JIRA issues that have referential solutions.
> > > - Improve error messages, refer [CALCITE-4265][1].
> > > - Add some functions in some libraries, refer [CALCIT-5826][2].
> > > - Unparse some functions in some SqlDialects, refer [CALCITE-6150][3].
> > > - ...
> > >
> > > # For future contributors
> > > You can filter[4] JIRA issues by an easy-fix label, then submit your
> PR.
> > > I saw that the easy-fix-list[5] of Apache Linkis received many
> > > contributions.
> > > Hope more people join Calcite.
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-4265
> > > [2] https://issues.apache.org/jira/browse/CALCITE-5826
> > > [3] https://issues.apache.org/jira/browse/CALCITE-6150
> > > [4]
> > >
> > >
> https://issues.apache.org/jira/browse/CALCITE-5525?jql=project%20%3D%20CALCITE%20AND%20labels%20%3D%20easy-fix
> > > [5] https://github.com/apache/linkis/issues/1161
> > >
>

[DISCUSS] Using easy-fix label in JIRA

[Jiajun Xie]

Hi, community:

Recently, more and more newcomers have started to contribute.
Improving documentation and adding unit tests are very easy tasks.
But some JIRA issues are also easy to fix.

# For current contributors
We can create or tag some JIRA issues that have referential solutions.
- Improve error messages, refer [CALCITE-4265][1].
- Add some functions in some libraries, refer [CALCIT-5826][2].
- Unparse some functions in some SqlDialects, refer [CALCITE-6150][3].
- ...

# For future contributors
You can filter[4] JIRA issues by an easy-fix label, then submit your PR.
I saw that the easy-fix-list[5] of Apache Linkis received many
contributions.
Hope more people join Calcite.

[1] https://issues.apache.org/jira/browse/CALCITE-4265
[2] https://issues.apache.org/jira/browse/CALCITE-5826
[3] https://issues.apache.org/jira/browse/CALCITE-6150
[4]
https://issues.apache.org/jira/browse/CALCITE-5525?jql=project%20%3D%20CALCITE%20AND%20labels%20%3D%20easy-fix
[5] https://github.com/apache/linkis/issues/1161

Re: [DISCUSS] Towards Calcite 1.36.0

[Jiajun Xie]

Hi, Benchao

Is the main branch still frozen?

I found a new commit[1], but 1.36.0 has not been released.

[1]
https://github.com/apache/calcite/commit/3157408a127de0d5c1d6b440bd2086f96f4f91ba

On Mon, 6 Nov 2023 at 14:59, Benchao Li  wrote:

> The release note PR is now available for review:
> https://github.com/apache/calcite/pull/3505
>
> Benchao Li  于2023年11月6日周一 12:46写道：
> >
> > Hi devs,
> >
> > All the issues targeted for 1.36.0 have been resolved, I'll start the
> > release process now. Therefore main branch is in code freeze until
> > further notice.
> >
> > Benchao Li  于2023年11月3日周五 10:50写道：
> > >
> > > @Mihai @Julian The only left issue is 5884 now, can we merge it and
> > > start the releasing process?
> > >
> > > LakeShen  于2023年11月1日周三 12:13写道：
> > > >
> > > > Hi Itiel,
> > > >
> > > > I have reviewd the https://github.com/apache/calcite/pull/3325,and
> left one
> > > > comments
> > > >
> > > > Benchao Li  于2023年11月1日周三 10:17写道：
> > > >
> > > > > Mihai,
> > > > > I've reviewed 5884, it looks good and I think we can have it in
> 1.36.0
> > > > >
> > > > > Itiel,
> > > > > https://github.com/apache/calcite/pull/3325 has been reviewed and
> even
> > > > > with an approval, I'll try to ping in the PR to try to get it
> merged
> > > > > in 1.36.0
> > > > >
> > > > > Mihai Budiu  于2023年11月1日周三 02:42写道：
> > > > > >
> > > > > > 5884 is mostly documentation fixes.
> > > > > > I expected it would be easy to get it in.
> > > > > > But we can leave it for later.
> > > > > >
> > > > > > Mihai
> > > > > >
> > > > > > 
> > > > > > From: Benchao Li 
> > > > > > Sent: Tuesday, October 31, 2023 6:50 AM
> > > > > > To: dev@calcite.apache.org 
> > > > > > Subject: Re: [DISCUSS] Towards Calcite 1.36.0
> > > > > >
> > > > > > Due to the inactivity of the review for CALCITE-5884 and
> CALCITE-4455,
> > > > > > I think we can move it out of 1.36.0 for now. What do you think?
> > > > > >
> > > > > > Other three issues look promising to be merged soon, then I'll
> start
> > > > > > the release process after they are merged.
> > > > > >
> > > > > > Julian Hyde  于2023年10月31日周二 02:31写道：
> > > > > > >
> > > > > > > I just merged CALCITE-{129,6052}. Five left.
> > > > > > >
> > > > > > > > On Oct 30, 2023, at 7:22 AM, Ran Tao 
> wrote:
> > > > > > > >
> > > > > > > > The expected release time at the end of the month seems to
> be coming
> > > > > soon.
> > > > > > > > I performed an update on the list compiled by Jiajun.
> > > > > > > >
> > > > > > > > #1.Last update (new comments or commits) within 2 days
> (almost
> > > > > ready):
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-129
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-5826
> > > > > > > >
> > > > > > > > #2. Last update exceeded 2 days
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-5990
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-5984
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-6052
> > > > > > > >
> > > > > > > > #3. Last update exceeded 7 days
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-5884
> > > > > > > > https://issues.apache.org/jira/browse/CALCITE-4455
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Ran Tao
> > > > > > > >
> > > > > > > >
> > > > > > > > Jiajun Xie  于2023年10月28日周六
> 13:34写道：
> > > > > > > >
> > > > > > > >> Hi, all:
> > > > > > > >> We're already very close to 1.36.0, all unresolved issues
> has been
> > > > > someone
> > 

Re: How about enable dependabot?

[Jiajun Xie]

Hi, Hongyu.

Your idea is great and you also introduced the steps to use it.

We need more feedback about benefits and risks from calcite users.
# What are the benefits?
- Quickly fix dependency vulnerabilities.
- Balancing the workload of each upgrade(Not 4.0 to 7.x).
- ...

# What are the risks?
- The latest version may be unstable.
- The burden of upgrading Calcite for users has increased.
- ...

For me, the risks are acceptable.
I am willing to help you complete this work.

On Sat, 4 Nov 2023 at 21:04, Hongyu Guo  wrote:

> Hi all,
>
> Recently, I opened 2 PRs about removing an unused library[1] and bumping
> various libraries[2]. I noticed that many dependencies of calcite are
> outdated. To address this issue, I suggest enabling dependabot[3] to
> automatically open "bump dependency" PRs and make calcite healthier.
>
> If we enable dependabot, what should we do?
>
> - Add `dependabot.yml` to `.github/`. It is straightforward, just follow
> the instructions in the documentation[4].
> - Refactor gradle project files: Dependabot's support for gradle is not
> sufficient as it only reads the text of `build.gradle.kts`, `build.gradle`,
> and `settings.gradle.kts` instead of running gradle. Additionally,
> dependabot can NOT read `gradle.properties`, so we need to refactor the
> gradle project files.
> - Ignore some dependencies: Some dependencies cannot be upgraded. For
> example, I attempted to bump javacc from 4.0 to 7.x, but due to
> incompatibility caused by the large version span, I had to give up. Also,
> we cannot upgrade elasticsearch due to licensing restrictions.
>
> What is your opinion on dependabot?
>
> [1]https://github.com/apache/calcite/pull/3502
> [2]https://github.com/apache/calcite/pull/3504
> [3]
>
> https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/about-dependabot-version-updates
> [4]
>
> https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#gradle
>
> Best,
> Hongyu
>

[jira] [Created] (CALCITE-6093) Unstable unit tes in Druid tests

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6093:
---

 Summary: Unstable unit tes in Druid tests
 Key: CALCITE-6093
 URL: https://issues.apache.org/jira/browse/CALCITE-6093
 Project: Calcite
  Issue Type: Improvement
  Components: druid-adapter
Reporter: Jiajun Xie


FAILURE  38.6sec,  235 completed,   1 failed,   6 skipped, 
org.apache.calcite.test.DruidAdapterIT
{code:java}
FAILURE   0.2sec, org.apache.calcite.test.DruidAdapter2IT > 
testInterleaveBetweenAggregateAndGroupOrderByOnMetrics()
285java.lang.AssertionError: 
286Expected: "store_state=CA; brand_name=King; A=21.4632\nstore_state=OR; 
brand_name=Symphony; A=32.176\nstore_state=CA; brand_name=Toretti; 
A=32.2465\nstore_state=WA; brand_name=King; A=34.6104\nstore_state=OR; 
brand_name=Toretti; A=36.3"
287 but: was "store_state=OR; brand_name=ADJ; 
A=83.8764\nstore_state=WA; brand_name=Akron; A=85.8402\nstore_state=OR; 
brand_name=American; A=86.7898\nstore_state=WA; brand_name=ADJ; 
A=97.6488\nstore_state=CA; brand_name=ADJ; A=98.0076" {code}

FAILURE  39.6sec,  453 completed,   2 failed,  12 skipped, Gradle Test Run 
:druid:test
{code:java}
FAILURE 28.7sec, 210 completed, 1 failed, 6 skipped, 
org.apache.calcite.test.DruidAdapter2IT 
308org.apache.calcite.test.DruidAdapterIT > 
testInterleaveBetweenAggregateAndGroupOrderByOnMetrics() failure marker 
331FAILURE 0.3sec, org.apache.calcite.test.DruidAdapterIT > 
testInterleaveBetweenAggregateAndGroupOrderByOnMetrics() 332 
java.lang.AssertionError: 333 Expected: "store_state=CA; brand_name=King; 
A=21.4632\nstore_state=OR; brand_name=Symphony; A=32.176\nstore_state=CA; 
brand_name=Toretti; A=32.2465\nstore_state=WA; brand_name=King; 
A=34.6104\nstore_state=OR; brand_name=Toretti; A=36.3" 334 but: was 
"store_state=OR; brand_name=ADJ; A=83.8764\nstore_state=WA; brand_name=Akron; 
A=85.8402\nstore_state=OR; brand_name=American; A=86.7898\nstore_state=WA; 
brand_name=ADJ; A=97.6488\nstore_state=CA; brand_name=ADJ; A=98.0076"
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6091) Char that in array is truncated in CASE WHEN statement

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-6091:
---

 Summary: Char that in array is truncated in CASE WHEN statement
 Key: CALCITE-6091
 URL: https://issues.apache.org/jira/browse/CALCITE-6091
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.35.0, 1.34.0
Reporter: Jiajun Xie
Assignee: Jiajun Xie






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Hongyu Guo

[Jiajun Xie]

Congrats, Hongyu!

On Fri, 3 Nov 2023 at 04:48, Francis Chuang 
wrote:

> Congrats, Hongyu!
>
> On 3/11/2023 2:09 am, LakeShen wrote:
> > Congratulations
> >
> > Runkang He  于2023年11月2日周四 22:51写道：
> >
> >> Congratulations, Hongyu!
> >>
> >> Best,
> >> Runkang He
> >>
> >> Benchao Li  于2023年11月2日周四 22:27写道：
> >>
> >>> Apache Calcite's Project Management Committee (PMC) has invited Hongyu
> >>> Guo to become a committer, and we are pleased to announce that he has
> >>> accepted.
> >>>
> >>> Hongyu's contributions are around the functions, parser, DDL(CREATE
> >>> TABLE LIKE, GRANT, REVOKE), and simplication. Apart from the code
> >>> contribution, Hongyu is also very active in the mailing list and code
> >>> reviews.
> >>>
> >>> Hongyu, welcome, thank you for your contributions, and we look forward
> >>> to your further interactions with the community! If you wish, please
> >>> feel free to tell us more about yourself and what you are working on.
> >>>
> >>> As your first commit, please add yourself to the contributors list [1]
> >>> and the community page will re-generate [2].
> >>>
> >>> Benchao Li (on behalf of the Apache Calcite PMC)
> >>>
> >>> [1]
> >>>
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> >>> [2] https://calcite.apache.org/community/#project-members
> >>>
> >>> --
> >>>
> >>> Best,
> >>> Benchao Li
> >>>
> >>
> >
>

Re: [ANNOUNCE] New committer: Runkang He

[Jiajun Xie]

Congrats, Runkang!

On Fri, 3 Nov 2023 at 04:46, Francis Chuang 
wrote:

> Congrats, Runkang!
>
> On 3/11/2023 3:54 am, Tanner Clary wrote:
> > Congratulations, Runkang!
> >
> > Best,
> > Tanner
> >
> > On Thu, Nov 2, 2023 at 9:11 AM Ran Tao  wrote:
> >
> >> Congrats, Runkang!
> >>
> >> Best Regards,
> >> Ran Tao
> >>
> >>
> >> Benchao Li  于2023年11月2日周四 22:27写道：
> >>
> >>> Apache Calcite's Project Management Committee (PMC) has invited
> >>> Runkang He to become a committer, and we are pleased to announce that
> >>> they have accepted.
> >>>
> >>> Runkang has added many new functions to Hive and Spark libraries, and
> >>> various improvements around simplification and sub-query. Apart from
> >>> code contributions, Runkang also helps a lot with code reviews.
> >>>
> >>> Runkang, welcome, thank you for your contributions, and we look
> >>> forward to your further interactions with the community! If you wish,
> >>> please feel free to tell us more about yourself and what you are
> >>> working on.
> >>>
> >>> As your first commit, please add yourself to the contributors list [1]
> >>> and the community page will re-generate [2].
> >>>
> >>> Benchao Li (on behalf of the Apache Calcite PMC)
> >>>
> >>> [1]
> >>>
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> >>> [2] https://calcite.apache.org/community/#project-members
> >>>
> >>> --
> >>>
> >>> Best,
> >>> Benchao Li
> >>>
> >>
> >
>

Re: [ANNOUNCE] New committer: Lei Shen

[Jiajun Xie]

Congratulations, Lei!

On Wed, 1 Nov 2023 at 23:23, Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Congratulations and welcome!
>
> On Wed, Nov 1, 2023, 12:58 Ran Tao  wrote:
>
> > Congrats, Lei!
> >
> > Best Regards,
> > Ran Tao
> >
> >
> > Runkang He  于2023年11月1日周三 17:26写道：
> >
> > > Congratulations, Lei!
> > >
> > > Best,
> > > Runkang He
> > >
> > > Ruben Q L  于2023年11月1日周三 16:49写道：
> > >
> > > > Congratulations!!
> > > >
> > > >
> > > > On Wed, Nov 1, 2023 at 12:09 AM Zhengqiang Duan <
> > > duanzhengqi...@apache.org
> > > > >
> > > > wrote:
> > > >
> > > > > Congratulations, Lei.
> > > > >
> > > > > Best regards,
> > > > > Zhengqiang
> > > > >
> > > > >
> > > > > Benchao Li  于2023年11月1日周三 07:50写道：
> > > > >
> > > > > > Congrats, Lei! Well deserved!
> > > > > >
> > > > > > Tanner Clary  于2023年11月1日周三
> > 05:12写道：
> > > > > > >
> > > > > > > Congratulations, Lei!
> > > > > > >
> > > > > > > Best,
> > > > > > > Tanner
> > > > > > >
> > > > > > > On Tue, Oct 31, 2023 at 1:52 PM Francis Chuang <
> > > > > francischu...@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Congrats, Lei!
> > > > > > > >
> > > > > > > > On 1/11/2023 4:34 am, Stamatis Zampetakis wrote:
> > > > > > > > > Apache Calcite's Project Management Committee (PMC) has
> > invited
> > > > Lei
> > > > > > Shen
> > > > > > > > to
> > > > > > > > > become a committer, and we are pleased to announce that
> they
> > > have
> > > > > > > > accepted.
> > > > > > > > >
> > > > > > > > > Lei has added many new optimizer rules to the project
> > enhancing
> > > > the
> > > > > > > > > query plans for Calcite and downstream projects and pushed
> > > > various
> > > > > > > > > enhancements around the TABLESAMPLE clause. Apart from code
> > > > > > > > > contributions, Lei is very active in the dev list
> > participating
> > > > in
> > > > > > > > > discussions and helping a lot with reviews.
> > > > > > > > >
> > > > > > > > > Lei, welcome, thank you for your contributions, and we look
> > > > forward
> > > > > > to
> > > > > > > > your
> > > > > > > > > further interactions with the community! If you wish,
> please
> > > feel
> > > > > > free
> > > > > > > > to tell
> > > > > > > > > us more about yourself and what you are working on.
> > > > > > > > >
> > > > > > > > > As your first commit, please add yourself to the
> contributors
> > > > list
> > > > > > [1]
> > > > > > > > > and the community page will re-generate [2].
> > > > > > > > >
> > > > > > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > >
> > > > > >
> > > >
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > > > > > > > [2] https://calcite.apache.org/community/#project-members
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best,
> > > > > > Benchao Li
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.36.0

[Jiajun Xie]

Hi, all:
We're already very close to 1.36.0, all unresolved issues has been someone
reviewed.
>
https://issues.apache.org/jira/browse/CALCITE-6047?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Open%20AND%20fixVersion%20%3D%201.36.0


- Some issues are currently being reviewed.
That's great! Thank you all for your efforts.
- Some issues have not been updated for a long time.
If anyone is willing to continue the review, I believe they should be
resolved soon.


#1.Last update (new comments or commits) within 2 days:
https://issues.apache.org/jira/browse/CALCITE-129
https://issues.apache.org/jira/browse/CALCITE-5990

#2. Last update exceeded 2 days
https://issues.apache.org/jira/browse/CALCITE-5984
https://issues.apache.org/jira/browse/CALCITE-6052

#3. Last update exceeded 7 days
https://issues.apache.org/jira/browse/CALCITE-5884
https://issues.apache.org/jira/browse/CALCITE-6037

#4. Last update exceeded 14 days
https://issues.apache.org/jira/browse/CALCITE-4455
https://issues.apache.org/jira/browse/CALCITE-5826

On Wed, 25 Oct 2023 at 11:40, Benchao Li  wrote:

> Thanks Mihai for the quick response, I'll take a look at the PR.
>
> Mihai Budiu  于2023年10月25日周三 05:02写道：
> >
> > For the blocker issues, I have filed two PRs which should collectively
> fix them.
> >
> > https://github.com/apache/calcite/pull/3481 fixes
> https://issues.apache.org/jira/browse/CALCITE-5990, but does not include
> tests, because the test fixture needs to be fixed as well for the tests to
> pass. There are tests in the next PR:
> > https://github.com/apache/calcite/pull/3471 fixes
> https://issues.apache.org/jira/browse/CALCITE-5921, but should be amended
> to reenable the disabled tests if we merge 3481 first.
> >
> > Mihai
> >
> > 
> > From: Benchao Li 
> > Sent: Tuesday, October 24, 2023 5:46 AM
> > To: dev@calcite.apache.org 
> > Subject: Re: [DISCUSS] Towards Calcite 1.36.0
> >
> > A quick update on the current status for issues targeted 1.36.0
> >
> > We need more reviewers for #2 and #3 since the end of October is
> approaching.
> > Please correct me if I'm mistaken, especially for issues in #1.
> >
> > #1. Issues already have reviewers, and seems promising to be merged
> shortly
> >
> > https://issues.apache.org/jira/browse/CALCITE-5884
> > https://issues.apache.org/jira/browse/CALCITE-5918
> > https://issues.apache.org/jira/browse/CALCITE-5860
> > https://issues.apache.org/jira/browse/CALCITE-6011
> > https://issues.apache.org/jira/browse/CALCITE-5949
> > https://issues.apache.org/jira/browse/CALCITE-6041
> > https://issues.apache.org/jira/browse/CALCITE-6038
> >
> > #2. Issues has been reviewed, and needs another reviewer
> >
> > https://issues.apache.org/jira/browse/CALCITE-5984
> > https://issues.apache.org/jira/browse/CALCITE-5825
> > https://issues.apache.org/jira/browse/CALCITE-5826
> > https://issues.apache.org/jira/browse/CALCITE-6037
> > https://issues.apache.org/jira/browse/CALCITE-4455
> > https://issues.apache.org/jira/browse/CALCITE-6042
> >
> > #3. PR is ready, no reviewer yet
> > https://issues.apache.org/jira/browse/CALCITE-6052
> > https://issues.apache.org/jira/browse/CALCITE-129
> >
> > #4. blocker issues status
> > https://issues.apache.org/jira/browse/CALCITE-5990 is marked blocked
> > by https://issues.apache.org/jira/browse/CALCITE-5921 now.
> > Both issues have a PR, but proposed to have a more thorough solution.
> > I'll keep an eye on them closely.
> >
> >
> > Ran Tao  于2023年10月18日周三 18:37写道：
> > >
> > > Hi, community.
> > >
> > > Recently I reported an issue[1] which revealed that calcite's MAP
> SubQuery
> > > doesn't work,
> > > considering this is a feature regression or bug, maybe we can fix it in
> > > 1.36.
> > > The PR is ready. If someone has time, pls help to review it. many
> thanks.
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-6041
> > >
> > > Best Regards,
> > > Ran Tao
> > >
> > >
> > > Benchao Li  于2023年10月8日周日 11:02写道：
> > >
> > > > It's been a bit more than 2 months since our last release [1] and
> > > > there are currently 90+ new commits in main branch.
> > > >
> > > > As usual, according to our Jira dashboard [2] and Github [3], there
> > > > are many pending issues that could / should be part of the release.
> > > > I'd propose to make a collective effort to try to clean up our 1.36
> > > > backlog and merge the PRs which are in a good state. I'd propose to
> > > > aim for **end of October** to release 1.36.0, this will give us about
> > > > 20 days to clean up pending PRs for next version. What do you think?
> > > >
> > > > According to [4], the following release managers would be:
> > > > - 1.36.0 Benchao Li
> > > > - 1.37.0 Sergey Nuyanzin
> > > > - 1.38.0 Julian Hyde
> > > >
> > > > So I'm supposed to be the release manager for 1.36.0
> > > >
> > > > And contributors, if you have any work that is in good shape and want
> > > > to be included in 1.36.0, please mark the fixVersion to 1.36.0, this
> > > > will inform the re

Re: [ANNOUNCE] New committer: Ran Tao

[Jiajun Xie]

Congrats, Ran Tao!

On Sat, 28 Oct 2023 at 10:49, Francis Chuang 
wrote:

> Congrats!
>
> On 28/10/2023 11:32 am, Benchao Li wrote:
> > Congrats, Ran!
> >
> > Runkang He  于2023年10月28日周六 07:34写道：
> >>
> >> Congrats Ran!
> >>
> >> Best,
> >> Runkang He
> >>
> >> Alessandro Solimando  于2023年10月28日周六
> >> 00:08写道：
> >>
> >>> Congrats Ran Tao and welcome!
> >>>
> >>> On Fri, 27 Oct 2023 at 17:42, Ruben Q L  wrote:
> >>>
>  Congrats!
> 
> 
>  On Fri, Oct 27, 2023 at 4:40 PM Tanner Clary   .invalid>
>  wrote:
> 
> > Congrats Ran!! Have learned a lot from your pull requests!
> >
> > Best,
> > Tanner
> >
> > On Fri, Oct 27, 2023 at 8:32 AM Stamatis Zampetakis <
> zabe...@gmail.com
> 
> > wrote:
> >
> >> Apache Calcite's Project Management Committee (PMC) has invited Ran
> >>> Tao
> > to
> >> become a committer, and we are pleased to announce that he has
>  accepted.
> >>
> >> Ran has implemented numerous improvements and fixes in the past few
> >> months. He has added new SQL functions to the Spark library,
> >>> addressed
> >> various issues related to casts and type inference, enhanced
> >> documentation, and made various changes with the aim of improving
> >>> code
> >> quality
> >>
> >> Ran, welcome, thank you for your contributions, and we look forward
> >>> to
> > your
> >> further interactions with the community! If you wish, please feel
> >>> free
>  to
> >> tell
> >> us more about yourself and what you are working on.
> >>
> >> As your first commit, please add yourself to the contributors list
> >>> [1]
> >> and the community page will re-generate [2].
> >>
> >> Stamatis (on behalf of the Apache Calcite PMC)
> >>
> >> [1]
> >>
> 
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> >> [2] https://calcite.apache.org/community/#project-members
> >>
> >
> 
> >>>
> >
> >
> >
>

Re: [ANNOUNCE] New committer: Mihai Budiu

[Jiajun Xie]

Congratulations Mihai!
Well deserved!

On Thu, 26 Oct 2023 at 20:08, Ran Tao  wrote:

> Congratulations, Mihai !
> and thanks for the review in the issue and PR !
>
> Best Regards,
> Ran Tao
>
> Stamatis Zampetakis  于2023年10月26日周四 16:57写道：
>
> > Apache Calcite's Project Management Committee (PMC) has invited Mihai
> > Budiu to become a committer, and we are pleased to announce that he
> > has accepted.
> >
> > Mihai has addressed numerous compile and runtime issues in SQL
> > functions and has made several significant improvements to the test
> > infrastructure to enhance code coverage, quality, and stability. In
> > addition to code contributions, Mihai is highly active on the mailing
> > list, where he answers questions, assists others, and shares ideas for
> > future improvements. Mihai has also been writing blogs and giving
> > talks about Calcite at various events and conferences, showcasing
> > Calcite in both simple and more advanced use cases, which can greatly
> > contribute to the growth of the community.
> >
> > Mihai, welcome, thank you for your contributions, and we look forward
> > to your further interactions with the community! If you wish, please
> > feel free to tell us more about yourself and what you are working on.
> >
> > As your first commit, please add yourself to the contributors list [1]
> > and the community page will re-generate [2].
> >
> > Stamatis (on behalf of the Apache Calcite PMC)
> >
> > [1]
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > [2] https://calcite.apache.org/community/#project-members
> >
>

Re: RelNode => Spark Logical Plan converter

[Jiajun Xie]

If your SQL is simple, you can refer to Kylin[1].

I like RelNode To Spark SQL because I had trouble with "RelNode=>Spark
Logical Plan":
1. Coupling Spark versions can result in difficulties during upgrades.
I have to spend a lot of time from Spark 2.3 to 3.0, and Spark 3.0
to 3.2.
2. Changes in metadata can cause problems
 After converting to RelNode, users may change metadata(Table or
Column).
Then you use old RelNode to generate Spark Logical Plan.
The Logical Plan may be error because Spark reads new metadata.
3. SQL functions require special handling
When you write SQL, you can use any function because it is a
string.
But when you use Spark Logical Plan, you should use the correct
spark expression.
More issues...
Coral[2] is a project that uses RelNode to SQL.

[1]
https://github.com/apache/kylin/tree/0e0d8a125ea81f735e66e38f32ef722622d77dc0/kylin-spark-project/kylin-spark-query/src/main/scala/org/apache/kylin/query/runtime/plans
[2] https://github.com/linkedin/coral

On Fri, 13 Oct 2023 at 10:27, LakeShen  wrote:

> Maybe you could see something in Hive,as far as I know, Hive uses Calcite,
> but eventually the plan will be transferred to Hive's own execution
> plan,Spark and Hive themselves are also very compatible.
> Now I don't find any project to convert Calcite Logical Plan to Spark
> Logical Plan directly,but you could do this by yourself.However, there are
> many differences between Calcite and Spark that need to be considered, such
> as differences in functions, operators of relational algebra trees,
> differences in data types, etc.
> I think that Calcite => Substrait => Spark is better way to do this,you
> could leverage the power of the Calcite community and the Substrait
> community.
>
> Best,
> LakeShen
>
>
> Julian Hyde  于2023年10月13日周五 08:20写道：
>
> > There aren’t any plans, but I like the idea of transforming between the
> > intermediate languages (Calcite algebra, Spark algebra, SQL of any
> dialect,
> > Substrate).
> >
> > In particular I was thinking of translating Calcite algebra —> Substrate,
> > and Substrate —> any supported SQL dialect. The former would be done in
> > Calcite, the latter could be done in a sister project of Substrate.
> >
> > Julian
> >
> >
> > > On Oct 12, 2023, at 9:07 AM, Guillaume Masse <
> > masse.guilla...@narrative.io.INVALID> wrote:
> > >
> > > Hi All,
> > >
> > > We use Apache Calcite to transform SQL then we want to run the logical
> > plan
> > > on our spark cluster. Currently we use RelToSqlConverter and execute
> the
> > > result with Spark SQL. I was wondering if we could just execute from a
> > > Spark logical plan (
> > >
> >
> https://github.com/apache/spark/blob/b0576fff9b72880cd81a9d22c044dec329bc67d0/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L211-L213
> > > ).
> > >
> > > Are there any project that goes from Calcite Logical Plan to Spark
> > Logical
> > > Plan directly?
> > >
> > > I know there is one possibility: Calcite => Substrait => Spark
> > > Substrait => Calcite via substrate-java
> > > 
> > > Substrait => Spark via gluten
> > > <
> >
> https://github.com/oap-project/gluten/tree/main/substrait/substrait-spark>
> > >
> > >
> > > --
> > > Guillaume Massé
> > > [Gee-OHM]
> > > (马赛卫)
> >
> >
>

Re: SQL Parsing

[Jiajun Xie]

Hi Thomas,

Cast map syntax is not supported in the current release version.
But we will support it in the 1.36.0 version.

FYI: https://issues.apache.org/jira/browse/CALCITE-5570

On Sun, 1 Oct 2023 at 22:38, Thomas Wang  wrote:

> Hi Mihai,
>
> Could you point me to more information w.r.t modifying the parser? Also, I
> noticed for the ARRAY type, I can do CAST(NULL AS BIGINT ARRAY), but the
> grammar doesn't seem to have MAP supported, does it? Did I miss something?
> Thanks.
>
> Thomas
>
> On Sat, Sep 30, 2023 at 7:59 PM Mihai Budiu  wrote:
>
> > Calcite is open source, so you can certainly modify the parser. The
> > architecture of the parser had been designed to be flexible and even
> > calcite has several parsers, the strandard one, server, and Babel. Let me
> > know if you need help figuring out how to roll your own.
> >
> > Mihai
> > 
> > From: Thomas Wang 
> > Sent: Saturday, September 30, 2023 7:42:19 PM
> > To: dev@calcite.apache.org 
> > Subject: Re: SQL Parsing
> >
> > Thanks Mihai, "BIGINT ARRAY" seems to work! Just curious if I can have
> > flexibility to change the behavior of the parse to parse "ARRAY"
> > instead?
> >
> > Thomas
> >
> > On Sat, Sep 30, 2023 at 6:32 PM Mihai Budiu  wrote:
> >
> > > The syntax is "bigint array"
> > > https://calcite.apache.org/docs/reference.html
> > > 
> > > From: Thomas Wang 
> > > Sent: Saturday, September 30, 2023 6:27:46 PM
> > > To: dev@calcite.apache.org 
> > > Subject: SQL Parsing
> > >
> > > Hi Apache Calcite Community,
> > >
> > > I'm new to Apache Calcite and trying to use it for SQL parsing and SQL
> > > rewriting.
> > >
> > > I tried to parse a couple of SELECT statements and it works pretty
> well.
> > > However, when I tried to parse a SELECT statement that contains a cast
> to
> > > array like below, it complains it cannot recognize ARRAY.
> > >
> > > SELECT CAST(NULL AS ARRAY) FROM schema.t1
> > >
> > > However, casting to BIGINT seems ok. The following SELECT is ok.
> > >
> > > SELECT CAST(NULL AS BIGINT) FROM schema.t1
> > >
> > > Is there any configuration I need to enable to enable/support parsing
> > > ARRAY? Thanks.
> > >
> > > Thomas
> > >
> >
>

[jira] [Created] (CALCITE-5944) RelMdRowCount can return more accurate rowCount

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5944:
---

 Summary: RelMdRowCount can return more accurate rowCount
 Key: CALCITE-5944
 URL: https://issues.apache.org/jira/browse/CALCITE-5944
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5943) RelMdSelectivity can return more precise selectivity for Sample

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5943:
---

 Summary: RelMdSelectivity can return more precise selectivity for 
Sample
 Key: CALCITE-5943
 URL: https://issues.apache.org/jira/browse/CALCITE-5943
 Project: Calcite
  Issue Type: Bug
Reporter: Jiajun Xie
Assignee: Jiajun Xie






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Community over Code (ApacheCon) East Asia

[Jiajun Xie]

Hi, all

Last year, I shared
MULTI-ENGINE VIRTUAL COLUMN TECHNOLOGY BASED ON APACHE CALCITE
at ApacheCon Asia 2022.

Next week(8.18), I will share
BEST PRACTICES FOR MULTI-ENGINE METRICS MANAGEMENT BASED ON APACHE CALCITE
[1]
at Community over Code 2023.

Welcome to communicate with me.

[1] https://apachecon.com/acasia2023/sessions/olap-1121.html

On Tue, 1 Aug 2023 at 21:08, Benchao Li  wrote:

> Thanks Julian for the nice suggestion!
>
> I've talked with the organisers about the stickers, they said that due to
> the limited funds, they might not be able to provide such a request. I'll
> try to find some local sticker shops to see if I can make it myself.
>
> About the BoF session, I'll consider it if there is a chance.
>
> Julian Hyde  于2023年8月1日周二 04:53写道：
>
> > Great news. Thank you for submitting a talk!
> >
> > I believe there’s a way to order Calcite laptop stickers before the
> event.
> > You can leave the stickers at the ComDev booth so that anyone attending
> can
> > have a sticker for their laptop. I’ve forgotten the details but it’s
> worth
> > starting the process a few weeks before the event.
> >
> > You could also consider hosting a Calcite BoF (birds of a feather)
> session.
> >
> > Julian
> >
> >
> > > On Jul 31, 2023, at 12:19 AM, Benchao Li  wrote:
> > >
> > > I'll attend, and give a talk[1] in the keynote track about my
> experiences
> > > contributing to Apache projects (Calcite and Flink).
> > >
> > > Hope to see you guys there!
> > >
> > > [1] https://apachecon.com/acasia2023/sessions/keynote-1234.html
> > >
> > > Julian Hyde  于2023年6月12日周一 05:15写道：
> > >
> > >> Community over Code (ApacheCon) East Asia is happening in August, and
> > >> the call for papers closes in LESS THAN A WEEK [1].
> > >>
> > >> I'm proud of how many Calcite community members we have in China and
> > >> East Asia, and I appreciate the constant stream of excellent and
> > >> innovative contributions.
> > >>
> > >> If you are one of those people, I encourage you to attend the
> > >> conference, or, better, submit a talk proposal. This is a great
> > >> opportunity to bring new people into the Calcite community, to
> > >> convince projects and companies based in China that Calcite is a good
> > >> core technology, or just to get to know each other better.
> > >>
> > >> I have attended several ApacheCon conferences in the US and Canada,
> > >> and they have helped me gain an understanding of what Apache does, and
> > >> make some lasting friendships with people in other projects.
> > >>
> > >> If you have an interesting talk about Calcite, please submit it!
> > >>
> > >> I believe that there is travel assistance available for those who need
> > >> financial help getting to the event [2].
> > >>
> > >> Julian
> > >>
> > >> [1] https://apachecon.com/acasia2023/cfp.html
> > >> [2] https://tac.apache.org/
> > >>
> > >
> > >
> > > --
> > >
> > > Best,
> > > Benchao Li
> >
> >
>
> --
>
> Best,
> Benchao Li
>

Re: [ANNOUNCE] New committer: Dan Zou

[Jiajun Xie]

Congratulations, Dan!

On Tue, 4 Jul 2023 at 19:48, Runkang He  wrote:

> Congratulations, Dan!
>
> Best,
> Runkang He
>
> Jacky Lau  于2023年7月4日周二 18:56写道：
>
> > Congratulations, Dan!
> >
> > Best Jacky!
> >
> > Benchao Li  于2023年7月4日周二 17:55写道：
> >
> > > Congratulations, Dan!
> > >
> > > Zhe Hu  于2023年7月4日周二 17:26写道：
> > >
> > > > Congrats! Dan.
> > > >  Replied Message 
> > > > | From | Francis Chuang |
> > > > | Date | 07/04/2023 15:18 |
> > > > | To | dev |
> > > > | Subject | Re: [ANNOUNCE] New committer: Dan Zou |
> > > > Congrats, Dan!
> > > >
> > > > On 4/07/2023 5:16 pm, Stamatis Zampetakis wrote:
> > > > > Apache Calcite's Project Management Committee (PMC) has invited Dan
> > Zou
> > > > to
> > > > > become a committer, and we are pleased to announce that they have
> > > > accepted.
> > > > >
> > > > > Dan has been doing some great work for the project over the past
> few
> > > > > months. They implemented and enabled multiple new SQL functions for
> > > > > BigQuery and MSSQL dialects, fixed some optimization rules, and
> > > > > improved documentation and test code.
> > > > >
> > > > > Dan, welcome, thank you for your contributions, and we look forward
> > to
> > > > your
> > > > > further interactions with the community! If you wish, please feel
> > free
> > > > to tell
> > > > > us more about yourself and what you are working on.
> > > > >
> > > > > As your first commit, please add yourself to the contributors list
> > [1]
> > > > > and the community page will re-generate [2].
> > > > >
> > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > >
> > > > > [1]
> > > >
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > > > [2] https://calcite.apache.org/community/#project-members
> > > >
> > >
> > >
> > > --
> > >
> > > Best,
> > > Benchao Li
> > >
> >
>

Re: [ANNOUNCE] New committer: TJ Banghart

[Jiajun Xie]

Congratulations, TJ!

On Wed, 5 Jul 2023 at 01:03, Tanner Clary 
wrote:

> Congrats and welcome, TJ!!
> On Tue, Jul 4, 2023 at 8:26 AM Runkang He  wrote:
>
> > Congratulations, TJ!
> >
> > Best, Runkang He
> >
> > xiong duan  于2023年7月4日周二 19:39写道：
> >
> > > Congratulations, TJ! Welcome!
> > >
> > > Jacky Lau  于2023年7月4日周二 18:56写道：
> > >
> > > > Congratulations, TJ!
> > > >
> > > > Best Jacky!
> > > >
> > > > Benchao Li  于2023年7月4日周二 17:55写道：
> > > >
> > > > > Congratulations, TJ!
> > > > >
> > > > > Zhe Hu  于2023年7月4日周二 17:26写道：
> > > > >
> > > > > > Congrats! TJ.
> > > > > >  Replied Message 
> > > > > > | From | Francis Chuang |
> > > > > > | Date | 07/04/2023 15:18 |
> > > > > > | To | dev |
> > > > > > | Subject | Re: [ANNOUNCE] New committer: TJ Banghart |
> > > > > > Congrats, TJ!
> > > > > >
> > > > > > On 4/07/2023 5:17 pm, Stamatis Zampetakis wrote:
> > > > > > > Apache Calcite's Project Management Committee (PMC) has invited
> > TJ
> > > > > > Banghart to
> > > > > > > become a committer, and we are pleased to announce that they
> have
> > > > > > accepted.
> > > > > > >
> > > > > > > TJ has been contributing to the community for about a year now.
> > > They
> > > > > > > introduced many new SQL functions for parsing and formatting
> > dates,
> > > > > > > times, and timestamps extending the capabilities of the
> BigQuery
> > > > > > > dialect. Furthermore, they pushed various improvements around
> > JSON
> > > > > > > serialization and deserialization, JDBC metadata, and lexical
> > > > > > > policies.
> > > > > > >
> > > > > > > TJ, welcome, thank you for your contributions, and we look
> > forward
> > > to
> > > > > > your
> > > > > > > further interactions with the community! If you wish, please
> feel
> > > > free
> > > > > > to tell
> > > > > > > us more about yourself and what you are working on.
> > > > > > >
> > > > > > > As your first commit, please add yourself to the contributors
> > list
> > > > [1]
> > > > > > > and the community page will re-generate [2].
> > > > > > >
> > > > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > > > >
> > > > > > > [1]
> > > > > >
> > > >
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > > > > > [2] https://calcite.apache.org/community/#project-members
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best,
> > > > > Benchao Li
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Zhe Hu

[Jiajun Xie]

Congratulations, Zhe Hu!

Welcome!


On Wed, 28 Jun 2023 at 19:39, xiong duan  wrote:

> Congratulations, Zhe Hu!
>
> Francis Chuang  于2023年6月28日周三 19:05写道：
>
> > Congrats, Zhe Hu!
> >
> > On 28/06/2023 9:04 pm, Stamatis Zampetakis wrote:
> > > Apache Calcite's Project Management Committee (PMC) has invited Zhe Hu
> to
> > > become a committer, and we are pleased to announce that they have
> > accepted.
> > >
> > > Zhe Hu has been contributing to the project for a while now. They
> > > improved the stability of the ElasticSearch adapter, worked on
> > > supporting new java versions, and landed various enhancements around
> > > CONCAT and CONVERT functions.
> > >
> > > Zhe Hu, welcome, thank you for your contributions, and we look forward
> > to your
> > > further interactions with the community! If you wish, please feel free
> > to tell
> > > us more about yourself and what you are working on.
> > >
> > > As your first commit, please add yourself to the contributors list [1]
> > > and the community page will re-generate [2].
> > >
> > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> > > [1]
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > [2] https://calcite.apache.org/community/#project-members
> >
>

Re: [ANNOUNCE] New committer: Jacky Lau

[Jiajun Xie]

Congratulations, Jacky!

Looking forward to your greater contribution in the future.

On Wed, 28 Jun 2023 at 19:40, xiong duan  wrote:

> Congratulations, Jacky!
>
> Francis Chuang  于2023年6月28日周三 18:57写道：
>
> > Congrats, Jacky!
> >
> > On 28/06/2023 8:46 pm, Stamatis Zampetakis wrote:
> > > Apache Calcite's Project Management Committee (PMC) has invited Jacky
> > Lau to
> > > become a committer, and we are pleased to announce that they have
> > accepted.
> > >
> > > Jacky has started contributing to the project very recently and in a
> > > very short time they landed many notable improvements around ARRAY and
> > > MAP functions with a particular focus on the Spark dialect.
> > >
> > > Jacky, welcome, thank you for your contributions, and we look forward
> to
> > your
> > > further interactions with the community! If you wish, please feel free
> > to tell
> > > us more about yourself and what you are working on.
> > >
> > > As your first commit, please add yourself to the contributors list [1]
> > > and the community page will re-generate [2].
> > >
> > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> > > [1]
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > [2] https://calcite.apache.org/community/#project-members
> >
>

[jira] [Created] (CALCITE-5805) SelectNamespace throws NullPointerException while validating MERGE statement

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5805:
---

 Summary: SelectNamespace throws NullPointerException while 
validating MERGE statement
 Key: CALCITE-5805
 URL: https://issues.apache.org/jira/browse/CALCITE-5805
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


{code:java}
final String sql = "merge into empnullables e "
+ "using (select * from emp where deptno is null) t "
+ "on e.empno = t.empno "
+ "when matched then update "
+ "set ename = t.ename, deptno = t.deptno, sal = t.sal * .1 "
+ "when not matched then insert (empno, ename, deptno, sal) "
+ "values(t.empno, t.ename, 10, t.sal * .15)";
sql(sql).ok(); // Expected it is ok, but failed{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5796) DELETE statement is not effective

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5796:
---

 Summary: DELETE statement is not effective 
 Key: CALCITE-5796
 URL: https://issues.apache.org/jira/browse/CALCITE-5796
 Project: Calcite
  Issue Type: Bug
  Components: core, linq4j
 Environment: I try to use `DELETE FROM t`, but the table was not 
affected.

 
{code:java}
static void erase(SqlIdentifier name, CalcitePrepare.Context context) {
  // Generate, prepare and execute an "DELETE FROM table" statement.
  // (It's a bit inefficient that we convert from SqlNode to SQL and back
  // again.)
  final FrameworkConfig config = Frameworks.newConfigBuilder()
  .defaultSchema(context.getRootSchema().plus())
  .build();
  final Planner planner = Frameworks.getPlanner(config);
  try {
final StringBuilder buf = new StringBuilder();
final SqlWriterConfig writerConfig =
SqlPrettyWriter.config().withAlwaysUseParentheses(false);
final SqlPrettyWriter w = new SqlPrettyWriter(writerConfig, buf);
buf.append("DELETE FROM ");
name.unparse(w, 0, 0);
final String sql = buf.toString();
final SqlNode query1 = planner.parse(sql);
final SqlNode query2 = planner.validate(query1);
final RelRoot r = planner.rel(query2);
final PreparedStatement prepare =
context.getRelRunner().prepareStatement(r.rel);
int rowCount = prepare.executeUpdate();
Util.discard(rowCount);
prepare.close();
  } catch (SqlParseException | ValidationException
   | RelConversionException | SQLException e) {
throw Util.throwAsRuntime(e);
  }
} {code}
 

Because we use `EnumerableDefaults#remove()` to delete target rows. 

The rows is object array, they can't match.

!image-2023-06-24-18-27-57-722.png!
    Reporter: Jiajun Xie
 Attachments: image-2023-06-24-18-27-57-722.png





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Oliver Lee

[Jiajun Xie]

Congratulations and welcome Oliver!

Re: Algebraic optimizations rule

[Jiajun Xie]

Hello, mbudiu:

Based on my experience, constant folding can occur in two stages.
- Enable it by `RelBuilder$Config$withSimplify`[1] in the convert stage
- Add `RelOptRules$CONSTANT_REDUCTION_RULES`[2] in the optimize stage

If your RexNode is not complex, you can directly use RexSimplify. For
example
```[3]
final RexSimplify simplify =
new RexSimplify(rexBuilder, RelOptPredicateList.EMPTY,
RexUtil.EXECUTOR)
.withParanoid(true);
return simplify.simplifyUnknownAs(e, RexUnknownAs.UNKNOWN);
```


[1]
https://github.com/apache/calcite/blob/8ea4160f10e95aca6c3b0029d505bbc56975a873/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L4841
[2]
https://github.com/apache/calcite/blob/8ea4160f10e95aca6c3b0029d505bbc56975a873/core/src/main/java/org/apache/calcite/plan/RelOptRules.java#L132
[3]
https://github.com/apache/calcite/blob/8ea4160f10e95aca6c3b0029d505bbc56975a873/core/src/test/java/org/apache/calcite/rex/RexProgramTestBase.java#L187

On Sat, 3 Jun 2023 at 04:33,  wrote:

> Hello,
>
>
>
> I was looking for rules that apply algebraic optimizations to RexNode
> expressions, such as 0 + x = x, but I couldn't find any.
>
> Is there such a rule?
>
>
>
> Mihai
>
>

Re: [ANNOUNCE] New committer: Tanner Clary

[Jiajun Xie]

Congratulations and welcome, Tanner!

On Sat, 27 May 2023 at 02:46, Sergey Nuyanzin  wrote:

> Congratulations, Tanner!
>
> On Fri, May 26, 2023 at 8:40 PM Bertil Chapuis  wrote:
>
> > Congratulations and welcome!
> >
> > Bertil
> >
> > > On 26 May 2023, at 18:58, Tanner Clary  .INVALID>
> > wrote:
> > >
> > > Hey everyone,
> > >
> > > Thank you all for the warm welcome! I have really enjoyed contributing
> to
> > > the project so far and look forward to continuing to do so.
> > >
> > > About me: I graduated in May 2022 and since then I've been working on
> > > Looker @ Google. I'm eager to learn more about other areas of the
> project
> > > and hopefully collaborate with many of you along the way.
> > >
> > > Thanks again,
> > > Tanner Clary
> > >
> > >
> > > On Fri, May 26, 2023 at 5:12 AM Michael Mior  wrote:
> > >
> > >> Congratulations and welcome Tanner!
> > >> --
> > >> Michael Mior
> > >> mm...@apache.org
> > >>
> > >>
> > >> On Fri, May 26, 2023 at 5:05 AM Stamatis Zampetakis <
> zabe...@gmail.com>
> > >> wrote:
> > >>
> > >>> Apache Calcite's Project Management Committee (PMC) has invited
> Tanner
> > >>> Clary to become a committer, and we are pleased to announce that they
> > >>> have accepted.
> > >>>
> > >>> In less than 5 months, Tanner has introduced, enabled, and tested
> over
> > >>> 30 SQL functions in Calcite. They have been a driving force in
> > >>> improving the BigQuery dialect and by now an expert in library and
> > >>> parser changes.
> > >>>
> > >>> Tanner, welcome, thank you for your contributions, and we look
> forward
> > >>> to your further interactions with the community! If you wish, please
> > >>> feel free to tell us more about yourself and what you are working on.
> > >>>
> > >>> As your first commit, please add yourself to the contributors list
> [1]
> > >>> and the community page will re-generate [2].
> > >>>
> > >>> Stamatis (on behalf of the Apache Calcite PMC)
> > >>>
> > >>> [1]
> > >>>
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > >>>
> > >>> [2] https://calcite.apache.org/community/#project-members
> > >>>
> > >>
> >
> >
>
> --
> Best regards,
> Sergey
>

Re: [DISCUSS] Sharing the load of reviewing PRs

[Jiajun Xie]

Thank Julian for your idea.
 Your plan helps to motivate new contributors.

“If there is no response to my PR,
I will be disappointed or even give up on continuing to contribute.”

I hope that every contributor will be encouraged,
and I also hope that the Calcite community will become stronger and
stronger.

+1, I am willing to join the pool of reviews.

On Tue, 11 Apr 2023 at 13:20, Benchao Li  wrote:

> Thanks Julian for starting the discussion!
>
> I'm spending my spare time to contribute to Calcite, usually at weekends,
> and sometimes in the break of weekdays, hence my time would be limited
> because the spare time may varies. Review work is not that simple for me
> because Calcite has many complicated components and evolves many years
> which means we need track a lot of background. I'm still learning some part
> while doing the review work.
>
> The complexity of PRs varies a lot, simple ones would be easier to get in
> because it only cost me minutes to hours to review. But the complex ones,
> usually I need to spend more time to understand the background, new design,
> the effect to the whole project, and the future direction we want to take.
> These kinds of PRs may be preempted by small ones, and finally do not
> getting reviewed for months, there is a example[1] which I would say sorry
> to the author that I still do not manage to give it a review till today.
>
> Any way to improve current status would be grateful. However, if the
> proposal from Julian may require more sustainable time, I'm not sure if it
> is suitable for guys like me who only devotes limited spare time to
> Calcite. Hence I'm +0 for this proposal. Of course, I would be happy to
> participate in the schema if we finally decide to do it.
>
> [1] https://issues.apache.org/jira/browse/CALCITE-5413
>
> Charles Givre  于2023年4月11日周二 12:43写道：
>
> > I for one would very much like to help with reviews.  I don't have a lot
> > of time this month, but next month should have more time.
> > Best,
> > -- C
> >
> > > On Apr 10, 2023, at 10:56 PM, Dan Zou  wrote:
> > >
> > > +1, thanks Julian for proposing this. From my observation, there are
> > many pending PRs in Calcite and only a few active committers, this puts a
> > lot of pressure on these committers. For example Julian have reviewed 34
> PR
> > in 2023 Q1, it is an unimaginable number. I am very supportive of
> achieving
> > a mechanism to improve the review efficiency of PRs, and also I would
> like
> > to make contribution in reviewing PRs.
> > >
> > > Best,
> > > Dan Zou
> > >
> > >
> > >
> > >
> > >
> > >> 2023年4月11日 01:56，Julian Hyde  写道：
> > >>
> > >> I don't enjoy reviewing and merging PRs. And every time I do, I feel
> > >> like a sucker, because there are over a few dozen committers who are
> > >> enjoying the project and not doing the work. (There is a small group
> > >> of committers who regularly review and merge PRs. I don't know how
> > >> they feel about the task, but I am immensely grateful.)
> > >>
> > >> I think I would review more PRs if I saw others doing the same.
> > >>
> > >> Can we figure out a fairer way to distribute the load? For release
> > >> managers (approximately the same amount of work, but compressed into a
> > >> few hours or days) we have successfully run a rota for several years.
> > >> Could we do something similar with PRs?
> > >>
> > >> I propose the following. For each calendar month, there is a PR
> > >> manager and 6 - 8 reviewers. The PR manager does not review PRs, but
> > >> assigns them to reviewers, and politely reminds reviews to keep the PR
> > >> moving.
> > >>
> > >> The PR manager's goals are:
> > >> * every non-draft PR is reviewed within 3 days of submission,
> > >> * every PR is merged within 3 days of being done;
> > >> * rotate duties so that no reviewer is asked to review more than 4
> > >> PRs per month;
> > >> * email a report at the end of the month;
> > >> * work down the backlog of historic PRs if it's a slow month.
> > >>
> > >> The PR manager rotates every month. The reviewers can rotate if they
> > >> wish, but I suspect most will stay in the pool for several months,
> > >> because the reviewing load is not very heavy, and because they see
> > >> others doing the work.
> > >>
> > >> Other notes:
> > >> * Non-committers would be welcome to join the pool of reviews (and
> > >> that would be a good way to earn the committer bit) and a committer
> > >> could merge when the PR is approved.
> > >> * If committers join the pool, that's a good way to earn PMC
> membership.
> > >> * Committers who are not in the pool are welcome to review PRs and
> > >> assign PRs to themselves (but expect to be nagged by the PR manager if
> > >> you don't review in a timely manner).
> > >>
> > >> What do you think? Would you join this scheme if we introduced it? If
> > >> you agree please +1; also happy to see revisions to this suggestion or
> > >> other ideas to share the work.
> > >>
> > >> Julian
> > >
> >
> >
>
> --
>
> Best,
> Benchao Li

[jira] [Created] (CALCITE-5630) Window with rows equivalence error in volcano planner

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5630:
---

 Summary: Window with rows equivalence error in volcano planner
 Key: CALCITE-5630
 URL: https://issues.apache.org/jira/browse/CALCITE-5630
 Project: Calcite
  Issue Type: Bug
Reporter: Jiajun Xie


Here is UT
{code:java}
@Test void testUnionWindow() {
  String selectSqlBase =
  "SELECT 'col{?}' as col, sum(\"salary\" ) OVER("
  + "PARTITION BY \"deptno\" "
  + "ORDER BY \"empid\" "
  + "ROWS BETWEEN CURRENT ROW AND {?} FOLLOWING) "
  + "FROM  \"hr\".\"emps\"";
  String selectSql1 = selectSqlBase.replace("{?}", "1"); // window1
  String selectSql2 = selectSqlBase.replace("{?}", "2"); // window2
  String sql = selectSql1 + "\nunion all\n" + selectSql2;
  CalciteAssert.hr().query(sql).runs()
  .returns("COL=col1; EXPR$1=8000.0\n"
  + "COL=col1; EXPR$1=21500.0\n"
  + "COL=col1; EXPR$1=18500.0\n"
  + "COL=col1; EXPR$1=7000.0\n"
  + "COL=col2; EXPR$1=8000.0\n"
  + "COL=col2; EXPR$1=21500.0\n"
  + "COL=col2; EXPR$1=18500.0\n"
  + "COL=col2; EXPR$1=7000.0\n"); 
 // sum that for col1 is same as sum that for col2, this is error
}
 {code}
Because RelDigest is same between window1 and window2,

volcano planner use window1 replace window2.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5597) Column that be order by should not be aggregate

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5597:
---

 Summary: Column that be order by should not be aggregate
 Key: CALCITE-5597
 URL: https://issues.apache.org/jira/browse/CALCITE-5597
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Jiajun Xie


Here is unit test
{code:java}
@Test void testDistinctOrderByRand() {
  final String sql = "select distinct deptno from emp order by rand()";
  sql(sql).ok();
} {code}
We will get error logical plan
{code:java}
LogicalProject(DEPTNO=[$0])
  LogicalSort(sort0=[$1], dir0=[ASC])
LogicalAggregate(group=[{0, 1}]) //rand() in group, result will be error
  LogicalProject(DEPTNO=[$7], EXPR$1=[RAND()])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5590) NullPointerException when converting 'in' expression that is used inside select list and group by

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5590:
---

 Summary: NullPointerException when converting  'in' expression 
that is used inside select list and group by
 Key: CALCITE-5590
 URL: https://issues.apache.org/jira/browse/CALCITE-5590
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.33.0
Reporter: Jiajun Xie


In CALCITE-5209, we changed sub-query handle way.

It can solve some problems, but it also introduces some new problems.

Here is a simple case to reproduce, it can work before 1.33.0
{code:java}
@Test void testGroupByCaseWhen() {
final String sql = "select case when deptno in (1) then 1 end as col\n"
+ "from emp\n"
+ "group by deptno, case when deptno in (1) then 1 else 0 end";
sql(sql).ok();
} {code}
{code:java}
while converting CASE WHEN `EMP`.`DEPTNO` IN (1) THEN 1 ELSE NULL END
java.lang.RuntimeException: while converting CASE WHEN `EMP`.`DEPTNO` IN (1) 
THEN 1 ELSE NULL END
at 
org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerNodeTypeMethod$0(ReflectiveConvertletTable.java:86)
at 
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:63)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5591)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4875)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5454)
at 
org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$9(StandardConvertletTable.java:205)
at 
org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:63)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5591)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4875)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5454)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.createAggImpl(SqlToRelConverter.java:3316)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertAgg(SqlToRelConverter.java:3158)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:784)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:682)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3680)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:597)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:639)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:758)
at 
org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:4502)
at 
org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:4494)
at 
org.apache.calcite.test.SqlToRelConverterTest.testGroupByCaseWhen(SqlToRelConverterTest.java:4028){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Jiajun Xie

[Jiajun Xie]

Thanks for the warm welcome to everyone!

A few words about myself:

I'm 23 years old and I often contribute code in my spare time.

In addition, there are many improvements for Calcite in my work,
but my leader does not support community cooperation.
I'm trying to persuade him, I hope to make more contributions in the future.

On Mon, 13 Feb 2023 at 15:16, Chunwei Lei  wrote:

> Congratulations, Jiajun!
>
>
> Best,
> Chunwei
>
>
> On Mon, Feb 13, 2023 at 2:36 PM Sergey Nuyanzin 
> wrote:
>
> > Congratulations, Jiajun!
> >
> > On Mon, Feb 13, 2023 at 3:06 AM 163  wrote:
> >
> > > Congratulations, Jiajun!
> > >
> > >
> > > Best,
> > > Dan Zou
> > >
> > >
> > >
> > >
> > >
> > > > 2023年2月11日 10:06，Benchao Li  写道：
> > > >
> > > > Congratulations
> > >
> > >
> >
> > --
> > Best regards,
> > Sergey
> >
>

[jira] [Created] (CALCITE-5525) GROUPING_ID() should be rewritten to GROUPING() in some dialects

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5525:
---

 Summary: GROUPING_ID() should be rewritten to GROUPING() in some 
dialects
 Key: CALCITE-5525
 URL: https://issues.apache.org/jira/browse/CALCITE-5525
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


Some databases not exists `grouping_id()`, `grouping()` is used as 
`grouping_id()`.
 - Postgresql
{code:java}
ERROR:  function grouping_id(integer, character varying) does not exist
LINE 1: select grouping(id), grouping(name), grouping_id(id, name), ...
 ^
HINT:  No function matches the given name and argument types. You might need to 
add explicit type casts
{code}

 - Presto
{code:java}
Function grouping_id not registered
{code}

 

Here is a simple query, `grouping_id()` is a legal function in Spark.
{code:java}
select  a,
        b,
        count(*),
        grouping(a) ga,
        grouping(b) gb,
        grouping_id(a, b)
from    (
            select  1 as a, 2 as b
        )
group by grouping sets((a, b), (b), ())
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5507) HAVING alias failed when aggregate function in condition

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5507:
---

 Summary: HAVING alias failed when aggregate function in condition
 Key: CALCITE-5507
 URL: https://issues.apache.org/jira/browse/CALCITE-5507
 Project: Calcite
  Issue Type: Improvement
Reporter: Jiajun Xie


We know that calcite can support HAVING alias by setting 
SqlConformanceEnum.LENIENT
{code:java}
sql("select count(empno) as e from emp having e > 10")
.withConformance(lenient).ok(); {code}
but when I add one aggregate function in HAVING clause, it will fail.
{code:java}
sql("select count(empno) as e from emp having ^e^ > 10 and count(empno) > 10 ")
    .withConformance(lenient).fails("Column 'E' not found in any table"); 
// I think it should be ok{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Jiajun Xie]

Congrats, Benchao!

Thanks for your help and your contributions!

On Sat, 28 Jan 2023 at 09:48, Chunwei Lei  wrote:

> Congrats, Benchao!
>
> Best,
> Chunwei
>
>
> On Fri, Jan 27, 2023 at 9:01 PM Michael Mior  wrote:
>
> > Congratulations and welcome Benchao!
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > On Fri, Jan 27, 2023 at 4:51 AM Stamatis Zampetakis 
> > wrote:
> >
> > > I am pleased to announce that Benchao has accepted an invitation to
> join
> > > the Calcite PMC. Benchao has been a consistent and helpful figure in
> > > the Calcite community for which we are very grateful. We look forward
> to
> > > the continued contributions and support.
> > >
> > > Please join me in congratulating Benchao!
> > >
> > > Stamatis (on behalf of the Calcite PMC)
> > >
> >
>

[jira] [Created] (CALCITE-5506) RelToSqlConverter get error result because RelFieldTrimmer lost aggregate function

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5506:
---

 Summary: RelToSqlConverter get error result because 
RelFieldTrimmer lost aggregate function
 Key: CALCITE-5506
 URL: https://issues.apache.org/jira/browse/CALCITE-5506
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie


Here is a sql

 
{code:java}
select  l.v as l_cost
from    (
            select  0 as v,
                    1 as k
        ) l
join    (
    select  sum("cost") as v,
                    1 as k
    from    (
        select 1 as "cost"
        union
        select 2 as "cost"
        )
) r
on l.k = r.k {code}
Before trimming, the RelNode is 

 

 
{code:java}
LogicalProject(L_COST=[$0])
  LogicalJoin(condition=[=($1, $3)], joinType=[inner])
    LogicalValues(tuples=[[{ 0, 1 }]])
    LogicalProject(V=[$0], K=[1])
      LogicalAggregate(group=[{}], V=[SUM($0)]) 
         LogicalUnion(all=[false])
          LogicalValues(tuples=[[{ 1 }]])
          LogicalValues(tuples=[[{ 2 }]]) {code}
After trimming, the RelNode is

 

 
{code:java}
LogicalProject(L_COST=[$0])
  LogicalJoin(condition=[=($1, $2)], joinType=[inner])
    LogicalValues(tuples=[[{ 0, 1 }]])
    LogicalProject(K=[1]) 
      LogicalAggregate(group=[{}], DUMMY=[COUNT()]) 
        LogicalUnion(all=[false])
          LogicalValues(tuples=[[{ 1 }]])
          LogicalValues(tuples=[[{ 2 }]]){code}
 

If we convert trimmed RelNode to sql, the sql will be
{code:java}
SELECT *
FROM 
  (VALUES (0, 1)) AS "t" ("V", "K")
INNER JOIN 
  (SELECT 1 AS "K" -- Missing SUM()
   FROM (SELECT *
 FROM (VALUES (1)) AS "t" ("cost")
 UNION
 SELECT *
 FROM (VALUES (2)) AS "t" ("cost")) AS "t2"
  ) AS "t4" 
ON "t"."K" = "t4"."K" {code}
The origin sql only has one row result, but the new sql that be trimmed has two 
row result.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5486) Subquery not support having alias in condition.

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5486:
---

 Summary: Subquery not support having alias in condition.
 Key: CALCITE-5486
 URL: https://issues.apache.org/jira/browse/CALCITE-5486
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie


{code:java}
sql("select * from emp where sal >\n"
+ " (select avg(sal) as s"
+ "from emp having ^s^ > 0"
+ "  )")
.withConformance(SqlConformanceEnum.LENIENT)
.fails("Column 'S' not found in any table");{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Why we cannot get all predicates for outer join query?

[Jiajun Xie]

Hi, Benchao and Renhe
Thanks for your information. They are useful for me.

Re: Why we cannot get all predicates for outer join query?

[Jiajun Xie]

Hi Benchao,
Thanks very much for your email.

I understand `RelMetadataQuery#getAllPredicates` cannot get all predicates
because some optimized cases cannot push/pull predicates.

But I think the function name and code comment[1] are misleading,
*getAllPredicates* cannot get all predicates.
And your case should be handled by `RelMetadataQuery#getPulledUpPredicates`[2]
instead of `RelMetadataQuery#getAllPredicates`.

[1]
https://github.com/apache/calcite/blob/413eded693a9087402cc1a6eefeca7a29445d337/core/src/main/java/org/apache/calcite/rel/metadata/RelMetadataQuery.java#L855
[2]
https://github.com/apache/calcite/blob/413eded693a9087402cc1a6eefeca7a29445d337/core/src/main/java/org/apache/calcite/rel/metadata/RelMetadataQuery.java#L836

On Sun, 15 Jan 2023 at 21:12, Benchao Li  wrote:

> Hi Jiajun,
>
> For outer join, the semantic is different for predicates in condition and
> where, for example:
> Q1: select * from emp left join dept on emp.deptno = dept.deptno
> Q2: select * from emp left join dept on true where emp.deptno = dept.deptno
>
> The semantic is different for Q1 and Q2. Q1 will output all the records
> from emp, including the records which fail to join from dept. However, Q2
> will only output the records which successfully join some records from
> dept.
>
> This is the reason why we cannot push/pull the predicates from outer joins
> conditions. Hope this helps.
>
> Jiajun Xie  于2023年1月12日周四 16:47写道：
>
> > Hello, all.
> >
> > I try to use RelMetadataQuery#getAllPredicates get predicate,
> > but I get null for outer join query that left column name is same as
> right
> > column name.
> > ```
> > final RelNode rel = sql("select name as dname from emp left outer join
> > dept"
> > + " on emp.deptno = dept.deptno").toRel();
> > final RelMetadataQuery mq = rel.getCluster().getMetadataQuery();
> > final RelOptPredicateList r = mq.getAllPredicates(rel);
> > assertNull(r);
> > ```
> >
> >
> > After commenting on two pieces of code:
> > 1. RelMdAllPredicates#getAllPredicates
> > ```
> > if (join.getJoinType().isOuterJoin()) {
> >   // We cannot map origin of this expression.
> >   return null;
> > }
> > 2. RelMdExpressionLineage#getExpressionLineage
> > ```
> > if (rel.getJoinType().isOuterJoin()) {
> >   // If we reference the inner side, we will bail out
> >   if (rel.getJoinType() == JoinRelType.LEFT) {
> > ImmutableBitSet rightFields = ImmutableBitSet.range(
> > nLeftColumns, rel.getRowType().getFieldCount());
> > if (inputFieldsUsed.intersects(rightFields)) {
> >   // We cannot map origin of this expression.
> >   return null;
> > }
> >   } else if (rel.getJoinType() == JoinRelType.RIGHT) {
> > ImmutableBitSet leftFields = ImmutableBitSet.range(
> > 0, nLeftColumns);
> > if (inputFieldsUsed.intersects(leftFields)) {
> >   // We cannot map origin of this expression.
> >   return null;
> > }
> >   } else {
> > // We cannot map origin of this expression.
> > return null;
> >   }
> > }
> > I can get the results I need
> > ```
> > final RelNode rel = sql("select name as dname from emp left outer join
> > dept"
> > + " on emp.deptno = dept.deptno").toRel();
> > final RelMetadataQuery mq = rel.getCluster().getMetadataQuery();
> > final RelOptPredicateList r = mq.getAllPredicates(rel);
> > assertThat(r.pulledUpPredicates.get(0).toString(),
> > equalTo("=([CATALOG, SALES, EMP].#0.$7, [CATALOG, SALES,
> > DEPT].#0.$0)"));
> > ```
> >
> >
> > It seems that we deliberately return null
> > in RelMetadataQuery#getAllPredicates . Can someone tell me why? Thanks!
> >
>
>
> --
>
> Best,
> Benchao Li
>

Why we cannot get all predicates for outer join query?

[Jiajun Xie]

Hello, all.

I try to use RelMetadataQuery#getAllPredicates get predicate,
but I get null for outer join query that left column name is same as right
column name.
```
final RelNode rel = sql("select name as dname from emp left outer join dept"
+ " on emp.deptno = dept.deptno").toRel();
final RelMetadataQuery mq = rel.getCluster().getMetadataQuery();
final RelOptPredicateList r = mq.getAllPredicates(rel);
assertNull(r);
```


After commenting on two pieces of code:
1. RelMdAllPredicates#getAllPredicates
```
if (join.getJoinType().isOuterJoin()) {
  // We cannot map origin of this expression.
  return null;
}
2. RelMdExpressionLineage#getExpressionLineage
```
if (rel.getJoinType().isOuterJoin()) {
  // If we reference the inner side, we will bail out
  if (rel.getJoinType() == JoinRelType.LEFT) {
ImmutableBitSet rightFields = ImmutableBitSet.range(
nLeftColumns, rel.getRowType().getFieldCount());
if (inputFieldsUsed.intersects(rightFields)) {
  // We cannot map origin of this expression.
  return null;
}
  } else if (rel.getJoinType() == JoinRelType.RIGHT) {
ImmutableBitSet leftFields = ImmutableBitSet.range(
0, nLeftColumns);
if (inputFieldsUsed.intersects(leftFields)) {
  // We cannot map origin of this expression.
  return null;
}
  } else {
// We cannot map origin of this expression.
return null;
  }
}
I can get the results I need
```
final RelNode rel = sql("select name as dname from emp left outer join dept"
+ " on emp.deptno = dept.deptno").toRel();
final RelMetadataQuery mq = rel.getCluster().getMetadataQuery();
final RelOptPredicateList r = mq.getAllPredicates(rel);
assertThat(r.pulledUpPredicates.get(0).toString(),
equalTo("=([CATALOG, SALES, EMP].#0.$7, [CATALOG, SALES,
DEPT].#0.$0)"));
```


It seems that we deliberately return null
in RelMetadataQuery#getAllPredicates . Can someone tell me why? Thanks!

[Review request] CALCITE-5394 and CALCITE-5395

[Jiajun Xie]

Hi all,
I fixed two bugs in RelToSql: CALCITE-5394[1] and CALCITE-5395[2].

Their code changes are very small, so it won't cost you too much time to
review.

Would anyone please review them? Thanks very much.

[1] https://issues.apache.org/jira/browse/CALCITE-5394
[2] https://issues.apache.org/jira/browse/CALCITE-5395

[jira] [Created] (CALCITE-5392) Support Snapshot in RelMdExpressionLineage

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5392:
---

 Summary: Support Snapshot in RelMdExpressionLineage
 Key: CALCITE-5392
 URL: https://issues.apache.org/jira/browse/CALCITE-5392
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


 For example:
{code:java}
select productid from products_temporal
for system_time as of TIMESTAMP '2022-11-19 00:00:00' 
{code}
The `productid` expression lineage is null because `RelMdExpressionLineage` 
does not implement the corresponding method.

 

We can implement method that for SNAPSHOT like other RelNode.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Dmitry Sysolyatin

[Jiajun Xie]

Congratulations, Dimitry! Your work looks great

On Wed, 9 Nov 2022 at 18:39, Dmitry Sysolyatin 
wrote:

> Hi!
> Thanks everyone for your kind words. I'm honored to be a part of the
> calcite community.
>
> Few words about myself. I'm currently working at Intertrust Technologies
> [1] company. The company has a lot of projects. Currently, I'm working on
> Intertrust Platform [2] which is an ecosystem to securely and efficiently
> manage distributed data.
>
> Besides work, I'm interested in random graphs and application of it in
> systems simulating.
>
> [1] https://www.intertrust.com/
> [2] https://www.intertrust.com/platform/
>
> On Wed, Nov 9, 2022 at 2:57 AM Benchao Li  wrote:
> >
> > Congratulations, Dimitry! Looking forward to further interactions with
> you.
> >
> > Bertil Chapuis  于2022年11月9日周三 06:23写道：
> >
> > > Congratulations, Dimitry!
> > >
> > > Bertil
> > >
> > > > On 8 Nov 2022, at 23:08, Andrei Sereda  wrote:
> > > >
> > > > Congratulations!
> > > >
> > > > On Tue, Nov 8, 2022, 16:08 Francis Chuang 
> > > wrote:
> > > >
> > > >> Congratulations, Dmitry!
> > > >>
> > > >> Francis
> > > >>
> > > >> On 9/11/2022 6:13 am, Julian Hyde wrote:
> > > >>> Congratulations, Dmitry! Thanks for the great work you’ve already
> done
> > > >> for the project.
> > > >>>
> > > >>> Julian
> > > >>>
> > > >>>
> > >  On Nov 8, 2022, at 10:56 AM, Alessandro Solimando <
> > > >> alessandro.solima...@gmail.com> wrote:
> > > 
> > >  Congrats Dmitry, welcome to Calcite!
> > > 
> > >  On Tue 8 Nov 2022, 18:44 Ruben Q L,  wrote:
> > > 
> > > > Apache Calcite's Project Management Committee (PMC) has invited
> > > Dmitry
> > > > Sysolyatin to become a committer, and we are pleased to announce
> he
> > > has
> > > > accepted the invitation.
> > > >
> > > > During the last months Dmitry has pushed a lot of high quality
> > > patches,
> > > > fixing and improving code around several places in Calcite core.
> > > Apart
> > > >> from
> > > > code contributions, he has been regularly involved in discussions
> > > >> regarding
> > > > multiple topics in the mailing list, always with a hardworking
> and
> > > > courteous spirit.
> > > >
> > > > Dmitry, welcome, thank you for your contributions, and we look
> > > forward
> > > >> to
> > > > your further interactions with the community! If you wish, please
> > > feel
> > > >> free
> > > > to tell us a bit more about yourself and what you are currently
> > > >> working on.
> > > > As your first commit, you can add yourself to the contributors
> list
> > > >> [1] and
> > > > the community page will re-generate [2].
> > > >
> > > > Ruben (on behalf of the Apache Calcite PMC)
> > > >
> > > > [1]
> > > >
> > > >>
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > > [2] https://calcite.apache.org/community/#project-members
> > > >
> > > >>>
> > > >>
> > >
> > >
> >
> > --
> >
> > Best,
> > Benchao Li
>

Re: What is the difference between TableMacro and TableFunction?

[Jiajun Xie]

Sorry, there is a problem with the format of the last message. Here is
new version:


I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
are very similar. - From their descriptions, TableMacro is for "compile
time",  TableFunction is for “execution time".
- From an email[3], I know TableFunction can be seen as a lightweight
relational operator.

But some descriptions do not seem to strictly distinguish between the two
interfaces. For example, ``` */** Table function that implements a view. It
returns the operator* * * tree of the view's SQL query. */* public class
ViewTableMacro implements TableMacro ``` ViewTableMacro[4] implements
TableMacro, but the description is a Table function. Is the description
inaccurate? Is anyone familiar with their other differences? Thanks very
much. Refs: [1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
[2]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
[3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65 [4]
https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00

On Mon, 24 Oct 2022 at 19:43, Jiajun Xie  wrote:

> Hi, all:
>
> I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
> are very similar.
> - From their descriptions, TableMacro is for *"compile time", * TableFunction
> is for “*execution time". *
> *- From an email[3], I know T*ableFunction can be seen as a lightweight
> relational operator.
>
> *But s*ome descriptions do not seem to strictly distinguish between the
> two interfaces. For example,
> ```
> */** Table function that implements a view. It returns the operator*
> * * tree of the view's SQL query. */*
> public class ViewTableMacro implements TableMacro
> ```
> ViewTableMacro[3] implements TableMacro, but the description is a Table
> function. Is the description inaccurate?
>
> Is anyone familiar with their other differences?  Thanks very much.
>
> Refs:
> [1]
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
> [2]
> https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
> [3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65
> [4] https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00
>
>

What is the difference between TableMacro and TableFunction?

[Jiajun Xie]

Hi, all:

I'm learning Calcite's TableMacro[1] and TableFunction[2] and I feel they
are very similar.
- From their descriptions, TableMacro is for *"compile time", * TableFunction
is for “*execution time". *
*- From an email[3], I know T*ableFunction can be seen as a lightweight
relational operator.

*But s*ome descriptions do not seem to strictly distinguish between the two
interfaces. For example,
```
*/** Table function that implements a view. It returns the operator*
* * tree of the view's SQL query. */*
public class ViewTableMacro implements TableMacro
```
ViewTableMacro[3] implements TableMacro, but the description is a Table
function. Is the description inaccurate?

Is anyone familiar with their other differences?  Thanks very much.

Refs:
[1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableMacro.java
[2]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/schema/TableFunction.java
[3] https://lists.apache.org/thread/twzmgl5sg701dxw0y24wkzwxonz0sy65
[4] https://lists.apache.org/thread/gz8w0gr9cp1nl1nslrw5c3pbxrog5l00

Re: Help with adding Spark functionality

[Jiajun Xie]

Hi Sophie~

FIRST and  LAST are keywords, but they also are nonReservedKeywords[1].

I found a JIRA ticket[2] about FIRST_VALUE/LAST_VALUE, maybe it is useful
for you.

Refs:
[1]
https://github.com/apache/calcite/blob/2feb16aee9f2710bd361a787a80e1cd74cf759bb/core/src/main/codegen/default_config.fmpp#L116
[2] https://issues.apache.org/jira/browse/CALCITE-883

On Thu, 20 Oct 2022 at 03:50, Julian Hyde  wrote:

> According to this [1] FIRST is a synonym for FIRST_VALUE, which
> Calcite already has.
>
> Please log a Jira case describing the feature you want. Then design
> discussions can occur against that case.
>
> Have a look through the commit log at how people have added
> dialect-specific functions. Usually adding functions doesn't require
> parser changes. But if FIRST and LAST are keywords maybe parser
> changes are required.
>
> Julian
>
> [1]
> https://docs.databricks.com/spark/latest/spark-sql/language-manual/functions/first.html
>
> On Wed, Oct 19, 2022 at 8:10 AM Sophie Mugridge White
>  wrote:
> >
> > Hi,
> >
> > I'm looking to add support for the Spark FIRST and LAST functions.
> >
> > Please can I have some pointers as to where to look? In particular in
> > reference to how it can be handled separately than the FIRST function
> > that's part of MATCH_RECOGNIZE
> >
> > Thanks,
> > Sophie
> >
> > --
> > This email is confidential, if you are not the intended recipient please
> > delete it and notify us immediately by emailing the sender. You should
> not
> > copy it or use it for any purpose nor disclose its contents to any other
> > person. Privitar Limited is registered in England with registered number
> > 09305666. Registered office Salisbury House, Station Road, Cambridge,
> > CB12LA.
>

Re: [ANNOUNCE] New committer: Bertil Chapuis

[Jiajun Xie]

Congrats, Bertil!

On Mon, 17 Oct 2022 at 08:28, Benchao Li  wrote:

> Congratulations, Bertil!
>
> Francis Chuang  于2022年10月17日周一 05:30写道：
>
> > Congrats, Bertil!
> >
> > On 17/10/2022 8:29 am, Julian Hyde wrote:
> > > Apache Calcite's Project Management Committee (PMC) has invited Bertil
> > > Chapuis to become a committer, and we are pleased to announce that he
> > > has accepted.
> > >
> > > Over a few months, Bertil has revitalized Calcite's geospatial
> > > support, with commits that switched our geospatial library from ESRI
> > > to the more modern JTS and proj4j libraries, and broadened our support
> > > for the industry-standard geospatial functions that are in PostGIS and
> > > Open Geospatial Consortium (OGC) Simple Features, including adding
> > > geospatial aggregate functions. He has been easy to work with, and his
> > > contributions are of a uniformly high quality.
> > >
> > > Incidentally, Bertil is one of the initial committers of the Baremaps
> > > project, which has just entered the Apache Incubator.
> > >
> > > Bertil, welcome, thank you for your contributions, and we look forward
> > > to your further interactions with the community! If you wish, please
> > > feel free to tell us more about yourself and what you are working on.
> > >
> > > As your first commit, please add yourself to the contributors list [1]
> > > and the community page will re-generate [2].
> > >
> > > Julian (on behalf of the Apache Calcite PMC)
> > >
> > > [1]
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > >
> > > [2] https://calcite.apache.org/community/#project-members
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: How to get lineage for virtual column

[Jiajun Xie]

My real goal is to know which virtual columns are used in query.

- I tried to use `getColumnOrigins`, here is one test in RelMetadataTest
```
@Test void testVirtualColumnOrigins() {
final String sql = "select E from VIRTUALCOLUMNS.VC_T1 "
+ "for system_time as of TIMESTAMP '2011-01-02 00:00:00'";
// Actual column name is A, but I need E.
sql(sql).withCatalogReaderFactory(MockCatalogReaderExtended::create)
.assertColumnOriginSingle("VC_T1", "E", false);
// I know that A is reasonable value because E = A + 1. (The root column
origin should be A.)
}
```

- Visit SqlNode is another solution, but there are some fields not be
trimmed.

I want to know if anyone has suggestions to get virtual column that query
used.

Thank you for your reply.

On Thu, 8 Sept 2022 at 14:55, Julian Hyde  wrote:

> You know, when people ask ‘Does Calcite support lineage?’, I’m never quite
> sure. People mean different things by lineage, and it takes a bit of effort
> to set up the required APIs.
>
> I think the way to solve this question is with unit tests.
>
> Can write a unit test that fails, or point to an existing test that almost
> does what you need? For example, is there a test to get the lineage of a
> view column?
>
> Julian
>
>
> > On Sep 5, 2022, at 2:52 AM, Jiajun Xie 
> wrote:
> >
> > Hi, all:
> > I want to know which virtual column is in use. For example,
> > ```
> > CREATE TABLE peoples(age int, virtual_age int as (age + 1) virtual);
> > SELECT virtual_age from peoples;
> > ```
> > After converting, the virtual column is expanded to expression.
> > ```
> > LogicalProject(virtual_age=[+($0, 1)])
> >  LogicalTableScan(table=[[default_ns, default, peoples]])
> > ```
> > So `RelMetadataQuery#getColumnOrigins` return RelColumnOrigin that is
> > age(not is virtual_age).
> >
> > I try to rewrite code in the implementation class of
> > `InitializerExpressionFactory`, but I can't know which column is in use.
> > When I confirm which column is in use, I can't know if the expression is
> > from a virtual column.
> >
> > Does anyone have relevant experience?  Thanks~
>
>

How to get lineage for virtual column

[Jiajun Xie]

Hi, all:
 I want to know which virtual column is in use. For example,
 ```
 CREATE TABLE peoples(age int, virtual_age int as (age + 1) virtual);
 SELECT virtual_age from peoples;
 ```
 After converting, the virtual column is expanded to expression.
 ```
 LogicalProject(virtual_age=[+($0, 1)])
  LogicalTableScan(table=[[default_ns, default, peoples]])
 ```
So `RelMetadataQuery#getColumnOrigins` return RelColumnOrigin that is
age(not is virtual_age).

I try to rewrite code in the implementation class of
`InitializerExpressionFactory`, but I can't know which column is in use.
When I confirm which column is in use, I can't know if the expression is
from a virtual column.

Does anyone have relevant experience?  Thanks~

Re: How can I convert a BindableTableScan with projects and Fitlers back to a Rel Tree with a project node, a filter node and a tablescan node?

[Jiajun Xie]

Pranav, there are `QueryExecution` that holding logicalPlan, optimizedPlan
and sparkPlan(physical) in Spark:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala

In fact, my code is similar to it. I have an object holding
ValidatedSqlNode, LogicalRelNode, physicalRelNode.


On Tue, 23 Aug 2022 at 23:03, Benchao Li  wrote:

> Pranav,
> There is a doc[1] which talks about how to write a rule. You can also take
> a look at existing
> rules such as FilterTableScanRule[2], ProjectTableScanRule[3] for
> reference.
>
> [1] https://calcite.apache.org/docs/howto.html#create-a-planner-rule
> [2]
>
> https://github.com/apache/calcite/blob/7de37aefaaa72959a0145053319fae2956fbb9f1/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java
> [3]
>
> https://github.com/apache/calcite/blob/7de37aefaaa72959a0145053319fae2956fbb9f1/core/src/main/java/org/apache/calcite/rel/rules/ProjectTableScanRule.java
>
> Pranav Deshpande  于2022年8月22日周一 23:00写道：
>
> > Hi Benchao,
> > Thank you very much for referring me to this document.
> >
> > Could you please provide me with a pointer on how to write the optimizer
> > rule(s) you mentioned in part I of your answer?
> >
> > Regards,
> > Pranav
> >
> > On Sat, Aug 20, 2022 at 11:35 PM Benchao Li 
> wrote:
> >
> > > I agree that a separate planner rule is better. Besides, if you are
> > > constructing the optimizing rules
> > > by yourself in your project, you can avoid this by not adding
> > > `FilterTableScanRule` and
> > > `ProjectTableScanRule`.
> > >
> > > Pranav,
> > > there is a doc[1] about how to contribute to Calcite.
> > >
> > > [1] https://calcite.apache.org/develop/#contributing
> > >
> > > Jiajun Xie  于2022年8月21日周日 09:00写道：
> > >
> > > > Agree with Hyde, there are many problems when converting a  physical
> > > > RelNode directly to SQL. For example,
> > > > https://issues.apache.org/jira/browse/CALCITE-2915
> > > >
> > > > In my project, I have an object  holding  Logical RelNode and
> Physical
> > > > RelNode. When I want to convert sql, I can use Logical RelNode. I
> hope
> > > it's
> > > > useful to you, Pranav.
> > > >
> > > > On Sun, 21 Aug 2022 at 02:50, Julian Hyde 
> > > wrote:
> > > >
> > > > > It doesn’t seem quite right to convert a physical RelNode directly
> to
> > > > SQL.
> > > > > It’s probably better to map it back to logical RelNode(s) first.
> > > > >
> > > > > I don’t know whether we have the software to do that mapping. If we
> > > > don’t,
> > > > > consider using planner rules. They’re often the right way in
> Calcite.
> > > > >
> > > > > Julian
> > > > >
> > > > > > On Aug 20, 2022, at 03:26, Pranav Deshpande <
> > > > > deshpande.v.pra...@gmail.com> wrote:
> > > > > >
> > > > > > Hi Bencaho,
> > > > > > Thank you very much for your reply. Could you please tell me the
> > > > > procedure
> > > > > > to create this issue on JIRA?
> > > > > >
> > > > > > Thanks & Regards,
> > > > > > Pranav
> > > > > >
> > > > > >> On Fri, Aug 19, 2022 at 11:13 PM Benchao Li <
> libenc...@apache.org
> > >
> > > > > wrote:
> > > > > >>
> > > > > >> Pranav,
> > > > > >>
> > > > > >> This is a good question. To me, I would take this as a bug, and
> we
> > > > could
> > > > > >> improve the
> > > > > >> RelToSqlConverter to treat BindableTableScan specially.
> > > > > >> Could you please help log a Jira issue? Contributions are
> welcome!
> > > > > >>
> > > > > >> Pranav Deshpande  于2022年8月20日周六
> > > > 02:13写道：
> > > > > >>
> > > > > >>> Hi Team,
> > > > > >>> How can I convert a BindableTableScan with projects and Filters
> > > back
> > > > > to a
> > > > > >>> RelTree with a project node, a filter node and a tablescan
> node?
> > > > > >>>
> > > > > >>> I am doing this because I encountered the following issue

Re: How can I convert a BindableTableScan with projects and Fitlers back to a Rel Tree with a project node, a filter node and a tablescan node?

[Jiajun Xie]

Agree with Hyde, there are many problems when converting a  physical
RelNode directly to SQL. For example,
https://issues.apache.org/jira/browse/CALCITE-2915

In my project, I have an object  holding  Logical RelNode and Physical
RelNode. When I want to convert sql, I can use Logical RelNode. I hope it's
useful to you, Pranav.

On Sun, 21 Aug 2022 at 02:50, Julian Hyde  wrote:

> It doesn’t seem quite right to convert a physical RelNode directly to SQL.
> It’s probably better to map it back to logical RelNode(s) first.
>
> I don’t know whether we have the software to do that mapping. If we don’t,
> consider using planner rules. They’re often the right way in Calcite.
>
> Julian
>
> > On Aug 20, 2022, at 03:26, Pranav Deshpande <
> deshpande.v.pra...@gmail.com> wrote:
> >
> > Hi Bencaho,
> > Thank you very much for your reply. Could you please tell me the
> procedure
> > to create this issue on JIRA?
> >
> > Thanks & Regards,
> > Pranav
> >
> >> On Fri, Aug 19, 2022 at 11:13 PM Benchao Li 
> wrote:
> >>
> >> Pranav,
> >>
> >> This is a good question. To me, I would take this as a bug, and we could
> >> improve the
> >> RelToSqlConverter to treat BindableTableScan specially.
> >> Could you please help log a Jira issue? Contributions are welcome!
> >>
> >> Pranav Deshpande  于2022年8月20日周六 02:13写道：
> >>
> >>> Hi Team,
> >>> How can I convert a BindableTableScan with projects and Filters back
> to a
> >>> RelTree with a project node, a filter node and a tablescan node?
> >>>
> >>> I am doing this because I encountered the following issue (steps
> detailed
> >>> below).
> >>>
> >>> 1. I have a query:eg. Select colA, colB from myTable where colA > 1;
> >>> 2. The logical plan for this is a filter on top of a project on top of
> a
> >>> TableScan.
> >>> 3. If the table we created is a ProjectableFilterableTable and we have
> >>> added the ProjectTableScan rule etc., then after optimization we have a
> >>> single node which is a tablescan that contains filters and project and
> >> can
> >>> be executed by the bindable convention.
> >>> 4. If I convert this node to a SQL, I get the wrong SQL.
> >>>
> >>> To get a SQL, I do the following:
> >>>
> >>> RelToSqlConverter converter = new
> >>> RelToSqlConverter(SqlDialect.DatabaseProduct.CALCITE.getDialect());
> >>> SqlNode sqlNode = converter.visitRoot(rablescan).asStatement();
> >>>
> >>> This only gives me Select * from myTable;
> >>>
> >>> However, if I only have a ScannableTable and we haven't added the
> >>> ProjectTableScan rule etc. then for the physical plan we also get a
> >>> BindableFilter on top of a BindableProject on top of a
> BindableTablescan.
> >>>
> >>> If i convert this back using the same code then I get the correct
> >> statement
> >>> back:  Select colA, colB from myTable where colA > 1
> >>>
> >>> Hence I would like to know how to convert a BindableTableScan with
> >> projects
> >>> and Filters back to a RelTree with a project node, a filter node and a
> >>> tablescan node & would appreciate the community's help on the same.
> >>>
> >>> Thanks & Regards,
> >>> Pranav
> >>>
> >>
> >>
> >> --
> >>
> >> Best,
> >> Benchao Li
> >>
>

Re: How can I convert a BindableTableScan with projects and Fitlers back to a Rel Tree with a project node, a filter node and a tablescan node?

[Jiajun Xie]

Hi benchao,
  I want to fix the bug, can I log a jira issue?

On Sat, 20 Aug 2022 at 11:13, Benchao Li  wrote:

> Pranav,
>
> This is a good question. To me, I would take this as a bug, and we could
> improve the
> RelToSqlConverter to treat BindableTableScan specially.
> Could you please help log a Jira issue? Contributions are welcome!
>
> Pranav Deshpande  于2022年8月20日周六 02:13写道：
>
> > Hi Team,
> > How can I convert a BindableTableScan with projects and Filters back to a
> > RelTree with a project node, a filter node and a tablescan node?
> >
> > I am doing this because I encountered the following issue (steps detailed
> > below).
> >
> > 1. I have a query:eg. Select colA, colB from myTable where colA > 1;
> > 2. The logical plan for this is a filter on top of a project on top of a
> > TableScan.
> > 3. If the table we created is a ProjectableFilterableTable and we have
> > added the ProjectTableScan rule etc., then after optimization we have a
> > single node which is a tablescan that contains filters and project and
> can
> > be executed by the bindable convention.
> > 4. If I convert this node to a SQL, I get the wrong SQL.
> >
> > To get a SQL, I do the following:
> >
> > RelToSqlConverter converter = new
> > RelToSqlConverter(SqlDialect.DatabaseProduct.CALCITE.getDialect());
> > SqlNode sqlNode = converter.visitRoot(rablescan).asStatement();
> >
> > This only gives me Select * from myTable;
> >
> > However, if I only have a ScannableTable and we haven't added the
> > ProjectTableScan rule etc. then for the physical plan we also get a
> > BindableFilter on top of a BindableProject on top of a BindableTablescan.
> >
> > If i convert this back using the same code then I get the correct
> statement
> > back:  Select colA, colB from myTable where colA > 1
> >
> > Hence I would like to know how to convert a BindableTableScan with
> projects
> > and Filters back to a RelTree with a project node, a filter node and a
> > tablescan node & would appreciate the community's help on the same.
> >
> > Thanks & Regards,
> > Pranav
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: Is column aliasing supported?

[Jiajun Xie]

Hi, Broeder:
There's a tricky situation: alias may be the same as column name. Some
users often make mistakes in company that I worked, so I have to point out
their problems.

If you want to support alias in the WHERE clause, I hope you can consider
how to handle this situation.

On Sat, 20 Aug 2022 at 07:31, Sean Broeder  wrote:

>
>
> > On Aug 19, 2022, at 3:57 PM, Sean Broeder  wrote:
> >
> > Thanks for the clarification
> >
> >> On Aug 19, 2022, at 3:23 PM, Julian Hyde  > wrote:
> >>
> >> Column aliases are only available in the ORDER BY clause. In some
> dialect-compliance settings, they are also available in the GROUP BY and
> HAVING clauses. But not in the WHERE clause.
> >
> Hi Julian,
> It looks like some databases do support column aliasing in where clauses,
> for example, Terradata documentation indicates it does.
>
> https://docs.teradata.com/r/Teradata-Database-SQL-Fundamentals/June-2017/Basic-SQL-Syntax/Referencing-Object-Names-in-a-Request/Using-a-Column-Alias
>
> I am interested in this feature and would contribute an enhancement to
> Calcite if it’s possible.
>
> Do you happen to know if the current lack of support is due to technical
> difficulties/limitations or it simply hasn’t been implemented yet?
>
> Also, if the latter, do you have a pointer to where I might start looking?
>
> Thanks,
> Sen

Re: [ANNOUNCE] Andrei Sereda joins Calcite PMC

[Jiajun Xie]

Congratulations, Andrei!

On Sun, 14 Aug 2022 at 10:50, Benchao Li  wrote:

> Congratulations Andrei!
>
> Francis Chuang  于2022年8月13日周六 09:53写道：
>
> > Congratulations Andrei!
> >
> > On 13/08/2022 5:29 am, Michael Mior wrote:
> > > Congratulations and welcome Andrei!
> > >
> > > --
> > > Michael Mior
> > > mm...@apache.org
> > >
> > >
> > > On Fri, Aug 12, 2022 at 1:48 PM Ruben Q L  wrote:
> > >
> > >> I am pleased to announce that Andrei has accepted an invitation to
> join
> > >> the Calcite PMC. Andrei has been a consistent and helpful figure in
> > >> the Calcite community for which we are very grateful. We look forward
> to
> > >> the continued contributions and support.
> > >>
> > >> Please join me in congratulating Andrei!
> > >>
> > >> Ruben (on behalf of the Calcite PMC)
> > >>
> > >
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: Technical Queries Regarding Pushing Down Joins & Unions To TableScans, Conventions

[Jiajun Xie]

Hi, Pranav,
  You can send an email to dev-subscr...@calcite.apache.org to subscribe.

  Actually, This way is listed in https://calcite.apache.org/community/



On Sat, 6 Aug 2022 at 23:02, Pranav Deshpande 
wrote:

> Hi Julian,
> I apologize for the same.
>
> It was not my intention to do so. I very much wanted to acknowledge the
> replies, however, I could not do so as mentioned on my 2nd post:
> https://lists.apache.org/thread/27v6pymkm8qfxfkrf2pojl8w1bpmjwzc (I asked
> for instructions on replying back to the mailing list on this as well).
>
> Failing to get a reply for that, I had to send emails to the mailing list
> again (I tried subscribing but the system is demanding some kind of SSO
> login).
>
> Could you please tell me how to subscribe? I will then reply to the
> particular queries I had on each post.
>
> Thanks & Regards,
> Pranav
>
>
>
>
>
> On Fri, Aug 5, 2022, 2:06 PM Julian Hyde  wrote:
>
> > Pranav,
> >
> > Please subscribe to this list. You have asked several questions, received
> > replies, not acknowledged those replies, and asked further questions.
> Also,
> > since you are not subscribed, each email you post has to go through
> manual
> > moderation.
> >
> > Julian
> >
> > > On Aug 5, 2022, at 9:38 AM, Pranav Deshpande <
> > deshpande.v.pra...@gmail.com> wrote:
> > >
> > > Dear Apache Calcite Team,
> > > I have 2 questions.
> > >
> > > -
> > > 1.
> > >
> > > There are plenty of examples on how to push down projects and filters
> > into
> > > the leaf nodes (tablescans).
> > >
> > > However, I could not find any examples to push down joins to TableScans
> > (or
> > > joins+filters+projects etc.) [this is helpful for data federation I
> > think].
> > >
> > > On the mailing list, many folks are suggesting that I use Drill.
> However,
> > > the purpose of my exercise is to gain knowledge about DBMS and Query
> > > processing etc.
> > >
> > > I tried debugging open source engines that use Calcite (Drill, Druid,
> > Trino
> > > etc.) but was completely lost.
> > >
> > > Any examples/pointers/guidance around the same would be appreciated.
> > > Example, pushing down a join with a filter to a DBMS(consider jdbc
> > > msql etc.)
> > >
> > > -
> > > 2.
> > >
> > > The 2nd question I have is regarding conventions and different DBMS.
> The
> > > cluster has a method to replace the trait convention(Bindable,JDBC
> etc.),
> > > and then we optimize and get the physical plan.
> > >
> > > But imagine I have both the MYSQL JDBC convnction and a cassandra
> > > convention and some user is trying to query both tables.
> > >
> > > Something like "SELECT users.username, specialdata.country from
> > > cassandraDB.user join mysqlDB.specialdata ON users.id
> > =specialdata.userid"
> > >
> > > Now, how will calcite do the optimization here? The planner is not
> > > accepting 2 different conventions.
> > >
> > > Thanks & Regards,
> > > Pranav
> >
> >
>

Review request: Babel parser doesn't parse IF(condition, then, else) statements

[Jiajun Xie]

  `IF(condition, then, else)` is a common expression, babel parser failed
because CALCITE-3946 put `IF` into the keywords list.

  I added it to reservedFunctionNames, here is PR:
https://github.com/apache/calcite/pull/2835

Re: HELP !

[Jiajun Xie]

Hi, xiaochen~
  As mentioned in calcite-5225, you can add it to SqlLibraryOperators. If
you have many functions to add, create your SqlOperatorTable and use
ChainedSqlOperatorTable is a more suitable way. For example,
https://github.com/apache/calcite/blob/0e57722208d842a1192f38f28c2a379ae88dcdf0/testkit/src/main/java/org/apache/calcite/test/MockSqlOperatorTable.java#L87



  In addition, I noticed that you use core SQL parser instead of Babel SQL
parser. Babel SQL parser is used to accept common dialects such as Oracle,
MySQL, PostgreSQL, BigQuery.
  Here is babel parser config:
```
SqlParser.Config c = SqlParser.config()
.withConformance(SqlConformanceEnum.BABEL)
.withParserFactory(SqlBabelParserImpl.FACTORY);
```

Re: [Discuss] CommonSubExpressions Optimize

[Jiajun Xie]

Yes, you are both correct.

I got an expected plan by using ProjectToCalcRule
```
PLAN=EnumerableCalc(expr#0..4=[{inputs}], expr#5=[12], expr#6=[*($t5,
$t3)], expr#7=[3], expr#8=[*($t7, $t3)], expr#9=[*($t3, $t5)],
expr#10=[*($t3, $t7)], expr#11=[+($t9, $t10)], salary=[$t3], EXPR$1=[$t6],
EXPR$2=[$t8], EXPR$3=[$t11], EXPR$4=[$t11])\n
 EnumerableTableScan(table=[[hr, emps]])\n\n
```

But I got an unexpected  query when using RelToSql. I hope to extract
CommonSubExpressions in SQL, because Trino does not support
CommonSubExpressions optimization. (Presto supports it since 0.245).
```
SELECT "salary", 12 * "salary", 3 * "salary", "salary" * 12 + "salary" * 3,
"salary" * 12 + "salary" * 3
FROM "hr"."emps"
```

I'm thinking about using the optimization of Calc, but change CalcToSql.
What do you think?
FYI: In my project, I convert HiveSQL to PrestoSQL, and use some
optimizations of Calcite.

Thanks for your reply!

On Fri, 15 Jul 2022 at 09:44, Benchao Li  wrote:

> Agree with Julian, RexProgram could do this work.
>
> I did this before in physical codegen based on Calc's RexProgram, and it
> works very well.
>
> Julian Hyde  于2022年7月14日周四 23:21写道：
>
> > It seems that you are tackling common scalar expressions (RexNodes)
> > whereas, based on the name the other rule is dealing with common
> relational
> > expressions (RelNode).
> >
> > For your rule, consider using RexProgram, RexProgramBuilder, and Calc.
> > They already convert a list of expressions to a DAG, so that no
> expression
> > is calculated more than once.
> >
> > Julian
> >
> > > On Jul 14, 2022, at 06:31, Jiajun Xie 
> wrote:
> > >
> > > Hello, all~
> > >  I am writing an RelOptRule that is for CommonSubExpressions. Here is
> an
> > > example that I completed.
> > >sql:  `select sal, sal * 12, sal * 3, sal * 12 + sal * 3 from emp`
> > >planBefore: ```LogicalProject(SAL=[$5], EXPR$1=[*($5, 12)],
> > > EXPR$2=[*($5, 3)], EXPR$3=[+(*($5, 12), *($5, 3))])
> > >  LogicalTableScan(table=[[CATALOG, SALES, EMP]])```
> > >planAfter: ```LogicalProject(SAL=[$0], EXPR$1=[$1], EXPR$2=[$2],
> > > EXPR$3=[+($1, $2)])
> > > LogicalProject(SAL=[$5], CSE$0=[*($5, 12)], CSE$1=[*($5, 3)])
> > >   LogicalTableScan(table=[[CATALOG, SALES, EMP]])```
> > >  I found a RelOptRule that is named as CommonRelSubExprRule, but it is
> > > abstract. I wonder if anyone has implemented it? Is it the same as
> what I
> > > am doing?
> > >  Thanks very much~
> >
>
>
> --
>
> Best,
> Benchao Li
>

[Discuss] CommonSubExpressions Optimize

[Jiajun Xie]

Hello, all~
  I am writing an RelOptRule that is for CommonSubExpressions. Here is an
example that I completed.
sql:  `select sal, sal * 12, sal * 3, sal * 12 + sal * 3 from emp`
planBefore: ```LogicalProject(SAL=[$5], EXPR$1=[*($5, 12)],
EXPR$2=[*($5, 3)], EXPR$3=[+(*($5, 12), *($5, 3))])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])```
planAfter: ```LogicalProject(SAL=[$0], EXPR$1=[$1], EXPR$2=[$2],
EXPR$3=[+($1, $2)])
 LogicalProject(SAL=[$5], CSE$0=[*($5, 12)], CSE$1=[*($5, 3)])
   LogicalTableScan(table=[[CATALOG, SALES, EMP]])```
  I found a RelOptRule that is named as CommonRelSubExprRule, but it is
abstract. I wonder if anyone has implemented it? Is it the same as what I
am doing?
  Thanks very much~

[jira] [Created] (CALCITE-5205) Supports hint option as string and numeric literal

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5205:
---

 Summary: Supports hint option as string and numeric literal
 Key: CALCITE-5205
 URL: https://issues.apache.org/jira/browse/CALCITE-5205
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


 Spark support hint that options contain string and number:
{code:java}
SELECT /*+ REPARTITION(3, c) */ * FROM t {code}
But calcite can't parse it:
{code:java}
Error while parsing SQL: select /*+ repartition(3, empno) */ empno, ename, 
deptno from emps
java.lang.RuntimeException: Error while parsing SQL: select /*+ repartition(3, 
empno) */ empno, ename, deptno from emps {code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Jing Zhang

[Jiajun Xie]

Congratulations, Jing Zhang.

On Wed, 6 Jul 2022 at 03:42, Stamatis Zampetakis  wrote:

> Congratulations Jing, welcome to the team!
>
> Best,
> Stamatis
>
> On Tue, Jul 5, 2022, 10:29 AM Yanjing Wang 
> wrote:
>
> > Congratulations, Jing Zhang.
> >
> > Zhengqiang Duan  于2022年7月5日周二 15:46写道：
> >
> > > Congratulations to Jing Zhang.
> > >
> > > ——
> > > Zhengqaing Duan (duanzhengqi...@apache.org)
> > > Apache ShardingSphere Committer
> > > GitHub@strongduanmu
> > >
> > > Jing Zhang  于2022年7月4日周一 22:07写道：
> > >
> > > > Thanks everyone,
> > > >
> > > > Calcite is really an awesome project.
> > > > It's my honour to be a committer of the Calcite community.
> > > >
> > > > I'm now working in the kuaishou company which serves to record and
> > share
> > > > videos taken by all users.
> > > > Besides, I have been active in the Flink community for several years.
> > > > In Apache Flink, we're currently heavily using Calcite, including SQL
> > > > parser, validator and optimizer.
> > > > I would like to contribute to further communication and collaboration
> > > > between the Calcite and Flink communities.
> > > >
> > > > Best,
> > > > Jing Zhang
> > > >
> > > > Alessandro Solimando  于2022年7月4日周一
> > > > 14:43写道：
> > > >
> > > > > Congratulations, Jing!
> > > > >
> > > > > Best regards,
> > > > > Alessandro
> > > > >
> > > > > On Mon, 4 Jul 2022 at 07:42, Chunwei Lei 
> > > wrote:
> > > > >
> > > > > > Congratulations!
> > > > > >
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Chunwei
> > > > > >
> > > > > >
> > > > > > On Mon, Jul 4, 2022 at 1:31 PM Forward Xu <
> forwardxu...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations!
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > ForwardXu
> > > > > > >
> > > > > > > Benchao Li  于2022年7月4日周一 13:11写道：
> > > > > > >
> > > > > > > > Congratulations Jing!
> > > > > > > >
> > > > > > > > Francis Chuang  于2022年7月4日周一
> > 10:43写道：
> > > > > > > >
> > > > > > > > > Congrats!
> > > > > > > > >
> > > > > > > > > On 4/07/2022 12:42 pm, Haisheng Yuan wrote:
> > > > > > > > > > Apache Calcite's Project Management Committee (PMC) has
> > > invited
> > > > > > Jing
> > > > > > > > > Zhang
> > > > > > > > > > to
> > > > > > > > > > become a committer, and we are pleased to announce that
> she
> > > has
> > > > > > > > accepted.
> > > > > > > > > >
> > > > > > > > > > Since Dec 2017, Jing has been an active and continuous
> > > > > contributor
> > > > > > to
> > > > > > > > the
> > > > > > > > > > Apache
> > > > > > > > > > Calcite project. She has pushed high quality patches,
> > fixing
> > > > and
> > > > > > > > > improving
> > > > > > > > > > code
> > > > > > > > > > around streaming plan.
> > > > > > > > > >
> > > > > > > > > > Jing, welcome, thank you for your contributions, and we
> > look
> > > > > > forward
> > > > > > > to
> > > > > > > > > your
> > > > > > > > > > further interactions with the community! If you wish,
> > please
> > > > feel
> > > > > > > free
> > > > > > > > to
> > > > > > > > > > tell
> > > > > > > > > > us more about yourself and what you are working on.
> > > > > > > > > >
> > > > > > > > > > Haisheng (on behalf of the Apache Calcite PMC)
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Benchao Li
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Benchao Li

[Jiajun Xie]

Congrat Benchao Li,  Well deserved!

On Mon, 4 Jul 2022 at 09:21, Haisheng Yuan  wrote:

> Apache Calcite's Project Management Committee (PMC) has invited Benchao Li
> to
> become a committer, and we are pleased to announce that he has accepted.
>
> Benchao has pushed a lot of high quality patches, fixing and improving code
> around plan simplification and rules. Apart from code contributions, he has
> been regularly involved in the discussion and helping out others in the
> mailing
> list.
>
> Benchao, welcome, thank you for your contributions, and we look forward to
> your
> further interactions with the community! If you wish, please feel free to
> tell
> us more about yourself and what you are working on.
>
> Haisheng (on behalf of the Apache Calcite PMC)
>

Review request: Babel parser doesn't parse IF(condition, then, else) statement

[Jiajun Xie]

Babel parser doesn't parse IF(condition, then, else) statement because
babel puts `IF` into the keyword set.

I put `IF` into the reserved function name set to fix the issue:
https://issues.apache.org/jira/browse/CALCITE-4802

Welcome to comment. Here is PR:
https://github.com/apache/calcite/pull/2835/files
  Thanks very much~

Pivots with pivotAgg without alias fail with Babel Parser Implementation

[Jiajun Xie]

I'm trying to fix the bug that babel parser failed(core parser succeed), I
wonder if there is a more suitable solution.

Babel parser use the word after PivotAgg as an alias when the user does not
write an alias.
ISSUE: https://issues.apache.org/jira/browse/CALCITE-4746

I hope those who are familiar with babel parser can review it. Thank you
very much.
PR: https://github.com/apache/calcite/pull/2842/files

Introduction of newcomers

[jiajun xie]

Hi, everyone. My name is JiajunXie, I recently subscribed to the mail
list. I graduated from college last year, and now I work in ByteDance.
In the past year, I didn't subscribe to email but just sent messages in
JIRA or GitHub. Maybe I added to your burden, I'm sorry. After that,
welcome to communicate in the mail list

[jira] [Created] (CALCITE-5164) Planner#parser can't parse TIMESTAMP() function

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5164:
---

 Summary: Planner#parser can't parse TIMESTAMP() function
 Key: CALCITE-5164
 URL: https://issues.apache.org/jira/browse/CALCITE-5164
 Project: Calcite
  Issue Type: Bug
  Components: babel
Affects Versions: 1.30.0
Reporter: Jiajun Xie
Assignee: Jiajun Xie


Both core and babel will parse fail.
{code:java}
  FrameworkConfig coreConfig = Frameworks.newConfigBuilder().build();
  Planner corePlanner = Frameworks.getPlanner(coreConfig);
  corePlanner.parse("SELECT TIMESTAMP('2022-05-21 08:00:00'");
  // Caused by: org.apache.calcite.sql.parser.babel.ParseException: 
Incorrect syntax near the keyword 'TIMESTAMP' at line 1, column 8.

  FrameworkConfig babelConfig = Frameworks.newConfigBuilder()
  .parserConfig(SqlParser.Config.DEFAULT.withParserFactory(
  SqlBabelParserImpl.FACTORY))
  .build();
  Planner babelPlanner = Frameworks.getPlanner(babelConfig);
  babelPlanner.parse("SELECT TIMESTAMP('2022-05-21 08:00:00'");
  // Caused by: org.apache.calcite.sql.parser.babel.ParseException: 
Incorrect syntax near the keyword 'TIMESTAMP' at line 1, column 8
{code}
Here are some databases that support TIMESTAMP function.
 - MySQL: 
[https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_timestamp]

{code:java}
select timestamp('2022-05-21 08:00:00')
// result
timestamp('2022-05-21 08:00:00') 
2022-05-21 08:00:00
{code}
 - Derby: [https://docs.oracle.com/javadb/10.6.2.1/ref/rreftimestampfunc.html]

Also, here are some databases that not support TIMESTAMP function:
 - Oracle: 
[https://docs.oracle.com/cd/B19306_01/server.102/b14225/ch4datetime.htm]

{code:java}
select timestamp('2022-05-21 08:00:00')
//ORA-00923: FROM keyword not found where expected
{code}
 - SQL Server: 
[https://docs.microsoft.com/en-us/sql/t-sql/functions/date-and-time-data-types-and-functions-transact-sql?view=sql-server-ver15]

{code:java}
select timestamp('2022-05-21 08:00:00')
// Msg 195 Level 15 State 10 Line 1
// 'timestamp' is not a recognized built-in function name.{code}
Is it necessary for us to support it in babel module?



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (CALCITE-5141) Implicit type conversion incomplete for insert values

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5141:
---

 Summary: Implicit type conversion incomplete for insert values
 Key: CALCITE-5141
 URL: https://issues.apache.org/jira/browse/CALCITE-5141
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Jiajun Xie
Assignee: Jiajun Xie


- Target table schema
{code:java}
// Register "T1" table.
final MockTable t1 =
MockTable.create(this, tSchema, "T1", false, 7.0, null);
t1.addColumn("t1_varchar20", f.varchar20Type, true);
t1.addColumn("t1_smallint", f.smallintType);
t1.addColumn("t1_int", f.intType);
t1.addColumn("t1_bigint", f.bigintType);
t1.addColumn("t1_float", f.floatType);
t1.addColumn("t1_double", f.doubleType);
t1.addColumn("t1_decimal", f.decimalType);
t1.addColumn("t1_timestamp", f.timestampType);
t1.addColumn("t1_date", f.dateType);
t1.addColumn("t1_binary", f.binaryType);
t1.addColumn("t1_boolean", f.booleanType);
registerTable(t1);
{code}

 - Insert query
{code:java}
insert into t1 values 
('a', 1, 1.0, 0, 0, 0, 0, TIMESTAMP '2021-11-28 00:00:00', date '2021-11-28', 
x'0A', false), 
('b', 2,  2, 0, 0, 0, 0, TIMESTAMP '2021-11-28 00:00:00', date '2021-11-28', 
x'0A', false), 
('c', CAST(3 AS SMALLINT),  3.0, 0, 0, 0, 0, TIMESTAMP '2021-11-28 00:00:00', 
date '2021-11-28', x'0A', false), 
('d', 4, 4.0, 0, 0, 0, 0, TIMESTAMP '2021-11-28 00:00:00', date '2021-11-28', 
x'0A', false), 
('e', 5, 5.0, 0, 0, 0, 0, TIMESTAMP '2021-11-28 00:00:00', date '2021-11-28', 
x'0A', false)
{code}

 - Incorrect converted plan: the data type is not matched with table schema
{code:java}
LogicalTableModify(table=[[CATALOG, SALES, T1]], operation=[INSERT], 
flattened=[false])
  LogicalValues(tuples=[[
  { 'a', 1, 1, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
  { 'b', 2, 2, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
  { 'c', 3, 3.0, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
  { 'd', 4, 4.0, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
  { 'e', 5, 5.0, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false }]])
{code}

 - Correct converted plan

{code:java}
LogicalTableModify(table=[[CATALOG, SALES, T1]], operation=[INSERT], 
flattened=[false])
  LogicalValues(tuples=[[
{ 'a', 1, 1, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
 { 'b', 2, 2, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
 { 'c', 3, 3, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false },
 { 'd', 4, 4, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false }, 
{ 'e', 5, 5, 0, 0, 0, 0, 2021-11-28 00:00:00, 2021-11-28, X'0a', false }]])
{code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (CALCITE-5091) RelMdRowCount can return more accurate rowCount when fetch is deterministic and offset is dynamic

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5091:
---

 Summary: RelMdRowCount can return more accurate rowCount when 
fetch is deterministic and offset is dynamic
 Key: CALCITE-5091
 URL: https://issues.apache.org/jira/browse/CALCITE-5091
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.30.0
Reporter: Jiajun Xie
Assignee: Jiajun Xie


The sql from https://issues.apache.org/jira/browse/CALCITE-5048
{code:java}
select * from emp order by ename limit 1 offset ?
{code}

*RelMdRowCount#getRowCount* will return 14d, because it always return rowCount 
when offset is dynamic.
{code:java}
if (rel.offset instanceof RexDynamicParam) {
  return rowCount;
}
{code}

Obviously, rowCount will not exceed 1 because* LIMIT 1*. We can improve it.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (CALCITE-5013) Parentheses should be reserved when the operand of SetOp has limit or offset

[Jiajun Xie (Jira)]

Jiajun Xie created CALCITE-5013:
---

 Summary: Parentheses should be reserved when the operand of SetOp 
has limit or offset
 Key: CALCITE-5013
 URL: https://issues.apache.org/jira/browse/CALCITE-5013
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Jiajun Xie


- In standard SQL, the operand of union should not have limit  or order by.So 
parse will fail in SqlParserTetest#testLimitUnion and 
SqlParserTetest#OrderUnion.

 - When users use parentheses, most engines allow it. See the 
discussion:[CALCITE-1892|https://issues.apache.org/jira/browse/CALCITE-1892].   



--
This message was sent by Atlassian Jira
(v8.20.1#820001)