Re: Set of k-cheapest plans

[Fan Liya]

IMO, there is no easy way, and the algorithm should depends on definition
of 'alternative plans'.

In general, the algorithm can proceed like this:
1. we use the volcano algoirthm to find the best plan
2. we make the self cost of some node to be infinite, and then apply the
volcano algorithm again to find the second best plan
3. we repeat the above steps, until we find the k best plans.

Best,
Liya Fan

On Thu, Mar 19, 2020 at 3:20 AM Julian Hyde  wrote:

> There’s no easy way. You could modify ‘buildCheapestPlan()’ to build all
> plans below a cost limit. (You’d have to carefully choose a cost limit only
> a little above the optimal cost, otherwise you’ll get huge numbers of
> plans.)
>
> I fear that you’ll get plans that are different in only trivial or minor
> respects (e.g. ordering of items in a Project) whereas you probably want
> plans with significant differences (e.g. different join orders).
>
> Julian
>
> > On Mar 18, 2020, at 12:01 PM, Rui Souto  wrote:
> >
> > Hi there!
> >
> > Just recently started to learn about Apache Calcite. What's the best way
> to
> > get a list of the k-cheapest alternative plans generated by the optimizer
> > for a given query? (being k an arbitrary number)
>
>

Apply for contribute permission

[Fan Liya]

Hi all,

I want to contribute to calcite, could you please give me the contribution
permission?
My jira id is fan_li_ya (Liya Fan).

Thank you!
Liya Fan

Re: Apply for contribute permission

[Fan Liya]

Thank you so much, Francis.

Best,
Liya Fan

On Mon, Apr 13, 2020 at 12:23 PM Francis Chuang 
wrote:

> Hey,
>
> I've added you as a contributor in jira.
>
> Francis
>
> On 13/04/2020 1:20 pm, Fan Liya wrote:
> > Hi all,
> >
> > I want to contribute to calcite, could you please give me the
> contribution
> > permission?
> > My jira id is fan_li_ya (Liya Fan).
> >
> > Thank you!
> > Liya Fan
> >
>

Calcite translates in-subqueries to inner joins

[Fan Liya]

Hi all,

Given a SQL query with an in subquery:

select a.name
from dept a
where a.deptno in (
select b.deptno * 2
from dept b)

Calcite translates it to an inner join, but the correct way should be
translating it to a left semi join.

According to the comments in the code, it seems this behavior is a design
choice.
So can you please help explain the rationale behind this? Do we have a plan
to fix it?

Best,
Liya Fan

Re: Calcite translates in-subqueries to inner joins

[Fan Liya]

Hi Stamatis,

Thanks a lot for the information.
If nobody is working on it, may I start to fix it? It is blocking an issue
we are working on.

Best,
Liya Fan

On Mon, Apr 13, 2020 at 9:51 PM Stamatis Zampetakis 
wrote:

> Hi Liya,
>
> Regarding this choice I guess you can have a look in the following
> discussions [1, 2].
> I think there is consensus that we should change this behavior in
> SqlToRelConverter and generate semi/anti joins directly when this is
> possible but I think that nobody is working on it at the moment.
>
> Best,
> Stamatis
>
> [1]
>
> https://lists.apache.org/thread.html/ra7d7a309592d327a5e912ded7d541c79a3c587fa2daba950cc463762%40%3Cdev.calcite.apache.org%3E
> [2]
>
> https://lists.apache.org/thread.html/f77bf1f946b8c026989163294a2134e485fe7897201c479bbb528c6d%40%3Cdev.calcite.apache.org%3E
>
> On Mon, Apr 13, 2020 at 2:11 PM Fan Liya  wrote:
>
> > Hi all,
> >
> > Given a SQL query with an in subquery:
> >
> > select a.name
> > from dept a
> > where a.deptno in (
> > select b.deptno * 2
> > from dept b)
> >
> > Calcite translates it to an inner join, but the correct way should be
> > translating it to a left semi join.
> >
> > According to the comments in the code, it seems this behavior is a design
> > choice.
> > So can you please help explain the rationale behind this? Do we have a
> plan
> > to fix it?
> >
> > Best,
> > Liya Fan
> >
>

Re: The Travis CI build failure

[Fan Liya]

According to my observations, many failures are caused by SocketTimeOut
exceptions.
So maybe we need to add timeout & retry logic for such test cases.

Best,
Liya Fan

On Mon, Apr 20, 2020 at 11:13 AM Francis Chuang 
wrote:

> Does the Github Actions tests always pass when this happens?
>
> I wonder if this is due to a network issue or a corrupt gradle cache on
> the travis nodes.
>
> See: https://github.com/facebook/react-native/issues/21130
>
> On 20/04/2020 12:58 pm, XING JIN wrote:
> > I have force pushed several times, but it keeps failing.
> >
> > Chunwei Lei  于2020年4月20日周一 上午10:49写道：
> >
> >> I am a little confused too. Sometimes the traivs ci build of master
> fails
> >> when I merge
> >> the pr which passes all tests.
> >>
> >>
> >>
> >> Best,
> >> Chunwei
> >>
> >>
> >> On Mon, Apr 20, 2020 at 10:33 AM XING JIN 
> wrote:
> >>
> >>> In PR https://github.com/apache/calcite/pull/1926 and
> >>> https://github.com/apache/calcite/pull/1925, The Travis CI build keeps
> >>> failing with below messages. The exception has nothing to do with the
> >>> change in PR. There seems to be something wrong with the building
> >>> environment.
> >>>
> >>> ...
> >>>
> >>>
> >>
> /home/travis/build/apache/calcite/spark/src/main/java/org/apache/calcite/adapter/spark/HttpServer.java:42:
> >>> error: cannot access java.lang
> >>> class HttpServer {
> >>> ^
> >>>zip END header not found
> >>>
> >>>
> >>
> /home/travis/build/apache/calcite/spark/src/main/java/org/apache/calcite/adapter/spark/HttpServer.java:43:
> >>> error: cannot find symbol
> >>>private static String localIpAddress;
> >>>   ^
> >>>symbol:   class String
> >>>location: class HttpServer
> >>>
> >>>
> >>
> /home/travis/build/apache/calcite/spark/src/main/java/org/apache/calcite/adapter/spark/HttpServer.java:45:
> >>> error: cannot find symbol
> >>>private final File resourceBase;
> >>>  ^
> >>>symbol:   class File
> >>>location: class HttpServer
> >>> ...
> >>>
> >>> Jin
> >>>
> >>
> >
>

Re: [ANNOUNCE] New committer: Vineet Garg

[Fan Liya]

Congratulations, Vineet!

Best,
Liya Fan

On Sun, Apr 26, 2020 at 10:30 AM Aman Sinha  wrote:

> Congratulations Vineet !
>
> -Aman
>
> On Sat, Apr 25, 2020 at 7:15 PM Feng Zhu  wrote:
>
> >  Congratulations, well deserved!
> >
> > best,
> > Feng
> >
> > Chunwei Lei  于2020年4月26日周日 上午10:12写道：
> >
> > > Congrats, Vineet!
> > >
> > >
> > > Best,
> > > Chunwei
> > >
> > >
> > > On Sun, Apr 26, 2020 at 8:24 AM Haisheng Yuan 
> wrote:
> > >
> > > > Congrats, Vineet!
> > > >
> > > > On 2020/04/25 22:18:35, Forward Xu  wrote:
> > > > > Congratulations
> > > > >
> > > > > best,
> > > > > Forward
> > > > >
> > > > > Francis Chuang  于2020年4月26日周日 上午6:04写道：
> > > > >
> > > > > > Congrats, Vineet!
> > > > > >
> > > > > > On 26/04/2020 7:52 am, Stamatis Zampetakis wrote:
> > > > > > > Apache Calcite's Project Management Committee (PMC) has invited
> > > > Vineet
> > > > > > > Garg to become a committer, and we are pleased to announce that
> > he
> > > > > > > has accepted.
> > > > > > >
> > > > > > > With the first code contribution in Calcite back in 2017,
> Vineet
> > is
> > > > > > > definitely
> > > > > > > not new to the project. Since then he has contributed many
> > patches,
> > > > > > > fixing and improving various modules of Calcite, notably things
> > > > around
> > > > > > > subqueries.
> > > > > > >
> > > > > > > Vineet, welcome, thank you for your contributions, and we look
> > > > forward
> > > > > > > your further interactions with the community! If you wish,
> please
> > > > feel
> > > > > > > free to tell us more about yourself and what you are working
> on.
> > > > > > >
> > > > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Re: [ANNOUNCE] New committer: Wang Yanlin

[Fan Liya]

Congratulations, Yanlin!

Best,
Liya Fan

On Wed, Apr 29, 2020 at 8:50 PM Wang Yanlin <1989yanlinw...@163.com> wrote:

> Hi, guys, thanks for your warm welcome.
>
>
>
> I'm working in Ant Finical, Alibaba  Group. Currently my team is working
> on building a system to process big data in form of sql.
> We use calcite to parse sql, optimize Relnode and rewrite SqlNode to
> execute on different engines, like Spark，MaxCompute, HBase and so on.
> Calcite is really a great community, and it's really an honor for me to
> become calcite committer, hops to make more contribution to calcite.
>
>
> Thanks again.
>
> --
>
> Best,
> Wang Yanlin
>
>
>
>
>
> 在 2020-04-29 13:58:35，"Zoltan Haindrich"  写道：
> >Congratulations!
> >
> >On 4/29/20 7:32 AM, Enrico Olivelli wrote:
> >> Congrats!
> >>
> >> Enrico
> >>
> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu  ha scritto:
> >>
> >>>   Congrations! Yanlin!
> >>>
> >>> best,
> >>> Feng
> >>>
> >>> Chunwei Lei  于2020年4月29日周三 上午10:16写道：
> >>>
>  Congrats, Yanlin!
> 
> 
>  Best,
>  Chunwei
> 
> 
>  On Wed, Apr 29, 2020 at 10:07 AM Forward Xu 
>  wrote:
> 
> > Congrats
> >
> >
> > Best,
> >
> > Forward
> >
> > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道：
> >
> >> Congrats, Wang Yanlin!
> >>
> >>
> >>
> >>
> >> ---Original---
> >> From: "Stamatis Zampetakis" >> Date: Wed, Apr 29, 2020 05:51 AM
> >> To: "dev" >> Subject: [ANNOUNCE] New committer: Wang Yanlin
> >>
> >>
> >> Apache Calcite's Project Management Committee (PMC) has invited Wang
> > Yanlin
> >> to
> >> become a committer, and we are pleased to announce that he has
>  accepted.
> >>
> >> Wang has pushed numerous fixes and improvements to the project,
> >>> landing
> > in
> >> total
> >> the impressive number of 30 commits to the master. Among other
> >>> things,
>  he
> >> contributed some important features in the Interpreter.
> >>
> >> Wang, welcome, thank you for your contributions, and we look forward
>  your
> >> further interactions with the community! If you wish, please feel
> >>> free
>  to
> >> tell
> >> us more about yourself and what you are working on.
> >>
> >> Stamatis (on behalf of the Apache Calcite PMC)
> >
> 
> >>>
> >>
>

Re: Re: [ANNOUNCE] New committer: Forward Xu

[Fan Liya]

Congratulations, Forward!

Best,
Liya Fan

On Wed, Apr 29, 2020 at 8:51 PM Wang Yanlin <1989yanlinw...@163.com> wrote:

>  Congrations!  Forward!--
>
> Best,
> Wang Yanlin
>
>
>
>
>
> At 2020-04-29 10:52:25, "Feng Zhu"  wrote:
> > Congrations! Forward!
> >
> >best,
> >Feng
> >
> >Chunwei Lei  于2020年4月29日周三 上午10:17写道：
> >
> >> Congrats, Forward!
> >>
> >>
> >>
> >> Best,
> >> Chunwei
> >>
> >>
> >> On Wed, Apr 29, 2020 at 6:46 AM Rui Wang  wrote:
> >>
> >> > Congrats!
> >> >
> >> >
> >> > -Rui
> >> >
> >> > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang <
> francischu...@apache.org
> >> >
> >> > wrote:
> >> >
> >> > > Congrats, Forward!
> >> > >
> >> > > Francis
> >> > >
> >> > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote:
> >> > > > Apache Calcite's Project Management Committee (PMC) has invited
> >> Forward
> >> > > Xu
> >> > > > to
> >> > > > become a committer, and we are pleased to announce that he has
> >> > accepted.
> >> > > >
> >> > > > Forward has been helping the project for some time now. He added
> many
> >> > new
> >> > > > SQL
> >> > > > functions to the project and is one of our JSON experts. On top of
> >> > that,
> >> > > and
> >> > > > other fixes, he is the one who added the Redis adapter to the
> >> project.
> >> > > >
> >> > > > Forward, welcome, thank you for your contributions, and we look
> >> forward
> >> > > to
> >> > > > your
> >> > > > further interactions with the community! If you wish, please feel
> >> free
> >> > to
> >> > > > tell
> >> > > > us more about yourself and what you are working on.
> >> > > >
> >> > > > Stamatis (on behalf of the Apache Calcite PMC)
> >> > > >
> >> > >
> >> >
> >>
>

Re: Re: [ANNOUNCE] New committer: Jin Xing

[Fan Liya]

Congratulations, Jin!

Best,
Liya Fan

On Wed, Apr 29, 2020 at 8:51 PM Wang Yanlin <1989yanlinw...@163.com> wrote:

> Congrats, Jin Xing!
>
>
> --
>
> Best,
> Wang Yanlin
>
>
>
>
>
> 在 2020-04-29 13:58:53，"Zoltan Haindrich"  写道：
> >Congrats!
> >
> >On 4/29/20 7:32 AM, Enrico Olivelli wrote:
> >> Congratulations!
> >>
> >> Enrico
> >>
> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu  ha scritto:
> >>
> >>>   Congrations!
> >>>
> >>> best,
> >>> Feng
> >>>
> >>> Chunwei Lei  于2020年4月29日周三 上午10:16写道：
> >>>
>  Congrats, Jin!
> 
> 
>  Best,
>  Chunwei
> 
> 
>  On Wed, Apr 29, 2020 at 10:07 AM Forward Xu 
>  wrote:
> 
> > Congrats
> >
> >
> > best,
> >
> > Forward
> >
> > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道：
> >
> >> Congrats, Jin Xing!
> >>
> >>
> >> ---Original---
> >> From: "Stamatis Zampetakis" >> Date: Wed, Apr 29, 2020 05:47 AM
> >> To: "dev" >> Subject: [ANNOUNCE] New committer: Jin Xing
> >>
> >>
> >> Apache Calcite's Project Management Committee (PMC) has invited Jin
>  Xing
> > to
> >> become a committer, and we are pleased to announce that he has
>  accepted.
> >>
> >> Jin has contributed a lot of code in the project and many
> >> recent improvements in
> >> materialized view matching have his signature on them. Apart from
> >>> code
> >> contributions, Jin provides valuable help to the community by doing
> > reviews
> >> and
> >> answering questions in the devlist.
> >>
> >> Jin, welcome, thank you for your contributions, and we look forward
> >>> to
> > your
> >> further interactions with the community! If you wish, please feel
> >>> free
>  to
> >> tell
> >> us more about yourself and what you are working on.
> >>
> >> Stamatis (on behalf of the Apache Calcite PMC)
> >
> 
> >>>
> >>
>

Re: [DISCUSS] Sarg (search argument) to generalize and replace IN in RexCall

[Fan Liya]

Hi Julian,

Thanks for opening the discussion.

In general, I am convinced of the value and importance of the proposal.

I want to discuss more about the algebra involved, as the simplified range
sets may not have the minimum computational costs.
Some examples:

1. Suppose we have expression x in (1, 2, 4, 5, 6).

The range set implementation may reduce it to (x >= 1 and x <= 2) or (x >=
4 and x <= 6), which represents 4 comparisons and 3 logical operations.
Another equivalent expression is x = 1 or x = 2 or x = 4 or x = 5 or x = 6,
which represents 5 comparisons and 4 logical operations.
It's hard to say which one has a smaller computational cost.

2. Suppose we have expression x in (1, 2, 4, 5)
The range set implementation may reduce it to (x >= 1 and x <= 2) or (x >=
4 and x <= 5).
Another (possibly cheaper) reduction should be x >= 1 and x <= 5 and x != 3.
I am not sure if the range set implementation supports this type of
simplification.

Therefore, to address above problems,
1. We may need a model to estimate the cost, and the simplification process
should be guided by the cost model.
2. Maybe we need to extend the range set implementation so it produces
expressions with minimum computational cost.

Best,
Liya Fan





On Mon, Aug 10, 2020 at 3:27 AM Julian Hyde  wrote:

> We have had several discussions over the years about how to represent
> IN-lists (e.g. “x IN (1, 3, 5)”) in RexNode-land.
>
> I have generally taken the position that we should expand to ORs (e.g. “x
> = 1 OR x = 3 OR x = 5”) but a few months ago accepted that we should allow
> IN in RexCall.
>
> I have given this some further thought, as part of a couple of RexSimplify
> bugs [1] [2] and I now think we should replace IN with something more
> powerful, namely sargs. A sarg (or search argument [3]) is an ordered set
> of intervals, and can be represented as a Guava ImmutableRangeSet, such as
> "[[0‥1), (1‥2], [3‥3], (5‥+∞)]". It can represent an IN-list of constants,
> but also ranges.
>
> Today you would write
>
>   RexCall(IN, RexInputRef(0), RexLiteral(1), RexLiteral(3), RexLiteral(5))
>
> With sargs, you would instead write
>
>   RexCall(IN_SARG, RexInputRef(0), RexSarg(ImmutableRangeSet(“[[1..1],
> [3..3], [5..5]]”)))
>
> There is a new operator IN_SARG, and a new node type RexSarg (it could
> almost be a RexLiteral).
>
> Sargs (and Guava RangeSets) have a powerful and consistent algebra, so if
> we invest in sarg support in RexSimplify and RelOptPredicateList, that
> investment is likely to pay dividends in better plans.
>
> Guava RangeSets, and hence sargs, have support for discrete domains, so
> they can easily optimize ">2 AND <4" to "3”.
>
> Sargs would be the preferred (therefore canonical) form for any AND or OR
> list that has more than one comparison on the same operand (e.g. "x > 3 AND
> x < 17 AND x <> 10”).
>
> This proposal would subsume IN and therefore we would stop supporting IN
> in RexCall.
>
> Julian
>
> [1] https://issues.apache.org/jira/browse/CALCITE-4155 <
> https://issues.apache.org/jira/browse/CALCITE-4155>
>
> [2] https://github.com/julianhyde/calcite/tree/4159-simplify <
> https://github.com/julianhyde/calcite/tree/4159-simplify>
>
> [3] https://en.wikipedia.org/wiki/Sargable <
> https://en.wikipedia.org/wiki/Sargable>

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Fan Liya]

Congratulations, Ruben!

Best,
Liya Fan


On Fri, Aug 14, 2020 at 2:49 AM Julian Hyde  wrote:

> Thanks for your hard work on the project and in the community, Ruben. Well
> deserved.
>
> Julian
>
>
> > On Aug 13, 2020, at 6:19 AM, Thomas Rebele 
> wrote:
> >
> > Congrats, Ruben!
> >
> > Cordialement / Best Regards,
> > *Thomas Rebele* | R&D Developer | 18 rue du 4 septembre, 75002 Paris,
> France
> > | www.tibco.com
> >
> >
> > On Thu, Aug 13, 2020 at 12:54 AM Francis Chuang <
> francischu...@apache.org>
> > wrote:
> >
> >> Congratulations, Ruben!
> >>
> >> Francis
> >>
> >> On 13/08/2020 3:43 am, Andrei Sereda wrote:
> >>> Congratulations, Ruben!
> >>>
> >>> On Wed, Aug 12, 2020 at 1:21 PM Rui Wang  wrote:
> >>>
>  Congrats, Ruben! Well deserved!
> 
> 
> 
>  -Rui
> 
>  On Wed, Aug 12, 2020 at 9:24 AM Enrico Olivelli 
>  wrote:
> 
> > Congrats Ruben!
> >
> > Enrico
> >
> > Il Mer 12 Ago 2020, 18:05 Michael Mior  ha
> scritto:
> >
> >> Congrats Reuben!
> >>
> >> --
> >> Michael Mior
> >> mm...@apache.org
> >>
> >> Le mer. 12 août 2020 à 09:32, Ruben Q L  a
> écrit :
> >>>
> >>> Thanks everyone!
> >>> It is an honor to join the Calcite PMC.
> >>>
> >>> Best regards,
> >>> Ruben
> >>>
> >>>
> >>> Le mer. 12 août 2020 à 03:03, Forward Xu 
> a
> >> écrit :
> >>>
>  Congrats, Ruben!
> 
> 
>  Best
> 
>  Forward
> 
>  XING JIN  于2020年8月12日周三 上午8:58写道：
> 
> > Congrats, Ruben!
> >
> > 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
> >
> >> Congratulations, Ruben!
> >>
> >>
> >> xzh
> >> -- 原始邮件 --
> >> 发件人:
> >>   "dev"
> >>
>   <
> >> zabe...@gmail.com>;
> >> 发送时间: 2020年8月12日(星期三) 凌晨5:53
> >> 收件人: "dev" >>
> >> 主题: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> >>
> >>
> >>
> >> I'm pleased to announce that Ruben has accepted an invitation
>  to
> >> join the Calcite PMC. Ruben has been a consistent and helpful
> >> figure in the Calcite community for which we are very grateful.
> > We
> >> look forward to the continued contributions and support.
> >>
> >> Please join me in congratulating Ruben!
> >>
> >> - Stamatis (on behalf of the Calcite PMC)
> >
> 
> >>
> >
> 
> >>>
> >>
>
>

Re: [ANNOUNCE] New committer: Rui Wang

[Fan Liya]

Congratulations, Rui!

Best,
Liya Fan

On Thu, Sep 10, 2020 at 9:45 AM Danny Chan  wrote:

> Congrats Rui!
>
> Best,
> Danny Chan
>
>
>
> > 在 2020年9月10日，上午6:28，Francis Chuang  写道：
> >
> > Congrats Rui!
>
>

Re: Disabling JaninoRelMetadataProvider

[Fan Liya]

Hi Enrico,

We have also observed the problem.

Janino compilation is time consuming. For each single Java class, it takes
tens of milli-seconds.
Moreover, the compilation will take place multiple times, because:
1. We have multiple types of metadata.
2. The class for a particular type of metadata may be regenerated and
recompiled, because a rel node was not registered at first.

We opened CALCITE-3901 and provided a PR [1] to address problem 2 by
reducing the number of compilations. However, so far it has not been
accepted yet.

To solve problem 1, I think we have two potential solutions:
1. Provide a metadata provider that is not based on codegen and Janino
compilation (Obviously, this will be a long term plan).
2. Save the generated Java class, and compile it with the client code, and
change the implementation of JaninoMetaDataProvider#load3 so that if a
handler can be found in the current class loader, then it skips the codegen
and compilation.

Best,
Liya Fan






[1] https://github.com/apache/calcite/pull/1901



On Fri, Sep 18, 2020 at 3:01 PM Enrico Olivelli  wrote:

> Hi,
> I am seeing that JaninoRelMetadataProvider is quite expensive, at least for
> the "boostrap" phase of the system.
>
> It is a cool piece of software and it is working well, but in some cases
> probably it could be quite overkill, for instance in test cases of
> applications that mostly run only one query and then end.
>
> You could argue that the price of the compilation is paid only once and
> there is no need to complicate things for some corner cases that probably
> do not affect production cases.
>
> In CALCITE-3901 we introduced calcite.enable.regenerate.metadata.handler
> system property in order to limit the number of times that Janino kicks in
> but not to drop it at all.
>
> From the code I see that it is a subclass of RelMetadataProvider, but there
> is no way to get rid of it in VolcanoPlanner.
>
> Also, Janino is a third party library, and having the ability of dropping
> it will help in having a smaller set of dependencies downstream (but this
> is not blocker at the moment, it is only a nice-to-have)
>
> Do you have any experience with this problem ? Is there any chance to add
> some configuration option to pass a RelMetadataProvider implementation
> that does not rely on dynamic code generation ?
> If the idea is valuable I will be happy to work on it.
>
> See this issue for reference, with a full stacktrace of the execution
> https://github.com/diennea/herddb/issues/517
>
> Regards
> Enrico
>

Re: Disabling JaninoRelMetadataProvider

[Fan Liya]

Hi Julian,

Thank you for the feedback and good suggestions.

The idea of lazy load is reasonable. However, the testing of this feature
is still tricky, because:
1. When we have changed the flag, all subsequent test cases are all also
affected (many components of JaninoRelMetadataProvider are declared as
static, which is used by all test cases in the same JVM, so it is difficult
to revert the changes made by a single test case).
2.  All test cases may be run in parallel, so when we change the flag,
other test cases running concurrently may fail. (The
JaninoRelMetadataProvider object is a singleton, which is shared by all
threads).

I have updated the PR [1] to implement the idea of lazy load. Could someone
please take a look?

Best,
Liya Fan

[1] https://github.com/apache/calcite/pull/1901

On Sat, Sep 19, 2020 at 1:50 AM Julian Hyde  wrote:

> CALCITE-3901 looks like a reasonable approach. I like the idea that it
> would throw and tell you the classes and metadata types that you have
> missed.
>
> The packaging in 3901 isn’t great. I don’t believe that the test needs to
> read a system property and not even via the CalciteSystemProperty
> mechanism.  We can do better.
>
> Maybe we have a class name and arguments for the metadata provider, which
> is loaded on demand. Will work for the test and also in production.
>
> Julian
>
> > On Sep 18, 2020, at 12:57 AM, Fan Liya  wrote:
> >
> > Hi Enrico,
> >
> > We have also observed the problem.
> >
> > Janino compilation is time consuming. For each single Java class, it
> takes
> > tens of milli-seconds.
> > Moreover, the compilation will take place multiple times, because:
> > 1. We have multiple types of metadata.
> > 2. The class for a particular type of metadata may be regenerated and
> > recompiled, because a rel node was not registered at first.
> >
> > We opened CALCITE-3901 and provided a PR [1] to address problem 2 by
> > reducing the number of compilations. However, so far it has not been
> > accepted yet.
> >
> > To solve problem 1, I think we have two potential solutions:
> > 1. Provide a metadata provider that is not based on codegen and Janino
> > compilation (Obviously, this will be a long term plan).
> > 2. Save the generated Java class, and compile it with the client code,
> and
> > change the implementation of JaninoMetaDataProvider#load3 so that if a
> > handler can be found in the current class loader, then it skips the
> codegen
> > and compilation.
> >
> > Best,
> > Liya Fan
> >
> >
> >
> >
> >
> >
> > [1] https://github.com/apache/calcite/pull/1901
> >
> >
> >
> >> On Fri, Sep 18, 2020 at 3:01 PM Enrico Olivelli 
> wrote:
> >>
> >> Hi,
> >> I am seeing that JaninoRelMetadataProvider is quite expensive, at least
> for
> >> the "boostrap" phase of the system.
> >>
> >> It is a cool piece of software and it is working well, but in some cases
> >> probably it could be quite overkill, for instance in test cases of
> >> applications that mostly run only one query and then end.
> >>
> >> You could argue that the price of the compilation is paid only once and
> >> there is no need to complicate things for some corner cases that
> probably
> >> do not affect production cases.
> >>
> >> In CALCITE-3901 we introduced calcite.enable.regenerate.metadata.handler
> >> system property in order to limit the number of times that Janino kicks
> in
> >> but not to drop it at all.
> >>
> >> From the code I see that it is a subclass of RelMetadataProvider, but
> there
> >> is no way to get rid of it in VolcanoPlanner.
> >>
> >> Also, Janino is a third party library, and having the ability of
> dropping
> >> it will help in having a smaller set of dependencies downstream (but
> this
> >> is not blocker at the moment, it is only a nice-to-have)
> >>
> >> Do you have any experience with this problem ? Is there any chance to
> add
> >> some configuration option to pass a RelMetadataProvider implementation
> >> that does not rely on dynamic code generation ?
> >> If the idea is valuable I will be happy to work on it.
> >>
> >> See this issue for reference, with a full stacktrace of the execution
> >> https://github.com/diennea/herddb/issues/517
> >>
> >> Regards
> >> Enrico
> >>
>

Re: Disabling JaninoRelMetadataProvider

[Fan Liya]

Hi Julian,

Thanks for your good suggestion.
The idea of delayed freeze sounds reasonable to me, and it provides more
flexibility.
I also like the idea of an output report which is helpful for performance
diagnostics.

Best,
Liya Fan


On Fri, Sep 25, 2020 at 7:39 AM Julian Hyde  wrote:

> Would it make sense to delay the 'freeze'? E.g. receive 1,000 requests
> (as measured using an AtomicInteger) and then freeze the
> JaninoRelMetadataProvider. That way, tests would not be competing with
> each other.
>
> Similarly, in production, you could create a production instance of
> JaninoRelMetadataProvider with what you believe to be a good list of
> classes and metadata. The first request that asks for a class or
> metadata will not fail. But when you have gathered, say, 100 missing
> classes (and therefore re-generated 100 times), at that point it can
> output a list of the classes that were missing from the initial
> config.
>
> Julian
>
> On Mon, Sep 21, 2020 at 4:53 AM Fan Liya  wrote:
> >
> > Hi Julian,
> >
> > Thank you for the feedback and good suggestions.
> >
> > The idea of lazy load is reasonable. However, the testing of this feature
> > is still tricky, because:
> > 1. When we have changed the flag, all subsequent test cases are all also
> > affected (many components of JaninoRelMetadataProvider are declared as
> > static, which is used by all test cases in the same JVM, so it is
> difficult
> > to revert the changes made by a single test case).
> > 2.  All test cases may be run in parallel, so when we change the flag,
> > other test cases running concurrently may fail. (The
> > JaninoRelMetadataProvider object is a singleton, which is shared by all
> > threads).
> >
> > I have updated the PR [1] to implement the idea of lazy load. Could
> someone
> > please take a look?
> >
> > Best,
> > Liya Fan
> >
> > [1] https://github.com/apache/calcite/pull/1901
> >
> > On Sat, Sep 19, 2020 at 1:50 AM Julian Hyde 
> wrote:
> >
> > > CALCITE-3901 looks like a reasonable approach. I like the idea that it
> > > would throw and tell you the classes and metadata types that you have
> > > missed.
> > >
> > > The packaging in 3901 isn’t great. I don’t believe that the test needs
> to
> > > read a system property and not even via the CalciteSystemProperty
> > > mechanism.  We can do better.
> > >
> > > Maybe we have a class name and arguments for the metadata provider,
> which
> > > is loaded on demand. Will work for the test and also in production.
> > >
> > > Julian
> > >
> > > > On Sep 18, 2020, at 12:57 AM, Fan Liya  wrote:
> > > >
> > > > Hi Enrico,
> > > >
> > > > We have also observed the problem.
> > > >
> > > > Janino compilation is time consuming. For each single Java class, it
> > > takes
> > > > tens of milli-seconds.
> > > > Moreover, the compilation will take place multiple times, because:
> > > > 1. We have multiple types of metadata.
> > > > 2. The class for a particular type of metadata may be regenerated and
> > > > recompiled, because a rel node was not registered at first.
> > > >
> > > > We opened CALCITE-3901 and provided a PR [1] to address problem 2 by
> > > > reducing the number of compilations. However, so far it has not been
> > > > accepted yet.
> > > >
> > > > To solve problem 1, I think we have two potential solutions:
> > > > 1. Provide a metadata provider that is not based on codegen and
> Janino
> > > > compilation (Obviously, this will be a long term plan).
> > > > 2. Save the generated Java class, and compile it with the client
> code,
> > > and
> > > > change the implementation of JaninoMetaDataProvider#load3 so that if
> a
> > > > handler can be found in the current class loader, then it skips the
> > > codegen
> > > > and compilation.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > [1] https://github.com/apache/calcite/pull/1901
> > > >
> > > >
> > > >
> > > >> On Fri, Sep 18, 2020 at 3:01 PM Enrico Olivelli <
> eolive...@gmail.com>
> > > wrote:
> > > >>
> > > >> Hi,
> > > >> I am seeing that JaninoRelMetadataProvider is quite expensi

Re: [DISCUSS][CALCITE-4199] Ensure nullness verification via checkerframework

[Fan Liya]

Sounds great! Thanks for your effort.
Does it require installing something in the developer's machine?

Best,
Liya Fan

On Sun, Sep 27, 2020 at 7:10 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Here's a sample issue:
>
> core/src/main/java/org/apache/calcite/sql/SqlBinaryOperator.java:161:
> error: [dereference.of.nullable] dereference of possibly-null reference
> call.getOperandLiteralValue(1, BigDecimal.class)
>   switch (call.getOperandLiteralValue(1,
> BigDecimal.class).signum()) {
>  ^
> The code in question is
>
> @Override public SqlMonotonicity getMonotonicity(SqlOperatorBinding call) {
>   if (getName().equals("/")) {
> final SqlMonotonicity mono0 = call.getOperandMonotonicity(0);
> final SqlMonotonicity mono1 = call.getOperandMonotonicity(1);
> if (mono1 == SqlMonotonicity.CONSTANT) {
>   if (call.isOperandLiteral(1, false)) {
> switch (call.getOperandLiteralValue(1, BigDecimal.class).signum())
> {
> case -1:
>
>
> The error message suggests that the test for 1/null is missing, and it
> turns out it is a true bug:
>
> @Test void testMonotonic() {
>   sql("select stream 1/null from orders")
>   .monotonic(SqlMonotonicity.CONSTANT);
>
>
> java.lang.NullPointerException
> at
>
> org.apache.calcite.sql.SqlBinaryOperator.getMonotonicity(SqlBinaryOperator.java:161)
> at org.apache.calcite.sql.SqlCall.getMonotonicity(SqlCall.java:193)
> at
>
> org.apache.calcite.sql.validate.SelectScope.getMonotonicity(SelectScope.java:159)
> at
>
> org.apache.calcite.sql.validate.SelectNamespace.getMonotonicity(SelectNamespace.java:76)
> at
>
> org.apache.calcite.sql.test.AbstractSqlTester.checkMonotonic(AbstractSqlTester.java:497)
> at
>
> org.apache.calcite.test.SqlValidatorTestCase$Sql.monotonic(SqlValidatorTestCase.java:373)
>
> Vladimir
>

DISCUSS: the concept of duplicate insensitive aggregate functions

[Fan Liya]

Hi all,

I would like to introduce the idea of duplicate insensitive aggregate
functions.

For such functions, the aggregation results remain the same even after
deduplication.

For example, given a sequence of data {1, 1, 2, 2, 3, 5, 5}, the
aggregation results of MIN are the same regardless of whether we perform
data deduplication first. That is,

MIN({1, 1, 2, 2, 3, 5, 5}) = MIN({1, 2, 3, 5})

So MIN is a *deduplicate insensitive function*.

On the other hand, function SUM is not duplicate insensitive, because

 SUM({1, 1, 2, 2, 3, 5, 5}) != SUM({1, 2, 3, 5})

The concept of deduplicate insensitiveness can help us in many optimization
scenarios.

For example, the curent implementation of AggregateMergeRule rules out any
aggregate calls for which the isDistict() method returns true. However, for
duplicate insensitive functions, the rule should be applicable.

Could you please give your valuable feedback?

Best,
Liya Fan

Re: DISCUSS: the concept of duplicate insensitive aggregate functions

[Fan Liya]

Hi Julian,

Thanks a lot for your feedback.
I think SqlAggFunction.getDistinctOptionality() is exactly what I
am looking for.

BTW, I think ANY_VALUE and SINGLE_VALUE also belong to the category of
duplicate insensitive functions.
What do you think?

Best,
Liya Fan



On Tue, Oct 13, 2020 at 4:55 PM Julian Hyde  wrote:

> We already have this concept. See SqlAggFunction.getDistinctOptionality(),
> added in https://issues.apache.org/jira/browse/CALCITE-3159 <
> https://issues.apache.org/jira/browse/CALCITE-3159>.
>
> Julian
>
>
> > On Oct 13, 2020, at 12:54 AM, Fan Liya  wrote:
> >
> > Hi all,
> >
> > I would like to introduce the idea of duplicate insensitive aggregate
> > functions.
> >
> > For such functions, the aggregation results remain the same even after
> > deduplication.
> >
> > For example, given a sequence of data {1, 1, 2, 2, 3, 5, 5}, the
> > aggregation results of MIN are the same regardless of whether we perform
> > data deduplication first. That is,
> >
> > MIN({1, 1, 2, 2, 3, 5, 5}) = MIN({1, 2, 3, 5})
> >
> > So MIN is a *deduplicate insensitive function*.
> >
> > On the other hand, function SUM is not duplicate insensitive, because
> >
> > SUM({1, 1, 2, 2, 3, 5, 5}) != SUM({1, 2, 3, 5})
> >
> > The concept of deduplicate insensitiveness can help us in many
> optimization
> > scenarios.
> >
> > For example, the curent implementation of AggregateMergeRule rules out
> any
> > aggregate calls for which the isDistict() method returns true. However,
> for
> > duplicate insensitive functions, the rule should be applicable.
> >
> > Could you please give your valuable feedback?
> >
> > Best,
> > Liya Fan
>
>

Re: DISCUSS: the concept of duplicate insensitive aggregate functions

[Fan Liya]

Hi Julian,

Thanks again for your feedback.

Since they are duplicate-insensitive, they should also be splittable
(SqlSplittableAggFunction), just like min/max, etc.
What do you think?

I want to fire a JIRA accordingly, so that more optimizations can be
applied.
Any feedback is appreciated.

Best,
Liya Fan



On Wed, Oct 14, 2020 at 2:59 AM Julian Hyde  wrote:

> I agree. ANY_VALUE and SINGLE_VALUE are duplicate-insensitive.
>
> > On Oct 13, 2020, at 2:17 AM, Fan Liya  wrote:
> >
> > Hi Julian,
> >
> > Thanks a lot for your feedback.
> > I think SqlAggFunction.getDistinctOptionality() is exactly what I
> > am looking for.
> >
> > BTW, I think ANY_VALUE and SINGLE_VALUE also belong to the category of
> > duplicate insensitive functions.
> > What do you think?
> >
> > Best,
> > Liya Fan
> >
> >
> >
> > On Tue, Oct 13, 2020 at 4:55 PM Julian Hyde 
> wrote:
> >
> >> We already have this concept. See
> SqlAggFunction.getDistinctOptionality(),
> >> added in https://issues.apache.org/jira/browse/CALCITE-3159 <
> >> https://issues.apache.org/jira/browse/CALCITE-3159>.
> >>
> >> Julian
> >>
> >>
> >>> On Oct 13, 2020, at 12:54 AM, Fan Liya  wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I would like to introduce the idea of duplicate insensitive aggregate
> >>> functions.
> >>>
> >>> For such functions, the aggregation results remain the same even after
> >>> deduplication.
> >>>
> >>> For example, given a sequence of data {1, 1, 2, 2, 3, 5, 5}, the
> >>> aggregation results of MIN are the same regardless of whether we
> perform
> >>> data deduplication first. That is,
> >>>
> >>> MIN({1, 1, 2, 2, 3, 5, 5}) = MIN({1, 2, 3, 5})
> >>>
> >>> So MIN is a *deduplicate insensitive function*.
> >>>
> >>> On the other hand, function SUM is not duplicate insensitive, because
> >>>
> >>> SUM({1, 1, 2, 2, 3, 5, 5}) != SUM({1, 2, 3, 5})
> >>>
> >>> The concept of deduplicate insensitiveness can help us in many
> >> optimization
> >>> scenarios.
> >>>
> >>> For example, the curent implementation of AggregateMergeRule rules out
> >> any
> >>> aggregate calls for which the isDistict() method returns true. However,
> >> for
> >>> duplicate insensitive functions, the rule should be applicable.
> >>>
> >>> Could you please give your valuable feedback?
> >>>
> >>> Best,
> >>> Liya Fan
> >>
> >>
>
>

Re: DISCUSS: the concept of duplicate insensitive aggregate functions

[Fan Liya]

Hi Julian,

Please see my comments inline:

> I suspect that any duplicate-insensitive function is very easily
> splittable. Especially ones that are their own rollup (which is true
> of min, max, any_value, single_value).

I agree. However, the opposite is not true. For example, COUNT can be
split into a COUNT and a SUM, but COUNT is not duplicate-insensitive.

> If getDistinctOptionality() returns IGNORED, does the author of the
> function need to write any further code to ensure that calls can be
> split?

Yes. We need to override the T unwrap(Class clazz) method, and
make it return an instance of SqlSplittableAggFunction.

> Lastly, I'll note that other systems (e.g. Algebird [1]) allow you to
> assign aggregate functions to algebraic structures - for example, sum
> and hyperLogLog are both monoids [2] and can therefore be 'rolled up'.
> We could do the same in Calcite. We could perhaps allow aggregate
> functions to declare themselves as belonging to a particular algebraic
> structure, and we could exploit the properties of those structures to
> perform optimizations.

Thanks for the information. It is super cool!
I would be glad to see algebraic structures introduced to Calcite, and
strongly believe that they will help us in more optimization scenarios.

Best,
Liya Fan

On Wed, Oct 14, 2020 at 1:19 PM Julian Hyde  wrote:

> I suspect that any duplicate-insensitive function is very easily
> splittable. Especially ones that are their own rollup (which is true
> of min, max, any_value, single_value).
>
> If getDistinctOptionality() returns IGNORED, does the author of the
> function need to write any further code to ensure that calls can be
> split?
>
> Lastly, I'll note that other systems (e.g. Algebird [1]) allow you to
> assign aggregate functions to algebraic structures - for example, sum
> and hyperLogLog are both monoids [2] and can therefore be 'rolled up'.
> We could do the same in Calcite. We could perhaps allow aggregate
> functions to declare themselves as belonging to a particular algebraic
> structure, and we could exploit the properties of those structures to
> perform optimizations.
>
> Julian
>
> [1] https://twitter.github.io/algebird/
>
> [2] https://en.wikipedia.org/wiki/Monoid
>
> On Tue, Oct 13, 2020 at 7:26 PM Fan Liya  wrote:
> >
> > Hi Julian,
> >
> > Thanks again for your feedback.
> >
> > Since they are duplicate-insensitive, they should also be splittable
> > (SqlSplittableAggFunction), just like min/max, etc.
> > What do you think?
> >
> > I want to fire a JIRA accordingly, so that more optimizations can be
> > applied.
> > Any feedback is appreciated.
> >
> > Best,
> > Liya Fan
> >
> >
> >
> > On Wed, Oct 14, 2020 at 2:59 AM Julian Hyde 
> wrote:
> >
> > > I agree. ANY_VALUE and SINGLE_VALUE are duplicate-insensitive.
> > >
> > > > On Oct 13, 2020, at 2:17 AM, Fan Liya  wrote:
> > > >
> > > > Hi Julian,
> > > >
> > > > Thanks a lot for your feedback.
> > > > I think SqlAggFunction.getDistinctOptionality() is exactly what I
> > > > am looking for.
> > > >
> > > > BTW, I think ANY_VALUE and SINGLE_VALUE also belong to the category
> of
> > > > duplicate insensitive functions.
> > > > What do you think?
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > >
> > > > On Tue, Oct 13, 2020 at 4:55 PM Julian Hyde 
> > > wrote:
> > > >
> > > >> We already have this concept. See
> > > SqlAggFunction.getDistinctOptionality(),
> > > >> added in https://issues.apache.org/jira/browse/CALCITE-3159 <
> > > >> https://issues.apache.org/jira/browse/CALCITE-3159>.
> > > >>
> > > >> Julian
> > > >>
> > > >>
> > > >>> On Oct 13, 2020, at 12:54 AM, Fan Liya 
> wrote:
> > > >>>
> > > >>> Hi all,
> > > >>>
> > > >>> I would like to introduce the idea of duplicate insensitive
> aggregate
> > > >>> functions.
> > > >>>
> > > >>> For such functions, the aggregation results remain the same even
> after
> > > >>> deduplication.
> > > >>>
> > > >>> For example, given a sequence of data {1, 1, 2, 2, 3, 5, 5}, the
> > > >>> aggregation results of MIN are the same regardless of whether we
> > > perform
> > > >>> data deduplication first. That is,
> > > >>>
> > > >>> MIN({1, 1, 2, 2, 3, 5, 5}) = MIN({1, 2, 3, 5})
> > > >>>
> > > >>> So MIN is a *deduplicate insensitive function*.
> > > >>>
> > > >>> On the other hand, function SUM is not duplicate insensitive,
> because
> > > >>>
> > > >>> SUM({1, 1, 2, 2, 3, 5, 5}) != SUM({1, 2, 3, 5})
> > > >>>
> > > >>> The concept of deduplicate insensitiveness can help us in many
> > > >> optimization
> > > >>> scenarios.
> > > >>>
> > > >>> For example, the curent implementation of AggregateMergeRule rules
> out
> > > >> any
> > > >>> aggregate calls for which the isDistict() method returns true.
> However,
> > > >> for
> > > >>> duplicate insensitive functions, the rule should be applicable.
> > > >>>
> > > >>> Could you please give your valuable feedback?
> > > >>>
> > > >>> Best,
> > > >>> Liya Fan
> > > >>
> > > >>
> > >
> > >
>

Re: Why ProjectJoinTransposeRule not in Calcite default rule set.

[Fan Liya]

I also think the ProjectJoinTransposeRule should be included in the main
optimizer process.

Best,
Liya Fan

On Fri, Oct 23, 2020 at 10:43 AM JiaTao Tao  wrote:

> This rule is only used in RelOptMaterializations#substitute, is not in
> Calcite's main optimizer process(RelOptUtil#registerDefaultRules), wonder
> why, seems a common rule.
>
>
> Regards!
>
> Aron Tao
>

Re: How to parse sql of DDL

[Fan Liya]

Hi Xiao,

It's hard to diagnose the problem without seeing your code.
My suggestion is that you can debug some test cases in our code base
(related to SqlParser), and check how it works.

Best,
Liya Fan


On Wed, Nov 4, 2020 at 10:34 AM xiao cai  wrote:

> Hi
> I just want to konw how to parse the sql statement of DDL. I had use
> SqlParser to do this, but it was failed.
> Is someone can help me , thank you very mush.
>
>
> Best xiao.

Re: How to parse sql of DDL

[Fan Liya]

Hi Xiao,

I am not sure if we support "alter table ... add column ... after ..."
statement, as some databases do not support this syntax.
We do support "alter table ... add column ..." statement. An example can be
found in MultiJdbcSchemaJoinTest#testSchemaConsistency, which may be
helpful.

Best,
Liya Fan

On Wed, Nov 4, 2020 at 10:53 AM xiao cai  wrote:

> Hi Liya：
> Thanks for your suggestion！
>
>
> Here is the demo i ran using SqlParser:
>
>
> --
> String sql = "ALTER TABLE `test`.`mysql_sink_test_1` ADD COLUMN
> `col_for_test` varchar(255) NULL AFTER `col_binary_1`;
>  SqlParser.Config mysqlConfig =
> SqlParser.configBuilder().setLex(Lex.MYSQL).build();
>  SqlParser sqlParser = SqlParser.create(sql.toLowerCase(), mysqlConfig);
>  SqlNode sqlNode = sqlParser.parseStmt();
> --
>
>
> Then I received the following error message：
>
>
> -
> org.apache.calcite.sql.parser.SqlParseException: Encountered "table" at
> line 1, column 7.
> Was expecting one of:
> "SESSION" ...
> "SYSTEM" ...
> at
> org.apache.calcite.sql.parser.impl.SqlParserImpl.convertException(SqlParserImpl.java:364)
> at
> org.apache.calcite.sql.parser.impl.SqlParserImpl.normalizeException(SqlParserImpl.java:149)
> at
> org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:148)
> at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:163)
> at org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:188)
> ———
>
>
>
>
>
>
>  原始邮件
> 发件人: Fan Liya
> 收件人: dev
> 发送时间: 2020年11月4日(周三) 10:40
> 主题: Re: How to parse sql of DDL
>
>
> Hi Xiao, It's hard to diagnose the problem without seeing your code. My
> suggestion is that you can debug some test cases in our code base (related
> to SqlParser), and check how it works. Best, Liya Fan On Wed, Nov 4, 2020
> at 10:34 AM xiao cai  wrote: > Hi > I just want to konw
> how to parse the sql statement of DDL. I had use > SqlParser to do this,
> but it was failed. > Is someone can help me , thank you very mush. > > >
> Best xiao.

Re: [DISCUSS] Make SqlNodeList implement List<SqlNode>

[Fan Liya]

IMO, the major advantage of Iterable over List is that we do not need to
know all the elements before traversing the collection.
According to the current implementation, however, SqlNodeList is internally
backed by a List, so the advantage of Iterable no longer exists.
Therefore, I think it is more natural to make it implement List.

Best,
Liya Fan


On Thu, Nov 5, 2020 at 11:44 AM Haisheng Yuan  wrote:

> The performance gain depends on the size of sqlnode list.
> I am in favor of making SqlNodeList implement List.
> +1 to the proposal.
>
> Haisheng Yuan
>
> On 2020/11/05 02:32:01, Chunwei Lei  wrote:
> > Thank you for raising this, Julian.
> >
> > How much performance improvement can we get?
> >
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Thu, Nov 5, 2020 at 5:32 AM Julian Hyde  wrote:
> >
> > > Currently class SqlNodeList [1] implements Iterable but it
> > > does not implement List. How do people feel about doing that?
> > > (Or Collection?)
> > >
> > > The main potential benefit is performance. Consider this code:
> > >
> > >   SqlNodeList nodeList;
> > >   ImmutableList immutableList1 = ImmutableList.copyOf(nodeList);
> > >
> > >   List list = nodeList.toList();
> > >   ImmutableList immutableList2 = ImmutableList.copyOf(list);
> > >
> > > Today, the second form is faster, because ImmutableList.copyOf can
> > > call size() and preallocate a list of the right size.
> > >
> > > The second benefit is that we can remove calls to '.toList()'.
> > >
> > > The downside is that a few locations are overloaded. For example,
> > > class Span has overloaded methods
> > >
> > >   static Span of(SqlNode node);
> > >   static Span of(Collection nodes)
> > >
> > > If SqlNodeList implements List, then the code
> > >
> > >   SqlNodeList nodeList;
> > >   Span s = Span.of(nodeList);
> > >
> > > becomes ambiguous. We can disambiguate by adding Span.of(SqlNodeList).
> > > But there may be other locations in client code that cannot be
> > > disambiguated.
> > >
> > > Julian
> > >
> > > [1]
> > >
> https://github.com/apache/calcite/blob/155276591288615c4d02d55fb7d77eceb2e24b2d/core/src/main/java/org/apache/calcite/sql/SqlNodeList.java#L38
> > >
> >
>

Re: [HELP] NPE on WebSphere 9 when starting application using calcite library (potential circular dependency in class initialization)

[Fan Liya]

We have encountered similar problems in other projects (please see
https://issues.apache.org/jira/browse/ARROW-9315).
The problem is not always reproducible, because it depends on the
particular order of class loading.
That means other systems using different versions of JVM may potentially be
affected by the problem.

Our solution was to change some class initialization task to be lazily
performed. FYI.

Best,
Liya Fan

On Fri, Nov 6, 2020 at 8:20 AM Ruben Q L  wrote:

> Thanks for the answers.
> Indeed, WebSphere uses IBM's own JVM, and my hypothesis is that its class
> loader follows a different (and unlucky) order to load Calcite classes.
> I will continue digging into it, see if I can isolate the problem, and open
> a Jira if there is in fact something to be addressed on Calcite's side.
> Ruben
>
>
> On Fri, Nov 6, 2020 at 12:10 AM Julian Hyde 
> wrote:
>
> > As you’re using Websphere, it’s possible you’re using IBM’s Java runtime.
> > Is that so?
> >
> > Julian
> >
> >
> > > On Nov 5, 2020, at 2:23 PM, Stamatis Zampetakis 
> > wrote:
> > >
> > > Hi Ruben,
> > >
> > > Since you mentioned WebSphere I am wondering if it is related to some
> > weird
> > > problem in the ClassLoader.
> > > If it was only a JVM problem I guess it could be reproduced outside of
> > > WebSphere just by referencing a few Calcite classes in some specific
> > order.
> > >
> > > Other than that, it is better to use JIRA when you have code and
> > > stacktraces; the formatting is helpful.
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Thu, Nov 5, 2020 at 10:00 AM Ruben Q L  wrote:
> > >
> > >> Hello everyone,
> > >>
> > >> I'm getting a strange error on WebSphere 9 (on Ubuntu) when trying to
> > start
> > >> an application that uses Calcite (shaded) library. The exception
> message
> > >> says:
> > >>
> > >> ** error [-]: Unexpected error.
> > >> **   ...
> > >> **   Caused by: java.lang.BootstrapMethodError:
> > >> java.lang.ExceptionInInitializerError
> > >> **
> > >>
> > >>
> >
> com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
> > >> **
> > >>
> > >>
> >
> com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
> > >> **
> > >>
> > >>
> >
> com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
> > >> **
> > >>
> > >>
> >
> com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
> > >> **   java.lang.Class.forNameImpl(Native Method)
> > >> **   java.lang.Class.forName(Class.java:348)
> > >> **
> > >>
> > >>
> >
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> > >> **
> > >> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> > >> **
> > >>
> >
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> > >> **
>  java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> > >> Method)
> > >> **
> > >>
> > java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
> > >> **   ... 74 more
> > >> **   Caused by: java.lang.ExceptionInInitializerError
> > >> **
> > >> java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
> > >> **
> > >>
> >
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
> > >> **   java.lang.Class.forNameImpl(Native Method)
> > >> **   java.lang.Class.forName(Class.java:348)
> > >> **
> > >>
> > >>
> >
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> > >> **
> > >> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> > >> **
> > >>
> >
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> > >> **
>  java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> > >> Method)
> > >> **
> > >>
> > java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
> > >> **   ... 85 more
> > >> **   Caused by: java.lang.NullPointerException
> > >> **
> > >>
> > >>
> >
> com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
> > >> **   ... 92 more
> > >> ** 1 errors, 5 infos.
> > >>
> > >>
> > >> As we can see there is a NPE in SqlCallBinding:60, when it tries to
> > call a
> > >> method on SqlStdOperatorTable.DEFAULT, which seems to be null [0].
> > >> According to the logs, what I think is happening is a circular
> > dependency
> > >> issue when classes get loaded [1].
> > >> However, so far I have only seen this error in WebSphere, in other
> > >> environments the same application is correctly initialized and
> launched,
> > >> and this exception does not happen.
> > >>
> > >> Has anyone ever seen an issue like this?
> > >> Thanks,
> > >> Ruben
> > >>
> > >> WebSphere version is:
> > >> Integrated Solutions Console, 9.0.0.1
> > >> Build Number: cf011636.02
> > >> Build Date: 9/6/16
> > >>
> > >> It uses java 8:
> > >> java version "1.8.0"

Re: [DISCUSS] State of the project 2020

[Fan Liya]

Hi Stamatis,

Thanks for your great work!
I am feeling lucky to have you as our chair in 2020!

Concerning the problem of PR reviewing, one method that comes to my mind is
to divide Calcite into a few sub-areas, and assign some owners to each
sub-area (based on code contribution, etc.). So once a PR is submitted, the
author could request review from the corresponding sub-area owners (github
could help with this as well), and the owner can decide to
review/triage/close the PR.  In this way, the responsibility for reviewers
could be made clearer.

+1 for voting Haisheng as our next chair. I believe he will be a great
chair!

Best,
Liya Fan


On Mon, Nov 9, 2020 at 7:30 AM Francis Chuang 
wrote:

> Hey Stamatis,
>
> Thanks for putting this together for 2020. Thank you for being our chair
> for 2020 and the excellent work you have done in this capacity.
>
> I agree that reviewing pull requests is still an issue with the project
> and while we have made a lot of progress it this area, it's still
> something we need to work on and improve.
>
> I am also glad that you raised the point regarding Avatica. I've been
> meaning to write a message to the list to see if there are things we can
> do to help Avatica as I've had some concerns about it for a while:
> - Avatica seems to be quite mature and the code quite stable.
> - There are 14 open PRs at the moment, so not a huge amount, but they
> don't seem to be reviewed although Danny and Josh has picked a few out
> to review during the year.
> - To an outside observer, it looks like the project is not under active
> development, other than the occasional commit every once in a while.
> - I believe Avatica is still a critical component for other projects,
> Phoenix is one that comes to mind, however, PRs not being reviewed can
> discourage potential contributors.
> - As there are only 14 open PRs at the moment, I think it would be
> really great if the community could spare some cycles and merge, review
> and close out those PRs. Quite a few of them have been hanging around
> for a few years and if they are no longer relevant, I think they should
> be closed.
> - I'd definitely love to see more contributors to Avatica in terms of
> reviewing PRs and merging. Does the community have any suggestions on
> how we can do this better?
>
> +1 for proposing Haisheng Yuan as our next chair, I think he will be an
> excellent choice!
>
> Francis
>
> On 5/11/2020 9:26 am, Stamatis Zampetakis wrote:
> > Hi Calcite community members,
> >
> > A bit more than five years ago (October 22, 2015) Calcite graduated to a
> > top-level Apache project[1]. At that time, the community decided to have
> an
> > annual “state of the project” discussion and to vote for a new PMC
> > chair/VP[2]. So, I’m kicking off both of those discussions.
> >
> > I think it was an excellent year so far in many aspects.
> >
> > We were lucky to have many high quality contributions including: notable
> > improvements in the Volcano planner (for speed, plan quality,
> > extensibility) bringing it a bit closer to Cascades and Columbia [6, 7,
> 8,
> > 9]; easier and more extensible parameterization of rules [3]; new
> dialects
> > such as ClickHouse [4], and Presto [5]; support for SQL hints [10]; new
> > adapters for querying Redis [11] and InnoDB [12] through Calcite; various
> > enhancements in streaming SQL. The previous list is by no means
> exhaustive.
> >
> > Apart from the new features, certainly worth mentioning is the
> > modernization of the build and test infrastructure (for both Calcite and
> > Avatica), with the migration from maven to gradle, JUnit4 to JUnit5, and
> > the introduction of GitHub actions as part of the CI.
> >
> > In terms of CI, I am happy to see a few more integration tests (IT)
> running
> > on a regular basis on GitHub. Eventually, it will be nice to have even
> more
> > IT tests to help us catch regressions early on and improve the quality of
> > our releases.
> >
> > We wouldn’t have so many great contributions, if we didn’t also have
> > prolific contributors.
> > Our community has grown with Danny, Haisheng, Ruben, joining the PMC,
> > Forward, Xing, Vineet, Yanlin, Feng, Rui, joining as committers, and many
> > more people chiming in discussions, reviews, and submitting pull
> requests,
> > who are not yet committers but I’m sure some of whom will become in the
> > near future.
> >
> > We have had five Calcite releases (1.22.0 to 1.26.0), one Avatica release
> > (1.17.0), and one Avatica Go (5.0.0) so far in 2020, and I think that is
> a
> > great tempo that we should strive to maintain in the years to come. One
> > thing to improve is the poor implication of other people than Francis on
> > the Avatica side; the rest of us, putting myself first, should try to be
> > more involved by reviewing PRs, preparing releases, voting etc.
> >
> > It was nice to see our community members giving talks to conferences such
> > as ApacheCon, and Flink Forward presenting Calcite and/or its
> 

Re: [DISCUSS][CALCITE-4199] Ensure nullness verification via checkerframework

[Fan Liya]

  Hi Vladimir,

Thanks for bringing this up. In general, I believe annotations for
nullability are beneficial for code readability and reasoning.
However, after reading the PR, I am having the same feelings as Nikolay.

Intuitively, the "@Nullable" annotation does not provide much information,
and it does not make the code more readable (IMO).

In addition, is it possible to break the PR into a number of small ones? A
large PR is difficult to review and difficult to merge.

Best,
Liya Fan

On Thu, Nov 19, 2020 at 5:38 AM Julian Hyde  wrote:

> Regardless of how it got there, the change to
> 2020-10-06-release-1.26.0.md is currently in the change that you are
> proposing to commit (see the top of
> https://patch-diff.githubusercontent.com/raw/apache/calcite/pull/2268.patch
> ),
> and it shouldn't be.
>
> Julian
>
> On Wed, Nov 18, 2020 at 1:12 PM Vladimir Sitnikov
>  wrote:
> >
> > Julian>Please remove the change you made to
> > Julian>site/_posts/2020-10-06-release-1.26.0.md in this PR
> >
> > Julian, I rebase checkerframework branch from time to time, and the
> commit
> > you mention was on master branch.
> >
> > I'm all ears if you have questions or suggestions regarding the checker
> > framework.
> >
> > Vladimir
>

Re: [DISCUSS][CALCITE-4199] Ensure nullness verification via checkerframework

[Fan Liya]

Hi Vladimir,

Thanks for your effort. I still believe in the value of nullable
annotations.

Please see my comments below

> I'm puzzled. First you say "annotations are beneficial for code
> readability", then
> you proceed with "annotations does not make the code more readable".

For ordinary Java code, variables are nullable by default, and only a few
variables are non-nullable (I think).
But Calcite is different, as most variables are non-nullable (as you have
indicated).
If so, it would be reasonable to annotate nullable variables only.
However, I think it would be nice if we could document it somewhere, so
other developers will follow the convention without being confused.

> It won't be difficult to merge. Git works just fine.

I mean that a large PR requires a long time to review.
During this time, you have to resolve conflicts many times, which is a
large amount of effort.
This is not a problem, of course, if you do not mind it.

Best,
Liya Fan


On Fri, Nov 20, 2020 at 7:35 AM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Liya Fan>is it possible to break the PR into a number of small ones?
>
> I can split the commit in two for review-only purposes: a commit that
> refactors the code, and the second commit that adds @Nullable annotations.
> Here you go: https://github.com/apache/calcite/pull/2273 (see
> [CALCITE-4199] Refactor: prepare code for nullability annotations)
> The "refactor" commit is still big though (757 files changed, 7487
> insertions(+), 3827 deletions(-))
>
> ---
>
> I tried logging findings as individual issues under CALCITE-4199, however,
> that resulted in quite an unfriendly "stop doing that" feedback from Julian
> (see [1]).
> * Nobody expressed "please create more of these"
> * It takes me time to create each ticket
> * That is why I stopped creating tickets and now I perform the refactoring
> as I add nullability annotations
>
> I would be glad if Julian admits the nullness checker finds useful bugs
> (e.g. see npeTest in [2], [3], etc).
> It would be great if Julian refrains from false allegations against me
> (e.g. "Vladimir has been trying to fix things that aren’t broken". "He did
> not leave time for anyone to review", etc, etc).
>
> I have provided countless samples when the nullness checker detects true
> bugs, and the detected bugs are both in old and in the recently added code.
> For instance, the recent discussion on SEARCH vs nullness (see [4]) was
> triggered by myself trying to rebase nullness annotations on top of Calcite
> 1.26.
> Search/Sarg failed nullness in a non-trivial way, and I asked Julian to fix
> the code.
>
> [1]:
>
> https://lists.apache.org/thread.html/rf2ee66ad385624a35829a4ce092ff13797e812e03abb80cb45b33421%40%3Cdev.calcite.apache.org%3E
> [2]:
>
> https://lists.apache.org/thread.html/rf863922cf4418f994a123ecbbb624a74e6e375aa421a4a7812ce52b0%40%3Cdev.calcite.apache.org%3E
> [3]: https://issues.apache.org/jira/browse/CALCITE-4234
> [4]:
>
> https://lists.apache.org/thread.html/rce6e84d0e0cd5a227d64c9f670b43650833f43e0cbccb80e04f5e0cf%40%3Cdev.calcite.apache.org%3E
>
> Vladimir
>

Re: [HELP] Local build fails

[Fan Liya]

Thanks for opening the discussion.
I am having the same problem, and my Java version is jdk1.8.0_201.

The problem can be reproduced by the simple command

gradlew build

I have tried to find a solution by Googling, but so far no solution has
worked for me.

Best,
Liya Fan


On Sat, Dec 5, 2020 at 3:11 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Please clarify the problem and the steps to reproduce it.
>
> Vladimir
>

Re: [HELP] Local build fails

[Fan Liya]

Thanks for the advice.
I have tried jdk1.8.0_271 and jdk1.9 (build 9+181), but the problem is
still there.

Best,
Liya Fan

On Mon, Dec 7, 2020 at 1:51 PM Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> wrote:

> Fan Liya, please update Java. It is the solution.
>
> Vladimir
>

Re: [HELP] Local build fails

[Fan Liya]

It is working for me too. Thanks for your kind help!

Best,
Liya Fan

On Thu, Dec 10, 2020 at 11:14 AM Chunwei Lei  wrote:

> Vladimir, Julian, Liya, thank you very much for your help. It works now.
>
>
>
> Best,
> Chunwei
>
>
> On Thu, Dec 10, 2020 at 4:12 AM Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
>
> > Fixed.
> >
> > Vladimir
> >
>

Re: [ANNOUNCE] New Calcite PMC chair: Haisheng Yuan

[Fan Liya]

Congratulations, Haisheng!
Looking forward to your great work in the coming year!

Stamatis, thanks for your great work in the past year!

Best,
Liya Fan


On Fri, Dec 18, 2020 at 11:11 AM Feng Zhu  wrote:

> Thanks for your work and effort, Stamatis!
> Congratulations, Haisheng!
>
> Stamatis Zampetakis  于2020年12月17日周四 下午9:49写道：
>
> > Calcite community members,
> >
> > I am pleased to announce that we have a new PMC chair and VP as per our
> > tradition of rotating the chair once a year. I have resigned, and
> > Haisheng was duly elected by the PMC and approved unanimously by the
> Board.
> >
> > Please join me in congratulating Haisheng!
> >
> > Best,
> > Stamatis
> >
>

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

It seems the problem is that the json format provided by Calcite does not
include properties specific to your operator?
If so, I think you can override RelNode#explainTerms in your operator to
attach the properties.

Best,
Liya Fan

On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal  wrote:

> Please let me know if we have an example for writing a custom visitor that
> I can use. I see a RelVisitor class that I can extend but how to make it to
> visit the specific Physical Operator.
>
> Thanks and Regards
> Bhavya
>
> On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal 
> wrote:
>
> > Thanks,
> >
> > I tried using the RelJsonWriter and tried to add my custom implementation
> > in the my custom Rel Nodes but I am not getting all the data in my child
> > operators. We are trying to use Calcite for a columnar data and we need
> the
> > information of the projections in the TableScan as well because we need
> to
> > pass those projections to read specific columns but I am not able to find
> > the projections in my Table scan. As per my understanding Calcite should
> be
> > passing this information as it is required for me reading the table.
> Please
> > let me know if I need to do something special to get that information in
> > the child operator.
> >
> > Regards
> > Bhavya
> >
> > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior  wrote:
> >
> >> That would likely be the best approach if you have some specific JSON
> >> format you're trying to generate. If you're happy with a JSON format
> >> defined by Calcite, you can look at RelJsonWriter.
> >>
> >> --
> >> Michael Mior
> >> mm...@apache.org
> >>
> >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana  a
> >> écrit :
> >> >
> >> > I would use a visitor to traverse the optimized/physical plan.
> >> >
> >> >
> >> >
> >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal 
> >> wrote:
> >> >
> >> > > Hi All,
> >> > >
> >> > > We need to generate a JSON object for the physical execution tree
> >> that has
> >> > > been created. Is there an option in Calcite that we can use to do
> >> this. I
> >> > > am not sure what is the right approach to do it. Please let me know
> if
> >> > > there are different ways to achieve this.
> >> > >
> >> > > Regards
> >> > > Bhavya
> >> > >
> >> > > --
> >> > > Your feedback matters - At Knoldus we aim to be very professional in
> >> our
> >> > > quality of work, commitment to results, and proactive communication.
> >> If
> >> > > you
> >> > > feel otherwise please share your feedback
> >> > >  and we would work on it.
> >> > >
> >>
> >
> >
> > --
> > *Bhavya Aggarwal*
> > CTO & Partner
> > Knoldus Inc. 
> > +91-9910483067
> > Canada - USA - India - Singapore
> >   >
> >  
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. 
> +91-9910483067
> Canada - USA - India - Singapore
>  
>  
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
>  and we would work on it.
>

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

Sorry I do not understand your question. Why is it difficult to pass sth.
to child operators?

Best,
Liya Fan


On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal  wrote:

> Hi Liya,
>
> Actually the question is how to pass the projection columns to Table Scan
> as right now in the LogicalTableScan there are no projection columns being
> passed so when I am trying to create my custom JSON , I do not have those
> projected columns or columns that are being used in the query. I want to
> understand what is the calcite mechanism for passing it to child operators
> without using Rules as it will be a lot of rule that we need to implement.
>
> Thanks and Regards
> Bhavya
>
> On Fri, Dec 18, 2020 at 9:47 AM Fan Liya  wrote:
>
> > Hi Bhavya,
> >
> > It seems the problem is that the json format provided by Calcite does not
> > include properties specific to your operator?
> > If so, I think you can override RelNode#explainTerms in your operator to
> > attach the properties.
> >
> > Best,
> > Liya Fan
> >
> > On Thu, Dec 17, 2020 at 10:45 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Please let me know if we have an example for writing a custom visitor
> > that
> > > I can use. I see a RelVisitor class that I can extend but how to make
> it
> > to
> > > visit the specific Physical Operator.
> > >
> > > Thanks and Regards
> > > Bhavya
> > >
> > > On Thu, Dec 17, 2020 at 7:43 PM Bhavya Aggarwal 
> > > wrote:
> > >
> > > > Thanks,
> > > >
> > > > I tried using the RelJsonWriter and tried to add my custom
> > implementation
> > > > in the my custom Rel Nodes but I am not getting all the data in my
> > child
> > > > operators. We are trying to use Calcite for a columnar data and we
> need
> > > the
> > > > information of the projections in the TableScan as well because we
> need
> > > to
> > > > pass those projections to read specific columns but I am not able to
> > find
> > > > the projections in my Table scan. As per my understanding Calcite
> > should
> > > be
> > > > passing this information as it is required for me reading the table.
> > > Please
> > > > let me know if I need to do something special to get that information
> > in
> > > > the child operator.
> > > >
> > > > Regards
> > > > Bhavya
> > > >
> > > > On Sun, Dec 13, 2020 at 11:32 PM Michael Mior 
> > wrote:
> > > >
> > > >> That would likely be the best approach if you have some specific
> JSON
> > > >> format you're trying to generate. If you're happy with a JSON format
> > > >> defined by Calcite, you can look at RelJsonWriter.
> > > >>
> > > >> --
> > > >> Michael Mior
> > > >> mm...@apache.org
> > > >>
> > > >> Le dim. 13 déc. 2020 à 05:36, Muhammad Gelbana  >
> > a
> > > >> écrit :
> > > >> >
> > > >> > I would use a visitor to traverse the optimized/physical plan.
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Sun, Dec 13, 2020 at 6:42 AM Bhavya Aggarwal <
> bha...@knoldus.com
> > >
> > > >> wrote:
> > > >> >
> > > >> > > Hi All,
> > > >> > >
> > > >> > > We need to generate a JSON object for the physical execution
> tree
> > > >> that has
> > > >> > > been created. Is there an option in Calcite that we can use to
> do
> > > >> this. I
> > > >> > > am not sure what is the right approach to do it. Please let me
> > know
> > > if
> > > >> > > there are different ways to achieve this.
> > > >> > >
> > > >> > > Regards
> > > >> > > Bhavya
> > > >> > >
> > > >> > > --
> > > >> > > Your feedback matters - At Knoldus we aim to be very
> professional
> > in
> > > >> our
> > > >> > > quality of work, commitment to results, and proactive
> > communication.
> > > >> If
> > > >> > > you
> > > >> > > feel otherwise please share your feedback
> > > >> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

Thanks for your explanation.

I do not think a rule like ProjectFilterTableScan solves the problem.
To see this, please note that the parent operator of the LogicalProject may
need other columns from the table
(e.g. the parent operator may be a join, which uses some other columns as
the join key).

So the column pruning of table scan should be a global decision, depending
on the whole plan.

Calcite has RelMdColumnOrigins, which can help in this scenario.
In particular, given an output column in the output operator, it gives the
dependent columns in the underlying source tables.
So columns not referenced from the final output can be pruned from the
source table.

Best,
Liya Fan


On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal  wrote:

> Hi Liya,
>
> Please see below , I have a query as given below
>
> SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
>
> Now I know that I have to fetch two columns from my columnar database i.e.
> ID, Name, so need to pass these parameters in my custom Table Scan.
> But when I convert the LogicalTableScan to my custom Table scan I do not
> have that information, please see the tree below, in the Logical Table Scan
> the Input is null, I was hoping that I will have the columns in the
> LogicalTableScan, The question is what do I need to do in Calcite to pass
> that information to the LogicalTableScan or in my physical operator. The
> way I have found is that I create a Rule like ProjectFilterTableScan rule
> and then extract what information I need. Is there a better way to do this?
>
> -Logical JSON Plan 
> > {
> >   "rels": [
> > {
> >   "id": "0",
> >   "relOp": "LogicalTableScan",
> >   "table": [
> > "USERS"
> >   ],
> >   "inputs": []
> > },
> > {
> >   "id": "1",
> >   "relOp": "LogicalFilter",
> >   "condition": {
> > "op": {
> >   "name": "=",
> >   "kind": "EQUALS",
> >   "syntax": "BINARY"
> > },
> > "operands": [
> >   {
> > "op": {
> >   "name": "CAST",
> >   "kind": "CAST",
> >   "syntax": "SPECIAL"
> > },
> > "operands": [
> >   {
> > "input": 1,
> > "name": "$1"
> >   }
> > ],
> > "type": {
> >   "type": "CHAR",
> >   "nullable": true,
> >   "precision": 5
> > }
> >   },
> >   {
> > "literal": "HELLO",
> > "type": {
> >   "type": "CHAR",
> >   "nullable": false,
> >   "precision": 5
> > }
> >   }
> > ]
> >   }
> > },
> > {
> >   "id": "2",
> >   "relOp": "LogicalProject",
> >   "fields": [
> > "ID"
> >   ],
> >   "exprs": [
> > {
> >   "input": 0,
> >   "name": "$0"
> > }
> >   ]
> > }
> >   ]
> > }
> >
> >> ---
> >
> >
> Thanks and regards
> Bhavya
>
> On Fri, Dec 18, 2020 at 11:54 AM Fan Liya  wrote:
>
> > Hi Bhavya,
> >
> > Sorry I do not understand your question. Why is it difficult to pass sth.
> > to child operators?
> >
> > Best,
> > Liya Fan
> >
> >
> > On Fri, Dec 18, 2020 at 12:38 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Hi Liya,
> > >
> > > Actually the question is how to pass the projection columns to Table
> Scan
> > > as right now in the LogicalTableScan there are no projection columns
> > being
> > > passed so when I am trying to create my custom JSON , I do not have
> those
> > > projected columns or columns that are being used in the query. I want
> to
> > > understand what is the calcite me

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Vladmir,

Thanks for your feedback.

I see your point. The example I gave above was not a good one.

I agree with you that making the decision locally solves most problems.
However, it may be sub-optimal for some scenarios:

The project operator may contain unnecessary columns (even if the columns
appear in the query). For example, the optimizer may determine that the
column is always a constant, and should only be included in the final write.

Best,
Liya Fan


On Fri, Dec 18, 2020 at 9:32 PM Vladimir Ozerov  wrote:

> Hi Liya,
>
> I do not think, this is a global decision. If you go bottom-up in the plan,
> and find the Project operator, than none parent operators will use any
> columns from the lower scans, other than those present in the Project.
>
> It is a common practice for Calcite integrations to restrict the number of
> columns returned from scans. It is achieved as follows:
> 1) There should be rules to extract projections from some operators (e.g.
> Joins)
> 2) There should be rules to transpose projects with child operators
> 3) Finally, you should create a rule to move project into scan for the
> pattern [Project<-Scan]
>
> When all three prerequisites are ready, the optimizer will produce missing
> projects (if any), then push them down the operator tree, and finally merge
> into the scan.
>
> Regards,
> Vladimir
>
> Пт, 18 дек. 2020 г. в 13:34, Fan Liya :
>
> > Hi Bhavya,
> >
> > Thanks for your explanation.
> >
> > I do not think a rule like ProjectFilterTableScan solves the problem.
> > To see this, please note that the parent operator of the LogicalProject
> may
> > need other columns from the table
> > (e.g. the parent operator may be a join, which uses some other columns as
> > the join key).
> >
> > So the column pruning of table scan should be a global decision,
> depending
> > on the whole plan.
> >
> > Calcite has RelMdColumnOrigins, which can help in this scenario.
> > In particular, given an output column in the output operator, it gives
> the
> > dependent columns in the underlying source tables.
> > So columns not referenced from the final output can be pruned from the
> > source table.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Fri, Dec 18, 2020 at 4:43 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Hi Liya,
> > >
> > > Please see below , I have a query as given below
> > >
> > > SELECT t_1.ID FROM USERS t_1 where NAME = 'HELLO';
> > >
> > > Now I know that I have to fetch two columns from my columnar database
> > i.e.
> > > ID, Name, so need to pass these parameters in my custom Table Scan.
> > > But when I convert the LogicalTableScan to my custom Table scan I do
> not
> > > have that information, please see the tree below, in the Logical Table
> > Scan
> > > the Input is null, I was hoping that I will have the columns in the
> > > LogicalTableScan, The question is what do I need to do in Calcite to
> pass
> > > that information to the LogicalTableScan or in my physical operator.
> The
> > > way I have found is that I create a Rule like ProjectFilterTableScan
> rule
> > > and then extract what information I need. Is there a better way to do
> > this?
> > >
> > > -Logical JSON Plan 
> > > > {
> > > >   "rels": [
> > > > {
> > > >   "id": "0",
> > > >   "relOp": "LogicalTableScan",
> > > >   "table": [
> > > > "USERS"
> > > >   ],
> > > >   "inputs": []
> > > > },
> > > > {
> > > >   "id": "1",
> > > >   "relOp": "LogicalFilter",
> > > >   "condition": {
> > > > "op": {
> > > >   "name": "=",
> > > >   "kind": "EQUALS",
> > > >   "syntax": "BINARY"
> > > > },
> > > > "operands": [
> > > >   {
> > > > "op": {
> > > >   "name": "CAST",
> > > >   "kind": "CAST",
> > > >   "syntax": "SPECIAL"
> > > > },
> > > > "operands": [
> > > > 

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

IMO, to solve the problem from a global view, the following steps needs to
be taken:

1. Generate a physical plan in the original way (without considering column
pruning in the table scan)
2. Modify all the table scans in the plan with the RelColumnMDOrigins
utility (the details have been described above)
3. Post process the plan with one of the following ways:
a) a plan visitor that adjusts other operators in the tree.
b) a light-weight planner (Hep or Volcano with limited rule sets)

Run the query with the finally generated plan.

Best,
Liya Fan


On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal  wrote:

> Hi Fan,
>
> I looked at the class RelColumnMDOrigins and it is giving me the origin of
> the column, but even if I want to take it as a global decision I am not
> sure how to proceed. Can you please elaborate on how to achieve this ? I am
> literally stuck as I do not want to use so many rules as in any case I have
> to pass these to the TableScan, even if the user does a select * from
> table, I need to add all those columns to the table scan.
>
> Regards
> Bhavya
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
>  and we would work on it.
>

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Bhavya,

Sorry I am not sure if I fully understand your question.

Let me try to answer it according to my understanding:

1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object, which
includes a RelOptTable object.
2. The table scan also has a RelOptTable object, and all table scans of the
plan can be found (e.g. through a visitor)
3. With the information of 1 and 2, given any output column, we can get to
know it is derived from which columns from which table scans.
4. With the information of 3, given a table scan, we can get to know which
column is never used in any output columns, and such columns can be pruned.

Best,
Liya Fan


On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal  wrote:

> Hi Liya,
>
> I had a look at the RelColumnMDOrigins and it is useful in determining
> which columns are from which table but still I am not sure how can I get
> the column information for TableScan without the rules. If you have any
> specific example where we have used this approach will be really helpful to
> me.
>
> Thanks and regards
> Bhavya
>
> On Mon, Dec 21, 2020 at 5:53 PM Fan Liya  wrote:
>
> > Hi Bhavya,
> >
> > IMO, to solve the problem from a global view, the following steps needs
> to
> > be taken:
> >
> > 1. Generate a physical plan in the original way (without considering
> column
> > pruning in the table scan)
> > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > utility (the details have been described above)
> > 3. Post process the plan with one of the following ways:
> > a) a plan visitor that adjusts other operators in the tree.
> > b) a light-weight planner (Hep or Volcano with limited rule sets)
> >
> > Run the query with the finally generated plan.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Hi Fan,
> > >
> > > I looked at the class RelColumnMDOrigins and it is giving me the origin
> > of
> > > the column, but even if I want to take it as a global decision I am not
> > > sure how to proceed. Can you please elaborate on how to achieve this ?
> I
> > am
> > > literally stuck as I do not want to use so many rules as in any case I
> > have
> > > to pass these to the TableScan, even if the user does a select * from
> > > table, I need to add all those columns to the table scan.
> > >
> > > Regards
> > > Bhavya
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>
>
> --
> *Bhavya Aggarwal*
> CTO & Partner
> Knoldus Inc. <http://www.knoldus.com/>
> +91-9910483067
> Canada - USA - India - Singapore
> <https://in.linkedin.com/company/knoldus> <https://twitter.com/Knolspeak>
> <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/>
>
> --
> Your feedback matters - At Knoldus we aim to be very professional in our
> quality of work, commitment to results, and proactive communication. If
> you
> feel otherwise please share your feedback
> <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
>

Re: Need a Jason output if physical tree

[Fan Liya]

Hi Vladmir,

Thanks for your feedback.

IMO, Root[$0] should depend on t.a and t.b, because the value of t.b
decides if some value could be included in Root[$0].

However, after investigating the code, I find you are right!
The current implementation of RelMdColumnOrigins#getColumnOrigins(Filter,
RelMetadataQuery, int) does not incorporate columns referenced in the
condition.

I am wondering if this is by design? Do we need another metadata query to
evaluate the relation between input and output columns?

Best,
Liya Fan


On Tue, Dec 22, 2020 at 3:50 PM Vladimir Ozerov  wrote:

> Hi Liya,
>
> This will not work AFAIK. Consider the query "SELECT a FROM t WHERE b>1".
> The top-level operator has only one column:
>
> Root[$0]
>   Project[$0]
> Filter[$1>1]
>   Scan[$0=a, $1=b]
>
> If you invoke RelMdColumnOrigins on Root[$0], you will get [t.a], but miss
> [t.b].
> To my knowledge, rules are the only way to reliably. constrain columns
> returned from the scan.
>
> Regards,
> Vladimir.
>
> вт, 22 дек. 2020 г. в 05:14, Fan Liya :
>
> > Hi Bhavya,
> >
> > Sorry I am not sure if I fully understand your question.
> >
> > Let me try to answer it according to my understanding:
> >
> > 1. Through RelColumnMDOrigins, we can get the RelColumnOrigin object,
> which
> > includes a RelOptTable object.
> > 2. The table scan also has a RelOptTable object, and all table scans of
> the
> > plan can be found (e.g. through a visitor)
> > 3. With the information of 1 and 2, given any output column, we can get
> to
> > know it is derived from which columns from which table scans.
> > 4. With the information of 3, given a table scan, we can get to know
> which
> > column is never used in any output columns, and such columns can be
> pruned.
> >
> > Best,
> > Liya Fan
> >
> >
> > On Mon, Dec 21, 2020 at 11:31 PM Bhavya Aggarwal 
> > wrote:
> >
> > > Hi Liya,
> > >
> > > I had a look at the RelColumnMDOrigins and it is useful in determining
> > > which columns are from which table but still I am not sure how can I
> get
> > > the column information for TableScan without the rules. If you have any
> > > specific example where we have used this approach will be really
> helpful
> > to
> > > me.
> > >
> > > Thanks and regards
> > > Bhavya
> > >
> > > On Mon, Dec 21, 2020 at 5:53 PM Fan Liya  wrote:
> > >
> > > > Hi Bhavya,
> > > >
> > > > IMO, to solve the problem from a global view, the following steps
> needs
> > > to
> > > > be taken:
> > > >
> > > > 1. Generate a physical plan in the original way (without considering
> > > column
> > > > pruning in the table scan)
> > > > 2. Modify all the table scans in the plan with the RelColumnMDOrigins
> > > > utility (the details have been described above)
> > > > 3. Post process the plan with one of the following ways:
> > > > a) a plan visitor that adjusts other operators in the tree.
> > > > b) a light-weight planner (Hep or Volcano with limited rule sets)
> > > >
> > > > Run the query with the finally generated plan.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > On Mon, Dec 21, 2020 at 3:33 PM Bhavya Aggarwal 
> > > > wrote:
> > > >
> > > > > Hi Fan,
> > > > >
> > > > > I looked at the class RelColumnMDOrigins and it is giving me the
> > origin
> > > > of
> > > > > the column, but even if I want to take it as a global decision I am
> > not
> > > > > sure how to proceed. Can you please elaborate on how to achieve
> this
> > ?
> > > I
> > > > am
> > > > > literally stuck as I do not want to use so many rules as in any
> case
> > I
> > > > have
> > > > > to pass these to the TableScan, even if the user does a select *
> from
> > > > > table, I need to add all those columns to the table scan.
> > > > >
> > > > > Regards
> > > > > Bhavya
> > > > >
> > > > > --
> > > > > Your feedback matters - At Knoldus we aim to be very professional
> in
> > > our
> > > > > quality of work, commitment to results, and proactive
> communication.
> > If
> > > > > you
> > > > > feel otherwise please share your feedback
> > > > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > > > >
> > > >
> > >
> > >
> > > --
> > > *Bhavya Aggarwal*
> > > CTO & Partner
> > > Knoldus Inc. <http://www.knoldus.com/>
> > > +91-9910483067
> > > Canada - USA - India - Singapore
> > > <https://in.linkedin.com/company/knoldus> <
> https://twitter.com/Knolspeak
> > >
> > > <https://www.facebook.com/KnoldusSoftware/> <https://blog.knoldus.com/
> >
> > >
> > > --
> > > Your feedback matters - At Knoldus we aim to be very professional in
> our
> > > quality of work, commitment to results, and proactive communication. If
> > > you
> > > feel otherwise please share your feedback
> > > <https://forms.gle/Ax1Te1DDpirAQuQ8A> and we would work on it.
> > >
> >
>

Re: [ANNOUNCE] New committer: Liya Fan

[Fan Liya]

Dear all,

I want to express my sincere gratitude to you for all your kind help.
I feel honored to have the opportunity of working with you in this awesome
community.

In the past years, I have been working on some different components of the
SQL engine, now in Alibaba, and previously in Microsoft.
Through this experience, I find the SQL optimizer especially interesting,
because it involves so many interesting problems.

In the future, I want to make greater contributions to the community and
help our users build better SQL engines.

Best,
Liya Fan


On Thu, Feb 11, 2021 at 1:54 AM Rui Wang  wrote:

> Congratulations Liya!
>
>
> -Rui
>
> On Wed, Feb 10, 2021 at 6:39 AM Michael Mior  wrote:
>
> > Congratulations and welcome Liya!
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le mer. 10 févr. 2021 à 05:23, Francis Chuang
> >  a écrit :
> > >
> > > Congrats Liya and well deserved!
> > >
> > > Francis
> > >
> > > On 10/02/2021 8:53 pm, Stamatis Zampetakis wrote:
> > > > Apache Calcite's Project Management Committee (PMC) has invited Liya
> > Fan to
> > > > become a committer, and we are pleased to announce that they have
> > accepted.
> > > >
> > > > Liya has made a lot of contributions to the project, enhancing
> > optimizer
> > > > rules,
> > > > improving the efficiency of graph traversal algorithms, as well as
> > fixing
> > > > various
> > > > bugs in the project.
> > > >
> > > > Liya, welcome, thank you for your contributions, and we look forward
> > to your
> > > > further interactions with the community! If you wish, please feel
> free
> > to
> > > > tell
> > > > us more about yourself and what you are working on.
> > > >
> > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > >
> >
>

Re: Nested loop joins

[Fan Liya]

Hi Priyendra,

We have FilterReduceExpressionsRule which reduces filter conditions.
It performs the simplification based on org.apache.calcite.rex.RexSimplify.

Best,
Liya Fan

On Mon, Mar 1, 2021 at 1:45 PM Priyendra Deshwal 
wrote:

> Hello friends,
>
> I am playing around with TPC-DS schema and playing with the following
> simplified variant of query41.
>
> select  distinct(i_product_name)
>  from item i1
>  where i_manufact_id between 738 and 738+40
>and (select count(*) as item_cnt
> from item
> where (i_manufact = i1.i_manufact and i_category = 'Women') or
>(i_manufact = i1.i_manufact and i_category = 'Men')) > 0
>  order by i_product_name
>  limit 100
>
> This results in the following optimized plan. Note that the join condition
> (i_manufact = i1.i_manufact) is not clearly expressed in this query since
> it is repeated in both OR clauses of the inner query. This results in a
> nested loop join and even the filter on i_category is not pushed all the
> way down to the query.
>
> EnumerableLimit(fetch=[100]): rowcount = 100.0, cumulative cost =
> 2.929266516286664E9, id = 408
>   EnumerableSort(sort0=[$0], dir0=[ASC]): rowcount = 5.4675E7, cumulative
> cost = 2.929266416286664E9, id = 406
> EnumerableAggregate(group=[{2}]): rowcount = 5.4675E7, cumulative cost
> = 2.874591416286664E9, id = 404
>   EnumerableCalc(expr#0..4=[{inputs}], expr#5=[IS NULL($t4)],
> expr#6=[0:BIGINT], expr#7=[0], expr#8=[>($t6, $t7)], expr#9=[AND($t5,
> $t8)], expr#10=[>($t4, $t7)], expr#11=[OR($t9, $t10)], proj#0..4=[{exprs}],
> $condition=[$t11]): rowcount = 5.4675E8, cumulative cost =
> 2.819916416286664E9, id = 410
> EnumerableHashJoin(condition=[=($1, $3)], joinType=[left]):
> rowcount = 2.187E9, cumulative cost = 2.273166416286664E9, id = 400
>   EnumerableInterpreter: rowcount = 18000.0, cumulative cost =
> 9000.0, id = 386
> BindableTableScan(table=[[default, ITEM]],
> filters=[[AND(>=($13, 738), <=($13, +(738, 40)))]], projects=[[13, 14,
> 21]]): rowcount = 18000.0, cumulative cost = 18.75, id = 360
>   EnumerableAggregate(group=[{2}], ITEM_CNT=[COUNT()]): rowcount =
> 81.0, cumulative cost = 8.193105E7, id = 398
> EnumerableNestedLoopJoin(condition=[OR(AND(=($1, $2), =($0,
> 'Women')), AND(=($1, $2), =($0, 'Men')))], joinType=[inner]): rowcount =
> 810.0, cumulative cost = 8.10198E7, id = 396
>   EnumerableInterpreter: rowcount = 18000.0, cumulative cost =
> 9000.0, id = 389
> BindableTableScan(table=[[default, ITEM]], projects=[[12,
> 14]]): rowcount = 18000.0, cumulative cost = 29.996, id = 231
>   EnumerableAggregate(group=[{0}]): rowcount = 1800.0,
> cumulative cost = 10800.0, id = 394
> EnumerableInterpreter: rowcount = 18000.0, cumulative cost
> = 9000.0, id = 392
>   BindableTableScan(table=[[default, ITEM]],
> filters=[[AND(>=($13, 738), <=($13, +(738, 40)))]], projects=[[14]]):
> rowcount = 18000.0, cumulative cost = 11.25, id = 364
>
> A simple rewrite of the query as follows where we "factor" out the join
> condition to the top level AND does make the plan significantly better
> (hash join with complete filter push down to the source).
>
> select  distinct(i_product_name)
>  from item i1
>  where i_manufact_id between 738 and 738+40
>and (select count(*) as item_cnt
> from item
> where i_manufact = i1.i_manufact and
>(i_category = 'Women' or i_category = 'Men')) > 0
>  order by i_product_name
>  limit 100
>
> EnumerableLimit(fetch=[100]): rowcount = 100.0, cumulative cost =
> 6523491.28666381, id = 336
>   EnumerableSort(sort0=[$0], dir0=[ASC]): rowcount = 121500.0, cumulative
> cost = 6523391.28666381, id = 334
> EnumerableAggregate(group=[{2}]): rowcount = 121500.0, cumulative cost
> = 6401891.28666381, id = 332
>   EnumerableCalc(expr#0..4=[{inputs}], expr#5=[IS NULL($t4)],
> expr#6=[0:BIGINT], expr#7=[0], expr#8=[>($t6, $t7)], expr#9=[AND($t5,
> $t8)], expr#10=[>($t4, $t7)], expr#11=[OR($t9, $t10)], proj#0..4=[{exprs}],
> $condition=[$t11]): rowcount = 1215000.0, cumulative cost =
> 6280391.28666381, id = 338
> EnumerableHashJoin(condition=[=($1, $3)], joinType=[left]):
> rowcount = 486.0, cumulative cost = 5065391.28666381, id = 328
>   EnumerableInterpreter: rowcount = 18000.0, cumulative cost =
> 9000.0, id = 321
> BindableTableScan(table=[[default, ITEM]],
> filters=[[AND(>=($13, 738), <=($13, +(738, 40)))]], projects=[[13, 14,
> 21]]): rowcount = 18000.0, cumulative cost = 18.75, id = 298
>   EnumerableAggregate(group=[{0}], ITEM_CNT=[COUNT()]): rowcount =
> 1800.0, cumulative cost = 11025.0, id = 326
> EnumerableInterpreter: rowcount = 18000.0, cumulative cost =
> 9000.0, id = 324
>   BindableTableScan(table=[[default, ITEM]],
> filters=[[OR(=($12, 'Women'), =($12, 'Men'))]], projects=[[14]]): rowcoun

Re: TopDown Optimizer Questions

[Fan Liya]

Hi Priyendra,

I am not sure if I fully understand your questions, and I want to try to
answer this one:

"*At what point does the enforcer operator come into play? Is it the
RelNodes responsibility *
*to add the enforcer operator for cases where it is unable to honor a
traitset or does the planner *
*somehow do that automatically?*"

When a new RelSubset is created, some AbstractConverter objects are created
connecting the
different RelSubsets in the same RelSet. Please see the code of
RelSubset#getOrCreateSubset,
and you can see it calls the RelSet#addConverters method, which creates
the  AbstractConverter
objects.

Later in the optimization process, (e.g. in the plan implementation phase),
the  AbstractConverter objects are converted into some physical nodes
depending on the specific context.
For example, when a Collation is required, the  AbstractConverter object
may be translated to a Sort operator.
The conversion is usually performed by a ConverterRule, and often depends
on the specific system.

After the  AbstractConverter objects are converted, a feasible physical
plan can be produced, and the
planner can choose the one with the minimum cost.

Best,
Liya Fan



On Sun, Mar 7, 2021 at 11:53 PM Priyendra Deshwal 
wrote:

> Hello friends,
>
> I have been learning about the top-down optimizer and I am trying to
> understand the semantics of the passThrough and derive calls. I have a few
> questions about those.
>
>1. The main passThrough call is `RelNode passThrough(RelTraitSet
>required)`. It is my understanding that this is called when the
> optimizer
>wants to "rewrite" a `RelNode` to have certain `required` traits. Few
>questions here:
>   1. This "rewrite" may involve passing the traits down to the
>   children. All the example implementations that I have come across
> only
>   propagate the traits down one depth from parent to children.
> That is, there
>   is no recursive `passThrough` call on the children. Is that recursive
>   propagation handled in the `TopDownRuleDriver` somewhere?
>   2. What should the behavior of a RelNode be when it is unable to
>   honor the requirements? Should it return null?
>   3. At what point does the enforcer operator come into play? Is it
>   the RelNodes responsibility to add the enforcer operator for
> cases where it
>   is unable to honor a traitset or does the planner somehow do that
>   automatically?
>   4. The current `RelNode` may already have certain traits. Should the
>   required traits overwrite the existing traits or should they be
> "merged"
>   into the existing traits?
>2. I am unclear on the semantics of the `derive` call. The various
>derive modes add to my confusion.
>   1. I tried searching for DeriveMode.BOTH and found no references in
>   the main planner machinery for it. Is the implementation of that
> currently
>   incomplete? Only LEFT_FIRST, RIGHT_FIRST and OMAKASE are referenced
> in
>   TopDownRuleDriver.java.
>   2. Can someone briefly explain what are the expected semantics of the
>   following calls? I am confused by the semantics because the first one
>   returns a single RelNode vs the second one returns a List<> and the
>   arguments are also quite different between the two methods.
>  1. RelNode derive(RelTraitSet childTraits, int childId);
>  2. List derive(List> inputTraits);
>  3. Looking at the default implementation of RelNode
>   derive(RelTraitSet childTraits, int childId) confused me a bit. My
>   understanding of derive was that it will take the child's traits and
> pass
>   those up the tree. But the default implementation is also
> calling `convert`
>   on the children. That suggests that even the children are being
> modified in
>   this call. Why is that necessary?
>
> Apologies for the barrage of questions and thanks in advance for in
> advance.
>
> Regards!
>

Re: Apache Arrow adapter

[Fan Liya]

Hi Michael,

Thanks for sharing the great work.
I believe it is important work for both communities.

Best,
Liya Fan


On Thu, Apr 8, 2021 at 3:30 AM Michael Mior  wrote:

> Hi all,
>
> I wanted to share some work one of my (now former) students, Karshit
> Shah, has done with integrating Apache Arrow into Calcite. Karshit has
> written an Arrow adapter that's able to perform filtering and
> projections natively on Arrow data using Gandiva so these expressions
> can be JITed using LLVM. The pull request[0] needs some cleanup, but
> the code is in relatively good shape.
>
> Right now, the adapter only reads from files, but I think there are a
> number of exciting extensions to this that are possible. For example,
> Arrow has a client-server framework Flight which could be connected
> with Calcite, perhaps via Avatica. (Andy Grove was doing some work on
> this last year[1] although I'm not sure of the progress.)
>
> The biggest blocker on this is actually not the Calcite code, but the
> availability of a suitably built Arrow dependency with Gandiva along
> with the appropriate CI configuration. I opened a JIRA on the Arrow
> project with some more details[2].
>
> I'd love some thoughts on the approach and some help in pushing this
> over the finish line.
>
> [0] https://github.com/apache/calcite/pull/2133
> [1]
> https://mail-archives.apache.org/mod_mbox/calcite-dev/202002.mbox/%3cCAJEf=X5xvXLQpJkX_VjJk=TnNRwT52v0=p28sczmid1tyce...@mail.gmail.com%3e
> [2] https://issues.apache.org/jira/browse/ARROW-11135
> --
> Michael Mior
> mm...@apache.org
>

Re: Using Calcite as a Distributed Optimizer

[Fan Liya]

I think you can refer to Flink

Best,
Liya Fan

On Fri, May 14, 2021 at 12:32 PM Atri Sharma  wrote:

> Is there an example I can refer to?
>
> On Fri, 14 May 2021, 04:59 Haisheng Yuan,  wrote:
>
> > Yes, definitely. Many distributed big data systems use Apache Calcite to
> > optimize queries and generate distributed plans.
> >
> > On 2021/05/13 23:16:10, Atri Sharma  wrote:
> > > Thank you.
> > >
> > > So my use case is such that I wish to use Calcite as a two phase
> > optimizer
> > > -- Get a SQL query,  compile it and optimize it and convert it to a SQL
> > > fragment.
> > >
> > > Then run the query on worker nodes, get results on master and merge
> > results.
> > >
> > > This question spans both Calcite and Avatica, but I wanted to
> understand
> > if
> > > achieving the above is possible with Calcite today.
> > >
> > > Atri
> > >
> > > On Fri, 14 May 2021, 01:20 Julian Hyde, 
> wrote:
> > >
> > > > Calcite has no user@ list.  So, ask away here!
> > > >
> > > > > On May 13, 2021, at 10:52 AM, Atri Sharma 
> > wrote:
> > > > >
> > > > > Sorry, didn't realize I had sent to the dev list. I will send to
> the
> > user
> > > > > list
> > > > >
> > > > > On Thu, 13 May 2021, 15:57 Atri Sharma, 
> wrote:
> > > > >
> > > > >> Hi All,
> > > > >>
> > > > >> Are there examples of using Calcite to compile and optimize
> queries
> > to
> > > > >> be run on a set of nodes, and then merge partial results back?
> > > > >>
> > > > >> Atri
> > > > >>
> > > > >> --
> > > > >> Regards,
> > > > >>
> > > > >> Atri
> > > > >> l'apprenant
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Release Managers

[Fan Liya]

Hi Haisheng,

I am interested in volunteering.

Best,
Liya Fan

On Fri, Jun 4, 2021 at 6:50 AM Julian Hyde  wrote:

> Yes, I am release manager for 1.28. I’ll aim to release first week of
> August.
>
> Somewhat related to the call for release managers: we need active
> committers. (Stamatis, can you run your stats on PRs and active
> committers?) On May 21, I noted that PR backlog is building up, and
> appealed for 5 committers to look at 5 PRs each. But since then only 2 PRs
> have been committed (thanks Ruben & Stamatis). We need to do better.
>
> Julian
>
>
> > On Jun 3, 2021, at 3:33 PM, Haisheng Yuan  wrote:
> >
> > Hi all,
> >
> > In the next half year, I hope we can get back to the pace of release for
> every 2 months. We need more release managers for the next few releases, is
> there any one interested in volunteering to be release manager?
> > Currently Julian is the release manager for 1.28.0, we need 3 more for
> 1.29.0 ~ 1.31.0.
> >
> > Thanks,
> > Haisheng Yuan
>
>

Re: [ANNOUNCE] New committer: Vladimir Ozerov

[Fan Liya]

Congratulations, Vladimir!
Thanks for the good articles.

Best,
Liya Fan

On Fri, Jun 25, 2021 at 9:07 AM Julian Hyde  wrote:

> Welcome, Vladimir!
>
> > On Jun 24, 2021, at 6:00 PM, Albert  wrote:
> >
> > Congrats.
> > just found the blog: https://www.querifylabs.com/author/vladimir-ozerov
> >
> > On Thu, Jun 24, 2021 at 2:27 PM Alessandro Solimando <
> > alessandro.solima...@gmail.com> wrote:
> >
> >> Congratulations Vladimir, well deserved, I had the chance to read some
> >> of the blog posts and I have appreciated them very much.
> >>
> >> Best regards,
> >> Alessandro
> >>
> >> On Thu, 24 Jun 2021 at 07:58, Viliam Durina 
> wrote:
> >>>
> >>> Congratulations!
> >>>
> >>> Viliam
> >>>
> >>> On Thu, 24 Jun 2021 at 06:58, Forward Xu 
> wrote:
> >>>
>  Congratulations!
> 
> 
>  Best，
> 
>  Forward
> 
>  Danny Chan  于2021年6月24日周四 上午11:51写道：
> 
> > Congrats, Vladimir!
> >
> > Best,
> > Danny Chan
> >
> > Yanjing Wang  于2021年6月24日周四 上午11:41写道：
> >
> >> Congrats, Vladimir!
> >>
> >> Roman Kondakov  于2021年6月24日周四
> >> 上午11:22写道：
> >>
> >>> Congratulations, Vladimir!
> >>>
> >>> Roman Kondakov
> >>>
> >>> On 24.06.2021 12:23, 段雄 wrote:
>  Congratulations!
> 
>  XING JIN  于2021年6月24日周四 上午10:21写道：
> 
> > Congratulations ~
> >
> > Best,
> > Jin
> >
> > guangyuan wang  于2021年6月24日周四
> >> 上午9:50写道：
> >
> >> Congratulations!
> >>
> >> Francis Chuang  于2021年6月24日周四
>  上午6:39写道：
> >>
> >>> Congrats, Vladimir!
> >>>
> >>> Francis
> >>>
> >>> On 24/06/2021 7:48 am, Haisheng Yuan wrote:
>  Congratulations and thanks for your contributions,
> >> Vladimir!
> 
>  Regards,
>  Haisheng
> 
>  On 2021/06/23 21:34:40, Stamatis Zampetakis <
> >> zabe...@gmail.com
> >
> > wrote:
> > Apache Calcite's Project Management Committee (PMC) has
>  invited
> >> Vladimir
> > Ozerov to
> > become a committer, and we are pleased to announce that
> >> he has
> >> accepted.
> >
> > Vladimir is among the few people who know very well the
>  internal
> >>> workings
> > of the
> > Calcite optimizer. He started and participated in many
> > discussions
> >> about
> > the core engine and contributed ideas and code for making
> >> it
> >> better.
> > Moreover, Vladimir has blogged and talked about Calcite in
> > various
> > conferences and meetups giving publicity and showcasing
> >> the
> >>> capabilities of
> > the project.
> >
> > Vladimir, welcome, thank you for your contributions, and
> >> we
>  look
> >>> forward to
> > your
> > further interactions with the community! If you wish,
> >> please
> > feel
> > free
> >>> to
> > tell
> > us more about yourself and what you are working on.
> >
> > Stamatis (on behalf of the Apache Calcite PMC)
> >
> >>>
> >>
> >
> 
> >>>
> >>
> >
> 
> >>>
> >>>
> >>> --
> >>> Viliam Durina
> >>> Jet Developer
> >>>  hazelcast®
> >>>
> >>>   2 W 5th Ave, Ste 300 | San Mateo, CA
> >> 94402 |
> >>> USA
> >>> +1 (650) 521-5453 | hazelcast.com 
> >>>
> >>> --
> >>> This message contains confidential information and is intended only for
> >> the
> >>> individuals named. If you are not the named addressee you should not
> >>> disseminate, distribute or copy this e-mail. Please notify the sender
> >>> immediately by e-mail if you have received this e-mail by mistake and
> >>> delete this e-mail from your system. E-mail transmission cannot be
> >>> guaranteed to be secure or error-free as information could be
> >> intercepted,
> >>> corrupted, lost, destroyed, arrive late or incomplete, or contain
> >> viruses.
> >>> The sender therefore does not accept liability for any errors or
> >> omissions
> >>> in the contents of this message, which arise as a result of e-mail
> >>> transmission. If verification is required, please request a hard-copy
> >>> version. -Hazelcast
> >>
> >
> >
> > --
> > ~~~
> > no mistakes
> > ~~
>
>

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Fan Liya]

Congratulations, Ruben!
Haisheng, thanks for your great work!

Best,
Liya Fan


Jing Zhang  于2022年1月24日周一 11:30写道：

> Congrats Ruben!
> Haisheng, thanks a lot for serving as a chair!
>
>
> Enrico Olivelli  于2022年1月23日周日 05:17写道：
>
> > Congrats!
> >
> > Enrico
> >
> > Il Sab 22 Gen 2022, 03:03 xiong duan  ha scritto:
> >
> > > Congratulations to Ruben and thanks to Haisheng!
> > >
> > > Danny Chan  于2022年1月21日周五 12:19写道：
> > >
> > > > Congratulations, Ruben, and good luck!
> > > >
> > > > Haisheng, Thank you for serving as Chair.
> > > >
> > > > Yanjing Wang 于2022年1月21日 周五下午12:16写道：
> > > >
> > > > > Congrats Ruben!
> > > > >
> > > > > Stamatis Zampetakis  于2022年1月21日周五 06:34写道：
> > > > >
> > > > > > Congrats Ruben! You're kind, fair, and knowledgeable, very well
> > > > deserved.
> > > > > >
> > > > > > Many thanks for serving as a chair Haisheng.
> > > > > >
> > > > > > Best,
> > > > > > Stamatis
> > > > > >
> > > > > > On Thu, Jan 20, 2022, 12:41 PM Forward Xu <
> forwardxu...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations to Ruben!  Thanks for serving as Chair during
> the
> > > > last
> > > > > > > year, Haisheng!
> > > > > > >
> > > > > > >
> > > > > > > forward
> > > > > > >
> > > > > > > Ruben Q L  于2022年1月20日周四 19:38写道：
> > > > > > >
> > > > > > > > Thanks everyone!
> > > > > > > > And thank you Haisheng for being our PMC Chair during last
> > year!
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jan 20, 2022 at 8:52 AM Alessandro Solimando <
> > > > > > > > alessandro.solima...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > Congratulations to Ruben and thanks a lot to Haisheng!
> > > > > > > > >
> > > > > > > > > On Thu, 20 Jan 2022 at 08:13, 953396112
> > > > <13282155...@qq.com.invalid
> > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations to Ruben!  
> > > > > > > > > > Thanks for serving as Chair, Haisheng!
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > > Zhaohui Xu
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > -- 原始邮件 --
> > > > > > > > > > 发件人:
> > > > > > > > > >   "dev"
> > > > > > > > > >
> > > >  <
> > > > > > > > > > hy...@apache.org>;
> > > > > > > > > > 发送时间: 2022年1月20日(星期四) 上午10:29
> > > > > > > > > > 收件人: "dev" > > > > > > > > >
> > > > > > > > > > 主题: [ANNOUNCE] New Calcite PMC chair: Ruben Q L
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Calcite community members,
> > > > > > > > > >
> > > > > > > > > > I am pleased to announce that we have a new PMC chair and
> > VP
> > > as
> > > > > per
> > > > > > > our
> > > > > > > > > > tradition of rotating the chair once a year. I have
> > resigned,
> > > > and
> > > > > > > Ruben
> > > > > > > > > Q L
> > > > > > > > > > was duly elected by the PMC and approved unanimously by
> the
> > > > > Board.
> > > > > > > > > >
> > > > > > > > > > Please join me in congratulating Ruben!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Haisheng Yuan
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.30.0

[Fan Liya]

Sure. If Andrei is not available, I think I am OK.

Best,
Liya Fan


Stamatis Zampetakis  于2022年2月24日周四 19:10写道：

> Thanks for driving this Ruben!
>
> Another alternative if Andrei is unavailable at the moment would be to see
> if they can switch places with Liya. Would this be possible Liya?
>
> Best,
> Stamatis
>
> On Thu, Feb 24, 2022 at 10:41 AM Ruben Q L  wrote:
>
> > Friendly reminder.
> >
> > The next release managers would be:
> > - Andrei Sereda for 1.30.0
> > - Liya Fan for 1.31.0
> >
> > Andrei, are you available to be RM for 1.30? When would it be a good date
> > for a RC?
> >
> > Ruben.
> >
> > PS: Do we have any other volunteers for the subsequent releases?
> >
> >
> > On Wed, Feb 16, 2022 at 12:59 AM xiong duan  wrote:
> >
> > > +1 to release 1.30.  Feb 28th sounds good to me.
> > >
> > > Julian Hyde  于2022年2月16日周三 08:27写道：
> > >
> > > > +1 to release 1.30 soon, to keep up the tempo. Feb 28th sounds good
> to
> > > > me. Thanks for pushing on this, Ruben.
> > > >
> > > > Andrei, Are you available to be RM for 1.30? When is a good date for
> > you?
> > > >
> > > > Julian
> > > >
> > > > On Mon, Feb 14, 2022 at 9:16 AM Ruben Q L  wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > It has been almost two months since our last release [1], and I
> think
> > > we
> > > > > should make an effort to continue keeping the rhythm of one release
> > > every
> > > > > two months approximately.
> > > > >
> > > > > If I am not mistaken, the next release managers would be [2]:
> > > > > - Andrei Sereda for 1.30.0
> > > > > - Liya Fan for 1.31.0
> > > > >
> > > > > Are you guys still able to act as release managers for these
> > versions?
> > > > > Do we have any other volunteers for the subsequent releases?
> > > > >
> > > > > As usual, according to our Jira dashboard [3] and Github [4], there
> > are
> > > > > many pending issues that could / should be part of the release.
> What
> > do
> > > > you
> > > > > think would be a reasonable date to propose for a RC? Perhaps 28th
> > > > > February? I would like to let us have plenty of time to
> collectively
> > > > review
> > > > > the PR candidates and "unblock" (to some extent) the PR situation
> > (see
> > > > also
> > > > > the recent separate thread on that matter [5]).
> > > > >
> > > > > Best regards,
> > > > > Ruben
> > > > >
> > > > >
> > > > > [1] https://calcite.apache.org/news/2021/12/25/release-1.29.0/
> > > > > [2]
> https://lists.apache.org/thread/kqslh8yqvxbolk74vov3ofw19h90xkp2
> > > > > [3]
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > > [4] https://github.com/apache/calcite/pulls
> > > > > [5]
> https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn
> > > >
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.30.0

[Fan Liya]

Thanks for your confirmation, Stamatis.
How about next Friday (March 4th) to cut the first release candidate?
Is that enough to get the PRs inside?

Best,
Liya Fan


Stamatis Zampetakis  于2022年2月27日周日 05:57写道：

> Many thanks Liya. Let's proceed like this then!
>
> Let us know when it would be the best time for you to cut the first release
> candidate so that we plan appropriately and get some PRs inside.
>
> Best,
> Stamatis
>
> On Fri, Feb 25, 2022 at 7:03 AM Fan Liya  wrote:
>
> > Sure. If Andrei is not available, I think I am OK.
> >
> > Best,
> > Liya Fan
> >
> >
> > Stamatis Zampetakis  于2022年2月24日周四 19:10写道：
> >
> > > Thanks for driving this Ruben!
> > >
> > > Another alternative if Andrei is unavailable at the moment would be to
> > see
> > > if they can switch places with Liya. Would this be possible Liya?
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Thu, Feb 24, 2022 at 10:41 AM Ruben Q L  wrote:
> > >
> > > > Friendly reminder.
> > > >
> > > > The next release managers would be:
> > > > - Andrei Sereda for 1.30.0
> > > > - Liya Fan for 1.31.0
> > > >
> > > > Andrei, are you available to be RM for 1.30? When would it be a good
> > date
> > > > for a RC?
> > > >
> > > > Ruben.
> > > >
> > > > PS: Do we have any other volunteers for the subsequent releases?
> > > >
> > > >
> > > > On Wed, Feb 16, 2022 at 12:59 AM xiong duan 
> > wrote:
> > > >
> > > > > +1 to release 1.30.  Feb 28th sounds good to me.
> > > > >
> > > > > Julian Hyde  于2022年2月16日周三 08:27写道：
> > > > >
> > > > > > +1 to release 1.30 soon, to keep up the tempo. Feb 28th sounds
> good
> > > to
> > > > > > me. Thanks for pushing on this, Ruben.
> > > > > >
> > > > > > Andrei, Are you available to be RM for 1.30? When is a good date
> > for
> > > > you?
> > > > > >
> > > > > > Julian
> > > > > >
> > > > > > On Mon, Feb 14, 2022 at 9:16 AM Ruben Q L 
> > wrote:
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > It has been almost two months since our last release [1], and I
> > > think
> > > > > we
> > > > > > > should make an effort to continue keeping the rhythm of one
> > release
> > > > > every
> > > > > > > two months approximately.
> > > > > > >
> > > > > > > If I am not mistaken, the next release managers would be [2]:
> > > > > > > - Andrei Sereda for 1.30.0
> > > > > > > - Liya Fan for 1.31.0
> > > > > > >
> > > > > > > Are you guys still able to act as release managers for these
> > > > versions?
> > > > > > > Do we have any other volunteers for the subsequent releases?
> > > > > > >
> > > > > > > As usual, according to our Jira dashboard [3] and Github [4],
> > there
> > > > are
> > > > > > > many pending issues that could / should be part of the release.
> > > What
> > > > do
> > > > > > you
> > > > > > > think would be a reasonable date to propose for a RC? Perhaps
> > 28th
> > > > > > > February? I would like to let us have plenty of time to
> > > collectively
> > > > > > review
> > > > > > > the PR candidates and "unblock" (to some extent) the PR
> situation
> > > > (see
> > > > > > also
> > > > > > > the recent separate thread on that matter [5]).
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Ruben
> > > > > > >
> > > > > > >
> > > > > > > [1] https://calcite.apache.org/news/2021/12/25/release-1.29.0/
> > > > > > > [2]
> > > https://lists.apache.org/thread/kqslh8yqvxbolk74vov3ofw19h90xkp2
> > > > > > > [3]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > > > > [4] https://github.com/apache/calcite/pulls
> > > > > > > [5]
> > > https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.30.0

[Fan Liya]

Got it. Thanks for the confirmation.

Best,
Liya Fan

Stamatis Zampetakis  于2022年3月2日周三 05:30写道：

> Since nobody objects, let's aim for Friday or whichever other day is
> convenient for you.
>
> Best,
> Stamatis
>
> On Sun, Feb 27, 2022 at 2:22 PM Fan Liya  wrote:
>
> > Thanks for your confirmation, Stamatis.
> > How about next Friday (March 4th) to cut the first release candidate?
> > Is that enough to get the PRs inside?
> >
> > Best,
> > Liya Fan
> >
> >
> > Stamatis Zampetakis  于2022年2月27日周日 05:57写道：
> >
> > > Many thanks Liya. Let's proceed like this then!
> > >
> > > Let us know when it would be the best time for you to cut the first
> > release
> > > candidate so that we plan appropriately and get some PRs inside.
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Fri, Feb 25, 2022 at 7:03 AM Fan Liya  wrote:
> > >
> > > > Sure. If Andrei is not available, I think I am OK.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > Stamatis Zampetakis  于2022年2月24日周四 19:10写道：
> > > >
> > > > > Thanks for driving this Ruben!
> > > > >
> > > > > Another alternative if Andrei is unavailable at the moment would be
> > to
> > > > see
> > > > > if they can switch places with Liya. Would this be possible Liya?
> > > > >
> > > > > Best,
> > > > > Stamatis
> > > > >
> > > > > On Thu, Feb 24, 2022 at 10:41 AM Ruben Q L 
> > wrote:
> > > > >
> > > > > > Friendly reminder.
> > > > > >
> > > > > > The next release managers would be:
> > > > > > - Andrei Sereda for 1.30.0
> > > > > > - Liya Fan for 1.31.0
> > > > > >
> > > > > > Andrei, are you available to be RM for 1.30? When would it be a
> > good
> > > > date
> > > > > > for a RC?
> > > > > >
> > > > > > Ruben.
> > > > > >
> > > > > > PS: Do we have any other volunteers for the subsequent releases?
> > > > > >
> > > > > >
> > > > > > On Wed, Feb 16, 2022 at 12:59 AM xiong duan  >
> > > > wrote:
> > > > > >
> > > > > > > +1 to release 1.30.  Feb 28th sounds good to me.
> > > > > > >
> > > > > > > Julian Hyde  于2022年2月16日周三 08:27写道：
> > > > > > >
> > > > > > > > +1 to release 1.30 soon, to keep up the tempo. Feb 28th
> sounds
> > > good
> > > > > to
> > > > > > > > me. Thanks for pushing on this, Ruben.
> > > > > > > >
> > > > > > > > Andrei, Are you available to be RM for 1.30? When is a good
> > date
> > > > for
> > > > > > you?
> > > > > > > >
> > > > > > > > Julian
> > > > > > > >
> > > > > > > > On Mon, Feb 14, 2022 at 9:16 AM Ruben Q L  >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > It has been almost two months since our last release [1],
> > and I
> > > > > think
> > > > > > > we
> > > > > > > > > should make an effort to continue keeping the rhythm of one
> > > > release
> > > > > > > every
> > > > > > > > > two months approximately.
> > > > > > > > >
> > > > > > > > > If I am not mistaken, the next release managers would be
> [2]:
> > > > > > > > > - Andrei Sereda for 1.30.0
> > > > > > > > > - Liya Fan for 1.31.0
> > > > > > > > >
> > > > > > > > > Are you guys still able to act as release managers for
> these
> > > > > > versions?
> > > > > > > > > Do we have any other volunteers for the subsequent
> releases?
> > > > > > > > >
> > > > > > > > > As usual, according to our Jira dashboard [3] and Github
> [4],
> > > > there
> > > > > > are
> > > > > > > > > many pending issues that could / should be part of the
> > release.
> > > > > What
> > > > > > do
> > > > > > > > you
> > > > > > > > > think would be a reasonable date to propose for a RC?
> Perhaps
> > > > 28th
> > > > > > > > > February? I would like to let us have plenty of time to
> > > > > collectively
> > > > > > > > review
> > > > > > > > > the PR candidates and "unblock" (to some extent) the PR
> > > situation
> > > > > > (see
> > > > > > > > also
> > > > > > > > > the recent separate thread on that matter [5]).
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > > Ruben
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > [1]
> > https://calcite.apache.org/news/2021/12/25/release-1.29.0/
> > > > > > > > > [2]
> > > > > https://lists.apache.org/thread/kqslh8yqvxbolk74vov3ofw19h90xkp2
> > > > > > > > > [3]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > > > > > > [4] https://github.com/apache/calcite/pulls
> > > > > > > > > [5]
> > > > > https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.30.0

[Fan Liya]

Hi Andrei,

Sure. No problem.

Best,
Liya Fan


Andrei Sereda  于2022年3月4日周五 12:28写道：

> Hello,
>
> Sorry for the delayed response. I was away for a couple of weeks.
>
> I see that 1.30.0 is being released by Liya.
>
> Can we swap Release Managers for 1.30 and 1.31 ? I'm happy to release
> Calcite 1.31.
>
> Thanks,
> Andrei.
>
>
>
> On Tue, Mar 1, 2022 at 9:40 PM Fan Liya  wrote:
>
> > Got it. Thanks for the confirmation.
> >
> > Best,
> > Liya Fan
> >
> > Stamatis Zampetakis  于2022年3月2日周三 05:30写道：
> >
> > > Since nobody objects, let's aim for Friday or whichever other day is
> > > convenient for you.
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Sun, Feb 27, 2022 at 2:22 PM Fan Liya  wrote:
> > >
> > > > Thanks for your confirmation, Stamatis.
> > > > How about next Friday (March 4th) to cut the first release candidate?
> > > > Is that enough to get the PRs inside?
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > > Stamatis Zampetakis  于2022年2月27日周日 05:57写道：
> > > >
> > > > > Many thanks Liya. Let's proceed like this then!
> > > > >
> > > > > Let us know when it would be the best time for you to cut the first
> > > > release
> > > > > candidate so that we plan appropriately and get some PRs inside.
> > > > >
> > > > > Best,
> > > > > Stamatis
> > > > >
> > > > > On Fri, Feb 25, 2022 at 7:03 AM Fan Liya 
> > wrote:
> > > > >
> > > > > > Sure. If Andrei is not available, I think I am OK.
> > > > > >
> > > > > > Best,
> > > > > > Liya Fan
> > > > > >
> > > > > >
> > > > > > Stamatis Zampetakis  于2022年2月24日周四 19:10写道：
> > > > > >
> > > > > > > Thanks for driving this Ruben!
> > > > > > >
> > > > > > > Another alternative if Andrei is unavailable at the moment
> would
> > be
> > > > to
> > > > > > see
> > > > > > > if they can switch places with Liya. Would this be possible
> Liya?
> > > > > > >
> > > > > > > Best,
> > > > > > > Stamatis
> > > > > > >
> > > > > > > On Thu, Feb 24, 2022 at 10:41 AM Ruben Q L 
> > > > wrote:
> > > > > > >
> > > > > > > > Friendly reminder.
> > > > > > > >
> > > > > > > > The next release managers would be:
> > > > > > > > - Andrei Sereda for 1.30.0
> > > > > > > > - Liya Fan for 1.31.0
> > > > > > > >
> > > > > > > > Andrei, are you available to be RM for 1.30? When would it
> be a
> > > > good
> > > > > > date
> > > > > > > > for a RC?
> > > > > > > >
> > > > > > > > Ruben.
> > > > > > > >
> > > > > > > > PS: Do we have any other volunteers for the subsequent
> > releases?
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Feb 16, 2022 at 12:59 AM xiong duan <
> > nobigo...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 to release 1.30.  Feb 28th sounds good to me.
> > > > > > > > >
> > > > > > > > > Julian Hyde  于2022年2月16日周三 08:27写道：
> > > > > > > > >
> > > > > > > > > > +1 to release 1.30 soon, to keep up the tempo. Feb 28th
> > > sounds
> > > > > good
> > > > > > > to
> > > > > > > > > > me. Thanks for pushing on this, Ruben.
> > > > > > > > > >
> > > > > > > > > > Andrei, Are you available to be RM for 1.30? When is a
> good
> > > > date
> > > > > > for
> > > > > > > > you?
> > > > > > > > > >
> > > > > > > > > > Julian
> > > > > > > > > >
> > > > > 

[Calcite Release 1.30.0]

[Fan Liya]

Hi all,

I am going to start the releasing process for Calcite 1.30.0. The following
JIRA items will be included in this release:

https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0

Please check. If there is no problem, I am going to freeze the code branch.

Best,
Liya Fan

Re: [Calcite Release 1.30.0]

[Fan Liya]

Done. Thanks for your kind reminder.

Best,
Liya Fan

Ruben Q L  于2022年3月4日周五 17:11写道：

> Thanks for pushing this forward.
>
> Just a small comment, according to our Jira dashboard [1] there are still
> several unresolved tickets with fix version 1.30.
> I guess they should be updated (remove fix version 1.30) if they are not
> going to be part of this release.
>
> Best,
> Ruben
>
> [1]
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
>
>
> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya  wrote:
>
> > Hi all,
> >
> > I am going to start the releasing process for Calcite 1.30.0. The
> following
> > JIRA items will be included in this release:
> >
> >
> >
> https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0
> >
> > Please check. If there is no problem, I am going to freeze the code
> branch.
> >
> > Best,
> > Liya Fan
> >
>

Re: [Calcite Release 1.30.0]

[Fan Liya]

Opened a PR for the release (https://github.com/apache/calcite/pull/2739).
Please take a look. Thanks.

Best,
Liya Fan


Fan Liya  于2022年3月4日周五 19:55写道：

> Done. Thanks for your kind reminder.
>
> Best,
> Liya Fan
>
> Ruben Q L  于2022年3月4日周五 17:11写道：
>
>> Thanks for pushing this forward.
>>
>> Just a small comment, according to our Jira dashboard [1] there are still
>> several unresolved tickets with fix version 1.30.
>> I guess they should be updated (remove fix version 1.30) if they are not
>> going to be part of this release.
>>
>> Best,
>> Ruben
>>
>> [1]
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
>>
>>
>> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya  wrote:
>>
>> > Hi all,
>> >
>> > I am going to start the releasing process for Calcite 1.30.0. The
>> following
>> > JIRA items will be included in this release:
>> >
>> >
>> >
>> https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0
>> >
>> > Please check. If there is no problem, I am going to freeze the code
>> branch.
>> >
>> > Best,
>> > Liya Fan
>> >
>>
>

Re: [Calcite Release 1.30.0]

[Fan Liya]

Hi all,

To sign the release package, it is required to upload my PGP key to the
KEYS file (https://dist.apache.org/repos/dist/release/calcite/KEYS)
However, only PMCs have the permission.
Is there anyone who can help? Thank you in advance.

The updated KEYS file is attached.

Best,
Liya Fan


Fan Liya  于2022年3月4日周五 22:28写道：

> Opened a PR for the release (https://github.com/apache/calcite/pull/2739).
>
> Please take a look. Thanks.
>
> Best,
> Liya Fan
>
>
> Fan Liya  于2022年3月4日周五 19:55写道：
>
>> Done. Thanks for your kind reminder.
>>
>> Best,
>> Liya Fan
>>
>> Ruben Q L  于2022年3月4日周五 17:11写道：
>>
>>> Thanks for pushing this forward.
>>>
>>> Just a small comment, according to our Jira dashboard [1] there are still
>>> several unresolved tickets with fix version 1.30.
>>> I guess they should be updated (remove fix version 1.30) if they are not
>>> going to be part of this release.
>>>
>>> Best,
>>> Ruben
>>>
>>> [1]
>>>
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
>>>
>>>
>>> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya  wrote:
>>>
>>> > Hi all,
>>> >
>>> > I am going to start the releasing process for Calcite 1.30.0. The
>>> following
>>> > JIRA items will be included in this release:
>>> >
>>> >
>>> >
>>> https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0
>>> >
>>> > Please check. If there is no problem, I am going to freeze the code
>>> branch.
>>> >
>>> > Best,
>>> > Liya Fan
>>> >
>>>
>>

Re: [Calcite Release 1.30.0]

[Fan Liya]

Hi Stamatis,

Thanks for your good suggestion.
I have uploaded the updated KEYS file here
https://gist.github.com/liyafan82/34472c56d29329c1d7b9185052c0376f

Best,
Liya Fan


Stamatis Zampetakis  于2022年3月4日周五 22:59写道：

> Hi Liya,
>
> Attachments are not allowed in most ASF lists.
>
> Export your public key according to the instructions in the KEYS file,
> upload it somewhere (e.g., your github repo), and send us the link to the
> key.
>
> Best,
> Stamatis
>
>
> On Fri, Mar 4, 2022 at 3:45 PM Fan Liya  wrote:
>
> > Hi all,
> >
> > To sign the release package, it is required to upload my PGP key to the
> > KEYS file (https://dist.apache.org/repos/dist/release/calcite/KEYS)
> > However, only PMCs have the permission.
> > Is there anyone who can help? Thank you in advance.
> >
> > The updated KEYS file is attached.
> >
> > Best,
> > Liya Fan
> >
> >
> > Fan Liya  于2022年3月4日周五 22:28写道：
> >
> >> Opened a PR for the release (
> https://github.com/apache/calcite/pull/2739).
> >>
> >> Please take a look. Thanks.
> >>
> >> Best,
> >> Liya Fan
> >>
> >>
> >> Fan Liya  于2022年3月4日周五 19:55写道：
> >>
> >>> Done. Thanks for your kind reminder.
> >>>
> >>> Best,
> >>> Liya Fan
> >>>
> >>> Ruben Q L  于2022年3月4日周五 17:11写道：
> >>>
> >>>> Thanks for pushing this forward.
> >>>>
> >>>> Just a small comment, according to our Jira dashboard [1] there are
> >>>> still
> >>>> several unresolved tickets with fix version 1.30.
> >>>> I guess they should be updated (remove fix version 1.30) if they are
> not
> >>>> going to be part of this release.
> >>>>
> >>>> Best,
> >>>> Ruben
> >>>>
> >>>> [1]
> >>>>
> >>>>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> >>>>
> >>>>
> >>>> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya  wrote:
> >>>>
> >>>> > Hi all,
> >>>> >
> >>>> > I am going to start the releasing process for Calcite 1.30.0. The
> >>>> following
> >>>> > JIRA items will be included in this release:
> >>>> >
> >>>> >
> >>>> >
> >>>>
> https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0
> >>>> >
> >>>> > Please check. If there is no problem, I am going to freeze the code
> >>>> branch.
> >>>> >
> >>>> > Best,
> >>>> > Liya Fan
> >>>> >
> >>>>
> >>>
>

Re: [Calcite Release 1.30.0]

[Fan Liya]

Hi Stamatis,

Thanks for your kind help and good suggestions.

Best,
Liya Fan

On Saturday, March 5, 2022, Stamatis Zampetakis  wrote:

> @Liya, @Jess: I added your signing keys to svn. Please check and ensure
> that everything looks good.
>
> @Liya: I saw you raised a PR for the release but this is not necessary.
> When you prepare the release candidate [1] people will review release notes
> and everything else as needed.
> Usually there is always an update of the release note post release so not
> need to worry too much about it.
>
> Best,
> Stamatis
>
> [1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate
>
> On Fri, Mar 4, 2022 at 5:04 PM Jess Balint  wrote:
>
> > Hi Stamatis,
> >
> > Can you add mine also please? I have appended it to Liya Fan's KEYS.txt:
> >
> > https://gist.github.com/jbalint/fe02278549c38ea037af44b3efdb5545
> >
> > Jess
> >
> > On Fri, Mar 4, 2022 at 9:41 AM Fan Liya  wrote:
> >
> > > Hi Stamatis,
> > >
> > > Thanks for your good suggestion.
> > > I have uploaded the updated KEYS file here
> > > https://gist.github.com/liyafan82/34472c56d29329c1d7b9185052c0376f
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > > Stamatis Zampetakis  于2022年3月4日周五 22:59写道：
> > >
> > > > Hi Liya,
> > > >
> > > > Attachments are not allowed in most ASF lists.
> > > >
> > > > Export your public key according to the instructions in the KEYS
> file,
> > > > upload it somewhere (e.g., your github repo), and send us the link to
> > the
> > > > key.
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > >
> > > > On Fri, Mar 4, 2022 at 3:45 PM Fan Liya 
> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > To sign the release package, it is required to upload my PGP key to
> > the
> > > > > KEYS file (https://dist.apache.org/repos/dist/release/calcite/KEYS
> )
> > > > > However, only PMCs have the permission.
> > > > > Is there anyone who can help? Thank you in advance.
> > > > >
> > > > > The updated KEYS file is attached.
> > > > >
> > > > > Best,
> > > > > Liya Fan
> > > > >
> > > > >
> > > > > Fan Liya  于2022年3月4日周五 22:28写道：
> > > > >
> > > > >> Opened a PR for the release (
> > > > https://github.com/apache/calcite/pull/2739).
> > > > >>
> > > > >> Please take a look. Thanks.
> > > > >>
> > > > >> Best,
> > > > >> Liya Fan
> > > > >>
> > > > >>
> > > > >> Fan Liya  于2022年3月4日周五 19:55写道：
> > > > >>
> > > > >>> Done. Thanks for your kind reminder.
> > > > >>>
> > > > >>> Best,
> > > > >>> Liya Fan
> > > > >>>
> > > > >>> Ruben Q L  于2022年3月4日周五 17:11写道：
> > > > >>>
> > > > >>>> Thanks for pushing this forward.
> > > > >>>>
> > > > >>>> Just a small comment, according to our Jira dashboard [1] there
> > are
> > > > >>>> still
> > > > >>>> several unresolved tickets with fix version 1.30.
> > > > >>>> I guess they should be updated (remove fix version 1.30) if they
> > are
> > > > not
> > > > >>>> going to be part of this release.
> > > > >>>>
> > > > >>>> Best,
> > > > >>>> Ruben
> > > > >>>>
> > > > >>>> [1]
> > > > >>>>
> > > > >>>>
> > > >
> > >
> > https://issues.apache.org/jira/secure/Dashboard.jspa?
> selectPageId=12333950
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya 
> > wrote:
> > > > >>>>
> > > > >>>> > Hi all,
> > > > >>>> >
> > > > >>>> > I am going to start the releasing process for Calcite 1.30.0.
> > The
> > > > >>>> following
> > > > >>>> > JIRA items will be included in this release:
> > > > >>>> >
> > > > >>>> >
> > > > >>>> >
> > > > >>>>
> > > >
> > >
> > https://issues.apache.org/jira/browse/CALCITE-1794?jql=
> project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%
> 20and%20fixVersion%20%3D%201.30.0
> > > > >>>> >
> > > > >>>> > Please check. If there is no problem, I am going to freeze the
> > > code
> > > > >>>> branch.
> > > > >>>> >
> > > > >>>> > Best,
> > > > >>>> > Liya Fan
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > >
> > >
> >
>

[VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Hi all,

I have created a build for Apache Calcite 1.30.0, release
candidate 0.

Thanks to everyone who has contributed to this release.

You can read the release notes here:
https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md

The commit to be voted upon:
https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e

Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e

Tag:
https://github.com/apache/calcite/tree/calcite-1.30.0-rc0

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
(revision 52873)

The hashes of the artifacts are as follows:
339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
*apache-calcite-1.30.0-src.tar.gz

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/COMMITTER_ID.asc
https://www.apache.org/dist/calcite/KEYS

To create the jars and test Apache Calcite: "gradle build"
(requires an appropriate Gradle/JDK installation)

Please vote on releasing this package as Apache Calcite 1.30.0.

The vote is open for the next 72 hours and passes if a majority of at
least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Calcite 1.30.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Here is my vote:

+1 (binding)

Best,
Liya Fan

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Since it is during the weekend, let's extend the voting period to at least
96 hours.

Best,
Liya Fan


Fan Liya  于2022年3月5日周六 17:56写道：

> Hi all,
>
> I have created a build for Apache Calcite 1.30.0, release
> candidate 0.
>
> Thanks to everyone who has contributed to this release.
>
> You can read the release notes here:
>
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
>
> The commit to be voted upon:
>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>
> Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>
> Tag:
> https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
> (revision 52873)
>
> The hashes of the artifacts are as follows:
>
> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
> *apache-calcite-1.30.0-src.tar.gz
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/COMMITTER_ID.asc
> https://www.apache.org/dist/calcite/KEYS
>
> To create the jars and test Apache Calcite: "gradle build"
> (requires an appropriate Gradle/JDK installation)
>
> Please vote on releasing this package as Apache Calcite 1.30.0.
>
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Calcite 1.30.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Here is my vote:
>
> +1 (binding)
>
> Best,
> Liya Fan
>
>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Hi Zhe,

Thanks for the feedback.
When preparing the build, the command seemed to finish successfully (no
error/exception reported, and the return value was 0).
However, by checking the log, it seems the pushRcTag failed to push the tag
to remote (maybe it is because I am using a different name for the apache
repository, which is not "origin").
Now I have manually pushed the tag, so it should be showing the release
note of 1.30.0. Please check.

Best,
Liya Fan


Zhe Hu  于2022年3月5日周六 19:11写道：

> Hi, Fan Liya.
> Thanks for your great job of releasing Calcite 1.30.0.
> I’m trying to read the release notes in
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md.
> However, the latest version of this page is still 1.29.0. I’m not sure if
> it’s my permission issue or not. But I can read it in your PR(
> https://github.com/apache/calcite/pull/2739/files).
> By the way, +1 for this release.
>
>
> Best,
> ZheHu
> On 03/5/2022 17:56，Fan Liya wrote：
> Hi all,
>
> I have created a build for Apache Calcite 1.30.0, release
> candidate 0.
>
> Thanks to everyone who has contributed to this release.
>
> You can read the release notes here:
>
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
>
> The commit to be voted upon:
>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>
> Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>
> Tag:
> https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
> (revision 52873)
>
> The hashes of the artifacts are as follows:
>
> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
> *apache-calcite-1.30.0-src.tar.gz
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/COMMITTER_ID.asc
> https://www.apache.org/dist/calcite/KEYS
>
> To create the jars and test Apache Calcite: "gradle build"
> (requires an appropriate Gradle/JDK installation)
>
> Please vote on releasing this package as Apache Calcite 1.30.0.
>
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Calcite 1.30.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Here is my vote:
>
> +1 (binding)
>
> Best,
> Liya Fan
>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Thanks for your feedback, Francis.
I will check.

Best,
Liya Fan


Francis Chuang  于2022年3月7日周一 09:36写道：

> Hey Liya,
>
> Thanks for being RM for this release.
>
> Can you check the commit to be voted upon?
>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> returns a 404
>
> Also, tag calcite-1.30.0-rc0 points to a39b9bf
> (https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0),
> which does not seem to be in the Calcite repository, see:
>
> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
>
> Thanks!
> Francis
>
> On 5/03/2022 8:56 pm, Fan Liya wrote:
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.30.0, release
> > candidate 0.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >
> > Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
> > (revision 52873)
> >
> > The hashes of the artifacts are as follows:
> >
> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
> > *apache-calcite-1.30.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/COMMITTER_ID.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.30.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.30.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> > Best,
> > Liya Fan
> >
>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

After some investigation, I could not resolve the 404 issue of the commit
link (In my computer, it is not 404, but an error with XML format).
So I have created a new build, which has resolved all the above issues
(according to the test on my local computer).

Let's start another vote based on the new build. Thanks.

Best,
Liya Fan


Fan Liya  于2022年3月7日周一 09:53写道：

> Thanks for your feedback, Francis.
> I will check.
>
> Best,
> Liya Fan
>
>
> Francis Chuang  于2022年3月7日周一 09:36写道：
>
>> Hey Liya,
>>
>> Thanks for being RM for this release.
>>
>> Can you check the commit to be voted upon?
>>
>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>> returns a 404
>>
>> Also, tag calcite-1.30.0-rc0 points to a39b9bf
>> (https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0),
>> which does not seem to be in the Calcite repository, see:
>>
>> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
>>
>> Thanks!
>> Francis
>>
>> On 5/03/2022 8:56 pm, Fan Liya wrote:
>> > Hi all,
>> >
>> > I have created a build for Apache Calcite 1.30.0, release
>> > candidate 0.
>> >
>> > Thanks to everyone who has contributed to this release.
>> >
>> > You can read the release notes here:
>> >
>> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
>> >
>> > The commit to be voted upon:
>> >
>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>> >
>> > Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>> >
>> > Tag:
>> > https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
>> >
>> > The artifacts to be voted on are located here:
>> >
>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
>> > (revision 52873)
>> >
>> > The hashes of the artifacts are as follows:
>> >
>> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
>> > *apache-calcite-1.30.0-src.tar.gz
>> >
>> > A staged Maven repository is available for review at:
>> >
>> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
>> >
>> > Release artifacts are signed with the following key:
>> > https://people.apache.org/keys/committer/COMMITTER_ID.asc
>> > https://www.apache.org/dist/calcite/KEYS
>> >
>> > To create the jars and test Apache Calcite: "gradle build"
>> > (requires an appropriate Gradle/JDK installation)
>> >
>> > Please vote on releasing this package as Apache Calcite 1.30.0.
>> >
>> > The vote is open for the next 72 hours and passes if a majority of at
>> > least three +1 PMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Calcite 1.30.0
>> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> > [ ] -1 Do not release this package because...
>> >
>> > Here is my vote:
>> >
>> > +1 (binding)
>> >
>> > Best,
>> > Liya Fan
>> >
>>
>

[VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Hi all,

I have created a build for Apache Calcite 1.30.0, release
candidate 0.

Thanks to everyone who has contributed to this release.

You can read the release notes here:
https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md

The commit to be voted upon:
https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=f14cf4c32b9079984a988bbad40230aa6a59b127

Its hash is f14cf4c32b9079984a988bbad40230aa6a59b127

Tag:
https://github.com/apache/calcite/tree/calcite-1.30.0-rc0

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
(revision 52896)

The hashes of the artifacts are as follows:
25527b5dfd3c28d4ac3c9d9a40b94dbcbce7feb17cc198ebe7c36a30c2df69a27ee5e4defab4edff1e1bc65c076bd0725bd8c378913d1d23f3d80f732e3e097f
*apache-calcite-1.30.0-src.tar.gz

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachecalcite-1149/org/apache/calcite/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/COMMITTER_ID.asc
https://www.apache.org/dist/calcite/KEYS

To create the jars and test Apache Calcite: "gradle build"
(requires an appropriate Gradle/JDK installation)

Please vote on releasing this package as Apache Calcite 1.30.0.

The vote is open for the next 72 hours and passes if a majority of at
least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Calcite 1.30.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Here is my vote:

+1 (binding)

Best,
Liya Fan

Fan Liya  于2022年3月7日周一 14:27写道：

> After some investigation, I could not resolve the 404 issue of the commit
> link (In my computer, it is not 404, but an error with XML format).
> So I have created a new build, which has resolved all the above issues
> (according to the test on my local computer).
>
> Let's start another vote based on the new build. Thanks.
>
> Best,
> Liya Fan
>
>
> Fan Liya  于2022年3月7日周一 09:53写道：
>
>> Thanks for your feedback, Francis.
>> I will check.
>>
>> Best,
>> Liya Fan
>>
>>
>> Francis Chuang  于2022年3月7日周一 09:36写道：
>>
>>> Hey Liya,
>>>
>>> Thanks for being RM for this release.
>>>
>>> Can you check the commit to be voted upon?
>>>
>>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>>> returns a 404
>>>
>>> Also, tag calcite-1.30.0-rc0 points to a39b9bf
>>> (https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0),
>>> which does not seem to be in the Calcite repository, see:
>>>
>>> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
>>>
>>> Thanks!
>>> Francis
>>>
>>> On 5/03/2022 8:56 pm, Fan Liya wrote:
>>> > Hi all,
>>> >
>>> > I have created a build for Apache Calcite 1.30.0, release
>>> > candidate 0.
>>> >
>>> > Thanks to everyone who has contributed to this release.
>>> >
>>> > You can read the release notes here:
>>> >
>>> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
>>> >
>>> > The commit to be voted upon:
>>> >
>>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>>> >
>>> > Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
>>> >
>>> > Tag:
>>> > https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
>>> >
>>> > The artifacts to be voted on are located here:
>>> >
>>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
>>> > (revision 52873)
>>> >
>>> > The hashes of the artifacts are as follows:
>>> >
>>> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
>>> > *apache-calcite-1.30.0-src.tar.gz
>>> >
>>> > A staged Maven repository is available for review at:
>>> >
>>> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
>>> >
>>> > Release artifacts are signed with the following key:
>>> > https://people.apache.org/keys/committer/COMMITTER_ID.asc
>>> > https://www.apache.org/dist/calcite/KEYS
>>> >
>>> > To create the jars and test Apache Calcite: "gradle build"
>>> > (requires an appropriate Gradle/JDK installation)
>>> >
>>> > Please vote on releasing this package as Apache Calcite 1.30.0.
>>> >
>>> > The vote is open for the next 72 hours and passes if a majority of at
>>> > least three +1 PMC votes are cast.
>>> >
>>> > [ ] +1 Release this package as Apache Calcite 1.30.0
>>> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
>>> > [ ] -1 Do not release this package because...
>>> >
>>> > Here is my vote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Best,
>>> > Liya Fan
>>> >
>>>
>>

[CANCEL][VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Cancel this thread, due to some problems in the build.
Thanks to Francis for the kind reminder.
Will start another vote based a new build soon.

Best,
Liya Fan

Francis Chuang  于2022年3月7日周一 14:31写道：

> Thanks so much Liya! Please cancel this vote by sending a [CANCEL]
> message as well, to formalize the cancellation of this vote.
>
> Francis
>
> On 7/03/2022 5:27 pm, Fan Liya wrote:
> > After some investigation, I could not resolve the 404 issue of the commit
> > link (In my computer, it is not 404, but an error with XML format).
> > So I have created a new build, which has resolved all the above issues
> > (according to the test on my local computer).
> >
> > Let's start another vote based on the new build. Thanks.
> >
> > Best,
> > Liya Fan
> >
> >
> > Fan Liya  于2022年3月7日周一 09:53写道：
> >
> >> Thanks for your feedback, Francis.
> >> I will check.
> >>
> >> Best,
> >> Liya Fan
> >>
> >>
> >> Francis Chuang  于2022年3月7日周一 09:36写道：
> >>
> >>> Hey Liya,
> >>>
> >>> Thanks for being RM for this release.
> >>>
> >>> Can you check the commit to be voted upon?
> >>>
> >>>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>> returns a 404
> >>>
> >>> Also, tag calcite-1.30.0-rc0 points to a39b9bf
> >>> (https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0),
> >>> which does not seem to be in the Calcite repository, see:
> >>>
> >>>
> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
> >>>
> >>> Thanks!
> >>> Francis
> >>>
> >>> On 5/03/2022 8:56 pm, Fan Liya wrote:
> >>>> Hi all,
> >>>>
> >>>> I have created a build for Apache Calcite 1.30.0, release
> >>>> candidate 0.
> >>>>
> >>>> Thanks to everyone who has contributed to this release.
> >>>>
> >>>> You can read the release notes here:
> >>>>
> >>>
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
> >>>>
> >>>> The commit to be voted upon:
> >>>>
> >>>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>>>
> >>>> Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>>>
> >>>> Tag:
> >>>> https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
> >>>>
> >>>> The artifacts to be voted on are located here:
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
> >>>> (revision 52873)
> >>>>
> >>>> The hashes of the artifacts are as follows:
> >>>>
> >>>
> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
> >>>> *apache-calcite-1.30.0-src.tar.gz
> >>>>
> >>>> A staged Maven repository is available for review at:
> >>>>
> >>>
> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
> >>>>
> >>>> Release artifacts are signed with the following key:
> >>>> https://people.apache.org/keys/committer/COMMITTER_ID.asc
> >>>> https://www.apache.org/dist/calcite/KEYS
> >>>>
> >>>> To create the jars and test Apache Calcite: "gradle build"
> >>>> (requires an appropriate Gradle/JDK installation)
> >>>>
> >>>> Please vote on releasing this package as Apache Calcite 1.30.0.
> >>>>
> >>>> The vote is open for the next 72 hours and passes if a majority of at
> >>>> least three +1 PMC votes are cast.
> >>>>
> >>>> [ ] +1 Release this package as Apache Calcite 1.30.0
> >>>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> >>>> [ ] -1 Do not release this package because...
> >>>>
> >>>> Here is my vote:
> >>>>
> >>>> +1 (binding)
> >>>>
> >>>> Best,
> >>>> Liya Fan
> >>>>
> >>>
> >>
> >
>

[VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Fan Liya]

Hi all,

I have created a build for Apache Calcite 1.30.0, release
candidate 1.

Thanks to everyone who has contributed to this release.

You can read the release notes here:
https://github.com/apache/calcite/blob/calcite-1.30.0-rc1/site/_docs/history.md

The commit to be voted upon:
https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=f14cf4c32b9079984a988bbad40230aa6a59b127

Its hash is f14cf4c32b9079984a988bbad40230aa6a59b127

Tag:
https://github.com/apache/calcite/tree/calcite-1.30.0-rc1

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc1
(revision 52897)

The hashes of the artifacts are as follows:
25527b5dfd3c28d4ac3c9d9a40b94dbcbce7feb17cc198ebe7c36a30c2df69a27ee5e4defab4edff1e1bc65c076bd0725bd8c378913d1d23f3d80f732e3e097f
*apache-calcite-1.30.0-src.tar.gz

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachecalcite-1150/org/apache/calcite/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/COMMITTER_ID.asc
https://www.apache.org/dist/calcite/KEYS

To create the jars and test Apache Calcite: "gradle build"
(requires an appropriate Gradle/JDK installation)

Please vote on releasing this package as Apache Calcite 1.30.0.

The vote is open for the next 72 hours and passes if a majority of at
least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Calcite 1.30.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Here is my vote:

+1 (binding)

Best,
Liya Fan

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Fan Liya]

Hi Xiong,

Thanks for your feedback. I will change the logs in a follow up MR.

Best,
Liya Fan


xiong duan  于2022年3月7日周一 15:30写道：

> Hi, Liya Fan.Thanks for your work.  +1.
> Just a little problem:
> I noticed the Bug-fixes logs all started with FixI think maybe we
> should stay the same as JIRA summary or PR commit?
>
> Fan Liya  于2022年3月7日周一 14:55写道：
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.30.0, release
> > candidate 1.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> >
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc1/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=f14cf4c32b9079984a988bbad40230aa6a59b127
> >
> > Its hash is f14cf4c32b9079984a988bbad40230aa6a59b127
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.30.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc1
> > (revision 52897)
> >
> > The hashes of the artifacts are as follows:
> >
> >
> 25527b5dfd3c28d4ac3c9d9a40b94dbcbce7feb17cc198ebe7c36a30c2df69a27ee5e4defab4edff1e1bc65c076bd0725bd8c378913d1d23f3d80f732e3e097f
> > *apache-calcite-1.30.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1150/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/COMMITTER_ID.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.30.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.30.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> > Best,
> > Liya Fan
> >
>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Hi Julian,

Thanks for your feedback. I will prepare another build, after addressing
these issues.

Best,
Liya Fan


Julian Hyde  于2022年3月8日周二 04:50写道：

> Liya, in the next iteration can you go back to the usual sections e.g
> “Bug-fixes, API changes and minor enhancements”. Also remove the “Fix”
> prefix from messages.
>
> Quite a few cases, e.g “ Fix the problem that JDBC adapter incorrectly
> adds ORDER BY columns to the SELECT list of generated SQL query” are in the
> gray area between bug and missing feature, and using the original message
> helps people make their own decision.
>
> I also think there’s value in putting changes to build and test in a
> separate section, as we used to do. But please move my “Fluent test
> fixtures” into the “new features” section - because it is a major new
> feature.
>
> Julian
>
> > On Mar 6, 2022, at 10:31 PM, Francis Chuang 
> wrote:
> >
> > Thanks so much Liya! Please cancel this vote by sending a [CANCEL]
> message as well, to formalize the cancellation of this vote.
> >
> > Francis
> >
> >> On 7/03/2022 5:27 pm, Fan Liya wrote:
> >> After some investigation, I could not resolve the 404 issue of the
> commit
> >> link (In my computer, it is not 404, but an error with XML format).
> >> So I have created a new build, which has resolved all the above issues
> >> (according to the test on my local computer).
> >> Let's start another vote based on the new build. Thanks.
> >> Best,
> >> Liya Fan
> >> Fan Liya  于2022年3月7日周一 09:53写道：
> >>> Thanks for your feedback, Francis.
> >>> I will check.
> >>>
> >>> Best,
> >>> Liya Fan
> >>>
> >>>
> >>> Francis Chuang  于2022年3月7日周一 09:36写道：
> >>>
> >>>> Hey Liya,
> >>>>
> >>>> Thanks for being RM for this release.
> >>>>
> >>>> Can you check the commit to be voted upon?
> >>>>
> >>>>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>>> returns a 404
> >>>>
> >>>> Also, tag calcite-1.30.0-rc0 points to a39b9bf
> >>>> (https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0),
> >>>> which does not seem to be in the Calcite repository, see:
> >>>>
> >>>>
> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
> >>>>
> >>>> Thanks!
> >>>> Francis
> >>>>
> >>>> On 5/03/2022 8:56 pm, Fan Liya wrote:
> >>>>> Hi all,
> >>>>>
> >>>>> I have created a build for Apache Calcite 1.30.0, release
> >>>>> candidate 0.
> >>>>>
> >>>>> Thanks to everyone who has contributed to this release.
> >>>>>
> >>>>> You can read the release notes here:
> >>>>>
> >>>>
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc0/site/_docs/history.md
> >>>>>
> >>>>> The commit to be voted upon:
> >>>>>
> >>>>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>>>>
> >>>>> Its hash is 28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> >>>>>
> >>>>> Tag:
> >>>>> https://github.com/apache/calcite/tree/calcite-1.30.0-rc0
> >>>>>
> >>>>> The artifacts to be voted on are located here:
> >>>>>
> >>>>
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc0
> >>>>> (revision 52873)
> >>>>>
> >>>>> The hashes of the artifacts are as follows:
> >>>>>
> >>>>
> 339b279426abfc09a73b0daf94479127aa469b1cd3fa1b1cf0842f57d64daf0e080d4e85ccdd2d32032ed424a39388790743adf2aec869480612e37003eabfc5
> >>>>> *apache-calcite-1.30.0-src.tar.gz
> >>>>>
> >>>>> A staged Maven repository is available for review at:
> >>>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachecalcite-1148/org/apache/calcite/
> >>>>>
> >>>>> Release artifacts are signed with the following key:
> >>>>> https://people.apache.org/keys/committer/COMMITTER_ID.asc
> >>>>> https://www.apache.org/dist/calcite/KEYS
> >>>>>
> >>>>> To create the jars and test Apache Calcite: "gradle build"
> >>>>> (requires an appropriate Gradle/JDK installation)
> >>>>>
> >>>>> Please vote on releasing this package as Apache Calcite 1.30.0.
> >>>>>
> >>>>> The vote is open for the next 72 hours and passes if a majority of at
> >>>>> least three +1 PMC votes are cast.
> >>>>>
> >>>>> [ ] +1 Release this package as Apache Calcite 1.30.0
> >>>>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> >>>>> [ ] -1 Do not release this package because...
> >>>>>
> >>>>> Here is my vote:
> >>>>>
> >>>>> +1 (binding)
> >>>>>
> >>>>> Best,
> >>>>> Liya Fan
> >>>>>
> >>>>
> >>>
>

Re: Bugs dropped on the floor

[Fan Liya]

Sorry. I will restore the fix-version.
It seems CALCITE-4976 is duplicate with CALCITE-5031. So I will close
CALCITE-5031.

Best,
Liya Fan


Julian Hyde  于2022年3月8日周二 06:25写道：

> Especially https://issues.apache.org/jira/browse/CALCITE-4976 <
> https://issues.apache.org/jira/browse/CALCITE-4976> whose title is
> literally “Release Calcite 1.30.0”, no longer has fix-version 1.30. Good
> grief.
>
>
> > On Mar 7, 2022, at 2:22 PM, Julian Hyde  wrote:
> >
> > There were a few bugs that had fix-version = 1.30 because they had PRs
> that looked like they were going to make it. Liya has removed the fix
> version.
> >
> > https://issues.apache.org/jira/browse/CALCITE-4908 <
> https://issues.apache.org/jira/browse/CALCITE-4908>
> > https://issues.apache.org/jira/browse/CALCITE-2552 <
> https://issues.apache.org/jira/browse/CALCITE-2552>
> > https://issues.apache.org/jira/browse/CALCITE-4987 <
> https://issues.apache.org/jira/browse/CALCITE-4987>
> > https://issues.apache.org/jira/browse/CALCITE-4913 <
> https://issues.apache.org/jira/browse/CALCITE-4913>
> >
> > Is that where we’re going to leave it?
> >
> > It breaks my heart that there we are ignoring good PRs.
> >
> > Julian
> >
>
>

Re: Bugs dropped on the floor

[Fan Liya]

Hi Ruben,

Good suggestion. Thanks.

I removed the fix version for above JIRAs because I could not assign 1.31.0
to them.
Version 1.31.0 has not been created, and I have no permission to create it.
Could someone with permission please create it?

Best,
Liya Fan


Fan Liya  于2022年3月8日周二 07:37写道：

> Sorry. I will restore the fix-version.
> It seems CALCITE-4976 is duplicate with CALCITE-5031. So I will close
> CALCITE-5031.
>
> Best,
> Liya Fan
>
>
> Julian Hyde  于2022年3月8日周二 06:25写道：
>
>> Especially https://issues.apache.org/jira/browse/CALCITE-4976 <
>> https://issues.apache.org/jira/browse/CALCITE-4976> whose title is
>> literally “Release Calcite 1.30.0”, no longer has fix-version 1.30. Good
>> grief.
>>
>>
>> > On Mar 7, 2022, at 2:22 PM, Julian Hyde  wrote:
>> >
>> > There were a few bugs that had fix-version = 1.30 because they had PRs
>> that looked like they were going to make it. Liya has removed the fix
>> version.
>> >
>> > https://issues.apache.org/jira/browse/CALCITE-4908 <
>> https://issues.apache.org/jira/browse/CALCITE-4908>
>> > https://issues.apache.org/jira/browse/CALCITE-2552 <
>> https://issues.apache.org/jira/browse/CALCITE-2552>
>> > https://issues.apache.org/jira/browse/CALCITE-4987 <
>> https://issues.apache.org/jira/browse/CALCITE-4987>
>> > https://issues.apache.org/jira/browse/CALCITE-4913 <
>> https://issues.apache.org/jira/browse/CALCITE-4913>
>> >
>> > Is that where we’re going to leave it?
>> >
>> > It breaks my heart that there we are ignoring good PRs.
>> >
>> > Julian
>> >
>>
>>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 0)

[Fan Liya]

Hi Ruben,

Thanks for your suggestion. Let's go ahead with it.

Best,
Liya Fan


Ruben Q L  于2022年3月8日周二 07:49写道：

> Hello Liya Fan,
>
> please be aware that, normally, when there are issues ONLY on the release
> notes, we do not require a new RC (they can be amended afterwards with a
> commit right after the release).
> So, I'd say that for the moment we can just cancel RC0 vote (an email
> "[CANCEL] [VOTE] Release Apache Calcite1.30.0 (release candidate 0)" should
> be sent to make it official), and keep RC1 vote open.
>
> Best,
> Ruben
>
>
> On Mon, Mar 7, 2022 at 11:34 PM Fan Liya  wrote:
>
> > Hi Julian,
> >
> > Thanks for your feedback. I will prepare another build, after addressing
> > these issues.
> >
> > Best,
> > Liya Fan
> >
> >
> > Julian Hyde  于2022年3月8日周二 04:50写道：
> >
> > > Liya, in the next iteration can you go back to the usual sections e.g
> > > “Bug-fixes, API changes and minor enhancements”. Also remove the “Fix”
> > > prefix from messages.
> > >
> > > Quite a few cases, e.g “ Fix the problem that JDBC adapter incorrectly
> > > adds ORDER BY columns to the SELECT list of generated SQL query” are in
> > the
> > > gray area between bug and missing feature, and using the original
> message
> > > helps people make their own decision.
> > >
> > > I also think there’s value in putting changes to build and test in a
> > > separate section, as we used to do. But please move my “Fluent test
> > > fixtures” into the “new features” section - because it is a major new
> > > feature.
> > >
> > > Julian
> > >
> > > > On Mar 6, 2022, at 10:31 PM, Francis Chuang <
> francischu...@apache.org>
> > > wrote:
> > > >
> > > > Thanks so much Liya! Please cancel this vote by sending a [CANCEL]
> > > message as well, to formalize the cancellation of this vote.
> > > >
> > > > Francis
> > > >
> > > >> On 7/03/2022 5:27 pm, Fan Liya wrote:
> > > >> After some investigation, I could not resolve the 404 issue of the
> > > commit
> > > >> link (In my computer, it is not 404, but an error with XML format).
> > > >> So I have created a new build, which has resolved all the above
> issues
> > > >> (according to the test on my local computer).
> > > >> Let's start another vote based on the new build. Thanks.
> > > >> Best,
> > > >> Liya Fan
> > > >> Fan Liya  于2022年3月7日周一 09:53写道：
> > > >>> Thanks for your feedback, Francis.
> > > >>> I will check.
> > > >>>
> > > >>> Best,
> > > >>> Liya Fan
> > > >>>
> > > >>>
> > > >>> Francis Chuang  于2022年3月7日周一 09:36写道：
> > > >>>
> > > >>>> Hey Liya,
> > > >>>>
> > > >>>> Thanks for being RM for this release.
> > > >>>>
> > > >>>> Can you check the commit to be voted upon?
> > > >>>>
> > > >>>>
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=28aacc0bbccfbcd821fece72ae3d92f6c1db8a5e
> > > >>>> returns a 404
> > > >>>>
> > > >>>> Also, tag calcite-1.30.0-rc0 points to a39b9bf
> > > >>>> (
> https://github.com/apache/calcite/releases/tag/calcite-1.30.0-rc0
> > ),
> > > >>>> which does not seem to be in the Calcite repository, see:
> > > >>>>
> > > >>>>
> > >
> >
> https://github.com/apache/calcite/commit/a39b9bf8b1cbab8f4462c41f6b4506753f859f96
> > > >>>>
> > > >>>> Thanks!
> > > >>>> Francis
> > > >>>>
> > > >>>> On 5/03/2022 8:56 pm, Fan Liya wrote:
> > > >>>>> Hi all,
> > > >>>>>
> > > >>>>> I have created a build for Apache Calcite 1.30.0, release
> > > >>>>> candidate 0.
> > > >>>>>
> > > >>>>> Thanks to everyone who has contributed to this release.
> > > >>>>>
> > > >>>>> You can read the release notes here:
> > > >>>>>
> > > >>>>
> > >
> >
> https://github.com/

Re: Bugs dropped on the floor

[Fan Liya]

Hi Francis and Ruben,

Thanks for your kind help.
I have set the fix version of the following JIRAs to 1.31.0 (their original
fix version was 1.30.0). Please check.

CALCITE-5021
CALCITE-4992
CALCITE-4990
CALCITE-4987
CALCITE-4976
CALCITE-4913
CALCITE-4908
CALCITE-4887
CALCITE-2552

Best,
Liya Fan


Ruben Q L  于2022年3月8日周二 07:54写道：

> I have created version 1.31.0 myself, it should be already visible.
>
> Ruben
>
>
> On Mon, Mar 7, 2022 at 11:51 PM Francis Chuang 
> wrote:
>
> > Hey Liya,
> >
> > I've added you to the administrator role, so you should be able to
> > create the 1.31.0 release version in jira.
> >
> > Francis
> >
> > On 8/03/2022 10:44 am, Fan Liya wrote:
> > > Hi Ruben,
> > >
> > > Good suggestion. Thanks.
> > >
> > > I removed the fix version for above JIRAs because I could not assign
> > 1.31.0
> > > to them.
> > > Version 1.31.0 has not been created, and I have no permission to create
> > it.
> > > Could someone with permission please create it?
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > > Fan Liya  于2022年3月8日周二 07:37写道：
> > >
> > >> Sorry. I will restore the fix-version.
> > >> It seems CALCITE-4976 is duplicate with CALCITE-5031. So I will close
> > >> CALCITE-5031.
> > >>
> > >> Best,
> > >> Liya Fan
> > >>
> > >>
> > >> Julian Hyde  于2022年3月8日周二 06:25写道：
> > >>
> > >>> Especially https://issues.apache.org/jira/browse/CALCITE-4976 <
> > >>> https://issues.apache.org/jira/browse/CALCITE-4976> whose title is
> > >>> literally “Release Calcite 1.30.0”, no longer has fix-version 1.30.
> > Good
> > >>> grief.
> > >>>
> > >>>
> > >>>> On Mar 7, 2022, at 2:22 PM, Julian Hyde 
> > wrote:
> > >>>>
> > >>>> There were a few bugs that had fix-version = 1.30 because they had
> PRs
> > >>> that looked like they were going to make it. Liya has removed the fix
> > >>> version.
> > >>>>
> > >>>> https://issues.apache.org/jira/browse/CALCITE-4908 <
> > >>> https://issues.apache.org/jira/browse/CALCITE-4908>
> > >>>> https://issues.apache.org/jira/browse/CALCITE-2552 <
> > >>> https://issues.apache.org/jira/browse/CALCITE-2552>
> > >>>> https://issues.apache.org/jira/browse/CALCITE-4987 <
> > >>> https://issues.apache.org/jira/browse/CALCITE-4987>
> > >>>> https://issues.apache.org/jira/browse/CALCITE-4913 <
> > >>> https://issues.apache.org/jira/browse/CALCITE-4913>
> > >>>>
> > >>>> Is that where we’re going to leave it?
> > >>>>
> > >>>> It breaks my heart that there we are ignoring good PRs.
> > >>>>
> > >>>> Julian
> > >>>>
> > >>>
> > >>>
> > >
> >
>

Re: Broken history after force pushing to master

[Fan Liya]

Hi all,

I think the broken history was caused by this:

1. In document "Making a release candidate [1]", it says "Make sure master
branch and site branch are in sync".
2. I checked the two branches, and find they have diverged. Some commits in
the site branch are not in the master branch.
3. I tried the method given in the document "git reset --hard site", but it
didn't work.
3. I tried to cherry-pick the commits to master, but it required resolving
conflicts, because the committing order was not correct.
4. So I used "git rebase -i" to insert the commits into the "right" place
of the master branch.
5. Finally, I pushed the result to the original master branch.

I think that is the reason for the broken history. Really sorry for the
trouble.
If needed, I can restore the original master branch. I have backed up the
branch.

Best,
Liya Fan

[1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate

xiong duan  于2022年3月9日周三 19:35写道：

> Hi. Stamatis. I agree we need to address this issue first.
> I find some relative descriptions at end of the email
> https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4. So it is
> a force push. Sorry I am not very good at Github job flow. But I think it
> describes what happened according to the appearances. So I hope this can
> help.
>
> This update added new revisions after undoing existing revisions. That is
> to say, some revisions that were in the old version of the branch are not
> in the new version. This situation occurs when a user --force pushes a
> change and generates a repository containing something like this: * -- * --
> B -- O -- O -- O (dcbc493) \ N -- N -- N refs/heads/master (c3dbf52) You
> should already have received notification emails for all of the O
> revisions, and so the following emails describe only the N revisions from
> the common base, B. Any revisions marked "omit" are not gone; other
> references still refer to them. Any revisions marked "discard" are gone
> forever. The 41 revisions listed above as "new" are entirely new to this
> repository and will be described in separate emails. The revisions listed
> as "add" were already present in the repository and have only been added to
> this reference.
>
> Stamatis Zampetakis  于2022年3月9日周三 18:08写道：
>
> > Hi all,
> >
> > Something happened during the generation of the 1.30.0 release candidate
> > and the git history is somewhat broken.
> >
> > If you use the GitHub repo and you try to pull (DON'T DO IT NOW) changes
> > from master to update your local copy you will see that a merge commit is
> > necessary which is not normal.
> >
> > Moreover, if you check the JIRAs resolved in this release (e.g.,
> > CALCITE-4991 [1]) you will notice that the comment [2] which indicates
> the
> > commit resolving the issue does not belong to any repository.
> >
> > From the above it seems there has been a force push to master. Looking at
> > the recent commits [3], I see something like a big rebase but not sure
> how
> > we ended up with this situation and why it was necessary.
> >
> > Going forward, I think the first step is to understand what happened so
> > that we avoid this reappearing in the future and the second step is to
> > restore the master branch (and others if affected) to its previous state
> > from someone's valid local copy; probably this will necessitate another
> > force-push.
> >
> > I am not doing anything for now till we agree on how we want to address
> > this issue.
> >
> > Best,
> > Stamatis
> >
> > [1] https://issues.apache.org/jira/browse/CALCITE-4991
> > [2]
> >
> >
> https://issues.apache.org/jira/browse/CALCITE-4991?focusedCommentId=17480091&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17480091
> > [3] https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4
> >
>

Re: Broken history after force pushing to master

[Fan Liya]

e we can force push master to site after the release is finalized.
> >
> > Best,
> > Stamatis
> >
> > [1] https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4
> > [2] https://lists.apache.org/thread/rvngk5tygfoyoc0klhwpo717mrngkdrw
> >
> >
> > On Wed, Mar 9, 2022 at 6:44 PM Ruben Q L  wrote:
> >
> >> Hello Liya,
> >>
> >> No worries, we all make mistakes.
> >> I think the sequence of steps that you describe looks like a plausible
> >> explanation for how we get into this situation. Do you know (from step
> 2)
> >> which commits were in site branch that were not in master?
> >> If in the future you (or anybody else) get blocked or experience any
> >> problem on a certain step during the release process, do not hesitate to
> >> send an email to the dev list with subject "[HELP] ..." describing the
> >> issue. In my experience, someone from the community will assist
> relatively
> >> fast.
> >>
> >> Any git expert with a clear idea on how to restore the master branch?
> >>
> >> Best,
> >> Ruben
> >>
> >>
> >> On Wed, Mar 9, 2022 at 1:32 PM Fan Liya  wrote:
> >>
> >>> Hi all,
> >>>
> >>> I think the broken history was caused by this:
> >>>
> >>> 1. In document "Making a release candidate [1]", it says "Make sure
> >> master
> >>> branch and site branch are in sync".
> >>> 2. I checked the two branches, and find they have diverged. Some
> commits
> >> in
> >>> the site branch are not in the master branch.
> >>> 3. I tried the method given in the document "git reset --hard site",
> but
> >> it
> >>> didn't work.
> >>> 3. I tried to cherry-pick the commits to master, but it required
> >> resolving
> >>> conflicts, because the committing order was not correct.
> >>> 4. So I used "git rebase -i" to insert the commits into the "right"
> place
> >>> of the master branch.
> >>> 5. Finally, I pushed the result to the original master branch.
> >>>
> >>> I think that is the reason for the broken history. Really sorry for the
> >>> trouble.
> >>> If needed, I can restore the original master branch. I have backed up
> the
> >>> branch.
> >>>
> >>> Best,
> >>> Liya Fan
> >>>
> >>> [1]
> >> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> >>>
> >>> xiong duan  于2022年3月9日周三 19:35写道：
> >>>
> >>>> Hi. Stamatis. I agree we need to address this issue first.
> >>>> I find some relative descriptions at end of the email
> >>>> https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4. So
> >> it
> >>> is
> >>>> a force push. Sorry I am not very good at Github job flow. But I think
> >> it
> >>>> describes what happened according to the appearances. So I hope this
> >> can
> >>>> help.
> >>>>
> >>>> This update added new revisions after undoing existing revisions. That
> >> is
> >>>> to say, some revisions that were in the old version of the branch are
> >> not
> >>>> in the new version. This situation occurs when a user --force pushes a
> >>>> change and generates a repository containing something like this: * --
> >> *
> >>> --
> >>>> B -- O -- O -- O (dcbc493) \ N -- N -- N refs/heads/master (c3dbf52)
> >> You
> >>>> should already have received notification emails for all of the O
> >>>> revisions, and so the following emails describe only the N revisions
> >> from
> >>>> the common base, B. Any revisions marked "omit" are not gone; other
> >>>> references still refer to them. Any revisions marked "discard" are
> gone
> >>>> forever. The 41 revisions listed above as "new" are entirely new to
> >> this
> >>>> repository and will be described in separate emails. The revisions
> >> listed
> >>>> as "add" were already present in the repository and have only been
> >> added
> >>> to
> >>>> this reference.
> >>>>
> >>>> Stamatis Zampetakis  于2022年3月9日周三 18:08写道：
> >>>>
> >

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Fan Liya]

Hi Ruben and Julian,

Thanks a lot for your feedback, I will prepare a PR addressing all the
issues described above.

Best,
Liya Fan


Julian Hyde  于2022年3月10日周四 08:03写道：

> Liya,
>
> I made much the same comments as Ruben regarding the release notes.
>
> Though we don’t usually use PRs for releases, I think it would be useful
> if we had a PR in which to review the release notes. It seems better than
> bike-shedding them during the actual release vote. If you agree, can you
> post that PR?
>
> Julian
>
>
> > On Mar 9, 2022, at 2:59 AM, Ruben Q L  wrote:
> >
> > Thanks Liya Fan for being the release manager.
> >
> > First of all, for the record regarding the RC itself:
> > - Checksum and signature: ok
> > - Gradle test: ok
> > - Calcite-based application test suite: ok
> >
> > IMO release notes would need some adjustments:
> > - Traditionally, at the beginning of the release note, before the
> > "Compatibility" paragraph, there should be another paragraph describing
> the
> > highlights of the release (see previous releases as an example).
> > - If I am not mistaken, there is a CVE to be mentioned in the release
> note,
> > which does not seem to be there.
> > - I would suggest to keep the traditional sections (see previous
> > releases): New features; Bug-fixes, API changes and minor
> > enhancements; Build and test suite; Dependency version upgrade; Web site
> > and documentation
> > - As Xiong said, bug fixes should not begin with "Fix..."; instead, the
> > commit message (or Jira title) should be used (this should be the norm
> for
> > all the tickets in the release note, not just the bug fixes).
> > - This particular item "[CALCITE-3673] Support ListTransientTable without
> > tables in the schema" is not a "New feature", but rather a "Minor
> > enhancement" or "Bug fix". Its title in the release note is also
> > misleading. I would suggest to keep the original Jira title / commit
> > message: "ListTransientTable should not leave tables in the schema".
> > - Apart from that, CALCITE-4976 mentions some modification to be made in
> > howto.md.
> >
> > Having said that, as Stamatis mentioned, I had also noticed something
> > strange on the repository while I was checking some of tickets that I had
> > resolved (the fact that the commit which resolved the issue does not
> belong
> > to any repository any more). As Stamatis suggests, I think we should put
> on
> > hold the RC process until we figure out what happened and how to handle
> the
> > situation.
> >
> > Best,
> > Ruben
> >
> >
> > On Wed, Mar 9, 2022 at 10:13 AM Stamatis Zampetakis 
> > wrote:
> >
> >> Thanks for preparing the RC Liya!
> >>
> >> The repository seems to be in some weird state (see [1]) so I don't feel
> >> comfortable getting a release out till we understand what exactly has
> >> happened.
> >> Let's continue the discussion under the respective thread.
> >>
> >> -1 (binding)
> >>
> >> Stamatis
> >>
> >> [1] https://lists.apache.org/thread/pw7q8tqn78rwq0sfxh6o2rxvwrjgjlkq
> >>
> >> On Mon, Mar 7, 2022 at 12:27 PM Enrico Olivelli 
> >> wrote:
> >>
> >>> +1 (non binding)
> >>>
> >>> Run all tests of HerdDB, all tests are passing without any code
> >>> change. (https://github.com/diennea/herddb/pull/779)
> >>>
> >>> I would like to note that previous tests of HerdDB did not pass with
> >>> 1.29.0 (and we were stuck to 1.28.0). With 1.29.0 tests on Github
> >>> actions hang during some classloading of some Calcite classes
> >>>
> >>> thank you Liya
> >>>
> >>> Enrico
> >>>
> >>> Il giorno lun 7 mar 2022 alle ore 09:20 Fan Liya
> >>>  ha scritto:
> >>>>
> >>>> Hi Xiong,
> >>>>
> >>>> Thanks for your feedback. I will change the logs in a follow up MR.
> >>>>
> >>>> Best,
> >>>> Liya Fan
> >>>>
> >>>>
> >>>> xiong duan  于2022年3月7日周一 15:30写道：
> >>>>
> >>>>> Hi, Liya Fan.Thanks for your work.  +1.
> >>>>> Just a little problem:
> >>>>> I noticed the Bug-fixes logs all started with FixI think maybe we
> >>>>> should stay the same as JIRA summary or PR commit?
> >>>>>
> >

[CANCEL][VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Fan Liya]

Due to the broken history problem, let's cancel this vote.
Thanks to all who have voted in this thread.

Best,
Liya Fan

Re: Broken history after force pushing to master

[Fan Liya]

Hi Julian,

Thanks a lot for your feedback. I will start a separate thread to discuss
the related problems when things are done, to avoid similar problems in the
future.

Best,
Liya Fan


Julian Hyde  于2022年3月10日周四 08:30写道：

> Yes, the ‘git rebase’ command works in mysterious ways. If it sees commits
> that have the same content it will ignore them, even if they have different
> hashes. Since you want to preserve the hashes in the master branch, the
> direction that you rebase (e.g. from master to site, versus from site to
> master) is crucial.
>
> As RM, you occasionally have to force-push. But force-pushes can be scary.
> It would be nice if there were some guidelines for when force-push  is
> necessary, and when it is not going to do much damage (e.g. is only going
> to change 1 commit).
>
> Julian
>
>
> > On Mar 9, 2022, at 4:19 PM, Fan Liya  wrote:
> >
> > Hi Ruben and Stamatis,
> >
> > Thanks a lot for your kind help. I should have sent an email asking for
> > help when I encountered the problem.
> > I must be careful in the future to never repeat such mistake again.
> >
> > In step 2, I found the following commits in site branch but not in master
> > branch (through git log master..site):
> >
> > commit d1324718fae633584a6c5f8e8f6238a0f851cbac
> > Author: Ruben Quesada Lopez 
> > Date:   Sat Jan 29 19:04:14 2022 +
> >
> >Site: update PMC Chair
> >
> > commit fabef057c536d56e10530b399543077abad03a24
> > Author: Jing Zhang 
> > Date:   Thu Jan 27 17:01:14 2022 +0800
> >
> >Site: Add external resources section in the community page
> >
> >Close apache/calcite#2703
> >
> > commit 48f4bf8596ebfa0f7460ce9358d30028f268cb8e
> > Author: Eugen Stan 
> > Date:   Fri Jan 28 12:02:09 2022 +0200
> >
> >Site: Add "calcite-clj - Use Calcite with Clojure" in talks section
> >
> >Close apache/calcite#2704
> >
> > commit d088cde8d74b9b9b0b157c2f5efe83fcedc6ffef
> > Author: Alessandro Solimando 
> > Date:   Mon Jan 10 11:48:29 2022 +0100
> >
> >Site: Add Alessandro Solimando as committer
> >
> > commit 664c4d3f9b0ec03071df6fef44b32169a275ea49
> > Author: mans2singh 
> > Date:   Wed Jan 5 19:41:56 2022 -0500
> >
> >Site: Fix typo in howto.md
> >
> > commit dd34953aa0b5af13e7e7fccd75b2f2e8fe9c8edd
> > Author: nobigo 
> > Date:   Fri Dec 31 21:28:20 2021 +0800
> >
> >Site: Change the javadoc title to Apache Calcite API
> >
> > commit d29aa09321ab14bd824e5465ddcfae8d16604134
> > Author: Guiyanakuang 
> > Date:   Wed Jan 5 10:12:03 2022 +0800
> >
> >Site: For tables that display results, center the content horizontally
> >
> >Close apache/calcite#2632
> >
> > commit da4fc3be015977fb02bf021583a06a5f58278b66
> > Author: Guiyanakuang 
> > Date:   Fri Dec 10 11:27:59 2021 +0800
> >
> >Site: Add syntax highlighting to SQL statements
> >
> >Close apache/calcite#2632
> >
> > commit cee9f670818116387810c5f89035975aa2694658
> > Author: Guiyanakuang 
> > Date:   Sun Dec 5 10:58:32 2021 +0800
> >
> >Site: Improve HTML tables display & update CSV tutorial
> >
> >1. Allow code pre-wrap in tables.
> >2. Display horizontal scrollbar in tables when content is too large
> >and cannot be wrapped.
> >3. Update CSV tutorial example based on current code.
> >
> >Close apache/calcite#2632
> >
> > Later, I found that these changes were also in the master branch (with
> > different commit hashes). However, since they entered the master branch
> in
> > a different order, they got different git commit hashes, and git
> recognized
> > them as different commits. So my operations above in effect changed the
> > order of some commits (and also their commit hashes).
> >
> > Stamatis, if you will help restore the master branch, I think I can help
> > verify the operation, because I have a local backup.
> >
> > When things are done, I think I will start another thread to discuss the
> > problem, to avoid such things happening again in the future.
> >
> > Thanks again for your kind help.
> >
> > Best,
> > Liya Fan
> >
> >
> >
> > Julian Hyde  于2022年3月10日周四 08:00写道：
> >
> >> That sounds good. I concur that "[CALCITE-5019] Avoid multiple scans
> when
> >> table is ProjectableFilterableTable and projections and filters act on
> >> different columns” is the last good commit and that its hash sho

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Fan Liya]

Hi all,

I have prepared a PR for the release note, after addressing related
problems (https://github.com/apache/calcite/pull/2741).
Please take a look. Thanks a lot.

Best,
Liya Fan


Fan Liya  于2022年3月10日周四 08:22写道：

> Hi Ruben and Julian,
>
> Thanks a lot for your feedback, I will prepare a PR addressing all the
> issues described above.
>
> Best,
> Liya Fan
>
>
> Julian Hyde  于2022年3月10日周四 08:03写道：
>
>> Liya,
>>
>> I made much the same comments as Ruben regarding the release notes.
>>
>> Though we don’t usually use PRs for releases, I think it would be useful
>> if we had a PR in which to review the release notes. It seems better than
>> bike-shedding them during the actual release vote. If you agree, can you
>> post that PR?
>>
>> Julian
>>
>>
>> > On Mar 9, 2022, at 2:59 AM, Ruben Q L  wrote:
>> >
>> > Thanks Liya Fan for being the release manager.
>> >
>> > First of all, for the record regarding the RC itself:
>> > - Checksum and signature: ok
>> > - Gradle test: ok
>> > - Calcite-based application test suite: ok
>> >
>> > IMO release notes would need some adjustments:
>> > - Traditionally, at the beginning of the release note, before the
>> > "Compatibility" paragraph, there should be another paragraph describing
>> the
>> > highlights of the release (see previous releases as an example).
>> > - If I am not mistaken, there is a CVE to be mentioned in the release
>> note,
>> > which does not seem to be there.
>> > - I would suggest to keep the traditional sections (see previous
>> > releases): New features; Bug-fixes, API changes and minor
>> > enhancements; Build and test suite; Dependency version upgrade; Web site
>> > and documentation
>> > - As Xiong said, bug fixes should not begin with "Fix..."; instead, the
>> > commit message (or Jira title) should be used (this should be the norm
>> for
>> > all the tickets in the release note, not just the bug fixes).
>> > - This particular item "[CALCITE-3673] Support ListTransientTable
>> without
>> > tables in the schema" is not a "New feature", but rather a "Minor
>> > enhancement" or "Bug fix". Its title in the release note is also
>> > misleading. I would suggest to keep the original Jira title / commit
>> > message: "ListTransientTable should not leave tables in the schema".
>> > - Apart from that, CALCITE-4976 mentions some modification to be made in
>> > howto.md.
>> >
>> > Having said that, as Stamatis mentioned, I had also noticed something
>> > strange on the repository while I was checking some of tickets that I
>> had
>> > resolved (the fact that the commit which resolved the issue does not
>> belong
>> > to any repository any more). As Stamatis suggests, I think we should
>> put on
>> > hold the RC process until we figure out what happened and how to handle
>> the
>> > situation.
>> >
>> > Best,
>> > Ruben
>> >
>> >
>> > On Wed, Mar 9, 2022 at 10:13 AM Stamatis Zampetakis 
>> > wrote:
>> >
>> >> Thanks for preparing the RC Liya!
>> >>
>> >> The repository seems to be in some weird state (see [1]) so I don't
>> feel
>> >> comfortable getting a release out till we understand what exactly has
>> >> happened.
>> >> Let's continue the discussion under the respective thread.
>> >>
>> >> -1 (binding)
>> >>
>> >> Stamatis
>> >>
>> >> [1] https://lists.apache.org/thread/pw7q8tqn78rwq0sfxh6o2rxvwrjgjlkq
>> >>
>> >> On Mon, Mar 7, 2022 at 12:27 PM Enrico Olivelli 
>> >> wrote:
>> >>
>> >>> +1 (non binding)
>> >>>
>> >>> Run all tests of HerdDB, all tests are passing without any code
>> >>> change. (https://github.com/diennea/herddb/pull/779)
>> >>>
>> >>> I would like to note that previous tests of HerdDB did not pass with
>> >>> 1.29.0 (and we were stuck to 1.28.0). With 1.29.0 tests on Github
>> >>> actions hang during some classloading of some Calcite classes
>> >>>
>> >>> thank you Liya
>> >>>
>> >>> Enrico
>> >>>
>> >>> Il giorno lun 7 mar 2022 alle ore 09:20 Fan Liya
>> >>>  ha scritto:
>> >>

Re: Broken history after force pushing to master

[Fan Liya]

Hi Stamatis,

Thanks a lot for your effort.
I just tested, and the master branch is in consistent state with my local
back-up. So I think it has restored to the original state.

I have submitted a PR for the release note (
https://github.com/apache/calcite/pull/2741).
After it gets merged, I will prepare another build (RC2) and restart the
voting process.

Thanks again for your kind help.

Best,
Liya Fan


Stamatis Zampetakis  于2022年3月10日周四 17:36写道：

> Thanks for the clarifications Liya.
>
> I restored the master branch to its previous state and aligned master and
> site.
> Now we can proceed with the release. Normally there shouldn't be a need to
> force-push again during this release.
>
> I don't think we want to disable force pushing altogether.
> Without force pushing if we accidentally merge things to the repo we
> wouldn't have a way to fix this easily.
> A trivial example that comes to mind is putting a wrong JIRA id in the
> commit message.
> Leaving the commit message as is, will not be the end of the world but it
> can be annoying when in the future people will get redirected to a wrong
> jira.
> Other examples include accidentally introducing merge commits etc.
>
> Best,
> Stamatis
>
>
> On Thu, Mar 10, 2022 at 6:28 AM stanilovsky evgeny <
> estanilovs...@gridgain.com> wrote:
>
> > Just notice - github allows to disable a force push into master branch.
> >
> > > It's not the first time that we have had small problems with history so
> > > no
> > > worries.
> > > Thankfully with the help of commit@calcite list we can always find a
> > way
> > > to
> > > fix things as long as we identify the problem soon enough.
> > >
> > > According to the change log [1] the last commit before force pushing
> was
> > > (dcbc493), which corresponds to CALCITE-5019.
> > >
> > > * -- * -- B -- O -- O -- O (dcbc493)
> > > \
> > > N -- N -- N refs/heads/master (c3dbf52)
> > >
> > > According to [2] the full commit id
> > > is dcbc493bf699d961427952c5efc047b76d859096.
> > >
> > > In order to restore the master branch in the state that it was before
> the
> > > force-push (before release) I plan to do the following steps:
> > >
> > > git fetch origin dcbc493bf699d961427952c5efc047b76d859096
> > > git checkout dcbc493bf699d961427952c5efc047b76d859096
> > > git branch -D master
> > > git switch -c master
> > > git push origin master -f
> > >
> > > I will apply the above sequence in 12h from now to give some time to
> > > others
> > > to react if necessary.
> > >
> > > Obviously this will nuke out any current release candidate so we will
> > > need
> > > to cancel existing votes and create an RC2.
> > >
> > > There has been a force push also to the site branch but doesn't matter
> > > much
> > > since we can force push master to site after the release is finalized.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1] https://lists.apache.org/thread/gkvn5hlmm3jlcklgw9k9nodyhxvqmsw4
> > > [2] https://lists.apache.org/thread/rvngk5tygfoyoc0klhwpo717mrngkdrw
> > >
> > >
> > > On Wed, Mar 9, 2022 at 6:44 PM Ruben Q L  wrote:
> > >
> > >> Hello Liya,
> > >>
> > >> No worries, we all make mistakes.
> > >> I think the sequence of steps that you describe looks like a plausible
> > >> explanation for how we get into this situation. Do you know (from step
> > >> 2)
> > >> which commits were in site branch that were not in master?
> > >> If in the future you (or anybody else) get blocked or experience any
> > >> problem on a certain step during the release process, do not hesitate
> to
> > >> send an email to the dev list with subject "[HELP] ..." describing the
> > >> issue. In my experience, someone from the community will assist
> > >> relatively
> > >> fast.
> > >>
> > >> Any git expert with a clear idea on how to restore the master branch?
> > >>
> > >> Best,
> > >> Ruben
> > >>
> > >>
> > >> On Wed, Mar 9, 2022 at 1:32 PM Fan Liya  wrote:
> > >>
> > >> > Hi all,
> > >> >
> > >> > I think the broken history was caused by this:
> > >> >
> > >> > 1. In document "Making a release candidate [1]", it says "Make sure
> > >&g

Re: Broken history after force pushing to master

[Fan Liya]

Hi Ruben,

Thanks a lot for your good comments. They are really helpful. I have added
them to the PR (https://github.com/apache/calcite/pull/2741)
Maybe later we can add more comments to make things clearer.

Best,
Liya Fan


Ruben Q L  于2022年3月10日周四 19:11写道：

> Hello,
>
> Thanks everyone for helping with the issue, especially Stamatis for
> carrying out the restore.
> I agree with leaving the possibility of force push into master, since it
> can be convenient in certain (very specific) cases.
>
> Regarding this:
>
> In step 2, I found the following commits in site branch but not in master
> branch (through git log master..site):
> ...
> Later, I found that these changes were also in the master branch (with
> different commit hashes) ...
>
>
> I can understand the confusion considering how the instruction is phrased
> [1]:
>   «Make sure master branch and site branch are in sync, i.e. there is no
> commit on site that has not been applied also to master.»
>
> I think it was me who introduced this sentence, it should be clarified to
> avoid the same problem in the future. Maybe by adding something along the
> lines:
>   «Make sure master branch and site branch are in sync, i.e. there is no
> commit on site that has not been applied also to master. We are talking
> about the commit content, you need to pay attention to the commit message
> and change, not hash: it is normal to have the same change in site and
> master, but with different hashes.»
>
> We could add this clarification later after the RC process, or we can take
> advantage of the RC PR [2], which already touches the howto.md file (where
> the problematic sentences is located). Liya Fan, as release manager, I
> think it's up to you.
>
> Best,
> Ruben
>
> [1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> [2] https://github.com/apache/calcite/pull/2741
>
>
> On Thu, Mar 10, 2022 at 9:50 AM Fan Liya  wrote:
>
> > Hi Stamatis,
> >
> > Thanks a lot for your effort.
> > I just tested, and the master branch is in consistent state with my local
> > back-up. So I think it has restored to the original state.
> >
> > I have submitted a PR for the release note (
> > https://github.com/apache/calcite/pull/2741).
> > After it gets merged, I will prepare another build (RC2) and restart the
> > voting process.
> >
> > Thanks again for your kind help.
> >
> > Best,
> > Liya Fan
> >
> >
> > Stamatis Zampetakis  于2022年3月10日周四 17:36写道：
> >
> > > Thanks for the clarifications Liya.
> > >
> > > I restored the master branch to its previous state and aligned master
> and
> > > site.
> > > Now we can proceed with the release. Normally there shouldn't be a need
> > to
> > > force-push again during this release.
> > >
> > > I don't think we want to disable force pushing altogether.
> > > Without force pushing if we accidentally merge things to the repo we
> > > wouldn't have a way to fix this easily.
> > > A trivial example that comes to mind is putting a wrong JIRA id in the
> > > commit message.
> > > Leaving the commit message as is, will not be the end of the world but
> it
> > > can be annoying when in the future people will get redirected to a
> wrong
> > > jira.
> > > Other examples include accidentally introducing merge commits etc.
> > >
> > > Best,
> > > Stamatis
> > >
> > >
> > > On Thu, Mar 10, 2022 at 6:28 AM stanilovsky evgeny <
> > > estanilovs...@gridgain.com> wrote:
> > >
> > > > Just notice - github allows to disable a force push into master
> branch.
> > > >
> > > > > It's not the first time that we have had small problems with
> history
> > so
> > > > > no
> > > > > worries.
> > > > > Thankfully with the help of commit@calcite list we can always
> find a
> > > > way
> > > > > to
> > > > > fix things as long as we identify the problem soon enough.
> > > > >
> > > > > According to the change log [1] the last commit before force
> pushing
> > > was
> > > > > (dcbc493), which corresponds to CALCITE-5019.
> > > > >
> > > > > * -- * -- B -- O -- O -- O (dcbc493)
> > > > > \
> > > > > N -- N -- N refs/heads/master (c3dbf52)
> > > > >
> > > > > According to [2] the full commit id
> > > > > is dcbc493bf699d961427952c5efc047b76d859096.

[HELP][Release 1.30.0]

[Fan Liya]

Hi all,

When preparing a release build, I ran the command according to the document
[1]:

*./gradlew prepareVote -Prc=2 -Pasf -Pasf.git.pushRepositoryProvider=GITBOX*

Then I got the following error message:



*Build calcite FAILURE reason:Execution failed for task ':pushRcTag':
  Caused by: org.eclipse.jgit.api.errors.TransportException:
https://gitbox.apache.org/repos/asf/calcite.git
: not authorized*

However, when I remove the last command line argument:

*./gradlew prepareVote -Prc=2 -Pasf*

It seemed to finish successfully.

Is this expected?
Thank you in advance.

Best,
Liya Fan

[1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate

Re: [HELP][Release 1.30.0]

[Fan Liya]

Hi Stamatis,

Thanks a lot for your clarification and confirmation.

Best,
Liya Fan


Stamatis Zampetakis  于2022年3月11日周五 21:39写道：

> Hi Liya,
>
> The asf.git.pushRepositoryProvider property controls the git repository
> that is used by the release plugin [1]. There are two options:
> * GITBOX -> https://gitbox.apache.org/repos/asf/calcite.git
> * GITHUB -> https://github.com/apache/calcite.git
>
> In the first attempt, you tried to push to GITBOX and it failed.
> In the second attempt, you tried to push to GITHUB and it was successful.
>
> Probably, the reason lies behind the credentials (asfGitSourceUsername,
> asfGitSourcePassword) that you are using for the release (GitHub vs ASF).
>
> When you push changes to a remote Calcite repository (that being regular
> commits, or tags, or anything else) you have to pick one of the two
> available options and that is sufficient.
>
> Best,
> Stamatis
>
> [1]
>
> https://github.com/vlsi/vlsi-release-plugins/commit/b8b3c60760f2065d791c0b095e1b675cdbcfacbd
>
> On Fri, Mar 11, 2022 at 2:09 PM Fan Liya  wrote:
>
> > Hi all,
> >
> > When preparing a release build, I ran the command according to the
> document
> > [1]:
> >
> > *./gradlew prepareVote -Prc=2 -Pasf
> > -Pasf.git.pushRepositoryProvider=GITBOX*
> >
> > Then I got the following error message:
> >
> >
> >
> > *Build calcite FAILURE reason:Execution failed for task ':pushRcTag':
> >   Caused by: org.eclipse.jgit.api.errors.TransportException:
> > https://gitbox.apache.org/repos/asf/calcite.git
> > <https://gitbox.apache.org/repos/asf/calcite.git>: not authorized*
> >
> > However, when I remove the last command line argument:
> >
> > *./gradlew prepareVote -Prc=2 -Pasf*
> >
> > It seemed to finish successfully.
> >
> > Is this expected?
> > Thank you in advance.
> >
> > Best,
> > Liya Fan
> >
> > [1]
> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> >
>

[VOTE] Release Apache Calcite 1.30.0 (release candidate 2)

[Fan Liya]

Hi all,

I have created a build for Apache Calcite 1.30.0, release
candidate 2.

Thanks to everyone who has contributed to this release.

You can read the release notes here:
https://github.com/apache/calcite/blob/calcite-1.30.0-rc2/site/_docs/history.md

The commit to be voted upon:
https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=170035fd97df1afdd0c0a499f63e0ca1606f7837

Its hash is 170035fd97df1afdd0c0a499f63e0ca1606f7837

Tag:
https://github.com/apache/calcite/tree/calcite-1.30.0-rc2

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc2
(revision 52982)

The hashes of the artifacts are as follows:
988c165bacb3ac20f9046fc80ebcd28eca66504ed3bdd3269ac2625ff59a80b06e3d7de12150ed9fe1e40def8b1a8290a87e280b7a7f0589a5b740fc79e273dd
*apache-calcite-1.30.0-src.tar.gz

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachecalcite-1152/org/apache/calcite/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/COMMITTER_ID.asc
https://www.apache.org/dist/calcite/KEYS

To create the jars and test Apache Calcite: "gradle build"
(requires an appropriate Gradle/JDK installation)

Please vote on releasing this package as Apache Calcite 1.30.0.

The vote is open for the next 72 hours and passes if a majority of at
least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Calcite 1.30.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Here is my vote:

+1 (non-binding)

Best,
Liya Fan

Re: Can't find java file for ImmutableCsvProjectTableScanRule

[Fan Liya]

Hi xiaobo,

It seems 'ImmutableCsvProjectTableScanRule' is a generated class.
You have to build the project first?

Best,
Liya Fan


xiaobo  于2022年3月13日周日 18:47写道：

> Hi,
>
> we are trying to make our adapter projectable now,  but we can't find the
> Java file for the ImmutableCsvProjectTableScanRule class in the CSV
> example, can you help with this.
>
>
> /** Rule configuration. */
>   @Value.Immutable(singleton = false)
>   public interface Config extends RelRule.Config {
>     Config DEFAULT =
> ImmutableCsvProjectTableScanRule.Config.builder()
>     .withOperandSupplier(b0 ->
>    
> b0.operand(LogicalProject.class).oneInput(b1 ->
>    
> b1.operand(CsvTableScan.class).noInputs()))
>     .build();
>
>     @Override default CsvProjectTableScanRule toRule() {
>   return new CsvProjectTableScanRule(this);
>     }
>   }

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 2)

[Fan Liya]

Hi Ruben,

Thanks a lot for your vote.

Hi all,

Since part of the voting period is during the weekend, let's extend the
voting period to at least 96 hours.

Best,
Liya Fan


Ruben Q L  于2022年3月14日周一 18:00写道：

> Thanks Liya Fan for being the release manager.
>
> My vote is: +1 (binding)
>
> - Checksum and signature: ok
> - Gradle test: ok
> - Calcite-based application test suite: ok
>
> Best,
> Ruben
>
>
> On Fri, Mar 11, 2022 at 1:54 PM Fan Liya  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.30.0, release
> > candidate 2.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> >
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc2/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=170035fd97df1afdd0c0a499f63e0ca1606f7837
> >
> > Its hash is 170035fd97df1afdd0c0a499f63e0ca1606f7837
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.30.0-rc2
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc2
> > (revision 52982)
> >
> > The hashes of the artifacts are as follows:
> >
> >
> 988c165bacb3ac20f9046fc80ebcd28eca66504ed3bdd3269ac2625ff59a80b06e3d7de12150ed9fe1e40def8b1a8290a87e280b7a7f0589a5b740fc79e273dd
> > *apache-calcite-1.30.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1152/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/COMMITTER_ID.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.30.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.30.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (non-binding)
> >
> > Best,
> > Liya Fan
> >
>

[RESULT] [VOTE] Release apache-calcite-1.30.0 (release candidate 2)

[Fan Liya]

Thanks to everyone who has tested the release candidate and given
their comments and votes.

The tally is as follows.

2 binding -1s:
Ruben,
Francis

2 non-binding +1s:
Liya,
Enrico

1 binding 0:
Stamatis

Therefore, I think this release candidate should be rejected, until
CALCITE-5040 is resolved.

Later, I will revert the release note commit
(170035fd97df1afdd0c0a499f63e0ca1606f7837,
maybe through git push force to master), and prepare another one after the
patch for
CALCITE-5040 is merged (if any).

Thanks everyone.

Liya Fan

Re: [RESULT] [VOTE] Release apache-calcite-1.30.0 (release candidate 2)

[Fan Liya]

Hi Julian,

Thanks for your feedback and your PR.

Best,
Liya Fan

Julian Hyde  于2022年3月15日周二 12:42写道：

> I have a PR for [CALCITE-5040]. Please review
> https://github.com/apache/calcite/pull/2744 <
> https://github.com/apache/calcite/pull/2744>.
>
> Liya, Sorry I didn’t find time to vote on this RC. I reviewed the release
> notes and they look good.
>
> > On Mar 14, 2022, at 8:24 PM, Fan Liya  wrote:
> >
> > Thanks to everyone who has tested the release candidate and given
> > their comments and votes.
> >
> > The tally is as follows.
> >
> > 2 binding -1s:
> > Ruben,
> > Francis
> >
> > 2 non-binding +1s:
> > Liya,
> > Enrico
> >
> > 1 binding 0:
> > Stamatis
> >
> > Therefore, I think this release candidate should be rejected, until
> > CALCITE-5040 is resolved.
> >
> > Later, I will revert the release note commit
> > (170035fd97df1afdd0c0a499f63e0ca1606f7837,
> > maybe through git push force to master), and prepare another one after
> the
> > patch for
> > CALCITE-5040 is merged (if any).
> >
> > Thanks everyone.
> >
> > Liya Fan
>
>

Re: [RESULT] [VOTE] Release apache-calcite-1.30.0 (release candidate 2)

[Fan Liya]

Sure. Thanks for your kind reminder.
I will do it and prepare a new build ASAP.

Best,
Liya  Fan


Julian Hyde  于2022年3月16日周三 01:38写道：

> Liya,
>
> The PR has a couple of LGTMs, and it has passed my internal tests. I think
> you should cherry-pick it into master, revise release notes, and start a
> new RC. (Will require a small force-push.)
>
> Julian
>
>
> On 2022/03/15 05:50:30 Fan Liya wrote:
> > Hi Julian,
> >
> > Thanks for your feedback and your PR.
> >
> > Best,
> > Liya Fan
> >
> > Julian Hyde  于2022年3月15日周二 12:42写道：
> >
> > > I have a PR for [CALCITE-5040]. Please review
> > > https://github.com/apache/calcite/pull/2744 <
> > > https://github.com/apache/calcite/pull/2744>.
> > >
> > > Liya, Sorry I didn’t find time to vote on this RC. I reviewed the
> release
> > > notes and they look good.
> > >
> > > > On Mar 14, 2022, at 8:24 PM, Fan Liya  wrote:
> > > >
> > > > Thanks to everyone who has tested the release candidate and given
> > > > their comments and votes.
> > > >
> > > > The tally is as follows.
> > > >
> > > > 2 binding -1s:
> > > > Ruben,
> > > > Francis
> > > >
> > > > 2 non-binding +1s:
> > > > Liya,
> > > > Enrico
> > > >
> > > > 1 binding 0:
> > > > Stamatis
> > > >
> > > > Therefore, I think this release candidate should be rejected, until
> > > > CALCITE-5040 is resolved.
> > > >
> > > > Later, I will revert the release note commit
> > > > (170035fd97df1afdd0c0a499f63e0ca1606f7837,
> > > > maybe through git push force to master), and prepare another one
> after
> > > the
> > > > patch for
> > > > CALCITE-5040 is merged (if any).
> > > >
> > > > Thanks everyone.
> > > >
> > > > Liya Fan
> > >
> > >
> >
>

[VOTE] Release Apache Calcite 1.30.0 (release candidate 3)

[Fan Liya]

Hi all,

I have created a build for Apache Calcite 1.30.0, release
candidate 3.

Thanks to everyone who has contributed to this release.

You can read the release notes here:
https://github.com/apache/calcite/blob/calcite-1.30.0-rc3/site/_docs/history.md

The commit to be voted upon:
https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=cd5229b9a371e20de207231d55bcbddf1ff39ec2

Its hash is cd5229b9a371e20de207231d55bcbddf1ff39ec2

Tag:
https://github.com/apache/calcite/tree/calcite-1.30.0-rc3

The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc3
(revision 53122)

The hashes of the artifacts are as follows:
6c9f65bcc75a05c226dbb0a74495e76761d738f6923086e2943fface1b8864697dff6d0de7aa0cb9b06f4f6aebe1322b5ecb829428f84d08c51dfb5773034040
*apache-calcite-1.30.0-src.tar.gz

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachecalcite-1153/org/apache/calcite/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/COMMITTER_ID.asc
https://www.apache.org/dist/calcite/KEYS

To create the jars and test Apache Calcite: "gradle build"
(requires an appropriate Gradle/JDK installation)

Please vote on releasing this package as Apache Calcite 1.30.0.

The vote is open for the next 72 hours and passes if a majority of at
least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Calcite 1.30.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...

Here is my vote:

+1 (non-binding)

Best,
Liya Fan

[RESULT] [VOTE] Release apache-calcite-1.30.0 (release candidate 3)

[Fan Liya]

Thanks to everyone who has tested the release candidate and given
their comments and votes.

The tally is as follows.

6 binding +1s:
Francis Chuang
Ruben Q L
Haisheng Yuan
Michael Mior
Julian Hyde
Stamatis Zampetakis

3 non-binding +1s:
Liya Fan
Alessandro Solimando
Enrico Olivelli

No 0s or -1s.

Therefore, I am delighted to announce that the proposal to release
Apache Calcite 1.30.0 has passed.

Thanks everyone. We’ll now roll the release out to the mirrors.

There was some issue during voting. I shall open a separate
thread to discuss.

Liya Fan

Re: [HELP][Release 1.30.0]

[Fan Liya]

Hi all,

I am having another problem when trying to publish the release.
After running the publishing command:

./gradlew publishDist -Prc=3 -Pasf

It seemed to have finished successfully.
However, the release was not propagated to the mirrors (it was supposed to
be propagated almost immediately):

https://dlcdn.apache.org/calcite/

Do you have any idea about the reason? Is it because my key was not signed,
as noted by Julian?

Thanks in advance

Best,
Liya Fan

Fan Liya  于2022年3月11日周五 21:49写道：

> Hi Stamatis,
>
> Thanks a lot for your clarification and confirmation.
>
> Best,
> Liya Fan
>
>
> Stamatis Zampetakis  于2022年3月11日周五 21:39写道：
>
>> Hi Liya,
>>
>> The asf.git.pushRepositoryProvider property controls the git repository
>> that is used by the release plugin [1]. There are two options:
>> * GITBOX -> https://gitbox.apache.org/repos/asf/calcite.git
>> * GITHUB -> https://github.com/apache/calcite.git
>>
>> In the first attempt, you tried to push to GITBOX and it failed.
>> In the second attempt, you tried to push to GITHUB and it was successful.
>>
>> Probably, the reason lies behind the credentials (asfGitSourceUsername,
>> asfGitSourcePassword) that you are using for the release (GitHub vs ASF).
>>
>> When you push changes to a remote Calcite repository (that being regular
>> commits, or tags, or anything else) you have to pick one of the two
>> available options and that is sufficient.
>>
>> Best,
>> Stamatis
>>
>> [1]
>>
>> https://github.com/vlsi/vlsi-release-plugins/commit/b8b3c60760f2065d791c0b095e1b675cdbcfacbd
>>
>> On Fri, Mar 11, 2022 at 2:09 PM Fan Liya  wrote:
>>
>> > Hi all,
>> >
>> > When preparing a release build, I ran the command according to the
>> document
>> > [1]:
>> >
>> > *./gradlew prepareVote -Prc=2 -Pasf
>> > -Pasf.git.pushRepositoryProvider=GITBOX*
>> >
>> > Then I got the following error message:
>> >
>> >
>> >
>> > *Build calcite FAILURE reason:Execution failed for task
>> ':pushRcTag':
>> >   Caused by: org.eclipse.jgit.api.errors.TransportException:
>> > https://gitbox.apache.org/repos/asf/calcite.git
>> > <https://gitbox.apache.org/repos/asf/calcite.git>: not authorized*
>> >
>> > However, when I remove the last command line argument:
>> >
>> > *./gradlew prepareVote -Prc=2 -Pasf*
>> >
>> > It seemed to finish successfully.
>> >
>> > Is this expected?
>> > Thank you in advance.
>> >
>> > Best,
>> > Liya Fan
>> >
>> > [1]
>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
>> >
>>
>

Re: [HELP][Release 1.30.0]

[Fan Liya]

Hi Francis,

Thanks a lot for your feedback.
I can see it propagated too.

However, it is not listed in

https://dlcdn.apache.org/calcite/

Maybe the list will be updated later.

Thanks again for your kind help.

Best,
Liya Fan


Francis Chuang  于2022年3月19日周六 16:37写道：

> Hi Liya,
>
> I can see that the release has propagated:
> https://dlcdn.apache.org/calcite/apache-calcite-1.30.0/
>
> We do need to sign your key though, using a virtual key signing party so
> that your key is part of the web of trust.
>
> Francis
>
> On 19/03/2022 7:28 pm, Fan Liya wrote:
> > Hi all,
> >
> > I am having another problem when trying to publish the release.
> > After running the publishing command:
> >
> > ./gradlew publishDist -Prc=3 -Pasf
> >
> > It seemed to have finished successfully.
> > However, the release was not propagated to the mirrors (it was supposed
> to
> > be propagated almost immediately):
> >
> > https://dlcdn.apache.org/calcite/
> >
> > Do you have any idea about the reason? Is it because my key was not
> signed,
> > as noted by Julian?
> >
> > Thanks in advance
> >
> > Best,
> > Liya Fan
> >
> > Fan Liya  于2022年3月11日周五 21:49写道：
> >
> >> Hi Stamatis,
> >>
> >> Thanks a lot for your clarification and confirmation.
> >>
> >> Best,
> >> Liya Fan
> >>
> >>
> >> Stamatis Zampetakis  于2022年3月11日周五 21:39写道：
> >>
> >>> Hi Liya,
> >>>
> >>> The asf.git.pushRepositoryProvider property controls the git repository
> >>> that is used by the release plugin [1]. There are two options:
> >>> * GITBOX -> https://gitbox.apache.org/repos/asf/calcite.git
> >>> * GITHUB -> https://github.com/apache/calcite.git
> >>>
> >>> In the first attempt, you tried to push to GITBOX and it failed.
> >>> In the second attempt, you tried to push to GITHUB and it was
> successful.
> >>>
> >>> Probably, the reason lies behind the credentials (asfGitSourceUsername,
> >>> asfGitSourcePassword) that you are using for the release (GitHub vs
> ASF).
> >>>
> >>> When you push changes to a remote Calcite repository (that being
> regular
> >>> commits, or tags, or anything else) you have to pick one of the two
> >>> available options and that is sufficient.
> >>>
> >>> Best,
> >>> Stamatis
> >>>
> >>> [1]
> >>>
> >>>
> https://github.com/vlsi/vlsi-release-plugins/commit/b8b3c60760f2065d791c0b095e1b675cdbcfacbd
> >>>
> >>> On Fri, Mar 11, 2022 at 2:09 PM Fan Liya  wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> When preparing a release build, I ran the command according to the
> >>> document
> >>>> [1]:
> >>>>
> >>>> *./gradlew prepareVote -Prc=2 -Pasf
> >>>> -Pasf.git.pushRepositoryProvider=GITBOX*
> >>>>
> >>>> Then I got the following error message:
> >>>>
> >>>>
> >>>>
> >>>> *Build calcite FAILURE reason:Execution failed for task
> >>> ':pushRcTag':
> >>>>Caused by: org.eclipse.jgit.api.errors.TransportException:
> >>>> https://gitbox.apache.org/repos/asf/calcite.git
> >>>> <https://gitbox.apache.org/repos/asf/calcite.git>: not authorized*
> >>>>
> >>>> However, when I remove the last command line argument:
> >>>>
> >>>> *./gradlew prepareVote -Prc=2 -Pasf*
> >>>>
> >>>> It seemed to finish successfully.
> >>>>
> >>>> Is this expected?
> >>>> Thank you in advance.
> >>>>
> >>>> Best,
> >>>> Liya Fan
> >>>>
> >>>> [1]
> >>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> >>>>
> >>>
> >>
> >
>

Re: [HELP][Release 1.30.0]

[Fan Liya]

Hi all,

When building the web site, I got another problem.

I followed the instructions in
https://github.com/apache/calcite/blob/master/site/README.md, and each step
finished successfully.
However, after running the following command:

docker-compose run build-site

I got the following error:

/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/spec_set.rb:87:in
`block in materialize': Could not find concurrent-ruby-1.1.9 in any of the
sources (Bundler::GemNotFound)

I tried the following methods to solve it, but none of them worked:
1. Remove the Gemfile.lock file and rebuild
2. Install concurrent-ruby-1.1.9 manually
3. Add local source in Gemfile
4. Rebuild index: sudo bundle --full-index

I changed another machine and restarted the whole process, but the problem
persists.

Do you have any idea about the problem? Thank you in advance.

Best,
Liya Fan




Fan Liya  于2022年3月19日周六 16:59写道：

> Hi Francis,
>
> Thanks a lot for your feedback.
> I can see it propagated too.
>
> However, it is not listed in
>
> https://dlcdn.apache.org/calcite/
>
> Maybe the list will be updated later.
>
> Thanks again for your kind help.
>
> Best,
> Liya Fan
>
>
> Francis Chuang  于2022年3月19日周六 16:37写道：
>
>> Hi Liya,
>>
>> I can see that the release has propagated:
>> https://dlcdn.apache.org/calcite/apache-calcite-1.30.0/
>>
>> We do need to sign your key though, using a virtual key signing party so
>> that your key is part of the web of trust.
>>
>> Francis
>>
>> On 19/03/2022 7:28 pm, Fan Liya wrote:
>> > Hi all,
>> >
>> > I am having another problem when trying to publish the release.
>> > After running the publishing command:
>> >
>> > ./gradlew publishDist -Prc=3 -Pasf
>> >
>> > It seemed to have finished successfully.
>> > However, the release was not propagated to the mirrors (it was supposed
>> to
>> > be propagated almost immediately):
>> >
>> > https://dlcdn.apache.org/calcite/
>> >
>> > Do you have any idea about the reason? Is it because my key was not
>> signed,
>> > as noted by Julian?
>> >
>> > Thanks in advance
>> >
>> > Best,
>> > Liya Fan
>> >
>> > Fan Liya  于2022年3月11日周五 21:49写道：
>> >
>> >> Hi Stamatis,
>> >>
>> >> Thanks a lot for your clarification and confirmation.
>> >>
>> >> Best,
>> >> Liya Fan
>> >>
>> >>
>> >> Stamatis Zampetakis  于2022年3月11日周五 21:39写道：
>> >>
>> >>> Hi Liya,
>> >>>
>> >>> The asf.git.pushRepositoryProvider property controls the git
>> repository
>> >>> that is used by the release plugin [1]. There are two options:
>> >>> * GITBOX -> https://gitbox.apache.org/repos/asf/calcite.git
>> >>> * GITHUB -> https://github.com/apache/calcite.git
>> >>>
>> >>> In the first attempt, you tried to push to GITBOX and it failed.
>> >>> In the second attempt, you tried to push to GITHUB and it was
>> successful.
>> >>>
>> >>> Probably, the reason lies behind the credentials
>> (asfGitSourceUsername,
>> >>> asfGitSourcePassword) that you are using for the release (GitHub vs
>> ASF).
>> >>>
>> >>> When you push changes to a remote Calcite repository (that being
>> regular
>> >>> commits, or tags, or anything else) you have to pick one of the two
>> >>> available options and that is sufficient.
>> >>>
>> >>> Best,
>> >>> Stamatis
>> >>>
>> >>> [1]
>> >>>
>> >>>
>> https://github.com/vlsi/vlsi-release-plugins/commit/b8b3c60760f2065d791c0b095e1b675cdbcfacbd
>> >>>
>> >>> On Fri, Mar 11, 2022 at 2:09 PM Fan Liya  wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> When preparing a release build, I ran the command according to the
>> >>> document
>> >>>> [1]:
>> >>>>
>> >>>> *./gradlew prepareVote -Prc=2 -Pasf
>> >>>> -Pasf.git.pushRepositoryProvider=GITBOX*
>> >>>>
>> >>>> Then I got the following error message:
>> >>>>
>> >>>>
>> >>>>
>> >>>> *Build calcite FAILURE reason:Execution failed for task
>> >>> ':pushRcTag':
>> >>>>Caused by: org.eclipse.jgit.api.errors.TransportException:
>> >>>> https://gitbox.apache.org/repos/asf/calcite.git
>> >>>> <https://gitbox.apache.org/repos/asf/calcite.git>: not authorized*
>> >>>>
>> >>>> However, when I remove the last command line argument:
>> >>>>
>> >>>> *./gradlew prepareVote -Prc=2 -Pasf*
>> >>>>
>> >>>> It seemed to finish successfully.
>> >>>>
>> >>>> Is this expected?
>> >>>> Thank you in advance.
>> >>>>
>> >>>> Best,
>> >>>> Liya Fan
>> >>>>
>> >>>> [1]
>> >>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
>> >>>>
>> >>>
>> >>
>> >
>>
>

Re: [HELP][Release 1.30.0]

[Fan Liya]

The problem was worked around by building the site manually.
If we find the root cause, maybe we need to adjust the document accordingly.

BTW, we need a PMC to update the latest release at
https://reporter.apache.org/addrelease.html?calcite
Is there anyone willing to help?
Thank you in advance.

Best,
Liya Fan



Fan Liya  于2022年3月20日周日 16:54写道：

> Hi all,
>
> When building the web site, I got another problem.
>
> I followed the instructions in
> https://github.com/apache/calcite/blob/master/site/README.md, and each
> step finished successfully.
> However, after running the following command:
>
> docker-compose run build-site
>
> I got the following error:
>
> /usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/spec_set.rb:87:in
> `block in materialize': Could not find concurrent-ruby-1.1.9 in any of the
> sources (Bundler::GemNotFound)
>
> I tried the following methods to solve it, but none of them worked:
> 1. Remove the Gemfile.lock file and rebuild
> 2. Install concurrent-ruby-1.1.9 manually
> 3. Add local source in Gemfile
> 4. Rebuild index: sudo bundle --full-index
>
> I changed another machine and restarted the whole process, but the problem
> persists.
>
> Do you have any idea about the problem? Thank you in advance.
>
> Best,
> Liya Fan
>
>
>
>
> Fan Liya  于2022年3月19日周六 16:59写道：
>
>> Hi Francis,
>>
>> Thanks a lot for your feedback.
>> I can see it propagated too.
>>
>> However, it is not listed in
>>
>> https://dlcdn.apache.org/calcite/
>>
>> Maybe the list will be updated later.
>>
>> Thanks again for your kind help.
>>
>> Best,
>> Liya Fan
>>
>>
>> Francis Chuang  于2022年3月19日周六 16:37写道：
>>
>>> Hi Liya,
>>>
>>> I can see that the release has propagated:
>>> https://dlcdn.apache.org/calcite/apache-calcite-1.30.0/
>>>
>>> We do need to sign your key though, using a virtual key signing party so
>>> that your key is part of the web of trust.
>>>
>>> Francis
>>>
>>> On 19/03/2022 7:28 pm, Fan Liya wrote:
>>> > Hi all,
>>> >
>>> > I am having another problem when trying to publish the release.
>>> > After running the publishing command:
>>> >
>>> > ./gradlew publishDist -Prc=3 -Pasf
>>> >
>>> > It seemed to have finished successfully.
>>> > However, the release was not propagated to the mirrors (it was
>>> supposed to
>>> > be propagated almost immediately):
>>> >
>>> > https://dlcdn.apache.org/calcite/
>>> >
>>> > Do you have any idea about the reason? Is it because my key was not
>>> signed,
>>> > as noted by Julian?
>>> >
>>> > Thanks in advance
>>> >
>>> > Best,
>>> > Liya Fan
>>> >
>>> > Fan Liya  于2022年3月11日周五 21:49写道：
>>> >
>>> >> Hi Stamatis,
>>> >>
>>> >> Thanks a lot for your clarification and confirmation.
>>> >>
>>> >> Best,
>>> >> Liya Fan
>>> >>
>>> >>
>>> >> Stamatis Zampetakis  于2022年3月11日周五 21:39写道：
>>> >>
>>> >>> Hi Liya,
>>> >>>
>>> >>> The asf.git.pushRepositoryProvider property controls the git
>>> repository
>>> >>> that is used by the release plugin [1]. There are two options:
>>> >>> * GITBOX -> https://gitbox.apache.org/repos/asf/calcite.git
>>> >>> * GITHUB -> https://github.com/apache/calcite.git
>>> >>>
>>> >>> In the first attempt, you tried to push to GITBOX and it failed.
>>> >>> In the second attempt, you tried to push to GITHUB and it was
>>> successful.
>>> >>>
>>> >>> Probably, the reason lies behind the credentials
>>> (asfGitSourceUsername,
>>> >>> asfGitSourcePassword) that you are using for the release (GitHub vs
>>> ASF).
>>> >>>
>>> >>> When you push changes to a remote Calcite repository (that being
>>> regular
>>> >>> commits, or tags, or anything else) you have to pick one of the two
>>> >>> available options and that is sufficient.
>>> >>>
>>> >>> Best,
>>> >>> Stamatis
>>> >>>
>>> >>> [1]
>>> >>>
>>> >>>
>>> https://github.com/vlsi/vlsi-release-plugins/commit/b8b3c60760f2065d791c0b095e1b675cdbcfacbd
>>> >>>
>>> >>> On Fri, Mar 11, 2022 at 2:09 PM Fan Liya  wrote:
>>> >>>
>>> >>>> Hi all,
>>> >>>>
>>> >>>> When preparing a release build, I ran the command according to the
>>> >>> document
>>> >>>> [1]:
>>> >>>>
>>> >>>> *./gradlew prepareVote -Prc=2 -Pasf
>>> >>>> -Pasf.git.pushRepositoryProvider=GITBOX*
>>> >>>>
>>> >>>> Then I got the following error message:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> *Build calcite FAILURE reason:Execution failed for task
>>> >>> ':pushRcTag':
>>> >>>>Caused by: org.eclipse.jgit.api.errors.TransportException:
>>> >>>> https://gitbox.apache.org/repos/asf/calcite.git
>>> >>>> <https://gitbox.apache.org/repos/asf/calcite.git>: not authorized*
>>> >>>>
>>> >>>> However, when I remove the last command line argument:
>>> >>>>
>>> >>>> *./gradlew prepareVote -Prc=2 -Pasf*
>>> >>>>
>>> >>>> It seemed to finish successfully.
>>> >>>>
>>> >>>> Is this expected?
>>> >>>> Thank you in advance.
>>> >>>>
>>> >>>> Best,
>>> >>>> Liya Fan
>>> >>>>
>>> >>>> [1]
>>> >>>
>>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
>>> >>>>
>>> >>>
>>> >>
>>> >
>>>
>>

Re: [Calcite Release 1.30.0]

[Fan Liya]

Hi all,

Code freeze for master branch is over and commits can resume.
Thanks a lot for all who have helped in the release process.

Best,
Liya Fan

Fan Liya  于2022年3月5日周六 08:27写道：
>
> Hi Stamatis,
>
> Thanks for your kind help and good suggestions.
>
> Best,
> Liya Fan
>
> On Saturday, March 5, 2022, Stamatis Zampetakis  wrote:
>>
>> @Liya, @Jess: I added your signing keys to svn. Please check and ensure
>> that everything looks good.
>>
>> @Liya: I saw you raised a PR for the release but this is not necessary.
>> When you prepare the release candidate [1] people will review release notes
>> and everything else as needed.
>> Usually there is always an update of the release note post release so not
>> need to worry too much about it.
>>
>> Best,
>> Stamatis
>>
>> [1] https://calcite.apache.org/docs/howto.html#making-a-release-candidate
>>
>> On Fri, Mar 4, 2022 at 5:04 PM Jess Balint  wrote:
>>
>> > Hi Stamatis,
>> >
>> > Can you add mine also please? I have appended it to Liya Fan's KEYS.txt:
>> >
>> > https://gist.github.com/jbalint/fe02278549c38ea037af44b3efdb5545
>> >
>> > Jess
>> >
>> > On Fri, Mar 4, 2022 at 9:41 AM Fan Liya  wrote:
>> >
>> > > Hi Stamatis,
>> > >
>> > > Thanks for your good suggestion.
>> > > I have uploaded the updated KEYS file here
>> > > https://gist.github.com/liyafan82/34472c56d29329c1d7b9185052c0376f
>> > >
>> > > Best,
>> > > Liya Fan
>> > >
>> > >
>> > > Stamatis Zampetakis  于2022年3月4日周五 22:59写道：
>> > >
>> > > > Hi Liya,
>> > > >
>> > > > Attachments are not allowed in most ASF lists.
>> > > >
>> > > > Export your public key according to the instructions in the KEYS file,
>> > > > upload it somewhere (e.g., your github repo), and send us the link to
>> > the
>> > > > key.
>> > > >
>> > > > Best,
>> > > > Stamatis
>> > > >
>> > > >
>> > > > On Fri, Mar 4, 2022 at 3:45 PM Fan Liya  wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > To sign the release package, it is required to upload my PGP key to
>> > the
>> > > > > KEYS file (https://dist.apache.org/repos/dist/release/calcite/KEYS)
>> > > > > However, only PMCs have the permission.
>> > > > > Is there anyone who can help? Thank you in advance.
>> > > > >
>> > > > > The updated KEYS file is attached.
>> > > > >
>> > > > > Best,
>> > > > > Liya Fan
>> > > > >
>> > > > >
>> > > > > Fan Liya  于2022年3月4日周五 22:28写道：
>> > > > >
>> > > > >> Opened a PR for the release (
>> > > > https://github.com/apache/calcite/pull/2739).
>> > > > >>
>> > > > >> Please take a look. Thanks.
>> > > > >>
>> > > > >> Best,
>> > > > >> Liya Fan
>> > > > >>
>> > > > >>
>> > > > >> Fan Liya  于2022年3月4日周五 19:55写道：
>> > > > >>
>> > > > >>> Done. Thanks for your kind reminder.
>> > > > >>>
>> > > > >>> Best,
>> > > > >>> Liya Fan
>> > > > >>>
>> > > > >>> Ruben Q L  于2022年3月4日周五 17:11写道：
>> > > > >>>
>> > > > >>>> Thanks for pushing this forward.
>> > > > >>>>
>> > > > >>>> Just a small comment, according to our Jira dashboard [1] there
>> > are
>> > > > >>>> still
>> > > > >>>> several unresolved tickets with fix version 1.30.
>> > > > >>>> I guess they should be updated (remove fix version 1.30) if they
>> > are
>> > > > not
>> > > > >>>> going to be part of this release.
>> > > > >>>>
>> > > > >>>> Best,
>> > > > >>>> Ruben
>> > > > >>>>
>> > > > >>>> [1]
>> > > > >>>>
>> > > > >>>>
>> > > >
>> > >
>> > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
>> > > > >>>>
>> > > > >>>>
>> > > > >>>> On Fri, Mar 4, 2022 at 6:44 AM Fan Liya 
>> > wrote:
>> > > > >>>>
>> > > > >>>> > Hi all,
>> > > > >>>> >
>> > > > >>>> > I am going to start the releasing process for Calcite 1.30.0.
>> > The
>> > > > >>>> following
>> > > > >>>> > JIRA items will be included in this release:
>> > > > >>>> >
>> > > > >>>> >
>> > > > >>>> >
>> > > > >>>>
>> > > >
>> > >
>> > https://issues.apache.org/jira/browse/CALCITE-1794?jql=project%20%3D%20CALCITE%20AND%20status%20%3D%20Resolved%20and%20fixVersion%20%3D%201.30.0
>> > > > >>>> >
>> > > > >>>> > Please check. If there is no problem, I am going to freeze the
>> > > code
>> > > > >>>> branch.
>> > > > >>>> >
>> > > > >>>> > Best,
>> > > > >>>> > Liya Fan
>> > > > >>>> >
>> > > > >>>>
>> > > > >>>
>> > > >
>> > >
>> >

Re: [HELP][Release 1.30.0]

[Fan Liya]

Hi Stamatis,

Thanks a lot for your kind help and precious time.
I am in UTC+8, and available today (2022-03-21) between 10:00 and 14:00 UTC.

Hi Jess,

The time slot you proposed is OK for me. I am afraid it is not
convenient for people in UTC+1.

Anyway, please propose another time slot if it is not convenient for
you. Thanks.

Best,
Liya Fan

Jess Balint  于2022年3月21日周一 06:27写道：
>
> I would also like to participate in this. Available 12:00 through 04:00 UTC.
>
> On Sun, Mar 20, 2022 at 3:18 PM Stamatis Zampetakis 
> wrote:
>
> > Hey Liya,
> >
> > I can sign your key. Feel free to propose a timeslot when you are available
> > and we can hold a virtual key signing party; I am in UTC+1.
> >
> > The docker-compose build works fine for me so not sure what went wrong in
> > your environment.
> >
> > I just added the 1.30.0 release in the reporter tool [1].
> >
> > Best,
> > Stamatis
> >
> > [1] https://reporter.apache.org/addrelease.html?calcite
> >
> >
> > On Sun, Mar 20, 2022 at 11:26 AM Fan Liya  wrote:
> >
> > > The problem was worked around by building the site manually.
> > > If we find the root cause, maybe we need to adjust the document
> > > accordingly.
> > >
> > > BTW, we need a PMC to update the latest release at
> > > https://reporter.apache.org/addrelease.html?calcite
> > > Is there anyone willing to help?
> > > Thank you in advance.
> > >
> > > Best,
> > > Liya Fan
> > >
> > >
> > >
> > > Fan Liya  于2022年3月20日周日 16:54写道：
> > >
> > > > Hi all,
> > > >
> > > > When building the web site, I got another problem.
> > > >
> > > > I followed the instructions in
> > > > https://github.com/apache/calcite/blob/master/site/README.md, and each
> > > > step finished successfully.
> > > > However, after running the following command:
> > > >
> > > > docker-compose run build-site
> > > >
> > > > I got the following error:
> > > >
> > > >
> > >
> > /usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/spec_set.rb:87:in
> > > > `block in materialize': Could not find concurrent-ruby-1.1.9 in any of
> > > the
> > > > sources (Bundler::GemNotFound)
> > > >
> > > > I tried the following methods to solve it, but none of them worked:
> > > > 1. Remove the Gemfile.lock file and rebuild
> > > > 2. Install concurrent-ruby-1.1.9 manually
> > > > 3. Add local source in Gemfile
> > > > 4. Rebuild index: sudo bundle --full-index
> > > >
> > > > I changed another machine and restarted the whole process, but the
> > > problem
> > > > persists.
> > > >
> > > > Do you have any idea about the problem? Thank you in advance.
> > > >
> > > > Best,
> > > > Liya Fan
> > > >
> > > >
> > > >
> > > >
> > > > Fan Liya  于2022年3月19日周六 16:59写道：
> > > >
> > > >> Hi Francis,
> > > >>
> > > >> Thanks a lot for your feedback.
> > > >> I can see it propagated too.
> > > >>
> > > >> However, it is not listed in
> > > >>
> > > >> https://dlcdn.apache.org/calcite/
> > > >>
> > > >> Maybe the list will be updated later.
> > > >>
> > > >> Thanks again for your kind help.
> > > >>
> > > >> Best,
> > > >> Liya Fan
> > > >>
> > > >>
> > > >> Francis Chuang  于2022年3月19日周六 16:37写道：
> > > >>
> > > >>> Hi Liya,
> > > >>>
> > > >>> I can see that the release has propagated:
> > > >>> https://dlcdn.apache.org/calcite/apache-calcite-1.30.0/
> > > >>>
> > > >>> We do need to sign your key though, using a virtual key signing party
> > > so
> > > >>> that your key is part of the web of trust.
> > > >>>
> > > >>> Francis
> > > >>>
> > > >>> On 19/03/2022 7:28 pm, Fan Liya wrote:
> > > >>> > Hi all,
> > > >>> >
> > > >>> > I am having another problem when trying to publish the release.
> > > >>> > After running the publishing command:
> > > >>> >

Re: Virtual key signing party

[Fan Liya]

Hi Stamatis,

I wan to attend it.
Thanks.

Best,
Liya Fan

pub   rsa4096 2019-03-15 [SC] [expires: 2023-03-15]
  0474 9577 FD93 4674 B9CD  45C5 D77C 3383 F192 7570
uid   [ unknown] Stamatis Zampetakis 
uid   [ unknown] Stamatis Zampetakis 
sub   rsa4096 2019-03-15 [E] [expires: 2023-03-15]

Stamatis Zampetakis  于2022年3月21日周一 18:27写道：
>
> Hello,
>
> Some of us (at least Liya and myself) are meeting today [1] (~1.5h from
> now) to sign each other's keys [2]. The meeting URL is the following:
>
> meet.google.com/byx-douk-qhn
>
> If you would like to attend, please reply to this thread with your public
> keys' fingerprint (gpg --fingerprint Stamatis) before the meeting.
>
> pub   rsa4096 2019-03-15 [SC] [expires: 2023-03-15]
>   0474 9577 FD93 4674 B9CD  45C5 D77C 3383 F192 7570
> uid   [ultimate] Stamatis Zampetakis 
> uid   [ultimate] Stamatis Zampetakis 
> sub   rsa4096 2019-03-15 [E] [expires: 2023-03-15]
>
> To verify your identity, please bring with you a government issued ID
> (preferably passport).
>
> As a reminder of the procedure, have a look at the notes [3] taken by
> Francis during a previous party!
>
> Anybody can participate (not need to be a committer to the project).
>
> Best,
> Stamatis
>
> [1] https://s.apache.org/f9snd
> [2]
> http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html
> [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84

Re: Virtual key signing party

[Fan Liya]

Sorry my public key finger print:

Best,
Liya Fan

pub   rsa4096 2022-03-03 [SC]
  F905 6357 2D7E 336E 0A0A  D095 7F99 D307 0C03 7870
uid   [ultimate] Liya Fan (CODE SIGNING KEY) 
sub   rsa4096 2022-03-03 [E]

Fan Liya  于2022年3月21日周一 18:43写道：
>
> Hi Stamatis,
>
> I wan to attend it.
> Thanks.
>
> Best,
> Liya Fan
>
> pub   rsa4096 2019-03-15 [SC] [expires: 2023-03-15]
>   0474 9577 FD93 4674 B9CD  45C5 D77C 3383 F192 7570
> uid   [ unknown] Stamatis Zampetakis 
> uid   [ unknown] Stamatis Zampetakis 
> sub   rsa4096 2019-03-15 [E] [expires: 2023-03-15]
>
> Stamatis Zampetakis  于2022年3月21日周一 18:27写道：
> >
> > Hello,
> >
> > Some of us (at least Liya and myself) are meeting today [1] (~1.5h from
> > now) to sign each other's keys [2]. The meeting URL is the following:
> >
> > meet.google.com/byx-douk-qhn
> >
> > If you would like to attend, please reply to this thread with your public
> > keys' fingerprint (gpg --fingerprint Stamatis) before the meeting.
> >
> > pub   rsa4096 2019-03-15 [SC] [expires: 2023-03-15]
> >   0474 9577 FD93 4674 B9CD  45C5 D77C 3383 F192 7570
> > uid   [ultimate] Stamatis Zampetakis 
> > uid   [ultimate] Stamatis Zampetakis 
> > sub   rsa4096 2019-03-15 [E] [expires: 2023-03-15]
> >
> > To verify your identity, please bring with you a government issued ID
> > (preferably passport).
> >
> > As a reminder of the procedure, have a look at the notes [3] taken by
> > Francis during a previous party!
> >
> > Anybody can participate (not need to be a committer to the project).
> >
> > Best,
> > Stamatis
> >
> > [1] https://s.apache.org/f9snd
> > [2]
> > http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html
> > [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84

[DISCUSS] Best practice for synchronizing master and site branches

[Fan Liya]

Hi all,

As part of the release process, we need to synchronize the master and
site branches (Please see
https://calcite.apache.org/docs/howto.html#making-a-release-candidate).
Usually, the site is behind the master branch by some commits.
If the existing commits in the site branch are in the same order as in
the master branch, the task is easy: just switch to the site branch,
and run

git rebase master

However, if some commits are in different orders, it can be tricky.
For example, the master branch may have the following commits (in
order):

A, B, X1, X2, ... , Xn.

and the site branch may have the following commits (in order):

B, A, X1, X2.

Basically we have two choices:

1. We can live with the out of order commits, because after
cherry-picking commits X3, X4, ... , Xn to the site branch, the file
contents will be consistent.

The problem is that, since the two branches have diverged, we cannot
use the rebase command. Instead, we have to manually cherry-pick
commits individually, which requires large effort. In addition, for
any subsequent release processes, we have to manually cherry-pick each
commit.

2. We need to make the commits order consistent, which will make it
easy for subsequent releases.
However, the problem is that, to make the commits order consistent,
some git force push command is unavoidable, which is risky to some
extent.

So what is the recommended way to do this? Thanks in advance for your feedback!

Best,
Liya Fan

Re: [DISCUSS] Best practice for synchronizing master and site branches

[Fan Liya]

Hi Francis,

Thanks for your feedback.

It seems we should choose option 2.
In addition, it seems less risky to run "git push --force" commands in
the site branch.

Best,
Liya Fan

Francis Chuang  于2022年3月25日周五 12:14写道：
>
> Hi Liya,
>
> Thanks for bringing this up. We have always done the following when
> committing:
> 1. Always commit to master.
> 2. If we need to publish the change to the site now (for example, new
> committer or announcement), cherry-pick the change into the site branch
> and publish it.
> 3. After a release, make the site branch the same as master (git reset
> --hard master) and force push (git push --force origin site).
>
> Francis
>
> On 25/03/2022 3:03 pm, Fan Liya wrote:
> > Hi all,
> >
> > As part of the release process, we need to synchronize the master and
> > site branches (Please see
> > https://calcite.apache.org/docs/howto.html#making-a-release-candidate).
> > Usually, the site is behind the master branch by some commits.
> > If the existing commits in the site branch are in the same order as in
> > the master branch, the task is easy: just switch to the site branch,
> > and run
> >
> > git rebase master
> >
> > However, if some commits are in different orders, it can be tricky.
> > For example, the master branch may have the following commits (in
> > order):
> >
> > A, B, X1, X2, ... , Xn.
> >
> > and the site branch may have the following commits (in order):
> >
> > B, A, X1, X2.
> >
> > Basically we have two choices:
> >
> > 1. We can live with the out of order commits, because after
> > cherry-picking commits X3, X4, ... , Xn to the site branch, the file
> > contents will be consistent.
> >
> > The problem is that, since the two branches have diverged, we cannot
> > use the rebase command. Instead, we have to manually cherry-pick
> > commits individually, which requires large effort. In addition, for
> > any subsequent release processes, we have to manually cherry-pick each
> > commit.
> >
> > 2. We need to make the commits order consistent, which will make it
> > easy for subsequent releases.
> > However, the problem is that, to make the commits order consistent,
> > some git force push command is unavoidable, which is risky to some
> > extent.
> >
> > So what is the recommended way to do this? Thanks in advance for your 
> > feedback!
> >
> > Best,
> > Liya Fan