Re: Calcite Release 1.31 started

[Charles Givre]

Thanks Andrei and Calcite team for 1.31!

From the Drill community we are very excited about this release as we have been 
working on a PR to get Drill back on the main version of Calcite. 
(https://github.com/apache/drill/pull/2602 
)  We've been testing Drill with 
Calcite 1.31 and we've seen very significant improvements in performance.

Best,
-- C




> On Aug 3, 2022, at 8:30 PM, Benchao Li  wrote:
> 
> Thanks Andrei for being the RM for 1.31.0
> 
> 
> Gavin Ray  于2022年8月4日周四 02:46写道：
> 
>> Hooray!!
>> 
>> On Wed, Aug 3, 2022 at 2:34 PM Michael Mior  wrote:
>> 
>>> Thanks for all your work Andrei!
>>> 
>>> --
>>> Michael Mior
>>> mm...@apache.org
>>> 
>>> 
>>> On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda  wrote:
>>> 
 1.31 has been released.
 
 You can resume dev work on the main branch.
 
 Thanks for your patience.
 
 
 On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda 
>> wrote:
 
> Hello,
> 
> I'm starting the Calcite 1.31 release. Main branch will be frozen
>> until
> completion.
> 
> Please avoid committing anything to main until further notice.
> 
> Andrei.
> 
 
>>> 
>> 
> 
> 
> -- 
> 
> Best,
> Benchao Li

Summary of experience using Calcite

[Sandeep N]

Hi all,

I wanted to share my experience with one of the current projects that I
tried to use Calcite with and through this experience hope to shed light on
some of the challenges I ran into and share my thoughts on how they could
be addressed.

Apologies for the long drawn explanation but I think context will help you
guys understand the challenges I hope to surface.

For the tldr version jump to the paragraph with heading "In Summary".

Let me first note that my use of Calcite  is unconventional in the sense
that I was looking to convert queries written in a homegrown DSL to SQL
that could be run on Snowflake. There were two main challenges : (a)
Mapping the DSL to SQL and (b) Generating SQL compliant to Snowflake with
support for custom data-types like VARIANT.

The approach I landed on based on feedback from this forum was to take the
AST from the DSL, use a tree walker to generate a relational algebra tree
which then can generate SQL using one of the Dialects (Snowflake to be
added).

The first task involved asking questions on the dev forum and reading
through previously asked questions. Here I have to extend thanks to
Stamatis for the pointers and links he provided which helped immensely. In
the end it took those pointers + step through debugging of the code via
unit-tests to figure how to use the relational algebra tree (RelNode and
RexNode).

The second task was to add Snowflake specific constructs and here again
thanks to  Stamatis and Julian I had pointers to start looking at the
pieces. But before I went down the path of extending relational algebra or
figuring adding snowflake functions I shared my prototype work with my team
and talked through the approach and the Calcite integration. My goal was to
get a general sense of whether my team thought this was a good path forward
(some of the reason for this will be obvious in the In Summary section).

A few of the team members looked at extending my prototype with Calcite and
after a couple of weeks of working with it we as a team arrived at the
decision that we will hold off on using  Calcite and instead build
something simpler and homegrown. "Simpler" because the types of queries we
needed to translate to SQL were very limited and did not exercise all the
possible types of SQL queries. We may revisit this in the future and come
back to Calcite.

In Summary : The feedback I have is as follows
(a) Calcite does not have enough examples, docs and how-to guides so it is
difficult to get started. For this one I think our usage of Calcite being
unconventional added to the complexity as the samples might not directly
map to what we were looking for - one needs to extrapolate from what is
done from SQL to applying the same for another DSL.

(b) Learning curve for Calcite is steep. - It takes a lot of time and
patience to debug through code and understand Calcite before using it. It
takes a certain mind-set of "don't give up" and "must figure this out" to
make headway. My experience with some of the technologies (having written
micro-containers and classloaders) have left me with a higher threshold of
patience - but that is not for everyone.

I will say that the dev forums are prompt to respond to queries but at
times it might not be adequate for newbies - One suggestion I can think of
is to pair a newbie with a seasoned Calcite contributor as a mentor for
tasks which need such a pairing e.g. adding a new Database Dialect. It is a
fairly big commitment/investment on the part of the mentor but it might
make inroads into the project easier. Mentors could assign tasks of
improving docs/samples/how-to guides to the newbies as they work with them.
It would address the challenges for future adopters and also build
confidence in the newbies to contribute.

I will end on a thank you note, just the process of figuring out the
relational algebra tree and translation taught me a lot. Thanks to you guys
for all the contributions. Like I said earlier I am still hopeful we will
come back to Calcite and thus plan to stay plugged in.

Cheers,

--Sandeep Nayak

Re: Calcite Release 1.31 started

[Benchao Li]

Thanks Andrei for being the RM for 1.31.0


Gavin Ray  于2022年8月4日周四 02:46写道：

> Hooray!!
>
> On Wed, Aug 3, 2022 at 2:34 PM Michael Mior  wrote:
>
> > Thanks for all your work Andrei!
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda  wrote:
> >
> > > 1.31 has been released.
> > >
> > > You can resume dev work on the main branch.
> > >
> > > Thanks for your patience.
> > >
> > >
> > > On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda 
> wrote:
> > >
> > > > Hello,
> > > >
> > > > I'm starting the Calcite 1.31 release. Main branch will be frozen
> until
> > > > completion.
> > > >
> > > > Please avoid committing anything to main until further notice.
> > > >
> > > > Andrei.
> > > >
> > >
> >
>


-- 

Best,
Benchao Li

Re: Calcite Release 1.31 started

[Julian Hyde]

Sure enough, GitHub’s dependabot has already noticed the release and
prompted me to upgrade to it [1].

Many thanks for being release manager, Andrei. It’s a lot of work, but a
steady release cadence is vital to keeping the community engaged and the
product secure.

Julian

[1]
https://github.com/hydromatic/morel/commit/edc9d6a43a2341190c920b8afe8b266c375054fb



On Aug 3, 2022, at 11:46 AM, Gavin Ray  wrote:

Hooray!!

On Wed, Aug 3, 2022 at 2:34 PM Michael Mior  wrote:

Thanks for all your work Andrei!

--
Michael Mior
mm...@apache.org


On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda  wrote:

1.31 has been released.

You can resume dev work on the main branch.

Thanks for your patience.


On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda  wrote:

Hello,

I'm starting the Calcite 1.31 release. Main branch will be frozen until
completion.

Please avoid committing anything to main until further notice.

Andrei.

Percentile_Disc return type

[Itiel Sadeh]

Hello,

First of all I just wanted to thank all of you for the work you are doing
on Calcite.

We want to add support for the percentile_disc aggregate function, but we
encounter a problem.
Our percentile_disc return type is dependent on the column of the "order
by" clause (just like postgresql
).
However, I don't see how to achieve that on calcite. The issue is that the
sort column argument is not passed to the percentile_disc function, Rather,
it is stored outside of it as a collation. Therefore, I cannot use the
SqlReturnTypeInference mechanism.

If someone has an idea on how to achieve this it would be very much
appreciated.
Thank you,

Itiel

Re: Calcite Release 1.31 started

[Gavin Ray]

Hooray!!

On Wed, Aug 3, 2022 at 2:34 PM Michael Mior  wrote:

> Thanks for all your work Andrei!
>
> --
> Michael Mior
> mm...@apache.org
>
>
> On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda  wrote:
>
> > 1.31 has been released.
> >
> > You can resume dev work on the main branch.
> >
> > Thanks for your patience.
> >
> >
> > On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda  wrote:
> >
> > > Hello,
> > >
> > > I'm starting the Calcite 1.31 release. Main branch will be frozen until
> > > completion.
> > >
> > > Please avoid committing anything to main until further notice.
> > >
> > > Andrei.
> > >
> >
>

Re: Calcite Release 1.31 started

[Michael Mior]

Thanks for all your work Andrei!

--
Michael Mior
mm...@apache.org


On Wed, Aug 3, 2022 at 12:44 PM Andrei Sereda  wrote:

> 1.31 has been released.
>
> You can resume dev work on the main branch.
>
> Thanks for your patience.
>
>
> On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda  wrote:
>
> > Hello,
> >
> > I'm starting the Calcite 1.31 release. Main branch will be frozen until
> > completion.
> >
> > Please avoid committing anything to main until further notice.
> >
> > Andrei.
> >
>

Jenkins build is back to normal : Calcite » Calcite-snapshots #197

[Apache Jenkins Server]

See 

[jira] [Created] (CALCITE-5226) Resolve security Vuln in Commons-DBCP

[Scott Reynolds (Jira)]

Scott Reynolds created CALCITE-5226:
---

 Summary: Resolve security Vuln in Commons-DBCP
 Key: CALCITE-5226
 URL: https://issues.apache.org/jira/browse/CALCITE-5226
 Project: Calcite
  Issue Type: Bug
Reporter: Scott Reynolds


In DBCP-562, the information is leaked. This was fixed in 2021. We should bump 
to to the latest version to resolve it.

 

[https://ossindex.sonatype.org/vulnerability/sonatype-2020-1349?component-type=maven&component-name=org.apache.commons%2Fcommons-dbcp2&utm_source=ossindex-client&utm_medium=integration&utm_content=1.7.0]

[https://ossindex.sonatype.org/vulnerability/sonatype-2020-0460?component-type=maven&component-name=org.apache.commons%2Fcommons-dbcp2&utm_source=ossindex-client&utm_medium=integration&utm_content=1.7.0]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: Calcite » Calcite-snapshots #196

[Apache Jenkins Server]

See 


Changes:

[25229979+asereda-gs] Prepare for next development iteration


--
[...truncated 5.02 KB...]
> Task :buildSrc:compileTestKotlin NO-SOURCE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:buildext:pluginDescriptors
> Task :buildSrc:javacc:pluginDescriptors
> Task :buildSrc:fmpp:pluginDescriptors
> Task :buildSrc:buildext:processResources
> Task :buildSrc:fmpp:processResources
> Task :buildSrc:javacc:processResources
> Task :buildSrc:javacc:autostyleKotlinCheck
> Task :buildSrc:buildext:autostyleKotlinCheck
> Task :buildSrc:fmpp:autostyleKotlinCheck
> Task :buildSrc:javacc:autostyleKotlinGradleCheck
> Task :buildSrc:javacc:autostyleCheck
> Task :buildSrc:javacc:processTestResources NO-SOURCE
> Task :buildSrc:buildext:autostyleKotlinGradleCheck
> Task :buildSrc:buildext:autostyleCheck
> Task :buildSrc:buildext:processTestResources NO-SOURCE
> Task :buildSrc:fmpp:autostyleKotlinGradleCheck
> Task :buildSrc:fmpp:autostyleCheck
> Task :buildSrc:fmpp:processTestResources NO-SOURCE

> Task :buildSrc:javacc:compileKotlin
w: 
:
 (66, 13): 'setter for main: String?' is deprecated. Deprecated in Java

> Task :buildSrc:buildext:compileKotlin
> Task :buildSrc:fmpp:compileKotlin
> Task :buildSrc:javacc:compileJava NO-SOURCE
> Task :buildSrc:javacc:classes
> Task :buildSrc:fmpp:compileJava NO-SOURCE
> Task :buildSrc:fmpp:classes
> Task :buildSrc:buildext:compileJava NO-SOURCE
> Task :buildSrc:buildext:classes
> Task :buildSrc:javacc:inspectClassesForKotlinIC
> Task :buildSrc:fmpp:inspectClassesForKotlinIC
> Task :buildSrc:buildext:inspectClassesForKotlinIC
> Task :buildSrc:javacc:jar
> Task :buildSrc:buildext:jar
> Task :buildSrc:fmpp:jar
> Task :buildSrc:javacc:assemble
> Task :buildSrc:fmpp:assemble
> Task :buildSrc:fmpp:compileTestKotlin NO-SOURCE
> Task :buildSrc:javacc:compileTestKotlin NO-SOURCE
> Task :buildSrc:buildext:assemble
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check
> Task :buildSrc:buildext:compileTestKotlin NO-SOURCE
> Task :buildSrc:build
> Task :buildSrc:javacc:pluginUnderTestMetadata
> Task :buildSrc:fmpp:pluginUnderTestMetadata
> Task :buildSrc:buildext:pluginUnderTestMetadata
> Task :buildSrc:javacc:compileTestJava NO-SOURCE
> Task :buildSrc:fmpp:compileTestJava NO-SOURCE
> Task :buildSrc:fmpp:testClasses UP-TO-DATE
> Task :buildSrc:buildext:compileTestJava NO-SOURCE
> Task :buildSrc:buildext:testClasses UP-TO-DATE
> Task :buildSrc:fmpp:test NO-SOURCE
> Task :buildSrc:javacc:testClasses UP-TO-DATE
> Task :buildSrc:buildext:test NO-SOURCE
> Task :buildSrc:javacc:test NO-SOURCE
> Task :buildSrc:buildext:validatePlugins
> Task :buildSrc:buildext:check
> Task :buildSrc:buildext:build
> Task :buildSrc:javacc:validatePlugins
> Task :buildSrc:javacc:check
> Task :buildSrc:javacc:build
> Task :buildSrc:fmpp:validatePlugins
> Task :buildSrc:fmpp:check
> Task :buildSrc:fmpp:build

> Configure project :
Building Apache Calcite 1.32.0-SNAPSHOT

> Task :assemble UP-TO-DATE
> Task :bom:assemble UP-TO-DATE
> Task :cassandra:processResources NO-SOURCE
> Task :example:assemble UP-TO-DATE
> Task :testkit:processResources NO-SOURCE
> Task :druid:processResources NO-SOURCE
> Task :file:processResources NO-SOURCE
> Task :elasticsearch:processResources NO-SOURCE
> Task :innodb:processResources NO-SOURCE
> Task :kafka:processResources NO-SOURCE
> Task :geode:processResources NO-SOURCE
> Task :mongodb:processResources NO-SOURCE
> Task :pig:processResources NO-SOURCE
> Task :redis:processResources NO-SOURCE
> Task :bom:autostyleConfigsCheck NO-SOURCE
> Task :example:autostyleConfigsCheck NO-SOURCE
> Task :plus:processResources
> Task :example:autostyleKotlinGradleCheck NO-SOURCE
> Task :spark:processResources NO-SOURCE
> Task :splunk:processResources NO-SOURCE
> Task :example:autostyleCheck UP-TO-DATE
> Task :example:check UP-TO-DATE
> Task :example:build UP-TO-DATE
> Task :kafka:autostyleConfigsCheck
> Task :plus:autostyleConfigsCheck
> Task :elasticsearch:autostyleConfigsCheck
> Task :pig:autostyleConfigsCheck
> Task :spark:autostyleConfigsCheck
> Task :mongodb:autostyleConfigsCheck
> Task :file:autostyleConfigsCheck
> Task :druid:autostyleConfigsCheck
> Task :innodb:autostyleConfigsCheck
> Task :testkit:autostyleConfigsCheck
> Task :cassandra:autostyleConfigsCheck
> Task :geode:autostyleConfigsCheck
> Task :redis:autostyleConfigsCheck
> Task :splunk:autostyleConfigsCheck
> Task :autostyleConfigsCheck
> Task :release:gitProps
> Task :release:getLicenses
> Task :ubenchmark:compileJava NO-SOURCE
> Task :release:license
> Task :release:licenseCopySpec
> Task :ubench

Re: Calcite Release 1.31 started

[Andrei Sereda]

1.31 has been released.

You can resume dev work on the main branch.

Thanks for your patience.


On Wed, Jul 20, 2022 at 2:52 AM Andrei Sereda  wrote:

> Hello,
>
> I'm starting the Calcite 1.31 release. Main branch will be frozen until
> completion.
>
> Please avoid committing anything to main until further notice.
>
> Andrei.
>

Build failed in Jenkins: Calcite » Calcite-snapshots #195

[Apache Jenkins Server]

See 


Changes:

[25229979+asereda-gs] [CALCITE-5096] Increase gradle build heap size from 512m 
to 1g

[25229979+asereda-gs] [CALCITE-5096] Release Calcite 1.31.0

[25229979+asereda-gs] [CALCITE-5096] Release announcement for Calcite 1.31


--
[...truncated 19.20 KB...]
> Task :buildSrc:buildext:check
> Task :buildSrc:buildext:build
> Task :buildSrc:javacc:validatePlugins
> Task :buildSrc:javacc:check
> Task :buildSrc:javacc:build
> Task :buildSrc:fmpp:validatePlugins
> Task :buildSrc:fmpp:check
> Task :buildSrc:fmpp:build

> Configure project :
Building Apache Calcite 1.31.0-SNAPSHOT

> Task :cassandra:processResources NO-SOURCE
> Task :assemble UP-TO-DATE
> Task :testkit:processResources NO-SOURCE
> Task :bom:assemble UP-TO-DATE
> Task :druid:processResources NO-SOURCE
> Task :bom:autostyleConfigsCheck NO-SOURCE
> Task :elasticsearch:processResources NO-SOURCE
> Task :example:assemble UP-TO-DATE
> Task :file:processResources NO-SOURCE
> Task :innodb:processResources NO-SOURCE
> Task :geode:processResources NO-SOURCE
> Task :kafka:processResources NO-SOURCE
> Task :mongodb:processResources NO-SOURCE
> Task :pig:processResources NO-SOURCE
> Task :redis:processResources NO-SOURCE
> Task :plus:processResources
> Task :spark:processResources NO-SOURCE
> Task :splunk:processResources NO-SOURCE
> Task :example:autostyleConfigsCheck NO-SOURCE
> Task :example:autostyleKotlinGradleCheck NO-SOURCE
> Task :example:autostyleCheck UP-TO-DATE
> Task :example:check UP-TO-DATE
> Task :example:build UP-TO-DATE
> Task :testkit:autostyleConfigsCheck
> Task :spark:autostyleConfigsCheck
> Task :file:autostyleConfigsCheck
> Task :pig:autostyleConfigsCheck
> Task :mongodb:autostyleConfigsCheck
> Task :kafka:autostyleConfigsCheck
> Task :druid:autostyleConfigsCheck
> Task :plus:autostyleConfigsCheck
> Task :geode:autostyleConfigsCheck
> Task :innodb:autostyleConfigsCheck
> Task :splunk:autostyleConfigsCheck
> Task :cassandra:autostyleConfigsCheck
> Task :redis:autostyleConfigsCheck
> Task :elasticsearch:autostyleConfigsCheck
> Task :autostyleConfigsCheck
> Task :release:gitProps
> Task :release:getLicenses
> Task :release:license
> Task :release:licenseCopySpec

> Task :piglet:javaCCMain
Java Compiler Compiler Version 4.0 (Parser Generator)
(type "javacc" with no arguments for help)
Reading from file 

 . . .
Warning: Output directory 
"
 does not exist. Creating the directory.
Note: UNICODE_INPUT option is specified. Please make sure you create the 
parser/lexer using a Reader with the correct character encoding.
File "TokenMgrError.java" does not exist.  Will create one.
File "ParseException.java" does not exist.  Will create one.
File "Token.java" does not exist.  Will create one.
File "SimpleCharStream.java" does not exist.  Will create one.
Parser generated with 0 errors and 1 warnings.

> Task :piglet:processResources NO-SOURCE
> Task :piglet:autostyleConfigsCheck
> Task :ubenchmark:compileJava NO-SOURCE
> Task :ubenchmark:processResources NO-SOURCE
> Task :ubenchmark:classes UP-TO-DATE
> Task :ubenchmark:jandexMain NO-SOURCE
> Task :ubenchmark:processJandexIndex NO-SOURCE
> Task :ubenchmark:jar
> Task :ubenchmark:sourcesJar
> Task :ubenchmark:assemble
> Task :ubenchmark:autostyleConfigsCheck
> Task :example:csv:processResources NO-SOURCE
> Task :example:csv:autostyleConfigsCheck
> Task :kafka:autostyleJavaCheck
> Task :redis:autostyleJavaCheck
> Task :ubenchmark:autostyleJavaCheck
> Task :splunk:autostyleJavaCheck
> Task :example:csv:autostyleJavaCheck
> Task :spark:autostyleJavaCheck
> Task :pig:autostyleJavaCheck
> Task :cassandra:autostyleJavaCheck
> Task :mongodb:autostyleJavaCheck
> Task :file:autostyleJavaCheck
> Task :innodb:autostyleJavaCheck
> Task :plus:autostyleJavaCheck
> Task :geode:autostyleJavaCheck
> Task :piglet:autostyleJavaCheck
> Task :elasticsearch:autostyleJavaCheck
> Task :kafka:autostyleKotlinGradleCheck
> Task :kafka:autostyleCheck
> Task :mongodb:autostyleKotlinGradleCheck
> Task :mongodb:autostyleCheck
> Task :spark:autostyleKotlinGradleCheck
> Task :spark:autostyleCheck
> Task :pig:autostyleKotlinGradleCheck
> Task :pig:autostyleCheck
> Task :redis:autostyleKotlinGradleCheck
> Task :redis:autostyleCheck
> Task :plus:autostyleKotlinGradleCheck
> Task :plus:autostyleCheck
> Task :elasticsearch:autostyleKotlinGradleCheck
> Task :elasticsearch:autostyleCheck
> Task :ubenchmark:autostyleKotlinGradleCheck
> Task :ubenchmark:autostyleCheck
> Task :splunk:autostyleKotlinGradleCheck
> Task :file:autostyleKotlinGradleCheck
> Task :geode:autostyleKotlinGradleCheck
> Task :file:autostyleCheck
> Task :innodb:autostyleKotlinGradleCheck
>