Re: [ANNOUNCE] New Calcite PMC chair: Benchao Li

[Enrico Olivelli]

Congratulations !

Enrico

Il giorno gio 21 dic 2023 alle ore 15:07 Tanner Clary
 ha scritto:
>
> Congrats, Benchao :)
>
> -Tanner
>
> On Thu, Dec 21, 2023 at 2:33 AM Alessandro Solimando <
> alessandro.solima...@gmail.com> wrote:
>
> > Congratulations Benchao, and thanks again to Stamatis for his great job!
> >
> > On Thu, 21 Dec 2023 at 11:18, Ruben Q L  wrote:
> >
> > > Congratulations Benchao!!
> > >
> > >
> > > On Thu, Dec 21, 2023 at 10:05 AM Sergey Nuyanzin 
> > > wrote:
> > >
> > > > Congratulations, Benchao!
> > > >
> > > > On Thu, Dec 21, 2023 at 10:47 AM Francis Chuang <
> > > francischu...@apache.org>
> > > > wrote:
> > > >
> > > > > Congratulations, Benchao!
> > > > >
> > > > > On 21/12/2023 8:24 pm, Guangdong Liu wrote:
> > > > > > Congratulations!
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best Regards
> > > > > >
> > > > > > 
> > > > > >
> > > > > > Liugddx
> > > > > > liug...@gmail.com
> > > > > >
> > > > > >
> > > > > > Stamatis Zampetakis  于2023年12月21日周四 17:18写道：
> > > > > >
> > > > > >> Calcite community members,
> > > > > >>
> > > > > >> I am pleased to announce that we have a new PMC chair and VP as
> > per
> > > > > >> our tradition of rotating the chair once a year. I have resigned,
> > > and
> > > > > >> Benchao Li was duly elected by the PMC and approved unanimously by
> > > the
> > > > > >> Board.
> > > > > >>
> > > > > >> Please join me in congratulating Benchao!
> > > > > >>
> > > > > >> Best regards,
> > > > > >> Stamatis
> > > > > >>
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Sergey
> > > >
> > >
> >

Re: [VOTE] Release Apache Calcite 1.31.0 (release candidate 3)

[Enrico Olivelli]

+1 (non binding)

All tests of HerdDB pass without any code changes
CI results on this PR:  https://github.com/diennea/herddb/pull/789


Thanks for driving the release
Enrico

Il giorno ven 29 lug 2022 alle ore 14:07 Andrei Sereda
 ha scritto:
>
> Hi all,
>
> I have created a build for Apache Calcite 1.31.0, release
> candidate 3.
>
> Thanks to everyone who has contributed to this release.
>
> You can read the release notes here:
> https://github.com/apache/calcite/blob/calcite-1.31.0-rc3/site/_docs/history.md
>
> The commit to be voted upon:
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=86d1bf40e4799665951065fd5f4ee4a771e7b655
>
> Its hash is 86d1bf40e4799665951065fd5f4ee4a771e7b655
>
> Tag:
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.31.0-rc3
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc3
> (revision 56034)
>
> The hashes of the artifacts are as follows:
> 477848037465e883bc7f45d71a59be5932ddfc37d4fd4571f17d7463afe6cc3df1ab5bd20f38fb28fd6468d5b23f559eba4ff3adde9d4f689622e95abd736260
> *apache-calcite-1.31.0-src.tar.gz
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachecalcite-1172/org/apache/calcite/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/sereda.asc
> https://www.apache.org/dist/calcite/KEYS
>
> To create the jars and test Apache Calcite: "gradle build"
> (requires an appropriate Gradle/JDK installation)
>
> Please vote on releasing this package as Apache Calcite 1.31.0.
>
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Calcite 1.31.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Here is my vote:
>
> +1 (non-binding)

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 3)

[Enrico Olivelli]

+1 (non binding)
Run HerdDB tests (https://github.com/diennea/herddb/pull/782) and a
few applications, no issues.

Calcite works like a charm, still with this RC (as expected)

Enrico

Il giorno mer 16 mar 2022 alle ore 16:49 Alessandro Solimando
 ha scritto:
>
> Hello everyone,
> thanks a lot Liya Fan for all the hard work on the release and for
> battle-testing our process documentation which is very important.
>
> My vote is +1 (non-binding), tested as follows:
>
> - verified gpg signature: OK
> $ curl "https://downloads.apache.org/calcite/KEYS; | gpg --import
> ...
> gpg: key 7F99D3070C037870: public key "Liya Fan (CODE SIGNING KEY) <
> liya...@apache.org>" imported
>
> $ gpg --verify apache-calcite-1.30.0-src.tar.gz.asc.txt
> apache-calcite-1.30.0-src.tar.gz
> gpg: Signature made Wed 16 Mar 04:26:23 2022 CET
> gpg:using RSA key F90563572D7E336E0A0AD0957F99D3070C037870
> gpg: Good signature from "Liya Fan (CODE SIGNING KEY) "
> [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg:  There is no indication that the signature belongs to the
> owner.
> Primary key fingerprint: F905 6357 2D7E 336E 0A0A  D095 7F99 D307 0C03 7870
>
> (we agreed on previous releases that this warning is OK)
>
> - verified package checksum: OK
> $ diff <(cat apache-calcite-1.30.0-src.tar.gz.sha512.txt) <(shasum -a 512
> apache-calcite-1.30.0-src.tar.gz)
> 1c1
> <
> 6c9f65bcc75a05c226dbb0a74495e76761d738f6923086e2943fface1b8864697dff6d0de7aa0cb9b06f4f6aebe1322b5ecb829428f84d08c51dfb5773034040
> *apache-calcite-1.30.0-src.tar.gz
> ---
> >
> 6c9f65bcc75a05c226dbb0a74495e76761d738f6923086e2943fface1b8864697dff6d0de7aa0cb9b06f4f6aebe1322b5ecb829428f84d08c51dfb5773034040
>  apache-calcite-1.30.0-src.tar.gz
>
> - verified gradle build: OK
> $ cp ~/git/calcite/gradlew .
> $ cp -r ~/git/calcite/gradle/wrapper gradle
> $ ./gradle build
> ...
> BUILD SUCCESSFUL in 5m 25s
>
> - checked release notes: OK
> - checked few modules in Nexus: OK
> - checking difference in folder: OK
> $ diff -qr . ~/git/calcite | grep -v gradle | grep -v idea
> (nothing worth mentioning)
>
> - environment used:
>
> $ sw_vers
> ProductName: macOS
> ProductVersion: 11.6
> BuildVersion: 20G165
>
> $ ./gradlew -version
>
> 
> Gradle 7.3
> 
>
> Build time:   2021-11-09 20:40:36 UTC
> Revision: 96754b8c44399658178a768ac764d727c2addb37
>
> Kotlin:   1.5.31
> Groovy:   3.0.9
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  1.8.0_292 (AdoptOpenJDK 25.292-b10)
> OS:   Mac OS X 10.16 x86_64
>
> $ java -version
> openjdk version "1.8.0_292"
> OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
> OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)
>
> Best regards,
> Alessandro
>
> On Wed, 16 Mar 2022 at 11:21, Ruben Q L  wrote:
>
> > Thanks again Liya Fan for being the release manager in this
> > particularly arduous one.
> >
> > My vote is: +1 (binding)
> >
> > - Checksum and signature: ok
> > - Gradle test: ok (including the problematic one in isolation)
> > - Calcite-based application test suite: ok
> >
> > Best,
> > Ruben
> >
> >
> > On Wed, Mar 16, 2022 at 5:01 AM Francis Chuang 
> > wrote:
> >
> > > Thanks, Liya!
> > >
> > > My vote is: +1 (binding)
> > >
> > > - Verified GPG signature - OK
> > > - Verified SHA512 - OK
> > > - Checked release notes on tag
> > > (
> > >
> > https://github.com/apache/calcite/blob/calcite-1.30.0-rc3/site/_docs/history.md
> > )
> > >
> > > - OK
> > > - Ran tests (gradle check) - OK
> > > - Ran failing test in rc2 (gradle cleanTest :core:test --tests
> > > SqlTypeFactoryTest.testUnknownCreateWithNullabilityTypeConsistency) - OK
> > > - Spot checked Nexus artifacts - OK
> > >
> > > Environment:
> > > Gradle 7.4.1 docker container in WSL2 (Ubuntu 20.04) on Windows 10 21h2
> > >
> > >  > docker version
> > > Client: Docker Engine - Community
> > >   Cloud integration: v1.0.22
> > >   Version:   20.10.13
> > >   API version:   1.41
> > >   Go version:go1.16.15
> > >   Git commit:a224086
> > >   Built: Thu Mar 10 14:08:15 2022
> > >   OS/Arch:   linux/amd64
> > >   Context:   default
> > >   Experimental:  true
> > >
> > > Server: Docker Desktop
> > >   Engine:
> > >Version:  20.10.13
> > >API version:  1.41 (minimum version 1.12)
> > >Go version:   go1.16.15
> > >Git commit:   906f57f
> > >Built:Thu Mar 10 14:06:05 2022
> > >OS/Arch:  linux/amd64
> > >Experimental: false
> > >   containerd:
> > >Version:  1.5.10
> > >GitCommit:2a1d4dbdb2a1030dc5b01e96fb110a9d9f150ecc
> > >   runc:
> > >Version:  1.0.3
> > >GitCommit:v1.0.3-0-gf46b6ba
> > >   docker-init:
> > >Version:  0.19.0
> > >GitCommit: 

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 2)

[Enrico Olivelli]

+1 non binding

HerdDB works like a charm without any changes
https://github.com/diennea/herddb/pull/782

Thanks for running the release

Enrico

Il Lun 14 Mar 2022, 12:02 Fan Liya  ha scritto:

> Hi Ruben,
>
> Thanks a lot for your vote.
>
> Hi all,
>
> Since part of the voting period is during the weekend, let's extend the
> voting period to at least 96 hours.
>
> Best,
> Liya Fan
>
>
> Ruben Q L  于2022年3月14日周一 18:00写道：
>
> > Thanks Liya Fan for being the release manager.
> >
> > My vote is: +1 (binding)
> >
> > - Checksum and signature: ok
> > - Gradle test: ok
> > - Calcite-based application test suite: ok
> >
> > Best,
> > Ruben
> >
> >
> > On Fri, Mar 11, 2022 at 1:54 PM Fan Liya  wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.30.0, release
> > > candidate 2.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > You can read the release notes here:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/calcite-1.30.0-rc2/site/_docs/history.md
> > >
> > > The commit to be voted upon:
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=170035fd97df1afdd0c0a499f63e0ca1606f7837
> > >
> > > Its hash is 170035fd97df1afdd0c0a499f63e0ca1606f7837
> > >
> > > Tag:
> > > https://github.com/apache/calcite/tree/calcite-1.30.0-rc2
> > >
> > > The artifacts to be voted on are located here:
> > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc2
> > > (revision 52982)
> > >
> > > The hashes of the artifacts are as follows:
> > >
> > >
> >
> 988c165bacb3ac20f9046fc80ebcd28eca66504ed3bdd3269ac2625ff59a80b06e3d7de12150ed9fe1e40def8b1a8290a87e280b7a7f0589a5b740fc79e273dd
> > > *apache-calcite-1.30.0-src.tar.gz
> > >
> > > A staged Maven repository is available for review at:
> > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1152/org/apache/calcite/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/COMMITTER_ID.asc
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > > To create the jars and test Apache Calcite: "gradle build"
> > > (requires an appropriate Gradle/JDK installation)
> > >
> > > Please vote on releasing this package as Apache Calcite 1.30.0.
> > >
> > > The vote is open for the next 72 hours and passes if a majority of at
> > > least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.30.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Here is my vote:
> > >
> > > +1 (non-binding)
> > >
> > > Best,
> > > Liya Fan
> > >
> >
>

Re: [VOTE] Release Apache Calcite 1.30.0 (release candidate 1)

[Enrico Olivelli]

+1 (non binding)

Run all tests of HerdDB, all tests are passing without any code
change. (https://github.com/diennea/herddb/pull/779)

I would like to note that previous tests of HerdDB did not pass with
1.29.0 (and we were stuck to 1.28.0). With 1.29.0 tests on Github
actions hang during some classloading of some Calcite classes

thank you Liya

Enrico

Il giorno lun 7 mar 2022 alle ore 09:20 Fan Liya
 ha scritto:
>
> Hi Xiong,
>
> Thanks for your feedback. I will change the logs in a follow up MR.
>
> Best,
> Liya Fan
>
>
> xiong duan  于2022年3月7日周一 15:30写道：
>
> > Hi, Liya Fan.Thanks for your work.  +1.
> > Just a little problem:
> > I noticed the Bug-fixes logs all started with FixI think maybe we
> > should stay the same as JIRA summary or PR commit?
> >
> > Fan Liya  于2022年3月7日周一 14:55写道：
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.30.0, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > You can read the release notes here:
> > >
> > >
> > https://github.com/apache/calcite/blob/calcite-1.30.0-rc1/site/_docs/history.md
> > >
> > > The commit to be voted upon:
> > >
> > >
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=f14cf4c32b9079984a988bbad40230aa6a59b127
> > >
> > > Its hash is f14cf4c32b9079984a988bbad40230aa6a59b127
> > >
> > > Tag:
> > > https://github.com/apache/calcite/tree/calcite-1.30.0-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.30.0-rc1
> > > (revision 52897)
> > >
> > > The hashes of the artifacts are as follows:
> > >
> > >
> > 25527b5dfd3c28d4ac3c9d9a40b94dbcbce7feb17cc198ebe7c36a30c2df69a27ee5e4defab4edff1e1bc65c076bd0725bd8c378913d1d23f3d80f732e3e097f
> > > *apache-calcite-1.30.0-src.tar.gz
> > >
> > > A staged Maven repository is available for review at:
> > >
> > >
> > https://repository.apache.org/content/repositories/orgapachecalcite-1150/org/apache/calcite/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/COMMITTER_ID.asc
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > > To create the jars and test Apache Calcite: "gradle build"
> > > (requires an appropriate Gradle/JDK installation)
> > >
> > > Please vote on releasing this package as Apache Calcite 1.30.0.
> > >
> > > The vote is open for the next 72 hours and passes if a majority of at
> > > least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.30.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Here is my vote:
> > >
> > > +1 (binding)
> > >
> > > Best,
> > > Liya Fan
> > >
> >

Re: [ANNOUNCE] New Calcite PMC chair: Ruben Q L

[Enrico Olivelli]

Congrats!

Enrico

Il Sab 22 Gen 2022, 03:03 xiong duan  ha scritto:

> Congratulations to Ruben and thanks to Haisheng!
>
> Danny Chan  于2022年1月21日周五 12:19写道：
>
> > Congratulations, Ruben, and good luck!
> >
> > Haisheng, Thank you for serving as Chair.
> >
> > Yanjing Wang 于2022年1月21日 周五下午12:16写道：
> >
> > > Congrats Ruben!
> > >
> > > Stamatis Zampetakis  于2022年1月21日周五 06:34写道：
> > >
> > > > Congrats Ruben! You're kind, fair, and knowledgeable, very well
> > deserved.
> > > >
> > > > Many thanks for serving as a chair Haisheng.
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > > On Thu, Jan 20, 2022, 12:41 PM Forward Xu 
> > > wrote:
> > > >
> > > > > Congratulations to Ruben!  Thanks for serving as Chair during the
> > last
> > > > > year, Haisheng!
> > > > >
> > > > >
> > > > > forward
> > > > >
> > > > > Ruben Q L  于2022年1月20日周四 19:38写道：
> > > > >
> > > > > > Thanks everyone!
> > > > > > And thank you Haisheng for being our PMC Chair during last year!
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 20, 2022 at 8:52 AM Alessandro Solimando <
> > > > > > alessandro.solima...@gmail.com> wrote:
> > > > > >
> > > > > > > Congratulations to Ruben and thanks a lot to Haisheng!
> > > > > > >
> > > > > > > On Thu, 20 Jan 2022 at 08:13, 953396112
> > <13282155...@qq.com.invalid
> > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Congratulations to Ruben!
> > > > > > > > Thanks for serving as Chair, Haisheng!
> > > > > > > >
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Zhaohui Xu
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --原始邮件--
> > > > > > > > 发件人:
> > > > > > > >   "dev"
> > > > > > > >
> >  <
> > > > > > > > hy...@apache.org;
> > > > > > > > 发送时间:2022年1月20日(星期四) 上午10:29
> > > > > > > > 收件人:"dev" > > > > > > >
> > > > > > > > 主题:[ANNOUNCE] New Calcite PMC chair: Ruben Q L
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Calcite community members,
> > > > > > > >
> > > > > > > > I am pleased to announce that we have a new PMC chair and VP
> as
> > > per
> > > > > our
> > > > > > > > tradition of rotating the chair once a year. I have resigned,
> > and
> > > > > Ruben
> > > > > > > Q L
> > > > > > > > was duly elected by the PMC and approved unanimously by the
> > > Board.
> > > > > > > >
> > > > > > > > Please join me in congratulating Ruben!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Haisheng Yuan
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Alessandro Solimando

[Enrico Olivelli]

Congrats!
Enrico

Il Sab 18 Dic 2021, 16:00 Alessandro Solimando <
alessandro.solima...@gmail.com> ha scritto:

> Hello everyone,
> it's an honour and privilege to be part of this great community, I have
> really learned a lot over the years by following discussions and
> contributing, I am looking forward to further collaborations.
>
> My academic background touched upon query processing over semi-structured
> data (XML) and for the Semantic Web (Sparql, descriptions logics,
> ontology-based data access, semantic query processing over DBMS).
>
> After several years of work as a data engineer (big data processing and
> machine learning), I have recently joined Cloudera's optimizer team, where
> I am working on Apache Hive (which extensively uses Apache Calcite).
>
> Best regards,
> Alessandro
>
> On Sat, 18 Dec 2021 at 15:08, Stamatis Zampetakis 
> wrote:
>
> > Apache Calcite's Project Management Committee (PMC) has invited
> Alessandro
> > Solimando to become a committer, and we are pleased to announce that he
> has
> > accepted.
> >
> > Alessandro has been in the community for quite some time now with his
> first
> > contribution dating back in 2018. Since then he pushed many fixes in both
> > Calcite and Avatica touching parts of the core engine,
> > the Cassandra adapter, and materialized view matching. Apart from code
> > contributions, Alessandro provides valuable help to the community by
> doing
> > reviews, answering questions in the devlist, and testing releases.
> >
> > Alessandro, welcome, thank you for your contributions, and we look
> forward
> > to your further interactions with the community! If you wish, please feel
> > free to tell us more about yourself and what you are working on.
> >
> > Stamatis (on behalf of the Apache Calcite PMC)
> >
>

Re: [ANNOUNCE] New Calcite PMC chair: Haisheng Yuan

[Enrico Olivelli]

Congrats!

And thank you Stamatis for your efforts as chair in the past year

Enrico

Il Gio 17 Dic 2020, 14:49 Stamatis Zampetakis  ha
scritto:

> Calcite community members,
>
> I am pleased to announce that we have a new PMC chair and VP as per our
> tradition of rotating the chair once a year. I have resigned, and
> Haisheng was duly elected by the PMC and approved unanimously by the Board.
>
> Please join me in congratulating Haisheng!
>
> Best,
> Stamatis
>

Re: Decouple core from linq4j and Avatica

[Enrico Olivelli]

Il Mer 25 Nov 2020, 05:57 Haisheng Yuan  ha scritto:

> > I would like to propose to decouple the "core" module from "ling4j" and
> Avatica.
> I like the idea.
>
> Moving Enumerable out of core may be time consuming and disruptive,
> because many core tests are using Enumerable to verify plan quality and
> correctness.
>
> Best,
> Haisheng
>
> On 2020/11/24 23:42:19, Stamatis Zampetakis  wrote:
> > Hi Vladimir,
> >
> > Personally, I like the idea.
> > I had similar thoughts in the past but it didn't try to break it down
> since
> > those dependencies were not causing any real trouble.
> >
> > Let's see what the others think.
>

I like the idea of having a smaller footprint.
In HerdDB project we need to reduce as much as possible the size of the
dependencies and Avatica is not useful but have to import it.
Also it would be better to make most of thirdparty dependencies optional if
you do not need them, I am talking about Json and geometry support for
instance.

Thank you for bringing up this proposal!

Enrico


>
> > Best,
> > Stamatis
> >
> >
> > On Tue, Nov 24, 2020 at 7:30 PM Vladimir Ozerov 
> wrote:
> >
> > > Hi colleagues,
> > >
> > > Many Calcite integrations use only part of the framework.
> Specifically, it
> > > is common to use only the parser/optimizer part. JDBC and runtime are
> used
> > > less frequently because they are not very well suited for mature
> processing
> > > engines (e.g. Enumerable runs out of memory easily).
> > >
> > > However, in order to use the parser/optimizer from the core module, you
> > > also need to add "linq4j" and Avatica modules to the classpath, which
> is
> > > not convenient - why to include modules, that you do not use?
> > >
> > > It turns out that most of the dependencies are indeed leaky
> abstractions,
> > > that could be decoupled easily. For example, the RelOptUtil class from
> the
> > > "core" depends on ... two string constants from the Avatica module.
> > >
> > > I would like to propose to decouple the "core" module from "ling4j" and
> > > Avatica. For example, we may introduce the new "common" module, that
> will
> > > hold common constants, utility classes, and interfaces (e.g. Meta).
> Then,
> > > we can organize the dependencies like this:
> > > common -> core
> > > common -> linq4j
> > > common -> Avatica
> > >
> > > Finally, we may shade and relocate the "common" module into the "core"
> > > during the build. In the end, we will have -2 runtime dependencies with
> > > relatively little effort. In principle, the same approach could be
> applied
> > > to Janino and Jackson dependencies, but it could be more complex, so my
> > > proposal is only about "linq4" and Avatica.
> > >
> > > How do you feel about it? Does this proposal sense to the community? If
> > > yes, I can try implementing the POC for this.
> > >
> > > Regards,
> > > Vladimir.
> > >
> >
>

Re: Search/Sarg: untested feature merged to the default branch

[Enrico Olivelli]

Stamatis,

Il Lun 16 Nov 2020, 10:20 Stamatis Zampetakis  ha
scritto:

> Hello,
>
> I don't want the current situation to be disheartening for anybody,
> especially those who spend significant time on these code changes (Julian,
> Vladimir).
> Our common goal is to release 1.27.0 in the best possible conditions so
> let's try to agree on the plan to move forward.
> To do this, I think it is important to answer the following questions:
>
> @Vladimir: Can you specify which JIRA issue(s) you would like to see solved
> for the next release?
> @all (committers or not): Did you postpone the upgrade to Calcite 1.26.0
> due to the problems found by Vladimir?
>

In HerdDB we still haven't updated Calcite because of Sarg, but only
because the change requires few changes and we still do not need to upgrade
yet, so the upgrade is only in our backlog.
Given Vladimir's precious findings we will wait for 1.27.


> As soon as we have the JIRA ids:
> @all (committers or not): Do we have somebody willing to work on the issues
> found by Vladimir?
>

I am sorry I don't have cycles

Enrico


> Best,
> Stamatis
>
> On Sat, Nov 14, 2020 at 4:55 PM Julian Hyde 
> wrote:
>
> > Vladimir,
> >
> > I’ve reverted your revert. A commit war is no way to proceed.
> >
> > Julian
> >
> >
> > > On Nov 14, 2020, at 2:55 AM, Vladimir Sitnikov <
> > sitnikov.vladi...@gmail.com> wrote:
> > >
> > > I've reverted SEARCH. Sorry for the inconvenience.
> > >
> > > Here's the PR that re-adds SEARCH:
> > > https://github.com/apache/calcite/pull/2263
> > > Please feel free to pick it up. I expect certain commits should be
> > squashed
> > > (e.g. search should be re-added as a single commit).
> > >
> > > I'm not sure I would have time to make it happen.
> > >
> > > Frankly speaking, I would suggest we should make SEARCH operator never
> > > return null.
> > > In other words, SEARCH(X, [Y, Z]) should be the same as "(X is not
> > distinct
> > > from Y) or (X is not distinct from Z)".
> > > The old semantics was like SEARCH(null, [42]) => unknown, SEARCH(null,
> > > [null, 42]) => false which results in wrong results in simplification.
> > >
> > > "Expand search" might want to receive unknownAs parameter.
> > >
> > > Vladimir
> >
> >
>

Re: [VOTE] Release apache-calcite-1.26.0 (release candidate 0)

[Enrico Olivelli]

+1 (non binding)
Run tests (AdoptOpen JDK14)
Verified signature and sha512
Run tests of HerdDB. I noticed the new hard dependency on Geometry (but
there is an hacky workaround), and the new SEARCH operator, that looks
cool, even if it will require a few cycles to adapt code to the new
Operator (RexUtil to the rescue!)


Ruben,
I have found this trick in order to drop Geometry and JSON Path
dependencies.
https://github.com/diennea/herddb/pull/707

Thank for driving the release

Enrico


Il giorno mar 6 ott 2020 alle ore 05:31 Andrei Sereda  ha
scritto:

> +1 (non-binding).
>
> - build locally: ok
> - check git hash: ok
> - check sha512 for artifacts: ok
> - check GPG signature: ok
>
> openjdk version "1.8.0_212"
>
> Note that I'm getting 404 for your public GPG key located here:
> https://people.apache.org/keys/committer/rubenql.asc
> So I've used Calcite KEYS instead.
>
> Thanks for RC, Ruben!
>
>
> On Mon, Oct 5, 2020 at 10:46 PM Danny Chan  wrote:
>
> > +1 (binding).
> >
> > - run tests locally: ok
> > - verify the commit hash in git tag: ok
> > - check sha512: ok
> >
> > Environment:
> > - java version 1.8.0_151
> > - MacOS Mojave 10.14.6
> >
> > I also find the new introduced ersi library when i do upgrade for Flink.
> I
> > solves the problem by adding a dependency for it. It is awesome if we can
> > avoid it. But it is not a blocker I guess.
> >
> > Enrico Olivelli 于2020年10月5日 周一下午10:58写道：
> >
> > > Il giorno lun 5 ott 2020 alle ore 16:43 Ruben Q L 
> ha
> > >
> > > scritto:
> > >
> > >
> > >
> > > > Enrico,
> > >
> > > >
> > >
> > > > thanks for your feedback. I also had a similar experience with my
> > >
> > > > downstream project.
> > >
> > > >
> > >
> > > > If I am not mistaken, esri library so far was not necessary because
> it
> > > was
> > >
> > > > used just on Calcite test; however, as you say, now it is required
> > since
> > > it
> > >
> > > > is used for Calcite build.
> > >
> > > > The root cause of this change seems to be CALCITE-1861 [1],
> implemented
> > > via
> > >
> > > > [2]. I think there is not much we can do about it.
> > >
> > > >
> > >
> > >
> > >
> > > So probably this is the line that introduces that hard dependency
> > >
> > >
> > >
> >
> https://github.com/apache/calcite/commit/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0#diff-39080ab40de0df7a4df59ca135c79d2fR377
> > >
> > >
> > >
> > > I will try to find some fix, at least to not fail the loading of
> Programs
> > >
> > > class
> > >
> > > I am going to file a JIRA for this case
> > >
> > >
> > >
> > >
> > >
> > > Enrico
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > >
> > >
> > > > Regarding the new SEARCH operator, I also had to adjust my code.
> > >
> > > > For me, the easiest way to "keep things working as before" was
> > basically
> > >
> > > > adding a "RexUtil#expandSearch" call at the appropriate place in
> order
> > to
> > >
> > > > convert the SEARCH into its equivalent non-search expression.
> > >
> > > >
> > >
> > > > Best,
> > >
> > > > Ruben
> > >
> > > >
> > >
> > > > [1] https://issues.apache.org/jira/browse/CALCITE-1861
> > >
> > > > [2]
> > >
> > > >
> > >
> > > >
> > >
> >
> https://github.com/apache/calcite/commit/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0
> > >
> > > >
> > >
> > > >
> > >
> > > > On Mon, Oct 5, 2020 at 3:17 PM Enrico Olivelli 
> > >
> > > > wrote:
> > >
> > > >
> > >
> > > > > Ruben,
> > >
> > > > > I am testing the RC, I found that now it is mandatory to have
> > >
> > > > > "com.esri.geometry:esri-geometry-api" on the classpath in order to
> > use
> > >
> > > > the
> > >
> > > > > Programs class.
> > >
> > > > >
> > >
> > > > > In HerdDB we were excluding that

Re: [VOTE] Release apache-calcite-1.26.0 (release candidate 0)

[Enrico Olivelli]

Il giorno lun 5 ott 2020 alle ore 16:43 Ruben Q L  ha
scritto:

> Enrico,
>
> thanks for your feedback. I also had a similar experience with my
> downstream project.
>
> If I am not mistaken, esri library so far was not necessary because it was
> used just on Calcite test; however, as you say, now it is required since it
> is used for Calcite build.
> The root cause of this change seems to be CALCITE-1861 [1], implemented via
> [2]. I think there is not much we can do about it.
>

So probably this is the line that introduces that hard dependency
https://github.com/apache/calcite/commit/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0#diff-39080ab40de0df7a4df59ca135c79d2fR377

I will try to find some fix, at least to not fail the loading of Programs
class
I am going to file a JIRA for this case


Enrico



>
> Regarding the new SEARCH operator, I also had to adjust my code.
> For me, the easiest way to "keep things working as before" was basically
> adding a "RexUtil#expandSearch" call at the appropriate place in order to
> convert the SEARCH into its equivalent non-search expression.
>
> Best,
> Ruben
>
> [1] https://issues.apache.org/jira/browse/CALCITE-1861
> [2]
>
> https://github.com/apache/calcite/commit/eab043f4ef43112c16a9f6708e6c53a15b1cfbe0
>
>
> On Mon, Oct 5, 2020 at 3:17 PM Enrico Olivelli 
> wrote:
>
> > Ruben,
> > I am testing the RC, I found that now it is mandatory to have
> > "com.esri.geometry:esri-geometry-api" on the classpath in order to use
> the
> > Programs class.
> >
> > In HerdDB we were excluding that third party dependency, and we would
> like
> > not to depend on it in order to bring in as few third party deps as
> > possible (and have smaller binaries).
> > So I would cast a -1
> > (also it would be super good to not have to depend
> > on com.jayway.jsonpath:json-path, but that needed dependency was there
> > before 1.26 and there is no need to change it now)
> >
> > I can try to work on this issue as soon as possible if you think it is
> > worth it.
> >
> > Apart from that we are trying to make it work to switch to the "SEARCH"
> > operator, that is a big behavior change.
> > I guess there is no way to disable it. But I am fine with that, I knew
> that
> > it is only a matter of implementing that operator as well.
> >
> > Enrico
> >
> > Il giorno lun 5 ott 2020 alle ore 14:42 Vladimir Sitnikov <
> > sitnikov.vladi...@gmail.com> ha scritto:
> >
> > > Thanks for preparing an RC.
> > >
> > > +1
> > >
> > > Build works for me modulo
> > > https://issues.apache.org/jira/browse/CALCITE-2816 PsTableFunction
> > > fails in Russian locale.
> > >
> > > Checksums match, signature validation passes.
> > > The release notes look good.
> > >
> > > changelog> Compatibility: Guava versions 19.0 to 29.0-jre
> > >
> > > I see build scripts are using Guava 29.0-jre only. I guess Guava 19
> > support
> > > comes like "hope it works".
> > >
> > > Vladimir
> > >
> >
>

Re: ApacheCon 2020 talks relevant to Apache Calcite

[Enrico Olivelli]

Stamatis,
I am going to talk about HerdDB inside BigData2 track.
https://www.apachecon.com/acah2020/tracks/bigdata-2.html
"After NoSQL discover CloudSQL databases
Romain Manni-Bucau, Enrico Olivelli"

We are going to cite Calcite somehow in a couple of slides as it is the
core of HerdDB SQL Planner.
Not that relevant, but it something

Cheers
Enrico

Il giorno lun 28 set 2020 alle ore 16:17 Stamatis Zampetakis <
zabe...@gmail.com> ha scritto:

> Hi all,
>
> ApacheCon is starting tomorrow and I was wondering if there are any talks
> relevant to Calcite?
> I had a quick look on the schedule but I didn't see something. If somebody
> is giving a talk please let us know.
>
> Best,
> Stamatis
>

Disabling JaninoRelMetadataProvider

[Enrico Olivelli]

Hi,
I am seeing that JaninoRelMetadataProvider is quite expensive, at least for
the "boostrap" phase of the system.

It is a cool piece of software and it is working well, but in some cases
probably it could be quite overkill, for instance in test cases of
applications that mostly run only one query and then end.

You could argue that the price of the compilation is paid only once and
there is no need to complicate things for some corner cases that probably
do not affect production cases.

In CALCITE-3901 we introduced calcite.enable.regenerate.metadata.handler
system property in order to limit the number of times that Janino kicks in
but not to drop it at all.

>From the code I see that it is a subclass of RelMetadataProvider, but there
is no way to get rid of it in VolcanoPlanner.

Also, Janino is a third party library, and having the ability of dropping
it will help in having a smaller set of dependencies downstream (but this
is not blocker at the moment, it is only a nice-to-have)

Do you have any experience with this problem ? Is there any chance to add
some configuration option to pass a RelMetadataProvider implementation
that does not rely on dynamic code generation ?
If the idea is valuable I will be happy to work on it.

See this issue for reference, with a full stacktrace of the execution
https://github.com/diennea/herddb/issues/517

Regards
Enrico

Re: Combining HepPLanner and VolcanoPlanner

[Enrico Olivelli]

Ruben,

Il giorno gio 17 set 2020 alle ore 14:57 Ruben Q L  ha
scritto:

> Enrico,
>
> I cannot understand how you can arrive at scenario C. Theoretically, if
> there are not enough rules to implement a certain plan, you should get a
> CannotPlanException from RelOptPlanner#findBest. I would expect this
> outcome if e.g. you do not have "rules that handle the LIMIT". Moreover,
> having "wrong plans computed for joins" seems like a bug. But it is not
> easy to assess the situation and evaluate what is going on without the
> actual query and rules that are applied.
>

I will send a demo when I have cycles.
Thanks for your response.

Enrico


>
> Best regards,
> Ruben
>
>
> Le mer. 16 sept. 2020 à 17:24, Enrico Olivelli  a
> écrit :
>
> > Ruben,
> > thank you very much, using your snippet I got the planner working !
> >
> > In HerdDB we are currently using
> > Programs.ofRules(Programs.RULE_SET) +RelOptUtils#registerDefaultRules
> (that
> > add lots of rules) and we are able to handle several cases of very
> complex
> > generic SQL.
> >
> > But If I try to reduce the number of rules I see this behaviour:
> > a) if I am lucky I get some error from the planner that tells that it
> can't
> > find a way to compute the plan -> I can switch to the full planner
> ruleset
> > b) If I am less lucky I have a plan that works but it is less efficient
> ->
> > (this is expected, and this is what I want, that is to trade planner
> speed
> > vs efficiency of plans
> > c) If I am unlucky the planner returns a plan that leads to wrong
> results !
> >
> > For instance you can see c) if you do not add rules that handle "LIMIT"
> and
> > simply the plan sometimes won't pick the LIMIT and the database will
> return
> > unexpected results
> > but I saw worse cases, like wrong plans computed for joins.
> >
> > So my next question is:
> > is there a way to create a minimal set of rules that covers simple SQL
> > (think about basic JPA stuff) and eventually fails the plan in case
> of
> > unhandled cases ?
> >
> > Best regards
> > Enrico
> >
> > Il giorno mer 16 set 2020 alle ore 09:57 Ruben Q L 
> ha
> > scritto:
> >
> > > Hi Enrico,
> > >
> > > not sure if this can be applied to your case, but I also have a use
> case
> > > with HepPlanner and VolcanoPlanner.
> > > In my case I have an initial phase of rules applied with HepPlanner.
> > These
> > > are rules that optimize the logical plan and are always a "quickwin"
> > (they
> > > always generate a better version of the plan, so it makes sense to
> apply
> > > them with a Hep).
> > > Then, I have a "main" phase, applied with VolcanoPlanner, that takes
> care
> > > of converting the plan into the most optimized Enumerable result.
> > > A simplified code snippet would be:
> > >
> > > private static final Program PHASE_1 =
> > > Programs.of(HepProgram.builder().addRuleCollection(
> > >  Arrays.asList(
> > >   CoreRules.FILTER_INTO_JOIN,
> > >   CoreRules.SORT_PROJECT_TRANSPOSE,
> > >   CoreRules.SORT_JOIN_TRANSPOSE,
> > >   CoreRules.FILTER_REDUCE_EXPRESSIONS,
> > >   ...
> > >   )).build(), false, MyRelMetadataProvider.INSTANCE);
> > >
> > > private static final List PHASE_2_RULES = Arrays.asList(
> > >  EnumerableRules.ENUMERABLE_VALUES_RULE,
> > >  EnumerableRules.ENUMERABLE_UNION_RULE,
> > >  EnumerableRules.ENUMERABLE_FILTER_TO_CALC_RULE,
> > >  EnumerableRules.ENUMERABLE_PROJECT_TO_CALC_RULE,
> > >  EnumerableRules.ENUMERABLE_COLLECT_RULE,
> > >  EnumerableRules.ENUMERABLE_UNCOLLECT_RULE,
> > >  ...);
> > > private static final Program PHASE_2 =
> > > Programs.of(RuleSets.ofList(PHASE_2_RULES));
> > >
> > >
> > > EnumerableRel optimizeLogicalPlan(RelNode query)
> > > {
> > >  Program optPhases = Programs.sequence(PHASE_1, PHASE_2);
> > >  RelTraitSet desiredTraitSet = query.getTraitSet()
> > >   .replace(EnumerableConvention.INSTANCE)
> > >   .simplify();
> > >  EnumerableRel result = (EnumerableRel) optPhases.run(
> > >   query.getCluster().getPlanner(), // this is a VolcanoPlanner
> > >   query,
> > >   desiredTraitSet,
> > >   Collections.emptyList(),
> > >   Collections.emptyList());
> > >  return result;
> > > }
> > >
> > >
> > > I hope it helps.
> > >
> &g

Re: Combining HepPLanner and VolcanoPlanner

[Enrico Olivelli]

Ruben,
thank you very much, using your snippet I got the planner working !

In HerdDB we are currently using
Programs.ofRules(Programs.RULE_SET) +RelOptUtils#registerDefaultRules (that
add lots of rules) and we are able to handle several cases of very complex
generic SQL.

But If I try to reduce the number of rules I see this behaviour:
a) if I am lucky I get some error from the planner that tells that it can't
find a way to compute the plan -> I can switch to the full planner ruleset
b) If I am less lucky I have a plan that works but it is less efficient ->
(this is expected, and this is what I want, that is to trade planner speed
vs efficiency of plans
c) If I am unlucky the planner returns a plan that leads to wrong results !

For instance you can see c) if you do not add rules that handle "LIMIT" and
simply the plan sometimes won't pick the LIMIT and the database will return
unexpected results
but I saw worse cases, like wrong plans computed for joins.

So my next question is:
is there a way to create a minimal set of rules that covers simple SQL
(think about basic JPA stuff) and eventually fails the plan in case of
unhandled cases ?

Best regards
Enrico

Il giorno mer 16 set 2020 alle ore 09:57 Ruben Q L  ha
scritto:

> Hi Enrico,
>
> not sure if this can be applied to your case, but I also have a use case
> with HepPlanner and VolcanoPlanner.
> In my case I have an initial phase of rules applied with HepPlanner. These
> are rules that optimize the logical plan and are always a "quickwin" (they
> always generate a better version of the plan, so it makes sense to apply
> them with a Hep).
> Then, I have a "main" phase, applied with VolcanoPlanner, that takes care
> of converting the plan into the most optimized Enumerable result.
> A simplified code snippet would be:
>
> private static final Program PHASE_1 =
> Programs.of(HepProgram.builder().addRuleCollection(
>  Arrays.asList(
>   CoreRules.FILTER_INTO_JOIN,
>   CoreRules.SORT_PROJECT_TRANSPOSE,
>   CoreRules.SORT_JOIN_TRANSPOSE,
>   CoreRules.FILTER_REDUCE_EXPRESSIONS,
>   ...
>   )).build(), false, MyRelMetadataProvider.INSTANCE);
>
> private static final List PHASE_2_RULES = Arrays.asList(
>  EnumerableRules.ENUMERABLE_VALUES_RULE,
>  EnumerableRules.ENUMERABLE_UNION_RULE,
>  EnumerableRules.ENUMERABLE_FILTER_TO_CALC_RULE,
>  EnumerableRules.ENUMERABLE_PROJECT_TO_CALC_RULE,
>  EnumerableRules.ENUMERABLE_COLLECT_RULE,
>  EnumerableRules.ENUMERABLE_UNCOLLECT_RULE,
>  ...);
> private static final Program PHASE_2 =
> Programs.of(RuleSets.ofList(PHASE_2_RULES));
>
>
> EnumerableRel optimizeLogicalPlan(RelNode query)
> {
>  Program optPhases = Programs.sequence(PHASE_1, PHASE_2);
>  RelTraitSet desiredTraitSet = query.getTraitSet()
>   .replace(EnumerableConvention.INSTANCE)
>   .simplify();
>  EnumerableRel result = (EnumerableRel) optPhases.run(
>   query.getCluster().getPlanner(), // this is a VolcanoPlanner
>   query,
>   desiredTraitSet,
>   Collections.emptyList(),
>   Collections.emptyList());
>  return result;
> }
>
>
> I hope it helps.
>
> Best regards,
> Ruben
>
>
> Le mar. 15 sept. 2020 à 17:43, Enrico Olivelli  a
> écrit :
>
> > Hi,
> >
> > I am trying to create a two stage planner using HepPlanner and then
> > VolcanoPlanner
> >
> > Which is the correct sequence of steps to pass from SQL to Enumerable ?
> > My goal is to use Hep for very simple queries like simple INSERTs,
> SELECT *
> > FROM TABLE, SELECT * FROM TABLE WHERE pk=?
> > Volcano is overkill for such stuff.
> >
> > This is my idea:
> > 1) get a Planner (that's Volcano)
> > Planner planner = Frameworks.getPlanner(config);
> >
> > 2) Get the logical plan
> >  SqlNode n = planner.parse(query);
> > n = planner.validate(n);
> > RelNode logicalPlan = planner.rel(n).project();
> > 3) Create HepPlanner
> > HepProgram hepProgram =
> > HepProgram.
> > builder()
> > . ?? which Rules ?
> > .build();
> >
> > HepPlanner hepPlanner = new HepPlanner(hepProgram);
> > hepPlanner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> >
> > hepPlanner.setRoot(logicalPlan);
> >
> > 4) Run HepPlanner
> > RelNode bestForHep = hepPlanner.findBestExp();
> >
> > 5) Pick Volcano
> >
> > final RelOptPlanner optPlanner = cluster.getPlanner();
> >
> > 6) Convert to Enumerable
> >
> > final RelOptPlanner optPlanner = cluster.getPlanner();
> >
> > optPlanner.addRule(CoreRule

Combining HepPLanner and VolcanoPlanner

[Enrico Olivelli]

Hi,

I am trying to create a two stage planner using HepPlanner and then
VolcanoPlanner

Which is the correct sequence of steps to pass from SQL to Enumerable ?
My goal is to use Hep for very simple queries like simple INSERTs, SELECT *
FROM TABLE, SELECT * FROM TABLE WHERE pk=?
Volcano is overkill for such stuff.

This is my idea:
1) get a Planner (that's Volcano)
Planner planner = Frameworks.getPlanner(config);

2) Get the logical plan
 SqlNode n = planner.parse(query);
n = planner.validate(n);
RelNode logicalPlan = planner.rel(n).project();
3) Create HepPlanner
HepProgram hepProgram =
HepProgram.
builder()
. ?? which Rules ?
.build();

HepPlanner hepPlanner = new HepPlanner(hepProgram);
hepPlanner.addRelTraitDef(ConventionTraitDef.INSTANCE);

hepPlanner.setRoot(logicalPlan);

4) Run HepPlanner
RelNode bestForHep = hepPlanner.findBestExp();

5) Pick Volcano

final RelOptPlanner optPlanner = cluster.getPlanner();

6) Convert to Enumerable

final RelOptPlanner optPlanner = cluster.getPlanner();

optPlanner.addRule(CoreRules.FILTER_REDUCE_EXPRESSIONS);
RelTraitSet desiredTraits =
cluster.traitSet()
.replace(EnumerableConvention.INSTANCE);
final RelCollation collation =
logicalPlan instanceof Sort
? ((Sort) logicalPlan).collation
: null;
if (collation != null) {
desiredTraits = desiredTraits.replace(collation);
}
final RelNode newRoot = optPlanner.changeTraits(logicalPlan,
desiredTraits);
optPlanner.setRoot(newRoot);
RelNode bestExp = optPlanner.findBestExp();

Any hint/pointer is very appreciated

The alternative is to detect such simple queries and use a little set of
Rules and not Programs.ofRules(Programs.RULE_SET)

Best regards
Enrico

Re: [ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC

[Enrico Olivelli]

Congrats Ruben!

Enrico

Il Mer 12 Ago 2020, 18:05 Michael Mior  ha scritto:

> Congrats Reuben!
>
> --
> Michael Mior
> mm...@apache.org
>
> Le mer. 12 août 2020 à 09:32, Ruben Q L  a écrit :
> >
> > Thanks everyone!
> > It is an honor to join the Calcite PMC.
> >
> > Best regards,
> > Ruben
> >
> >
> > Le mer. 12 août 2020 à 03:03, Forward Xu  a
> écrit :
> >
> > > Congrats, Ruben!
> > >
> > >
> > > Best
> > >
> > > Forward
> > >
> > > XING JIN  于2020年8月12日周三 上午8:58写道：
> > >
> > > > Congrats, Ruben!
> > > >
> > > > 953396112 <953396...@qq.com> 于2020年8月12日周三 上午7:47写道：
> > > >
> > > > > Congratulations,Ruben!
> > > > >
> > > > >
> > > > > xzh
> > > > > --原始邮件--
> > > > > 发件人:
> > > > >   "dev"
> > > > > <
> > > > > zabe...@gmail.com;
> > > > > 发送时间:2020年8月12日(星期三) 凌晨5:53
> > > > > 收件人:"dev" > > > >
> > > > > 主题:[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
> > > > >
> > > > >
> > > > >
> > > > > I'm pleased to announce that Ruben has accepted an invitation to
> > > > > join the Calcite PMC. Ruben has been a consistent and helpful
> > > > > figure in the Calcite community for which we are very grateful. We
> > > > > look forward to the continued contributions and support.
> > > > >
> > > > > Please join me in congratulating Ruben!
> > > > >
> > > > > - Stamatis (on behalf of the Calcite PMC)
> > > >
> > >
>

Re: [VOTE] Release apache-calcite-1.25.0 (release candidate 0)

[Enrico Olivelli]

+1 (non binding)
run tests locally on Fedora + JDK14
run tests of HerdDB just by switching from 1.24 without any change

Enrico

Il giorno dom 9 ago 2020 alle ore 05:22 Andrei Sereda  ha
scritto:

> Hi all,
>
> I have created a build for Apache Calcite 1.25.0, release
> candidate 0.
>
> Thanks to everyone who has contributed to this release.
>
> You can read the release notes here:
> https://apache.github.io/calcite-site-preview/docs/history.html
>
> The commit to be voted upon:
>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555
>
> Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555
>
> Tag:
> https://github.com/apache/calcite/tree/calcite-1.25.0-rc0
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0
> (revision 40922)
>
> RAT report:
> https://apache.github.io/calcite-site-preview/rat/rat-report.txt
>
> Site preview is here:
> https://apache.github.io/calcite-site-preview/
>
> JavaDoc API preview is here:
> https://apache.github.io/calcite-site-preview/api
>
> The hashes of the artifacts are as follows:
>
> a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87
> *apache-calcite-1.25.0-src.tar.gz
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/sereda.asc
> https://www.apache.org/dist/calcite/KEYS
>
> N.B.
> To create the jars and test Apache Calcite: "./gradlew build".
>
> If you do not have a Java environment available, you can run the tests
> using docker. To do so, install docker and docker-compose, then run
> "docker-compose run test" from the root of the directory.
>
> Please vote on releasing this package as Apache Calcite 1.25.0.
>
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Calcite 1.25.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (non binding)
>

Re: Avatica and HepPlanner

[Enrico Olivelli]

Is there some documentation or best practice doc about how using both of
the two planners ?

I guess you have to pass a set of rules to the HepPlanner in order to
perform some simpler steps
and then pass other rules to VolcanoPlanner in order to create the physical
plan.
What is the RelNode to pass from the HepPlanner to the VolcanoPlanner ?

// parse
SqlNode n = planner.parse(query);
// validate
n = planner.validate(n);
// create "logical plan"
RelNode logicalPlan = planner.rel(n).project();

// get Volcano from "logicalPlan"
RelOptCluster cluster = logicalPlan.getCluster();
elOptPlanner optPlanner = cluster.getPlanner();

// perform physical planning
 RelNode bestExp = optPlanner.findBestExp();


Regards
Enrico

Il giorno mar 4 ago 2020 alle ore 08:05 Meron Avigdor <
meron.avig...@gigaspaces.com> ha scritto:

> Thanks Rui and Stamatis for the clarification. I was probably misled by the
> documentation,
> as it stated that there are 2 implementations for planner rules. Thus I
> figured that this is configurable in some way.
> Also, it was mentioned that the Heuristic approach carried by the
> HepPlanner has less overhead.
>
> If you say the two are complementary to each other, I will approach it
> differently.
> Thank you for the assistance.
>
>
> On Tue, Aug 4, 2020 at 12:50 AM Stamatis Zampetakis 
> wrote:
>
> > Hi Meron,
> >
> > The two planners are not equivalent so the comparison is not totally
> fair.
> > Even if you apply the same ruleset it does not mean that you are going
> > to obtain the same plan.
> >
> > The two planners are complementary to each other and they are used in
> > different stages of the optimization process thus it is not possible to
> use
> > exclusively one or the other. For the same reason I think it will be
> tricky
> > to
> > add such configuration.
> >
> > If you notice a performance problem with the existing planners or have
> > ideas to improve them feel free to file a JIRA.
> >
> > Best,
> > Stamatis
> >
> > On Sun, Aug 2, 2020 at 10:38 PM Rui Wang  wrote:
> >
> > > I tried to dig into the Calcite codebase and found there is likely no
> way
> > > to enable JDBC codepath:
> > > 1. There is no planner config in CalciteConnectionProperty [1].
> > > 2.  In CalcitePrepareImpl, there is only one createPlannerFactory with
> > one
> > > createPlanner, in which VolcanoPlanner is used [2].
> > >
> > >
> > > If my understanding is correct, then there could be an improvement that
> > > adds a config to CalciteConnectionProperty, and use that config to
> > control
> > > which planner to use by JDBC approach.
> > >
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/config/CalciteConnectionProperty.java#L37
> > > [2]:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/2088488ac8327b19512a76a122cae2961fc551c3/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L427
> > >
> > >
> > > -Rui
> > >
> > > On Sun, Aug 2, 2020 at 8:55 AM Meron Avigdor <
> > meron.avig...@gigaspaces.com
> > > >
> > > wrote:
> > >
> > > > Hi.
> > > > I have implemented an extension of org.apache.calcite.jdbc.Driver,
> and
> > I
> > > > have the model handler extend org.apache.calcite.avatica.HandlerImpl
> > > >
> > > > For performance optimizations, I would like to test VolcanoPlanner
> vs.
> > > > HepPlanner
> > > > as the documentation says that the latter is faster.
> > > >
> > > > How is this to be configured? I could not find a hook point to return
> > the
> > > > planner to be used.
> > > >
> > > > Thank you.
> > > > Meron
> > > >
> > >
> >
>

Re: [VOTE] Release apache-calcite-1.24.0 (release candidate 0)

[Enrico Olivelli]

+1 (non binding)
- verified hashes and checksums
- built from sources and run tests (JDK14 on Linux)
- run tests of HerdDB and some client application

I only had to fix a deprecation warning, changing from
ReduceExpressionsRule.FILTER_INSTANCE to
CoreRules.FILTER_REDUCE_EXPRESSIONS, see [1] below
without the change of CoreRules.FILTER_REDUCE_EXPRESSIONS all of the tests
of HerdDB failed with a NPE,
I debugged the issue with a debugger and
the ReduceExpressionsRule.FILTER_INSTANCE at runtime is null, I can't
understand why.

Not a big deal, changing to CoreRules.FILTER_REDUCE_EXPRESSIONS fixes the
issue

java.lang.NullPointerException
at
org.apache.calcite.plan.AbstractRelOptPlanner.addRule(AbstractRelOptPlanner.java:147)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.addRule(VolcanoPlanner.java:416)
at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:576)
at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:331)
at herddb.core.TestUtils.scan(TestUtils.java:70)

[1]
https://github.com/diennea/herddb/pull/665/files#diff-ca87d7835fc281efa58a8809669017a9R576


Enrico

Il giorno mar 21 lug 2020 alle ore 06:12 Francis Chuang <
francischu...@apache.org> ha scritto:

> Thanks for making this release available for voting, Chunwei!
>
> Verified GPG Signature - OK
> Verified SHA512 - OK
> Ran tests per HOWTO (./gradlew check) - OK
> Quickly skimmed release notes - Looks good, but I agree with Julian's
> comments.
> Spotted checked a few JARs in the Maven repository - OK
>
> Environment (OpenJDK:latest docker container):
> Gradle 6.3 (via gradlew)
> Oracle Linux Server 7.8
> openjdk version "14.0.2" 2020-07-14
> OpenJDK Runtime Environment (build 14.0.2+12-46)
> OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)
>
> My vote is: +1 (binding)
>
> Francis
>
> On 21/07/2020 12:07 pm, Haisheng Yuan wrote:
> > Environment:
> > Mac OS X 10.15.1, JDK 1.8.0_162
> >
> > - Checked signatures and checksums, OK
> > - Ran unit tests (./gradlew build), OK
> >
> > +1 (binding)
> >
> >> * why is 4032 'breaking'?
> > With that change, the CalcMergeRule won't match PhysicalNode(including
> EnumerableCalc) in VolcanoPlanner. Perhaps I should elaborate in the
> release notes.
> >
> >> * why is 3786 breaking? (recomputeDigest was not present in 1.23; the
> >> remarks about caching digests are useful, so why aren't they in the
> >> javadoc?)
> > recomputeDigest() has been there since b0dab68 (2012-05-07). I will add
> the remarks into the javadoc after release.
> >
> > Thanks,
> > Haisheng
> >
> > On 2020/07/21 01:14:17, Julian Hyde  wrote:
> >> Downloaded, checked hashes, built and ran tests on Ubuntu/JDK 14;
> >> checked distro against git (see issue 1); reviewed release notes (see
> >> issue 2).
> >>
> >> +1 (binding) but issues 1 and 2 need to be fixed right after the
> release.
> >>
> >> Issue 1. License file is not the same as in source control:
> >>
> >> diff -r ./LICENSE /tmp/apache-calcite-1.24.0-src/LICENSE
> >> 177a178,189
> >>>
> >>> Additional License files can be found in the 'licenses' folder located
> in the same directory as the LICENSE file (i.e. this file)
> >>>
> >>> - Software produced outside the ASF which is available under other
> licenses (not Apache-2.0)
> >>>
> >>> MIT
> >>> * cobyism:html5shiv:3.7.2
> >>> * font-awesome:font-awesome-code:4.2.0
> >>> * gridsim:gridsim:
> >>> * jekyll:jekyll:
> >>> * normalize:normalize:3.0.2
> >>> * respond:respond:1.4.2
> >>
> >> Can you fix the release instructions that the generated LICENSE needs
> >> to be committed (probably at the same time you revise the release
> >> notes).
> >>
> >> Issue 2. Release notes
> >>
> >> For the 'highlights', I prefer a paragraph with hyperlinks over a list
> >> (see
> https://github.com/apache/calcite/blob/calcite-1.24.0-rc0/site/_docs/history.md#1180--2018-12-21
> ).
> >>
> >> Regarding categorization:
> >> * why is 4032 'breaking'?
> >> * why is 3786 breaking? (recomputeDigest was not present in 1.23; the
> >> remarks about caching digests are useful, so why aren't they in the
> >> javadoc?)
> >> * we need a note that a bunch of methods are deprecated in this
> >> release and will be removed before 1.25 (see 3923, 4023 and 4079).
> >> This will break semantic versioning in 1.25, so is a big deal.
> >> * 4073, 3224, 4056, 4008, 3972, 4060 are listed as new features, but I
> >> think they are bug fixes or improved implementations
> >> * 3946, 4089, 4087 are listed as fixes but could be listed as new
> features
> >> * 4075 should be under 'test suite'
> >> * 4094 description does not need 'follow-up after review comments'
> >> * 4086 is an upgrade, so should be in 'bug fixes', not documentation
> >> * A few places SQL and Java keywords are not in code font (e.g. NPE,
> >> IllegalArgumentException, RexNode, Expression, HAVING, ARRAY, MAP,
> >> CAST)
> >>
> >> Julian
> >>
> >> On Mon, Jul 20, 2020 at 12:01 PM Michael Mior  wrote:
> >>>
> >>> +1
> >>>
> >>> Checked hash and signature and compiled and ran tests. 

Re: [DISCUSS] New RexNode: RexListCmp

[Enrico Olivelli]

Il Lun 20 Lug 2020, 03:00 Haisheng Yuan  ha scritto:

> Hi all,
>
> This is a rough idea, I'd like to see how the community think about it.
>
> RexListCmp extends RexNode / RexCall {
>   public final SqlOperator op;
>   public final RexNode left;
>   public final ImmutableList list;
>   public final RexQuantifier quantifier;
>   public final RelDataType type;
> }
>
> Enum RexQuantifier {
>   ALL,
>   ANY
> }
>
> Background:
>
> It is not uncommon that the query contains large number of constant IN
> list, e.g.
> 1) SELECT * FROM foo WHERE a NOT IN (1, 2, 3, , 1);
> 2) SELECT * FROM bar WHERE b IN (1, 2, 3, , 1);
>
> Currently, Calcite either translates it into a Join, or expand to OR/AND,
> which is inefficient, and may cause problems.
>

Yes. It is not efficient.

I would love to see this new feature

Thanks
Enrico




> With RexListCmp, the predicate in query 1) will be represented as:
> RexListCmp {
>   op = "<>",
>   left = "a"
>   list = "1,2,3...1"
>   quantifier = "ALL"
> }
>
> The predicate in query 2) will be represented as:
> RexListCmp {
>   op = "=",
>   left = "b"
>   list = "1,2,3...1"
>   quantifier = "ANY"
> }
>
> It may also be used to represent the predicate in the following query:
>
> SELECT * FROM bar WHERE (a,b) IN / NOT IN ((1,1), (2,2), (3,3), ... (1000,
> 1000));
>
> Further more, it is extensible. The op is not limited to be equals or not
> equals, it also be >, <, >=, <=, IDF, INDF or even customized sql operator
> like geospatial operator intersect:
> boolean &&( geometry A , geometry B )
>
> Thoughts?
>
> Thanks,
> Haisheng Yuan
>
>
>

Re: [DISCUSS] Binary files for testing InnoDB adapter

[Enrico Olivelli]

Il Gio 16 Lug 2020, 21:28 Julian Hyde  ha scritto:

> Michael,
>
> I think that sentence covers the situation where, say, we ship a
> .class file without shipping the .java it was generated from.
>
> In our case, we are shipping the equivalent of the .class file and the
> .java file (namely EMP.ibd and the scott.sql that MySQL generated it
> from). We have to ship the .class file equivalent (.ibd) because we
> don't require that people running the tests have the javac equivalent
> (MySQL).
>
> The concern about auditability remains.
>

You can wrap the files in a text based format with a readable license
header and the test case can unpack the files and use them. It is tricky
but it can work


Enrico


> Julian
>
> On Thu, Jul 16, 2020 at 8:34 AM Michael Mior  wrote:
> >
> > I don't personally have a problem with this, but it seems as though it
> > might violate the release policy. Specifically the statement "It is
> > also necessary for the PMC to ensure that the source package is
> > sufficient to build any binary artifacts associated with the release."
> >
> > https://www.apache.org/legal/release-policy.html#what
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le mer. 15 juil. 2020 à 20:02, Julian Hyde  a écrit :
> > >
> > > TL;DR: PMC members, would you vote for a release 1.24 if it includes
> > > binary files necessary for testing?
> > >
> > > I would like to include the InnoDB adapter [1] in release 1.24. It is
> > > well written, well documented, and it is ready.
> > >
> > > There is one problem: there are some binary files (in InnoDB format)
> > > [2] that are included for testing. As a general rule, Apache does not
> > > release binary files as part of the source release because they are
> > > difficult to audit for provenance.
> > >
> > > I think we should make an exception, for just release 1.24, because
> > > the files are small (just EMP and DEPT tables) and generated by hand.
> > >
> > > I have asked the author to do a follow-up task to remove the files
> > > before next release.
> > >
> > > PMC members, please reply to this email and indicate whether this
> > > would cause you to vote -1 on the upcoming release.
> > >
> > > Julian
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> > >
> > > [2]
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
>

Re: [DISCUSS] Binary files for testing InnoDB adapter

[Enrico Olivelli]

IMHO If you need those files for tests and you don't have a way to generate
them it is allowed to keep them.
You can add some readme file that explain their nature. You can also add a
check sum file.


Just my 2 cents
Enrico

Il Gio 16 Lug 2020, 05:40 Francis Chuang  ha
scritto:

> I am +1 for including the files in this release as long as they are
> removed in the next release.
>
> Francis
>
> On 16/07/2020 1:08 pm, Haisheng Yuan wrote:
> > +1
> >
> > I am fine to make an exception for this release.
> > Let's see what's author's plan to remove the binary files in next
> release.
> >
> > On 2020/07/16 00:02:26, Julian Hyde  wrote:
> >> TL;DR: PMC members, would you vote for a release 1.24 if it includes
> >> binary files necessary for testing?
> >>
> >> I would like to include the InnoDB adapter [1] in release 1.24. It is
> >> well written, well documented, and it is ready.
> >>
> >> There is one problem: there are some binary files (in InnoDB format)
> >> [2] that are included for testing. As a general rule, Apache does not
> >> release binary files as part of the source release because they are
> >> difficult to audit for provenance.
> >>
> >> I think we should make an exception, for just release 1.24, because
> >> the files are small (just EMP and DEPT tables) and generated by hand.
> >>
> >> I have asked the author to do a follow-up task to remove the files
> >> before next release.
> >>
> >> PMC members, please reply to this email and indicate whether this
> >> would cause you to vote -1 on the upcoming release.
> >>
> >> Julian
> >>
> >> [1] https://issues.apache.org/jira/browse/CALCITE-4034
> >>
> >> [2]
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
> >>
>

Re: [DISCUSS] Make RexNode serializable

[Enrico Olivelli]

Rui

Il Mar 7 Lug 2020, 20:30 Rui Wang  ha scritto:

> Hi Community,
>
> In Apache Beam we are facing a use case where we need to keep RexNode in
> our distributed primitives. Because of the nature of distributed computing,
> Beam requires the usage of those primitives be serializable (thus those
> primitives can be sent over the network to backend/workers for
> further execution).
>
> In the Java world this requirement means to make RexNode implement the Java
> Serializable interface.
>
> A workaround right now is to create a bunch of classes to "clone" RexNode
> while making those classes implement the Serializable interface.
>

Did you evaluate to use some framework like Kryo that allows you to
serialize Jon serializable classes?

I think that in general Java serialisation is not efficient as it is too
general purpose.
It also brings in a few Security issues.

Maybe an alternative idea is to add some serialisation ad-hoc mechanism in
RexNode.
We should also ensure that every RexNode will be able to be serialized and
deserialized.

Enrico


> So what do you think of the idea that makes RexNode implement the
> Serializable interface?
>
>
> -Rui
>

Re: Deploy Calcite to local Maven Repository

[Enrico Olivelli]

Il giorno ven 3 lug 2020 alle ore 16:26 953396112 <953396...@qq.com> ha
scritto:

> You can use the following command to publish jars to your local Maven
> repository. I hope it can help you.
> ./gradlew :core:publishToMavenLocal
>


It works !
Thank you very much

Enrico


>
>
>
> Best regards
> xzh
>
>
> ------原始邮件--
> 发件人:"Enrico Olivelli" 发送时间:2020年7月3日(星期五) 晚上10:15
> 收件人:"dev"
> 主题:Deploy Calcite to local Maven Repository
>
>
>
> Hi,
> I am trying to build Calcite and put jars into local Maven repository
>
> Usually this is easy with the Maven plugin
> https://docs.gradle.org/current/userguide/maven_plugin.html
>
> ./gradlew install -x test
>
> I am trying to add the 'maven' plugin to Gradle configuration but without
> effect
>
> Is there any way to perform this task ?
> I would like to test current HerdDB master against current Calcite master,
> as probably Calcite 1.24 is going to be released soon
>
>
> Best regards
> Enrico

Deploy Calcite to local Maven Repository

[Enrico Olivelli]

Hi,
I am trying to build Calcite and put jars into local Maven repository

Usually this is easy with the Maven plugin
https://docs.gradle.org/current/userguide/maven_plugin.html

./gradlew install -x test

I am trying to add the 'maven' plugin to Gradle configuration but without
effect

Is there any way to perform this task ?
I would like to test current HerdDB master against current Calcite master,
as probably Calcite 1.24 is going to be released soon


Best regards
Enrico

Re: Why migrate from Maven to Gradle

[Enrico Olivelli]

Il Mar 16 Giu 2020, 05:31 Haisheng Yuan  ha scritto:

> Hi Zoltan,
>
> > For example I use NetBeans, and things just work with maven.
>

Hi
If you use NetBeans 11.3+ you can open Calcite again. Better 12.0

Enrico


Have you tried intellij? It works fine with Gradle. And it is free.
>
> > Gradle will add friction to contributors familiar with maven and
> unfamiliar with gradle...
> Your case is far better than mine, at least you have knowledge of maven. I
> have near 0 knowledge for both maven and gradle. But that doesn't prevent
> me from making contribution to Calcite. The only command I run daily is
> ./gradlew build.  And I am happy with it. :) So don't worry about that.
> Feel free to ask if you need help, I think Vladimir and other community
> members are more than happy to help.
>
> Thanks,
> Haisheng
>
> On 2020/06/15 23:27:21, Zoltan Farkas 
> wrote:
> > Vladimir,
> >
> > A new user will probably be impacted more by the IDE experience.
> >
> > Some IDEs do better at handling maven than gradle... For example I use
> NetBeans, and things just work with maven.
> >
> > In my experience, Maven is simply a more mature tool (use it since
> 2002), more plugins, good IDE support, with more developers being fluent
> with it.
> >
> > Gradle will add friction to contributors familiar with maven and
> unfamiliar with gradle...
> >
> > Hope it’s worth it...
> >
> > --z
> >
> > > On Jun 15, 2020, at 2:49 PM, Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
> > >
> > > 
> > >>
> > >> Gradle is not that user friendly for new uses
> > >
> > > Can you please elaborate?
> > >
> > > Gradle's command line is easier to follow as it provides help with the
> > > usable tasks and descriptions,
> > > and it requires much less ceremony.
> > > For instance, with Maven I often had issues like
> "ExtenderSqlParserImpl not
> > > found" (== I had to build the project before importing to the IDE),
> > > and Gradle fixes that.
> > >
> > > Vladimir
> >
> >
>

Re: [ANNOUNCE] Apache Calcite 1.23.0 released

[Enrico Olivelli]

Great


Enrico

Il Gio 28 Mag 2020, 05:12 Chunwei Lei  ha scritto:

> Nice job, Haisheng!
>
>
> Best,
> Chunwei
>
>
> On Thu, May 28, 2020 at 10:42 AM XING JIN  wrote:
>
> > Thanks a lot for driving this, Haisheng!
> >
> > Best,
> > Jin
> >
> > Stamatis Zampetakis  于2020年5月28日周四 上午7:14写道：
> >
> > > Many thanks again Haisheng, the release has your signature on it and it
> > is
> > > in many places :)
> > >
> > > On Wed, May 27, 2020 at 9:34 PM Haisheng Yuan 
> wrote:
> > >
> > > > The Apache Calcite team is pleased to announce the release of Apache
> > > > Calcite 1.23.0.
> > > >
> > > > Calcite is a dynamic data management framework. Its cost-based
> > > > optimizer converts queries, represented in relational algebra, into
> > > > executable plans. Calcite supports many front-end languages and
> > > > back-end data engines, and includes an SQL parser and, as a
> > > > sub-project, the Avatica JDBC driver.
> > > >
> > > > This release comes two months after 1.22.0. It includes more than
> > > > 100 resolved issues, comprising of a few new features as well as
> > > > general improvements and bug-fixes. It includes support for top
> > > > down trait request and trait enforcement without abstract converter,
> > > > ClickHouse dialect, SESSION and HOP Table functions, and many
> > > > more bug fixes and improvements.
> > > >
> > > > You can start using it in Maven by simply updating your dependency
> to:
> > > >
> > > >   
> > > > org.apache.calcite
> > > > calcite-core
> > > > 1.23.0
> > > >   
> > > >
> > > > If you'd like to download the source release, you can find it here:
> > > >
> > > >   https://calcite.apache.org/downloads/
> > > >
> > > > You can read more about the release (including release notes) here:
> > > >
> > > >   https://calcite.apache.org/news/2020/05/23/release-1.23.0/
> > > >
> > > > We welcome your help and feedback. For more information on how to
> > > > report problems, and to get involved, visit the project website at:
> > > >
> > > >   https://calcite.apache.org/
> > > >
> > > > Thanks to everyone involved!
> > > >
> > >
> >
>

Re: [VOTE] Release apache-calcite-1.23.0 (release candidate 1)

[Enrico Olivelli]

Vladimir o Haisheng
could you please open a ticket to INFRA ?
I can do it myself, with your permission (I am not in Calcite
PMC/committers group)

Otherwise it is not possible to use Calcite 1.23.0 from downstream projects

Enrico

Il giorno mar 26 mag 2020 alle ore 10:34 Vladimir Sitnikov <
sitnikov.vladi...@gmail.com> ha scritto:

> ASF releases are shared with repository.apache.org:
>
> https://repository.apache.org/service/local/repositories/releases/content/org/apache/calcite/calcite-core/1.23.0/calcite-core-1.23.0.pom
>
> It might be there are issues with ASF Nexus and Sonatype Nexus sync.
>
> Vladimir
>

Re: [VOTE] Release apache-calcite-1.23.0 (release candidate 1)

[Enrico Olivelli]

Haisheng
I still can't see Calcite 1.23.0 on Maven Central
https://search.maven.org/search?q=a:calcite-core

did the release procedure complete successfully?

Enrico



Il giorno dom 24 mag 2020 alle ore 14:35 JiaTao Tao  ha
scritto:

> Vote: +1 (non-binding)
>
> CALCITE-3819  really
> helps infinite planning in our use case. And I'm looking forward
> to CALCITE-3997.
>
> ---
> Environment: Mac-OS
> JDK version: 1.8.0_212
> Build with tests: OK
>
>
> Regards!
>
> Aron Tao
>
>
> Vladimir Sitnikov  于2020年5月24日周日 上午4:35写道：
>
> > >Looks like it is expected:
> > >https://github.com/gradle/gradle/issues/9758
> >
> > It is not expected.
> >
> > The way to fix it is to apply the following patch:
> >
> > --- a/buildSrc/build.gradle.kts
> > +++ b/buildSrc/build.gradle.kts
> > @@ -19,7 +19,6 @@ import com.github.vlsi.gradle.properties.dsl.props
> >  import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
> >
> >  plugins {
> > -java
> >  `kotlin-dsl` apply false
> >  id("com.github.autostyle")
> >  id("com.github.vlsi.gradle-extensions")
> > @@ -75,6 +74,6 @@ fun Project.applyKotlinProjectConventions() {
> >
> >  dependencies {
> >  subprojects.forEach {
> > -runtimeOnly(project(it.path))
> > +"runtimeOnly"(project(it.path))
> >  }
> >  }
> >
> > Vladimir
> >
>

Re: [VOTE] Release apache-calcite-1.23.0 (release candidate 1)

[Enrico Olivelli]

+1 (non binding)
Local tests passes on Linux (Fedora 31) + JDK14
Checksums and signatures looks good.

All tests are passing on HerdDB.
Thank you very much Haisheng !

Enrico

Il giorno sab 16 mag 2020 alle ore 23:21 Anton Haidai <
anton.hai...@gmail.com> ha scritto:

> Hello,
>
> Local Calcite build with tests enabled on Linux: *OK*
> Calcite-based system (Zoomdata) test suite: *OK*
>
> Vote:
> +1 (non-binding)
>
> On Sat, May 16, 2020 at 7:02 AM Haisheng Yuan  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.23.0, release
> > candidate 1.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> >
> https://github.com/apache/calcite/blob/calcite-1.23.0-rc1/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=b708fdc46d4c5fd4c5a6c7a398823318a7b4dce3
> >
> > Its hash is b708fdc46d4c5fd4c5a6c7a398823318a7b4dce3
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.23.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.23.0-rc1
> > (revision 39622)
> >
> > The hashes of the artifacts are as follows:
> >
> >
> 961c4f13199e199c669a6168ba655a9492bdd80d644da375a684b732c0b628b8a2ffacea5da97c82e8702a8e3bf7a1f58784baa49509fb3c48ef593259e11f46
> > *apache-calcite-1.23.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1089/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://dist.apache.org/repos/dist/release/calcite/KEYS
> >
> > N.B.
> > To create the jars and test Apache Calcite: "./gradlew build".
> >
> > If you do not have a Java environment available, you can run the tests
> > using docker. To do so, install docker and docker-compose, then run
> > "docker-compose run test" from the root of the directory.
> >
> > Please vote on releasing this package as Apache Calcite 1.23.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.23.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> > Thanks,
> > Haisheng Yuan
> >
>
>
> --
> Best regards,
> Anton.
>

Re: [DISCUSS] DDL parsing, and relationship between "server" and "babel" modules

[Enrico Olivelli]

Il Ven 15 Mag 2020, 22:26 Julian Hyde  ha scritto:

> Currently, there is no DDL in Calcite's "core" module (or its SQL
> parser) and the SQL parser in the "server" module adds DDL extensions
> for Calcite's object types.
>
> There is a PR [1][2] to add support for "CREATE TABLE" to Babel, and
> it makes "babel" extend the "server" module. In particular, it uses
> classes SqlDdlNodes and SqlCreateTable from "server".
>
> I am uncomfortable with that approach, because "server" does other
> things besides parse DDL (it creates a stateful server and handles
> RPC). Also, the needs of other DBMSs' DDL might make Calcite's DDL
> classes more complicated. So, it all seems to be unnecessary coupling.
>
> The alternative is to copy-paste the DDL classes (currently
> SqlDdlNodes, SqlCreateTable and SqlColumnDeclaration) from "babel"
> into "server".
>
> Another alternative would be to move all DDL classes into "core".
>
> None of the alternatives are great, but I prefer the copy-paste
> approach. What do y'all think?
>

It would be interesting for HerdDB community to have DDL in core Calcite


Enrico


> Julian
>
> [1] https://github.com/apache/calcite/pull/1938
> [2] https://issues.apache.org/jira/browse/CALCITE-3946
>

Re: Problems on HerdDB with 1.23 was [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

[Enrico Olivelli]

All of the two tickets have been fixed on current master!
The former was a regression
The latter was an improvement in Calcite that needed only a fix in a test
in HerdDB suite
check the JIRA for more details

We are re running all of the tests locally, of HerdDB and of some known
downstream applications

Thank you !
Enrico

Il giorno mer 13 mag 2020 alle ore 15:05 Enrico Olivelli <
eolive...@gmail.com> ha scritto:

> Tickets:
> https://issues.apache.org/jira/browse/CALCITE-3997
> https://issues.apache.org/jira/browse/CALCITE-3998
>
> I will try to create the reproducer, but maybe you will be smarter than me
> :-)
>
>
> Enrico
>
> Il giorno mer 13 mag 2020 alle ore 14:44 Haisheng Yuan 
> ha scritto:
>
>> > Yesterday I was trying to create a test case in Calcite codebase.
>> > But I did not find where to put it.
>> > Can you please give me an hint?
>> Maybe JdbcTest.java, take testMergeJoin() as an example.
>>
>> > Otherwise I will try to create a minimal Java block of code that
>> reproduces
>> > the problem. I did that way last time and Stamatis was able to create
>> the
>> > test in Calcite code
>> >
>> > Does this approach work for you?
>> That would also work.
>>
>> Thanks,
>> Haisheng
>> On 2020/05/13 12:31:26, Enrico Olivelli  wrote:
>> > Il Mer 13 Mag 2020, 13:45 Haisheng Yuan  ha scritto:
>> >
>> > > Hi Enrico,
>> > >
>> > > > Is it possibile to disable it? I will check. Any suggestion is
>> welcome
>> > > Disabling it won't help. It is a Calcite bug. There is nothing wrong
>> in
>> > > HerdDB. Can you help us log a JIRA and provide a reproducible test
>> case?
>> > >
>> >
>> > Sorry for the delay.
>> > I has another problem today. I will do as soon as possible.
>> >
>> > Yesterday I was trying to create a test case in Calcite codebase.
>> > But I did not find where to put it.
>> > Can you please give me an hint?
>> > Otherwise I will try to create a minimal Java block of code that
>> reproduces
>> > the problem. I did that way last time and Stamatis was able to create
>> the
>> > test in Calcite code
>> >
>> > Does this approach work for you?
>> >
>> > Enrico
>> >
>> >
>> > > > Do you think that I can safely disable those rules?
>> > > You have to create your own rule instances. But let Calcite do it for
>> you.
>> > >
>> > > Thanks,
>> > > Haisheng Yuan
>> > >
>> > > On 2020/05/13 08:15:30, Enrico Olivelli  wrote:
>> > > > Haisheng,
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > Il Mar 12 Mag 2020, 16:38 Haisheng Yuan  ha
>> scritto:
>> > > >
>> > > > > Hi Enrico,
>> > > > >
>> > > > > Thanks for reporting issues so quick for calcite-1.23.0-rc0.
>> > > Appreciate it.
>> > > > > Can you log JIRA for these issues? We will fix them.
>> > > > >
>> > > > Doing it non
>> > > >
>> > > >
>> > > > > Regarding with the first issue, I guess several factors are
>> > > contributing
>> > > > > to the issue.
>> > > > > 1. Trait enforcement is enabled for EnumerableConvention by
>> default in
>> > > > > 1.23.0, now it can generate mergejoins. We can disable it again if
>> > > people
>> > > > > would like.
>> > > > >
>> > > >
>> > > > Is it possibile to disable it? I will check. Any suggestion is
>> welcome
>> > > >
>> > > >
>> > > > > 2. RelBuilder hasn't been able to handle physical operator's
>> trait well
>> > > > > yet, especially for Project.
>> > > > >
>> > > > > 3. Logical operator has been doing some work that it is not
>> expected to
>> > > > > do, but physical operator should do. Here when creating
>> > > LogicalProject, it
>> > > > > is trying to deduce its collation from input MergeJoin. Project
>> is a
>> > > > > frequently created operator, but profiler shows that
>> > > > > RelTraitSet.replaceIfs() take 65% in the total runtime of
>> > > > > LogicalProject.create(). That is not only inappropriate
>> operation, but
>> > &

Re: Problems on HerdDB with 1.23 was [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

[Enrico Olivelli]

Tickets:
https://issues.apache.org/jira/browse/CALCITE-3997
https://issues.apache.org/jira/browse/CALCITE-3998

I will try to create the reproducer, but maybe you will be smarter than me
:-)


Enrico

Il giorno mer 13 mag 2020 alle ore 14:44 Haisheng Yuan 
ha scritto:

> > Yesterday I was trying to create a test case in Calcite codebase.
> > But I did not find where to put it.
> > Can you please give me an hint?
> Maybe JdbcTest.java, take testMergeJoin() as an example.
>
> > Otherwise I will try to create a minimal Java block of code that
> reproduces
> > the problem. I did that way last time and Stamatis was able to create the
> > test in Calcite code
> >
> > Does this approach work for you?
> That would also work.
>
> Thanks,
> Haisheng
> On 2020/05/13 12:31:26, Enrico Olivelli  wrote:
> > Il Mer 13 Mag 2020, 13:45 Haisheng Yuan  ha scritto:
> >
> > > Hi Enrico,
> > >
> > > > Is it possibile to disable it? I will check. Any suggestion is
> welcome
> > > Disabling it won't help. It is a Calcite bug. There is nothing wrong in
> > > HerdDB. Can you help us log a JIRA and provide a reproducible test
> case?
> > >
> >
> > Sorry for the delay.
> > I has another problem today. I will do as soon as possible.
> >
> > Yesterday I was trying to create a test case in Calcite codebase.
> > But I did not find where to put it.
> > Can you please give me an hint?
> > Otherwise I will try to create a minimal Java block of code that
> reproduces
> > the problem. I did that way last time and Stamatis was able to create the
> > test in Calcite code
> >
> > Does this approach work for you?
> >
> > Enrico
> >
> >
> > > > Do you think that I can safely disable those rules?
> > > You have to create your own rule instances. But let Calcite do it for
> you.
> > >
> > > Thanks,
> > > Haisheng Yuan
> > >
> > > On 2020/05/13 08:15:30, Enrico Olivelli  wrote:
> > > > Haisheng,
> > > >
> > > >
> > > >
> > > >
> > > > Il Mar 12 Mag 2020, 16:38 Haisheng Yuan  ha
> scritto:
> > > >
> > > > > Hi Enrico,
> > > > >
> > > > > Thanks for reporting issues so quick for calcite-1.23.0-rc0.
> > > Appreciate it.
> > > > > Can you log JIRA for these issues? We will fix them.
> > > > >
> > > > Doing it non
> > > >
> > > >
> > > > > Regarding with the first issue, I guess several factors are
> > > contributing
> > > > > to the issue.
> > > > > 1. Trait enforcement is enabled for EnumerableConvention by
> default in
> > > > > 1.23.0, now it can generate mergejoins. We can disable it again if
> > > people
> > > > > would like.
> > > > >
> > > >
> > > > Is it possibile to disable it? I will check. Any suggestion is
> welcome
> > > >
> > > >
> > > > > 2. RelBuilder hasn't been able to handle physical operator's trait
> well
> > > > > yet, especially for Project.
> > > > >
> > > > > 3. Logical operator has been doing some work that it is not
> expected to
> > > > > do, but physical operator should do. Here when creating
> > > LogicalProject, it
> > > > > is trying to deduce its collation from input MergeJoin. Project is
> a
> > > > > frequently created operator, but profiler shows that
> > > > > RelTraitSet.replaceIfs() take 65% in the total runtime of
> > > > > LogicalProject.create(). That is not only inappropriate operation,
> but
> > > also
> > > > > time-wasting operation.
> > > > >
> > > > > 4. Transformation rules can match with physical operator. In this
> case,
> > > > > JoinPushExpressionsRule matched with EnumerableMergeJoin, but the
> rule
> > > > > can't deal with physical operator well, because the traits is not
> > > properly
> > > > > handled. This not only happens on JoinPushExpressionsRule, if you
> > > tweak the
> > > > > query, you might be able to see similar assertion error when
> applying
> > > rule
> > > > > FilterIntoJoinRule. The problem has been there since their
> inception,
> > > but
> > > > > it is just disclosed today by HerdDB, does that mean no one use
> > > Calcite's
> > > > > def

[jira] [Created] (CALCITE-3998) Bad datatype for sum(n), it should be BIGINT but it is sometimes INTEGER

[Enrico Olivelli (Jira)]

Enrico Olivelli created CALCITE-3998:


 Summary: Bad datatype for sum(n), it should be BIGINT but it is 
sometimes INTEGER
 Key: CALCITE-3998
 URL: https://issues.apache.org/jira/browse/CALCITE-3998
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.23.0
Reporter: Enrico Olivelli


I also noted that sometimes the type of sum(N) where N is an INTEGER column 
sometimes it is now reported by Calcite as INTEGER and sometimes as a BIGINT. 
In 1.22 every time is reported as BIGINT.
So we have another test failing.

SELECT sum(n1), count(*) as cc, k1
FROM tblspace1.tsql
GROUP by k1
ORDER BY sum(n1)

Here sum(n1) is reported now a INTEGER, previously it was a BIGINT. I would 
prefer to see it as a BIGINT in order to prevent overflows

Here are the plans:
INFO: Query: SELECT sum(n1), count(*) as cc, k1  FROM tblspace1.tsql GROUP by 
k1 ORDER BY sum(n1) -- Logical Plan
LogicalSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost = 
{10.52595367432 rows, 37.0 cpu, 0.0 io}, id = 1038
  LogicalProject(EXPR$0=[$1], CC=[$2], K1=[$0]): rowcount = 2.0, cumulative 
cost = {8.52595367432 rows, 13.0 cpu, 0.0 io}, id = 1037
LogicalAggregate(group=[{0}], EXPR$0=[SUM($1)], CC=[COUNT()]): rowcount = 
2.0, cumulative cost = {6.52595367432 rows, 7.0 cpu, 0.0 io}, id = 1035
  LogicalProject(K1=[$0], n1=[$1]): rowcount = 2.0, cumulative cost = {4.0 
rows, 7.0 cpu, 0.0 io}, id = 1034
LogicalTableScan(table=[[tblspace1, tsql]]): rowcount = 2.0, cumulative 
cost = {2.0 rows, 3.0 cpu, 0.0 io}, id = 1032

May 12, 2020 11:07:37 AM herddb.sql.CalcitePlanner runPlanner
INFO: Query: SELECT sum(n1), count(*) as cc, k1  FROM tblspace1.tsql GROUP by 
k1 ORDER BY sum(n1) -- Best  Plan
EnumerableSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost = {5.0 
rows, 31.0 cpu, 0.0 io}, id = 1245
  EnumerableProject(EXPR$0=[$1], CC=[1:BIGINT], K1=[$0]): rowcount = 2.0, 
cumulative cost = {3.0 rows, 7.0 cpu, 0.0 io}, id = 1244
EnumerableInterpreter: rowcount = 2.0, cumulative cost = {1.0 rows, 1.0 
cpu, 0.0 io}, id = 1243
  BindableTableScan(table=[[tblspace1, tsql]], projects=[[0, 1]]): rowcount 
= 2.0, cumulative cost = {0.016 rows, 0.024 cpu, 0.0 io}, id = 1055


Within the same test case with the same tables the result of this query is not 
changed
SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM tblspace1.tsql
INFO: Query: SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM 
tblspace1.tsql -- Logical Plan
LogicalAggregate(group=[{}], SS=[SUM($0)], MI=[MIN($0)], MA=[MAX($0)]): 
rowcount = 1.0, cumulative cost = {5.387500047683716 rows, 5.0 cpu, 0.0 io}, id 
= 1253
  LogicalProject(n1=[$1]): rowcount = 2.0, cumulative cost = {4.0 rows, 5.0 
cpu, 0.0 io}, id = 1252
LogicalTableScan(table=[[tblspace1, tsql]]): rowcount = 2.0, cumulative 
cost = {2.0 rows, 3.0 cpu, 0.0 io}, id = 1250

May 12, 2020 11:08:48 AM herddb.sql.CalcitePlanner runPlanner
INFO: Query: SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM 
tblspace1.tsql -- Best  Plan
EnumerableAggregate(group=[{}], SS=[SUM($0)], MI=[MIN($0)], MA=[MAX($0)]): 
rowcount = 1.0, cumulative cost = {2.387500047683716 rows, 1.0 cpu, 0.0 io}, id 
= 1295
  EnumerableInterpreter: rowcount = 2.0, cumulative cost = {1.0 rows, 1.0 cpu, 
0.0 io}, id = 1294
BindableTableScan(table=[[tblspace1, tsql]], projects=[[1]]): rowcount = 
2.0, cumulative cost = {0.012 rows, 0.018002 cpu, 0.0 io}, id = 1265

This is the test on HerdDB
https://github.com/diennea/herddb/blob/vote-calcite-123/herddb-core/src/test/java/herddb/sql/SimplerPlannerTest.java#L237



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-3997) Problem with MERGE JOIN: java.lang.AssertionError: cannot merge join: left input is not sorted on left keys

[Enrico Olivelli (Jira)]

Enrico Olivelli created CALCITE-3997:


 Summary: Problem with MERGE JOIN: java.lang.AssertionError: cannot 
merge join: left input is not sorted on left keys
 Key: CALCITE-3997
 URL: https://issues.apache.org/jira/browse/CALCITE-3997
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.23.0
Reporter: Enrico Olivelli


I have a couple of problems with HerdDB.

1) JOIN order unsorted columns in presence of a WHERE over other columns
This is my case:

CREATE TABLE tblspace1.table1 (k1 string primary key,n1 int,s1 string)
CREATE TABLE tblspace1.table3 (k1 string primary key,n3 int,s3 string)
SELECT t1.k1 as first, t2.k1 as second
FROMtblspace1.table1 t1 
 INNER JOIN tblspace1.table3 t2 ON t1.k1=t2.k1
 WHERE t1.n1 + 1 = t2.n3

In this case for table1 and table3 no column is physically sorted (no column 
with a collation)  

I have this Planner error:
java.lang.AssertionError: cannot merge join: left input is not sorted on left 
keys
at 
org.apache.calcite.rel.metadata.RelMdCollation.mergeJoin(RelMdCollation.java:457)
at 
org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:153)
at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
at GeneratedMetadataHandler_Collation.collations(Unknown Source)
at 
org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:539)
at 
org.apache.calcite.rel.metadata.RelMdCollation.project(RelMdCollation.java:273)
at 
org.apache.calcite.rel.logical.LogicalProject.lambda$create$0(LogicalProject.java:122)
at org.apache.calcite.plan.RelTraitSet.replaceIfs(RelTraitSet.java:242)
at org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:121)
at org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:111)
at 
org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:172)
at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1464)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1258)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1230)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1219)
at 
org.apache.calcite.plan.RelOptUtil.pushDownJoinConditions(RelOptUtil.java:3620)
at 
org.apache.calcite.rel.rules.JoinPushExpressionsRule.onMatch(JoinPushExpressionsRule.java:59)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:221)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:519)
at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:535)
at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:292) 

If I remove the "WHERE" clause then no error is reported.
we have many  other test cases about JOINs and this one is the only one that 
fails

This is the failing test case on HerdDB
https://github.com/diennea/herddb/blob/vote-calcite-123/herddb-core/src/test/java/herddb/core/SimpleJoinTest.java#L522

We are using the default set of rules Programs.ofRules(Programs.RULE_SET)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Problems on HerdDB with 1.23 was [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

[Enrico Olivelli]

Il Mer 13 Mag 2020, 13:45 Haisheng Yuan  ha scritto:

> Hi Enrico,
>
> > Is it possibile to disable it? I will check. Any suggestion is welcome
> Disabling it won't help. It is a Calcite bug. There is nothing wrong in
> HerdDB. Can you help us log a JIRA and provide a reproducible test case?
>

Sorry for the delay.
I has another problem today. I will do as soon as possible.

Yesterday I was trying to create a test case in Calcite codebase.
But I did not find where to put it.
Can you please give me an hint?
Otherwise I will try to create a minimal Java block of code that reproduces
the problem. I did that way last time and Stamatis was able to create the
test in Calcite code

Does this approach work for you?

Enrico


> > Do you think that I can safely disable those rules?
> You have to create your own rule instances. But let Calcite do it for you.
>
> Thanks,
> Haisheng Yuan
>
> On 2020/05/13 08:15:30, Enrico Olivelli  wrote:
> > Haisheng,
> >
> >
> >
> >
> > Il Mar 12 Mag 2020, 16:38 Haisheng Yuan  ha scritto:
> >
> > > Hi Enrico,
> > >
> > > Thanks for reporting issues so quick for calcite-1.23.0-rc0.
> Appreciate it.
> > > Can you log JIRA for these issues? We will fix them.
> > >
> > Doing it non
> >
> >
> > > Regarding with the first issue, I guess several factors are
> contributing
> > > to the issue.
> > > 1. Trait enforcement is enabled for EnumerableConvention by default in
> > > 1.23.0, now it can generate mergejoins. We can disable it again if
> people
> > > would like.
> > >
> >
> > Is it possibile to disable it? I will check. Any suggestion is welcome
> >
> >
> > > 2. RelBuilder hasn't been able to handle physical operator's trait well
> > > yet, especially for Project.
> > >
> > > 3. Logical operator has been doing some work that it is not expected to
> > > do, but physical operator should do. Here when creating
> LogicalProject, it
> > > is trying to deduce its collation from input MergeJoin. Project is a
> > > frequently created operator, but profiler shows that
> > > RelTraitSet.replaceIfs() take 65% in the total runtime of
> > > LogicalProject.create(). That is not only inappropriate operation, but
> also
> > > time-wasting operation.
> > >
> > > 4. Transformation rules can match with physical operator. In this case,
> > > JoinPushExpressionsRule matched with EnumerableMergeJoin, but the rule
> > > can't deal with physical operator well, because the traits is not
> properly
> > > handled. This not only happens on JoinPushExpressionsRule, if you
> tweak the
> > > query, you might be able to see similar assertion error when applying
> rule
> > > FilterIntoJoinRule. The problem has been there since their inception,
> but
> > > it is just disclosed today by HerdDB, does that mean no one use
> Calcite's
> > > default rule implementation to match trait aware physical operators,
> > > intentionally? Can we safely stop matching physical operators in these
> > > rules? (ProjectMerge can be an exception, some people use it on
> physical
> > > operator for post processing).
> > >
> >
> > Do you think that I can safely disable those rules?
> >
> > Enrico
> >
> >
> > > Thanks,
> > > Haisheng
> > >
> > >
> > > On 2020/05/12 09:10:31, Enrico Olivelli  wrote:
> > > > Haisheng,
> > > > I am sorry, I have a couple of problems with HerdDB.
> > > >
> > > > 1) JOIN order unsorted columns in presence of a WHERE over other
> columns
> > > > This is my case:
> > > >
> > > > CREATE TABLE tblspace1.table1 (k1 string primary key,n1 int,s1
> string)
> > > > CREATE TABLE tblspace1.table3 (k1 string primary key,n3 int,s3
> string)
> > > > SELECT t1.k1 as first, t2.k1 as second
> > > > FROMtblspace1.table1 t1
> > > >  INNER JOIN tblspace1.table3 t2 ON t1.k1=t2.k1
> > > >  WHERE t1.n1 + 1 = t2.n3
> > > >
> > > > In this case for table1 and table3 no column is physically sorted (no
> > > > column with a collation)
> > > >
> > > > I have this Planner error:
> > > > java.lang.AssertionError: cannot merge join: left input is not
> sorted on
> > > > left keys
> > > > at
> > > >
> > >
> org.apache.calcite.rel.metadata.RelMdCollation.mergeJoin(RelMdCollation.java:457)
> > > > at
> > &

Re: Problems on HerdDB with 1.23 was [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

[Enrico Olivelli]

Haisheng,




Il Mar 12 Mag 2020, 16:38 Haisheng Yuan  ha scritto:

> Hi Enrico,
>
> Thanks for reporting issues so quick for calcite-1.23.0-rc0. Appreciate it.
> Can you log JIRA for these issues? We will fix them.
>
Doing it non


> Regarding with the first issue, I guess several factors are contributing
> to the issue.
> 1. Trait enforcement is enabled for EnumerableConvention by default in
> 1.23.0, now it can generate mergejoins. We can disable it again if people
> would like.
>

Is it possibile to disable it? I will check. Any suggestion is welcome


> 2. RelBuilder hasn't been able to handle physical operator's trait well
> yet, especially for Project.
>
> 3. Logical operator has been doing some work that it is not expected to
> do, but physical operator should do. Here when creating LogicalProject, it
> is trying to deduce its collation from input MergeJoin. Project is a
> frequently created operator, but profiler shows that
> RelTraitSet.replaceIfs() take 65% in the total runtime of
> LogicalProject.create(). That is not only inappropriate operation, but also
> time-wasting operation.
>
> 4. Transformation rules can match with physical operator. In this case,
> JoinPushExpressionsRule matched with EnumerableMergeJoin, but the rule
> can't deal with physical operator well, because the traits is not properly
> handled. This not only happens on JoinPushExpressionsRule, if you tweak the
> query, you might be able to see similar assertion error when applying rule
> FilterIntoJoinRule. The problem has been there since their inception, but
> it is just disclosed today by HerdDB, does that mean no one use Calcite's
> default rule implementation to match trait aware physical operators,
> intentionally? Can we safely stop matching physical operators in these
> rules? (ProjectMerge can be an exception, some people use it on physical
> operator for post processing).
>

Do you think that I can safely disable those rules?

Enrico


> Thanks,
> Haisheng
>
>
> On 2020/05/12 09:10:31, Enrico Olivelli  wrote:
> > Haisheng,
> > I am sorry, I have a couple of problems with HerdDB.
> >
> > 1) JOIN order unsorted columns in presence of a WHERE over other columns
> > This is my case:
> >
> > CREATE TABLE tblspace1.table1 (k1 string primary key,n1 int,s1 string)
> > CREATE TABLE tblspace1.table3 (k1 string primary key,n3 int,s3 string)
> > SELECT t1.k1 as first, t2.k1 as second
> > FROMtblspace1.table1 t1
> >  INNER JOIN tblspace1.table3 t2 ON t1.k1=t2.k1
> >  WHERE t1.n1 + 1 = t2.n3
> >
> > In this case for table1 and table3 no column is physically sorted (no
> > column with a collation)
> >
> > I have this Planner error:
> > java.lang.AssertionError: cannot merge join: left input is not sorted on
> > left keys
> > at
> >
> org.apache.calcite.rel.metadata.RelMdCollation.mergeJoin(RelMdCollation.java:457)
> > at
> >
> org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:153)
> > at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
> > at GeneratedMetadataHandler_Collation.collations(Unknown Source)
> > at
> >
> org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:539)
> > at
> >
> org.apache.calcite.rel.metadata.RelMdCollation.project(RelMdCollation.java:273)
> > at
> >
> org.apache.calcite.rel.logical.LogicalProject.lambda$create$0(LogicalProject.java:122)
> > at org.apache.calcite.plan.RelTraitSet.replaceIfs(RelTraitSet.java:242)
> > at
> >
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:121)
> > at
> >
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:111)
> > at
> >
> org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:172)
> > at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1464)
> > at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1258)
> > at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1230)
> > at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1219)
> > at
> >
> org.apache.calcite.plan.RelOptUtil.pushDownJoinConditions(RelOptUtil.java:3620)
> > at
> >
> org.apache.calcite.rel.rules.JoinPushExpressionsRule.onMatch(JoinPushExpressionsRule.java:59)
> > at
> >
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:221)
> > at
> >
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:519)
> > at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:535)
> > at herddb.sql.CalciteP

Problems on HerdDB with 1.23 was [VOTE] Release apache-calcite-1.23.0 (release candidate 0)

[Enrico Olivelli]

Haisheng,
I am sorry, I have a couple of problems with HerdDB.

1) JOIN order unsorted columns in presence of a WHERE over other columns
This is my case:

CREATE TABLE tblspace1.table1 (k1 string primary key,n1 int,s1 string)
CREATE TABLE tblspace1.table3 (k1 string primary key,n3 int,s3 string)
SELECT t1.k1 as first, t2.k1 as second
FROMtblspace1.table1 t1
 INNER JOIN tblspace1.table3 t2 ON t1.k1=t2.k1
 WHERE t1.n1 + 1 = t2.n3

In this case for table1 and table3 no column is physically sorted (no
column with a collation)

I have this Planner error:
java.lang.AssertionError: cannot merge join: left input is not sorted on
left keys
at
org.apache.calcite.rel.metadata.RelMdCollation.mergeJoin(RelMdCollation.java:457)
at
org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:153)
at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
at GeneratedMetadataHandler_Collation.collations(Unknown Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:539)
at
org.apache.calcite.rel.metadata.RelMdCollation.project(RelMdCollation.java:273)
at
org.apache.calcite.rel.logical.LogicalProject.lambda$create$0(LogicalProject.java:122)
at org.apache.calcite.plan.RelTraitSet.replaceIfs(RelTraitSet.java:242)
at
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:121)
at
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:111)
at
org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:172)
at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1464)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1258)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1230)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1219)
at
org.apache.calcite.plan.RelOptUtil.pushDownJoinConditions(RelOptUtil.java:3620)
at
org.apache.calcite.rel.rules.JoinPushExpressionsRule.onMatch(JoinPushExpressionsRule.java:59)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:221)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:519)
at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:535)
at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:292)

*If I remove the "WHERE" clause then no error is reported.*
we have many  other test cases about JOINs and this one is the only one
that fails

This is the failing test case on HerdDB
https://github.com/diennea/herddb/blob/vote-calcite-123/herddb-core/src/test/java/herddb/core/SimpleJoinTest.java#L522

We are using the default set of rules Programs.ofRules(Programs.RULE_SET)

I will try to create a reproducer in Calcite core test suite, in order to
understand if it is a bug in HerdDB or in Calcite
but I am reporting the problem as early as possible.
We wanted to create a daily job that tests HerdDB against current Calcite
master but unfortunately we still have not find the time to do it.

2) Changed the data type of sum(N) from BIGINT to INTEGER

I also noted that sometimes the type of sum(N) where N is an INTEGER column
sometimes it is now reported by Calcite as INTEGER and sometimes as
a BIGINT. In 1.22 every time is reported as BIGINT.
So we have another test failing.

SELECT sum(n1), count(*) as cc, k1
FROM tblspace1.tsql
GROUP by k1
ORDER BY sum(n1)

Here sum(n1) is reported now a INTEGER, previously it was a BIGINT. I would
prefer to see it as a BIGINT in order to prevent overflows

Here are the plans:
INFO: Query: SELECT sum(n1), count(*) as cc, k1  FROM tblspace1.tsql GROUP
by k1 ORDER BY sum(n1) -- Logical Plan
LogicalSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost =
{10.52595367432 rows, 37.0 cpu, 0.0 io}, id = 1038
  LogicalProject(EXPR$0=[$1], CC=[$2], K1=[$0]): rowcount = 2.0, cumulative
cost = {8.52595367432 rows, 13.0 cpu, 0.0 io}, id = 1037
LogicalAggregate(group=[{0}], EXPR$0=[SUM($1)], CC=[COUNT()]): rowcount
= 2.0, cumulative cost = {6.52595367432 rows, 7.0 cpu, 0.0 io}, id =
1035
  LogicalProject(K1=[$0], n1=[$1]): rowcount = 2.0, cumulative cost =
{4.0 rows, 7.0 cpu, 0.0 io}, id = 1034
LogicalTableScan(table=[[tblspace1, tsql]]): rowcount = 2.0,
cumulative cost = {2.0 rows, 3.0 cpu, 0.0 io}, id = 1032

May 12, 2020 11:07:37 AM herddb.sql.CalcitePlanner runPlanner
INFO: Query: SELECT sum(n1), count(*) as cc, k1  FROM tblspace1.tsql GROUP
by k1 ORDER BY sum(n1) -- Best  Plan
EnumerableSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost =
{5.0 rows, 31.0 cpu, 0.0 io}, id = 1245
  EnumerableProject(EXPR$0=[$1], CC=[1:BIGINT], K1=[$0]): rowcount = 2.0,
cumulative cost = {3.0 rows, 7.0 cpu, 0.0 io}, id = 1244
EnumerableInterpreter: rowcount = 2.0, cumulative cost = {1.0 rows, 1.0
cpu, 0.0 io}, id = 1243
  BindableTableScan(table=[[tblspace1, tsql]], projects=[[0, 1]]):
rowcount = 2.0, cumulative cost = {0.016 rows, 0.024 cpu, 0.0 io}, id = 1055


Within the same test 

Re: [ANNOUNCE] New committer: Wang Yanlin

[Enrico Olivelli]

Congrats!

Enrico

Il Mer 29 Apr 2020, 04:51 Feng Zhu  ha scritto:

>  Congrations! Yanlin!
>
> best,
> Feng
>
> Chunwei Lei  于2020年4月29日周三 上午10:16写道：
>
> > Congrats, Yanlin!
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Wed, Apr 29, 2020 at 10:07 AM Forward Xu 
> > wrote:
> >
> > > Congrats
> > >
> > >
> > > Best,
> > >
> > > Forward
> > >
> > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道：
> > >
> > > > Congrats, Wang Yanlin!
> > > >
> > > >
> > > >
> > > >
> > > > ---Original---
> > > > From: "Stamatis Zampetakis" > > > Date: Wed, Apr 29, 2020 05:51 AM
> > > > To: "dev" > > > Subject: [ANNOUNCE] New committer: Wang Yanlin
> > > >
> > > >
> > > > Apache Calcite's Project Management Committee (PMC) has invited Wang
> > > Yanlin
> > > > to
> > > > become a committer, and we are pleased to announce that he has
> > accepted.
> > > >
> > > > Wang has pushed numerous fixes and improvements to the project,
> landing
> > > in
> > > > total
> > > > the impressive number of 30 commits to the master. Among other
> things,
> > he
> > > > contributed some important features in the Interpreter.
> > > >
> > > > Wang, welcome, thank you for your contributions, and we look forward
> > your
> > > > further interactions with the community! If you wish, please feel
> free
> > to
> > > > tell
> > > > us more about yourself and what you are working on.
> > > >
> > > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> >
>

Re: [ANNOUNCE] New committer: Forward Xu

[Enrico Olivelli]

Congrats!

Enrico

Il Mer 29 Apr 2020, 04:52 Feng Zhu  ha scritto:

>  Congrations! Forward!
>
> best,
> Feng
>
> Chunwei Lei  于2020年4月29日周三 上午10:17写道：
>
> > Congrats, Forward!
> >
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Wed, Apr 29, 2020 at 6:46 AM Rui Wang  wrote:
> >
> > > Congrats!
> > >
> > >
> > > -Rui
> > >
> > > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang <
> francischu...@apache.org
> > >
> > > wrote:
> > >
> > > > Congrats, Forward!
> > > >
> > > > Francis
> > > >
> > > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote:
> > > > > Apache Calcite's Project Management Committee (PMC) has invited
> > Forward
> > > > Xu
> > > > > to
> > > > > become a committer, and we are pleased to announce that he has
> > > accepted.
> > > > >
> > > > > Forward has been helping the project for some time now. He added
> many
> > > new
> > > > > SQL
> > > > > functions to the project and is one of our JSON experts. On top of
> > > that,
> > > > and
> > > > > other fixes, he is the one who added the Redis adapter to the
> > project.
> > > > >
> > > > > Forward, welcome, thank you for your contributions, and we look
> > forward
> > > > to
> > > > > your
> > > > > further interactions with the community! If you wish, please feel
> > free
> > > to
> > > > > tell
> > > > > us more about yourself and what you are working on.
> > > > >
> > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Jin Xing

[Enrico Olivelli]

Congratulations!

Enrico

Il Mer 29 Apr 2020, 04:51 Feng Zhu  ha scritto:

>  Congrations!
>
> best,
> Feng
>
> Chunwei Lei  于2020年4月29日周三 上午10:16写道：
>
> > Congrats, Jin!
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Wed, Apr 29, 2020 at 10:07 AM Forward Xu 
> > wrote:
> >
> > > Congrats
> > >
> > >
> > > best,
> > >
> > > Forward
> > >
> > > 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道：
> > >
> > > > Congrats, Jin Xing!
> > > >
> > > >
> > > > ---Original---
> > > > From: "Stamatis Zampetakis" > > > Date: Wed, Apr 29, 2020 05:47 AM
> > > > To: "dev" > > > Subject: [ANNOUNCE] New committer: Jin Xing
> > > >
> > > >
> > > > Apache Calcite's Project Management Committee (PMC) has invited Jin
> > Xing
> > > to
> > > > become a committer, and we are pleased to announce that he has
> > accepted.
> > > >
> > > > Jin has contributed a lot of code in the project and many
> > > > recent improvements in
> > > > materialized view matching have his signature on them. Apart from
> code
> > > > contributions, Jin provides valuable help to the community by doing
> > > reviews
> > > > and
> > > > answering questions in the devlist.
> > > >
> > > > Jin, welcome, thank you for your contributions, and we look forward
> to
> > > your
> > > > further interactions with the community! If you wish, please feel
> free
> > to
> > > > tell
> > > > us more about yourself and what you are working on.
> > > >
> > > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> >
>

Re: [ANNOUNCE] New committer: Vineet Garg

[Enrico Olivelli]

Congratulations Vineet!

Enrico

Il Dom 26 Apr 2020, 07:18 XING JIN  ha scritto:

> Congrats, Vineet!
>
> Best,
> Jin
>
> Fan Liya  于2020年4月26日周日 上午11:45写道：
>
> > Congratulations, Vineet!
> >
> > Best,
> > Liya Fan
> >
> > On Sun, Apr 26, 2020 at 10:30 AM Aman Sinha  wrote:
> >
> > > Congratulations Vineet !
> > >
> > > -Aman
> > >
> > > On Sat, Apr 25, 2020 at 7:15 PM Feng Zhu 
> wrote:
> > >
> > > >  Congratulations, well deserved!
> > > >
> > > > best,
> > > > Feng
> > > >
> > > > Chunwei Lei  于2020年4月26日周日 上午10:12写道：
> > > >
> > > > > Congrats, Vineet!
> > > > >
> > > > >
> > > > > Best,
> > > > > Chunwei
> > > > >
> > > > >
> > > > > On Sun, Apr 26, 2020 at 8:24 AM Haisheng Yuan 
> > > wrote:
> > > > >
> > > > > > Congrats, Vineet!
> > > > > >
> > > > > > On 2020/04/25 22:18:35, Forward Xu 
> wrote:
> > > > > > > Congratulations
> > > > > > >
> > > > > > > best,
> > > > > > > Forward
> > > > > > >
> > > > > > > Francis Chuang  于2020年4月26日周日
> > 上午6:04写道：
> > > > > > >
> > > > > > > > Congrats, Vineet!
> > > > > > > >
> > > > > > > > On 26/04/2020 7:52 am, Stamatis Zampetakis wrote:
> > > > > > > > > Apache Calcite's Project Management Committee (PMC) has
> > invited
> > > > > > Vineet
> > > > > > > > > Garg to become a committer, and we are pleased to announce
> > that
> > > > he
> > > > > > > > > has accepted.
> > > > > > > > >
> > > > > > > > > With the first code contribution in Calcite back in 2017,
> > > Vineet
> > > > is
> > > > > > > > > definitely
> > > > > > > > > not new to the project. Since then he has contributed many
> > > > patches,
> > > > > > > > > fixing and improving various modules of Calcite, notably
> > things
> > > > > > around
> > > > > > > > > subqueries.
> > > > > > > > >
> > > > > > > > > Vineet, welcome, thank you for your contributions, and we
> > look
> > > > > > forward
> > > > > > > > > your further interactions with the community! If you wish,
> > > please
> > > > > > feel
> > > > > > > > > free to tell us more about yourself and what you are
> working
> > > on.
> > > > > > > > >
> > > > > > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: TableModify does not keep UPSERT keyword

[Enrico Olivelli]

It looks like we managed to access SqlInsert#isUpsert just by
considering the original SqlNode while processing the plan
https://github.com/diennea/herddb/pull/582
Having it in the Logical Plan would be better, but there is no need to
complicate things by now.

In my case UPSERT is to be executed atomically with a row-level lock,
so in my opinion it is different that a "MERGE INTO" that is more
complicated.

Julian:
Having INSERT ... ON CONFLICT UPDATE would be a good option for me as
well, but we should still support "UPSERT" keyword, at least in Babel,
for compatibility reasons

Thank you all

Il giorno mer 18 mar 2020 alle ore 20:49 Julian Hyde
 ha scritto:
>
> The fact that PostgreSQL implements MERGE and “INSERT … ON CONFLICT” 
> (so-called UPSERT) differently with regard to consistency is an 
> implementation detail in PostgreSQL, and I refuse to accept it as rationale 
> for what we do in Calcite. (If we try to emulate the behavior of systems that 
> suck… well, it’s a race to mediocrity.)
>
> But I do accept that if we parse an UPSERT command, we should not throw away 
> the information in the SqlNode or RelNode representations.
>
> I think we should do what PostgreSQL has done[1]: add a ‘conflictBehavior’ 
> field, whose value is one of 3 enum values: ‘FAIL’ or ‘DO_NOTHING’ or 
> ‘UPDATE’, or something. So, the UPSERT command we implemented for Phoenix 
> would be "INSERT … ON CONFLICT UPDATE”.
>
> But we should not add complex actions like "ON CONFLICT (did) DO UPDATE SET 
> dname = EXCLUDED.dname” to INSERT. If people want complex actions, they 
> should use Calcite's MERGE command. If you want to translate a Calcite MERGE 
> to a PostgreSQL ‘INSERT … ON CONFLICT UPDATE …’, have at it.
>
> By the way, I don’t like the word ‘UPSERT’. It probably started an in-joke 
> among kernel developers. PostgreSQL doesn’t use it in their SQL (only in 
> their documentation), and neither should we.
>
> We had to add UPSERT to support Phoenix’s dialect (because Phoenix inherited 
> limitations on consistency from the underlying HBase engine) but we should 
> deprecate UPSERT and move to INSERT … ON CONFLICT instead.
>
> Julian
>
> [1] https://www.postgresql.org/docs/current/sql-insert.html 
> <https://www.postgresql.org/docs/current/sql-insert.html>
>
> > On Mar 18, 2020, at 2:07 AM, Enrico Olivelli  wrote:
> >
> > Il Mer 18 Mar 2020, 08:35 Christian Beikov  > <mailto:christian.bei...@gmail.com>> ha
> > scritto:
> >
> >> I'd argue that these technical details are in fact the reason why this
> >> is a different functionality, that deserves special handling.
> >>
> >> I expect an upsert statement like `INSERT INTO tbl(a, b) VALUES(1, 2) ON
> >> CONFLICT DO NOTHING` to never produce a constraint violation.
> >>
> >> This is functionally different from a statement like `INSERT INTO tbl(a,
> >> b) SELECT a, b (VALUES(1, 2)) x(a, b) WHERE NOT EXISTS(SELECT 1 FROM tbl
> >> y WHERE x.a = y.a AND x.b = y.b)` which might cause the constraint
> >> violation.
> >>
> >> On the other hand, an upsert statement like `INSERT INTO tbl(a, b)
> >> VALUES(1, 2) ON CONFLICT DO UPDATE` guarantees that at the end of the
> >> statement, there is the tuple `(1, 2)`. There are other variants that
> >> provide different functionality(conflict handler conditions, special
> >> update clause, etc.) and overall, for DBMS that do not support the ON
> >> CONFLICT clause, it is necessary to fallback to PL/SQL to handle
> >> constraint violations in exception handlers within loops to ensure the
> >> same behvaior.
> >>
> >> If Calcite transforms such an UPSERT statement to a MERGE statement, it
> >> must at least be flagged to require atomicity to be able to generate the
> >> correct logic for the backend that this is running on.
> >>
> >
> > Please don't do this.
> > They are different features
> > MERGE is more powerful and it is also for moving data from a table to
> > another one (but I have never used MERGE)
> >
> > Enrico
> >
> >
> >> Am 17.03.2020 um 22:28 schrieb Julian Hyde:
> >>> I don't think there's a significant difference between the UPSERT and
> >>> MERGE. The differences are in marketing (which keyword to use) and in
> >>> technical details (e.g. concurrency semantics). Not worth splitting a
> >>> core concept over. We spend a lot of effort keeping like-things-like.
> >>>
> >>> On Tue, Mar 17, 2020 at 1:11 AM Christian Beikov
> >>>  wrote:
> >>>> AFAIK MERGE has different concurrency

Re: TableModify does not keep UPSERT keyword

[Enrico Olivelli]

Il Mer 18 Mar 2020, 08:35 Christian Beikov  ha
scritto:

> I'd argue that these technical details are in fact the reason why this
> is a different functionality, that deserves special handling.
>
> I expect an upsert statement like `INSERT INTO tbl(a, b) VALUES(1, 2) ON
> CONFLICT DO NOTHING` to never produce a constraint violation.
>
> This is functionally different from a statement like `INSERT INTO tbl(a,
> b) SELECT a, b (VALUES(1, 2)) x(a, b) WHERE NOT EXISTS(SELECT 1 FROM tbl
> y WHERE x.a = y.a AND x.b = y.b)` which might cause the constraint
> violation.
>
> On the other hand, an upsert statement like `INSERT INTO tbl(a, b)
> VALUES(1, 2) ON CONFLICT DO UPDATE` guarantees that at the end of the
> statement, there is the tuple `(1, 2)`. There are other variants that
> provide different functionality(conflict handler conditions, special
> update clause, etc.) and overall, for DBMS that do not support the ON
> CONFLICT clause, it is necessary to fallback to PL/SQL to handle
> constraint violations in exception handlers within loops to ensure the
> same behvaior.
>
> If Calcite transforms such an UPSERT statement to a MERGE statement, it
> must at least be flagged to require atomicity to be able to generate the
> correct logic for the backend that this is running on.
>

Please don't do this.
They are different features
MERGE is more powerful and it is also for moving data from a table to
another one (but I have never used MERGE)

Enrico


> Am 17.03.2020 um 22:28 schrieb Julian Hyde:
> > I don't think there's a significant difference between the UPSERT and
> > MERGE. The differences are in marketing (which keyword to use) and in
> > technical details (e.g. concurrency semantics). Not worth splitting a
> > core concept over. We spend a lot of effort keeping like-things-like.
> >
> > On Tue, Mar 17, 2020 at 1:11 AM Christian Beikov
> >  wrote:
> >> AFAIK MERGE has different concurrency semantics than what some DBMS call
> >> UPSERT. PostgreSQL for example has a guaranteed insert or update
> >> semantics whereas MERGE could end with constraint violation errors:
> >> https://wiki.postgresql.org/wiki/UPSERT
> >>
> >> Maybe it's worth adding that to the relational model after all?
> >>
> >> Am 17.03.2020 um 07:17 schrieb Enrico Olivelli:
> >>> Il Lun 16 Mar 2020, 23:55 Julian Hyde  ha
> scritto:
> >>>
> >>>> Change the unparse operation for the dialect so that you generate
> UPSERT
> >>>> rather than MERGE.
> >>>>
> >>>> IIRC we did this for another dialect - maybe Phoenix.
> >>>>
> >>> Julian,
> >>> In my case (HerdDB) I need to see the presence of UPSERT in the RelNode
> >>> (EnumerableTableModify oŕ LogicalTableModify)
> >>> I saw the JIRA where you introduced UPSERT for Phoenix
> >>> I will check deeper, thanks
> >>>
> >>> Enrico
> >>>
> >>>
> >>>
> >>>> Julian
> >>>>
> >>>>> On Mar 16, 2020, at 1:22 AM, Enrico Olivelli 
> >>>> wrote:
> >>>>> Il Lun 16 Mar 2020, 09:06 Stamatis Zampetakis 
> ha
> >>>>> scritto:
> >>>>>
> >>>>>> Hi Enrico,
> >>>>>>
> >>>>>> I have the impression that UPSERT is currently supported only at the
> >>>> parser
> >>>>>> level [1] so it seems normal that you don't find something relevant
> in
> >>>>>> LogicalTableModify etc. Note that the SQL standard equivalent is the
> >>>> MERGE
> >>>>>> statement [2] but this also seems to be supported only at the
> >>>>>> parser/validator level [2].
> >>>>>> I guess it is not necessary to introduce more things in TableModify
> >>>> since
> >>>>>> UPSERT seems to be syntactic sugar. I think that most of the work
> can be
> >>>>>> done in RelToSqlConverter [4] and possibly the rest of the code can
> be
> >>>> left
> >>>>>> intact.
> >>>>>>
> >>>>> I would like to sens a patch that introduces the ability to keep the
> >>>>> SqlInsert 'keywords' and pass them to TableModify?
> >>>>> Would it be a good approach?
> >>>>>
> >>>>> The alternative is to introduce a new Operation but it would be a
> more
> >>>>> invasive change and I think it is not worth
> >>>>>
> >>>>>
> >>>>> Enrico
> >>>>>
> >>>>>
> >>>>>> Best,
> >>>>>> Stamatis
> >>>>>>
> >>>>>> [1] https://issues.apache.org/jira/browse/CALCITE-492
> >>>>>> [2] https://en.wikipedia.org/wiki/Merge_(SQL)
> >>>>>> [3] https://issues.apache.org/jira/browse/CALCITE-985
> >>>>>> [4]
> >>>>>>
> >>>>>>
> >>>>
> https://github.com/apache/calcite/blob/d234626227954eefffe49f42abec65c649ffe3a6/core/src/test/java/org/apache/calcite/test/SqlToRelConverterTest.java#L2395
> >>>>>>> On Sun, Mar 15, 2020 at 6:53 PM Enrico Olivelli <
> eolive...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>> I am trying to use UPSERT but it seems to me that in TableModify
> (or
> >>>>>>> best LogicalTableModify or EnumerableTableModify) there is no way
> to
> >>>>>>> distinguish an INSERT from an UPSERT.
> >>>>>>>
> >>>>>>> Am I missing something?
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> Enrico
> >>>>>>>
>

Re: TableModify does not keep UPSERT keyword

[Enrico Olivelli]

Il Lun 16 Mar 2020, 23:55 Julian Hyde  ha scritto:

> Change the unparse operation for the dialect so that you generate UPSERT
> rather than MERGE.
>
> IIRC we did this for another dialect - maybe Phoenix.
>

Julian,
In my case (HerdDB) I need to see the presence of UPSERT in the RelNode
(EnumerableTableModify oŕ LogicalTableModify)
I saw the JIRA where you introduced UPSERT for Phoenix
I will check deeper, thanks

Enrico



> Julian
>
> > On Mar 16, 2020, at 1:22 AM, Enrico Olivelli 
> wrote:
> >
> > Il Lun 16 Mar 2020, 09:06 Stamatis Zampetakis  ha
> > scritto:
> >
> >> Hi Enrico,
> >>
> >> I have the impression that UPSERT is currently supported only at the
> parser
> >> level [1] so it seems normal that you don't find something relevant in
> >> LogicalTableModify etc. Note that the SQL standard equivalent is the
> MERGE
> >> statement [2] but this also seems to be supported only at the
> >> parser/validator level [2].
> >> I guess it is not necessary to introduce more things in TableModify
> since
> >> UPSERT seems to be syntactic sugar. I think that most of the work can be
> >> done in RelToSqlConverter [4] and possibly the rest of the code can be
> left
> >> intact.
> >>
> >
> > I would like to sens a patch that introduces the ability to keep the
> > SqlInsert 'keywords' and pass them to TableModify?
> > Would it be a good approach?
> >
> > The alternative is to introduce a new Operation but it would be a more
> > invasive change and I think it is not worth
> >
> >
> > Enrico
> >
> >
> >> Best,
> >> Stamatis
> >>
> >> [1] https://issues.apache.org/jira/browse/CALCITE-492
> >> [2] https://en.wikipedia.org/wiki/Merge_(SQL)
> >> [3] https://issues.apache.org/jira/browse/CALCITE-985
> >> [4]
> >>
> >>
> https://github.com/apache/calcite/blob/d234626227954eefffe49f42abec65c649ffe3a6/core/src/test/java/org/apache/calcite/test/SqlToRelConverterTest.java#L2395
> >>
> >>> On Sun, Mar 15, 2020 at 6:53 PM Enrico Olivelli 
> >>> wrote:
> >>>
> >>> Hi,
> >>> I am trying to use UPSERT but it seems to me that in TableModify (or
> >>> best LogicalTableModify or EnumerableTableModify) there is no way to
> >>> distinguish an INSERT from an UPSERT.
> >>>
> >>> Am I missing something?
> >>>
> >>> Regards
> >>> Enrico
> >>>
> >>
>

Re: TableModify does not keep UPSERT keyword

[Enrico Olivelli]

Il Lun 16 Mar 2020, 09:06 Stamatis Zampetakis  ha
scritto:

> Hi Enrico,
>
> I have the impression that UPSERT is currently supported only at the parser
> level [1] so it seems normal that you don't find something relevant in
> LogicalTableModify etc. Note that the SQL standard equivalent is the MERGE
> statement [2] but this also seems to be supported only at the
> parser/validator level [2].
> I guess it is not necessary to introduce more things in TableModify since
> UPSERT seems to be syntactic sugar. I think that most of the work can be
> done in RelToSqlConverter [4] and possibly the rest of the code can be left
> intact.
>

I would like to sens a patch that introduces the ability to keep the
SqlInsert 'keywords' and pass them to TableModify?
 Would it be a good approach?

The alternative is to introduce a new Operation but it would be a more
invasive change and I think it is not worth


Enrico


> Best,
> Stamatis
>
> [1] https://issues.apache.org/jira/browse/CALCITE-492
> [2] https://en.wikipedia.org/wiki/Merge_(SQL)
> [3] https://issues.apache.org/jira/browse/CALCITE-985
> [4]
>
> https://github.com/apache/calcite/blob/d234626227954eefffe49f42abec65c649ffe3a6/core/src/test/java/org/apache/calcite/test/SqlToRelConverterTest.java#L2395
>
> On Sun, Mar 15, 2020 at 6:53 PM Enrico Olivelli 
> wrote:
>
> > Hi,
> > I am trying to use UPSERT but it seems to me that in TableModify (or
> > best LogicalTableModify or EnumerableTableModify) there is no way to
> > distinguish an INSERT from an UPSERT.
> >
> > Am I missing something?
> >
> > Regards
> > Enrico
> >
>

TableModify does not keep UPSERT keyword

[Enrico Olivelli]

Hi,
I am trying to use UPSERT but it seems to me that in TableModify (or
best LogicalTableModify or EnumerableTableModify) there is no way to
distinguish an INSERT from an UPSERT.

Am I missing something?

Regards
Enrico

Re: [DISCUSS] Code reviews

[Enrico Olivelli]

Hi,
I see that Calcite is a core and mission critical component for many
applications and libraries, in many core libraries they follow
strictly Review-Than-Commit and this rule is very useful.

There is no hurry in committing a fix or a new feature, we are a great
opensource project, without time bounds and no need for "hotfixes".

I see that also new committers feel better with RTC because when it is
the first time you commit to such an important project like Calcite
anyone is really scared to commit
some regression that is not caught by tests !

Trivial patches can go without review, but most of the time it is not
worth to hurry,
I totally suggest to at least add the rule that no-one can commit
patches from himself.
It is not a pain to gently ask "please help merging this patch" and
wait for a fellow committer to push the change to master branch.

Just my 2cents
Enrico

Il giorno sab 14 mar 2020 alle ore 17:19 Michael Mior
 ha scritto:
>
> Thanks for raising the discussion Stamatis! I agree that breaking and
> other significant changes should be reviewed before committing. I'm
> hesitant about saying that *all* issues should be open for 72h before
> committing. Sometimes I'll come up with a small bug fix or enhancement
> that I'd just like to get out and forcing that to happen on separate
> days would significantly slow things down.
> --
> Michael Mior
> mm...@apache.org
>
> Le sam. 14 mars 2020 à 12:13, Stamatis Zampetakis  a écrit 
> :
> >
> > Hi all,
> >
> > In the recent discussion about the quality of the commit messages [1] it
> > was brought up the question of having a specific process for code reviews.
> > I do believe that this is an important subject so I decided to start a
> > separate thread around this topic.
> >
> > Some people lean towards a commit then review (CTR) approach while others
> > seem to prefer more the review then commit (RTC), which basically means at
> > least one +1 vote before pushing to master.
> >
> > Personally, I don't this it is necessary to go strictly for one or the
> > other. As a project we have rather high standards on who do we invite to be
> > a committer so we do put a lot of trust on our members. That is to say that
> > the committer should be able to judge when it is appropriate to commit
> > directly and when it is necessary to wait for a review.
> >
> > As a rule of thumb if a change does not require a JIRA case then it does
> > not require a review either. Typical examples would be: typos, code style,
> > fixing warnings, adding/skipping tests, small doc and site improvements,
> > announcements, etc.
> >
> > For those changes that do justify a JIRA, we can possibly identify a few
> > that fall either to CTR or RTC. Nevertheless, I think it would be nice to
> > have a minimum wait time (72h?) from the moment that a JIRA case is opened
> > before committing the change to master. This gives plenty of time for
> > people to react and declare the intention to review the change before
> > merging.
> >
> > From my perspective any change that breaks backward compatibility should
> > follow RTC and receive at least one +1 vote from another committer before
> > being merged to the master. Obviously, not every change in a public class
> > should be considered as a breaking change. However, changing the behaviour
> > or the API of classes/interfaces such as RelOptRule, RelOptPlanner,
> > RelNode, RelBuilder, RexBuilder, RexSimplify, Schema, Table, SqlParser,
> > RelToSqlConverter, SqlToRelConverter, SqlValidator and their main
> > extensions/implementations can have potentially a big impact on clients so
> > it would be better if we wait for a +1 before merging.
> >
> > For bug fixes that do not break compatibility and do not introduce big
> > changes in the code base I think it is fine to commit without strictly
> > waiting for a review. Bug fixes should (almost) always be accompanied with
> > a test case that reproduces the problem.
> >
> > In terms of evolutions, if the change is localized and small (e.g., new
> > method in RelBuilder, new SQL function, etc.) I think it is safe to use
> > CTR. In some cases, if there is no review and to be on the safe side, it
> > might make sense to mark the new API as experimental.
> >
> > Needless to say that for big evolutions the code should receive at least
> > one positive vote. For the latter, I think we are doing pretty well so far
> > since such kind of evolutions usually pass first from a discussion in the
> > dev list and I think it is a good idea to continue to do so.
> >
> > Finally regarding the content and quality of a review there are many nice
> > articles out there so I am not going to outline a big list with "Dos" and
> > "Donts" here. I will just mention the three filters approach [2] which I
> > think is a helpful tool to keep in mind when doing reviews.
> >
> > The above describe more or less the way I operate my self so it is normal
> > if people feel and want to operate differently. I just wanted to 

Re: [Needs Help] How to sync the jar to mvn repository

[Enrico Olivelli]

It is automatic.
Just wait.
It is not important.
Downstream projects are able to already use Calcite 1.22.0

Enrico

Il giorno ven 6 mar 2020 alle ore 08:42 Danny Chan
 ha scritto:
>
> I release the staging repository yesterday and it appears here
>
> https://search.maven.org/artifact/org.apache.calcite/calcite-core/1.22.0/jar
>
> But how to sync it to the mvnrepository.com ? Sorry I’m first time for it.
> https://mvnrepository.com/artifact/org.apache.calcite/calcite-core
>
> Best,
> Danny Chan

Re: [ANNOUNCE] New committer: Feng Zhu

[Enrico Olivelli]

Welcome !

Enrico

Il giorno mer 4 mar 2020 alle ore 10:42 Matt Wang  ha scritto:
>
> Congrats Feng!
>
>
> ---
> Best,
> Matt Wang
>
>
> On 03/4/2020 11:30，Xin Wang wrote：
> Congrats Feng!
>
> - Xin
>
> Kevin Risden  于2020年3月4日周三 上午2:32写道：
>
> Congrats Feng!
>
> Sounds like an interesting project.
>
> Kevin Risden
>
>
> On Sun, Mar 1, 2020 at 5:34 AM Feng Zhu  wrote:
>
> Thanks Julian.
> Let me make a brief introduction about the project.
> SuperSQL relies on both Calcite and Avatica (with inner branch). It aims
> to
> take the role as a unified entrance and middleware to connect various
> data
> platforms in Tencent, including
> RDBMS、ES、Hive、Flink、Spark、Presto、ClickHouse
> and some inner-built systems. They may be integrated as datasource or
> execution engine.
>
> As we are still busy on migrating massive workloads and supporting
> in-house
> business with SuperSQL, current now we do not have enough resources on
> public release (e.g., presentation, talks or posts). Therefore, it is not
> suprising that you wasn't aware of it before.
>
> Is it better to add the paragraph when we have website or an open-sourced
> version (maybe several months later)?
>
> Tencent's logo can be found in bottom right-hand corner in (
> https://www.tencent.com/en-us/index.html).
> I think you can just pick up the english part (i.e. "Tencent").
>
> Best regards,
> Feng
>
> Julian Hyde  于2020年3月1日周日 上午7:33写道：
>
> Congratulations, thanks, and welcome!
>
> Feng, I wasn’t aware of Tencent’s SuperSQL project. It would be a great
> addition to the “powered by” page. If you would like to add it, please
> create a PR to add a paragraph to that page (no need for a JIRA case),
> and
> provide a link to a logo for Tencent that I can add to the picture.
>
> Julian
>
>
>
> On Feb 29, 2020, at 3:15 PM, Rui Wang  wrote:
>
> Congratulations Feng! Well deserved!
>
>
>
> -Rui
>
> On Sat, Feb 29, 2020 at 2:10 AM Feng Zhu 
> wrote:
>
> Thank you everyone for your warm welcome!
> Currently I am working at SuperSQL team of Tencent in Shenzhen,
> Guangdong,
> China.
> The project relies heavily on Calcite to analyze data residing in
> thousands
> of heterogeneous data sources.
>
> It is my greate honor to be part of the community. Calcite is an
> excellent
> and promising project!
> I will do my best to contribute to the project.
>
> Best regards,
> Feng
>
> Chunwei Lei  于2020年2月29日周六 下午4:46写道：
>
> Congratulations！
>
>
> Best,
> Chunwei
>
>
> On Sat, Feb 29, 2020 at 4:41 PM Danny Chan 
> wrote:
>
> Congratulations!
>
> Francis Chuang 于2020年2月29日 周六下午4:35写道：
>
> Congrats, well-deserved!
>
> On 29/02/2020 6:26 pm, Stamatis Zampetakis wrote:
> Apache Calcite's Project Management Committee (PMC) has invited
> Feng
> Zhu to become a committer, and we are pleased to announce that
> he
> has accepted.
>
> Feng is an active contributor and has contributed a lot of code
> to
> some
> fairly complex parts of Calcite. He has been very helpful in
> discussions
> and reviews, especially around code generation, and his work is
> always
> of high quality.
>
> Feng, welcome, thank you for your contributions, and we look
> forward
> your further interactions with the community! If you wish,
> please
> feel
> free to tell us more about yourself and what you are working on.
>
> Stamatis (on behalf of the Apache Calcite PMC)
>
>
>
>
>
>
>
>
>
>
>
> --
> Thanks,
> Xin

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 3)

[Enrico Olivelli]

+1 (non binding)
Run tests locally (on Mac + JDK13)
Run tests of HerdDB and a bunch of know applications that are using HerdDB

Thank you Danny

Enrico

Il giorno mar 3 mar 2020 alle ore 11:06 Feng Zhu
 ha scritto:
>
> Hi Danny, thanks for your continuous effort!
>
> +1 (non-binding)
>
> - Build and Test (./gradlew build) - OK
> - Checked Release Notes - OK
> - Checked README - OK
> - Validate gpg signature - OK
> Environment 1: Windows 7, JDK 1.8.0_121
> Environment 2: Mac OS X 10.15.1, JDK 1.8.0_231_b11
>
> Bests,
> Feng
>
> Danny Chan  于2020年3月2日周一 下午9:28写道：
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.22.0, release
> > candidate 3.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> > https://github.com/apache/calcite/blob/v1.22.0-rc3/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=537b8dbb4b58c61b6c573eb07a51b8d38896a1ff
> >
> > Its hash is 537b8dbb4b58c61b6c573eb07a51b8d38896a1ff
> >
> > Tag:
> > https://github.com/apache/calcite/tree/v1.22.0-rc3
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc3
> > (revision 38350)
> >
> > The hashes of the artifacts are as follows:
> >
> > a7dface824287756fcc66f6f075286975e409e7ea9a0af97ad973f90bc2832250c09e0477bc571bd78f6ee3721883cce76cada9a3e9601e1dd497fb02679647f
> > *apache-calcite-1.22.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> > https://repository.apache.org/content/repositories/orgapachecalcite-1080/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/danny0405.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > N.B.
> > To create the jars and test Apache Calcite: "./gradlew build".
> >
> > If you do not have a Java environment available, you can run the tests
> > using docker. To do so, install docker and docker-compose, then run
> > "docker-compose run test" from the root of the directory.
> >
> > Please vote on releasing this package as Apache Calcite 1.22.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.22.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> > Best,
> > Danny Chan
> >

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

[Enrico Olivelli]

With the help of Stamatis
we have created a reproducer
the attached test case (in the last comment)  fails on current master
(and 1.22.0rc) but does not fail on 1.21
so it can be considered a real regression

Stamatis created a test case for Calcite code base, that can be used
by a developer to fix the issue
https://issues.apache.org/jira/browse/CALCITE-3826

I am sorry I am not able to work on it these days

Enrico


Il giorno gio 27 feb 2020 alle ore 09:29 Julian Hyde
 ha scritto:
>
> Danny
>
> I agree that a -1 is disheartening. Especially when given for mistaken 
> reasons.
>
> Technically a -1 does not kill an RC. We just need 3 more +1 votes than -1 
> votes.
>
> I urge everyone to give detailed rationale for their vote. Even if you vote 
> +1, describe what you checked.
>
> Julian
>
> > On Feb 26, 2020, at 03:14, Danny Chan  wrote:
> >
> > Gives a -1 when you are sure that it is a bug, or the voting would be very 
> > depressing, anyone can give a -1 for any reasons.
> >
> > Best,
> > Danny Chan
> > 在 2020年2月26日 +0800 PM3:04，Enrico Olivelli ，写道：
> >> Danny,
> >> I have created https://issues.apache.org/jira/browse/CALCITE-3826
> >> for the problem about bind variables in UPDATE statements.
> >>
> >> I feel this it can be a serious regression that can lead to data
> >> corruption (wrong data type in DML statements).
> >> AFAICT There is no way to workaround the problem in the application,
> >> because Calcite produces a wrong model for the query.
> >>
> >> It seems to be a regression introduced in
> >> https://issues.apache.org/jira/browse/CALCITE-3672 but I am not sure.
> >> we should confirm the bug and fix it or demonstrate that the bug is not 
> >> valid
> >>
> >> -1 (non binding)
> >>
> >> Enrico
> >>
> >>> Il giorno mer 26 feb 2020 alle ore 07:34 Feng Zhu
> >>>  ha scritto:
> >>>
> >>> Hi Danny,
> >>> Thanks for your efforts.
> >>>
> >>> +1 (non-binding)
> >>>
> >>> Environment: Windows 7, JDK 1.8.0_121
> >>> - Build and Test (./gradlew build) - OK
> >>> - Checked Release Notes - OK
> >>> - Checked README - OK
> >>>
> >>> Best,
> >>> Feng
> >>>
> >>> Danny Chan  于2020年2月26日周三 下午1:23写道：
> >>>
> >>>> Hi all,
> >>>>
> >>>> I have created a build for Apache Calcite 1.22.0, release candidate 2.
> >>>>
> >>>> Thanks to everyone who has contributed to this release.
> >>>>  You can read the release notes here:
> >>>> https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> >>>>
> >>>> The commit to be voted upon:
> >>>>
> >>>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=e397e47b0752c0647590097584903a9561a646eb
> >>>>
> >>>> Its hash is e397e47b0752c0647590097584903a9561a646eb.
> >>>>
> >>>> The artifacts to be voted on are located here:
> >>>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc2/
> >>>>
> >>>> The hashes of the artifacts are as follows:
> >>>> src.tar.gz.sha512
> >>>>
> >>>> 049a7650645106589b8fa65adf36ebbe90accf8359bcc05f40d7b50b00df9327b24b189c4cd1c0e12a702635cc1dc41f9e8b60238073bf83dfcff9c0ef60e66a
> >>>> *apache-calcite-1.22.0-src.tar.gz
> >>>>
> >>>> A staged Maven repository is available for review at:
> >>>> https://repository.apache.org/content/repositories/orgapachecalcite-1079
> >>>>
> >>>> Release artifacts are signed with the following key:
> >>>> https://people.apache.org/keys/committer/danny0405.asc
> >>>>
> >>>> Please vote on releasing this package as Apache Calcite 1.22.0.
> >>>>
> >>>> The vote is open for the next 72 hours and passes if a majority of
> >>>> at least three +1 PMC votes are cast.
> >>>>
> >>>> [ ] +1 Release this package as Apache Calcite 1.22.0
> >>>> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> >>>> [ ] -1 Do not release this package because...
> >>>>
> >>>> Here is my vote:
> >>>>
> >>>> +1 (binding)
> >>>>
> >>>> Best,
> >>>> Danny Chan
> >>>>

Re: Problems on RC with HerdDB: was: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

I have attached a reproducer testcase for my problem.
The problem is not in the planner itself but in the Sql to Rel conversion

https://issues.apache.org/jira/browse/CALCITE-3826

I am sorry I am not able to open Calcite with IntelliJ nor NetBeans.
I can push the producer to a github repo if you prefer, it is a
simpler JUnit based test case

Hope this helps

Enrico

Il giorno mar 25 feb 2020 alle ore 14:50 Enrico Olivelli
 ha scritto:
>
> Il giorno mar 25 feb 2020 alle ore 14:43 Vladimir Sitnikov
>  ha scritto:
> >
> > >Does any ring bell ?
> >
> > Is it related to [CALCITE-3672] Support implicit type coercion for insert
> > and update ?
> > https://issues.apache.org/jira/browse/CALCITE-3672
> > https://github.com/apache/calcite/pull/1720
>
>
> Vladimir, Danny,
> yes I think that that patch can be the trigger of the regression.
> I see that tests do not deal with bind variables.
>
> I wonder if there is some way to disable the new behaviour
> otherwise I feel this can be a real blocker for the release.
>
>
>
> >
> > >I am now trying to install IntelliJ, but it won't be so immediate
> >
> > AFAIK it should be seamless, so please follow up if you run into issues.
>
> I am able to open the project, I just have to get used to IntelliJ, no 
> problem.
>
> Enrico
>
> >
> > Vladimir

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

[Enrico Olivelli]

Il giorno mer 26 feb 2020 alle ore 12:14 Danny Chan
 ha scritto:
>
> Gives a -1 when you are sure that it is a bug, or the voting would be very 
> depressing, anyone can give a -1 for any reasons.

I am sorry.
I am doing my best in helping with the validation of the release, it
is in the interest of every one that the new release is stable.
I would like to see that there is no issue in Calcite and the problem
is in my application,.

I know the weight of a -1 (I have been release manager for a few
Apache projects and releases)
Btw my vote is "non binding",
It is up to you PMC to accept formally the release.

Enrico

>
> Best,
> Danny Chan
> 在 2020年2月26日 +0800 PM3:04，Enrico Olivelli ，写道：
> > Danny,
> > I have created https://issues.apache.org/jira/browse/CALCITE-3826
> > for the problem about bind variables in UPDATE statements.
> >
> > I feel this it can be a serious regression that can lead to data
> > corruption (wrong data type in DML statements).
> > AFAICT There is no way to workaround the problem in the application,
> > because Calcite produces a wrong model for the query.
> >
> > It seems to be a regression introduced in
> > https://issues.apache.org/jira/browse/CALCITE-3672 but I am not sure.
> > we should confirm the bug and fix it or demonstrate that the bug is not 
> > valid
> >
> > -1 (non binding)
> >
> > Enrico
> >
> > Il giorno mer 26 feb 2020 alle ore 07:34 Feng Zhu
> >  ha scritto:
> > >
> > > Hi Danny,
> > > Thanks for your efforts.
> > >
> > > +1 (non-binding)
> > >
> > > Environment: Windows 7, JDK 1.8.0_121
> > > - Build and Test (./gradlew build) - OK
> > > - Checked Release Notes - OK
> > > - Checked README - OK
> > >
> > > Best,
> > > Feng
> > >
> > > Danny Chan  于2020年2月26日周三 下午1:23写道：
> > >
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Calcite 1.22.0, release candidate 2.
> > > >
> > > > Thanks to everyone who has contributed to this release.
> > > >  You can read the release notes here:
> > > > https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> > > >
> > > > The commit to be voted upon:
> > > >
> > > > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=e397e47b0752c0647590097584903a9561a646eb
> > > >
> > > > Its hash is e397e47b0752c0647590097584903a9561a646eb.
> > > >
> > > > The artifacts to be voted on are located here:
> > > > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc2/
> > > >
> > > > The hashes of the artifacts are as follows:
> > > > src.tar.gz.sha512
> > > >
> > > > 049a7650645106589b8fa65adf36ebbe90accf8359bcc05f40d7b50b00df9327b24b189c4cd1c0e12a702635cc1dc41f9e8b60238073bf83dfcff9c0ef60e66a
> > > > *apache-calcite-1.22.0-src.tar.gz
> > > >
> > > > A staged Maven repository is available for review at:
> > > > https://repository.apache.org/content/repositories/orgapachecalcite-1079
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/danny0405.asc
> > > >
> > > > Please vote on releasing this package as Apache Calcite 1.22.0.
> > > >
> > > > The vote is open for the next 72 hours and passes if a majority of
> > > > at least three +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache Calcite 1.22.0
> > > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Here is my vote:
> > > >
> > > > +1 (binding)
> > > >
> > > > Best,
> > > > Danny Chan
> > > >

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

[Enrico Olivelli]

Il giorno mer 26 feb 2020 alle ore 08:10 Francis Chuang
 ha scritto:
>
> Hey Enrico,
>
> If a regression was introduced and broke existing behavior and is not a
> breaking change, then it's probably something that we should fix before
> the release.
>
> Are you able to open a PR with a test case to confirm the bug so that it
> can be fixed?

I am trying to setup a new dev env because Gradle + Kotlin is not
supported by my IDE.
I guess it will take time.
The JIRA contains enough information and I am sure that a skilled
Calcite developer
is able to setup such test in a few minutes.

>
> The release is running a bit late already, so hopefully this won't be a
> complicated fix.

Yes, but it is worth to assess the problem, or we will deliver huge
problems to our clients

Enrico

>
> Francis
>
> On 26/02/2020 6:03 pm, Enrico Olivelli wrote:
> > Danny,
> > I have created https://issues.apache.org/jira/browse/CALCITE-3826
> > for the problem about bind variables in UPDATE statements.
> >
> > I feel this it can be a serious regression that can lead to data
> > corruption (wrong data type in DML statements).
> > AFAICT There is no way to workaround the problem in the application,
> > because Calcite produces a wrong model for the query.
> >
> > It seems to be a regression introduced in
> > https://issues.apache.org/jira/browse/CALCITE-3672 but I am not sure.
> > we should confirm the bug and fix it or demonstrate that the bug is not 
> > valid
> >
> > -1 (non binding)
> >
> > Enrico
> >
> > Il giorno mer 26 feb 2020 alle ore 07:34 Feng Zhu
> >  ha scritto:
> >>
> >> Hi Danny,
> >> Thanks for your efforts.
> >>
> >> +1 (non-binding)
> >>
> >> Environment: Windows 7, JDK 1.8.0_121
> >> - Build and Test (./gradlew build) - OK
> >> - Checked Release Notes - OK
> >> - Checked README - OK
> >>
> >> Best,
> >> Feng
> >>
> >> Danny Chan  于2020年2月26日周三 下午1:23写道：
> >>
> >>> Hi all,
> >>>
> >>> I have created a build for Apache Calcite 1.22.0, release candidate 2.
> >>>
> >>> Thanks to everyone who has contributed to this release.
> >>>  You can read the release notes here:
> >>> https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> >>>
> >>> The commit to be voted upon:
> >>>
> >>> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=e397e47b0752c0647590097584903a9561a646eb
> >>>
> >>> Its hash is e397e47b0752c0647590097584903a9561a646eb.
> >>>
> >>> The artifacts to be voted on are located here:
> >>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc2/
> >>>
> >>> The hashes of the artifacts are as follows:
> >>> src.tar.gz.sha512
> >>>
> >>> 049a7650645106589b8fa65adf36ebbe90accf8359bcc05f40d7b50b00df9327b24b189c4cd1c0e12a702635cc1dc41f9e8b60238073bf83dfcff9c0ef60e66a
> >>> *apache-calcite-1.22.0-src.tar.gz
> >>>
> >>> A staged Maven repository is available for review at:
> >>> https://repository.apache.org/content/repositories/orgapachecalcite-1079
> >>>
> >>> Release artifacts are signed with the following key:
> >>> https://people.apache.org/keys/committer/danny0405.asc
> >>>
> >>> Please vote on releasing this package as Apache Calcite 1.22.0.
> >>>
> >>> The vote is open for the next 72 hours and passes if a majority of
> >>> at least three +1 PMC votes are cast.
> >>>
> >>> [ ] +1 Release this package as Apache Calcite 1.22.0
> >>> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> >>> [ ] -1 Do not release this package because...
> >>>
> >>> Here is my vote:
> >>>
> >>> +1 (binding)
> >>>
> >>> Best,
> >>> Danny Chan
> >>>

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 2)

[Enrico Olivelli]

Danny,
I have created https://issues.apache.org/jira/browse/CALCITE-3826
for the problem about bind variables in UPDATE statements.

I feel this it can be a serious regression that can lead to data
corruption (wrong data type in DML statements).
AFAICT There is no way to workaround the problem in the application,
because Calcite produces a wrong model for the query.

It seems to be a regression introduced in
https://issues.apache.org/jira/browse/CALCITE-3672 but I am not sure.
we should confirm the bug and fix it or demonstrate that the bug is not valid

-1 (non binding)

Enrico

Il giorno mer 26 feb 2020 alle ore 07:34 Feng Zhu
 ha scritto:
>
> Hi Danny,
> Thanks for your efforts.
>
> +1 (non-binding)
>
> Environment: Windows 7, JDK 1.8.0_121
> - Build and Test (./gradlew build) - OK
> - Checked Release Notes - OK
> - Checked README - OK
>
> Best,
> Feng
>
> Danny Chan  于2020年2月26日周三 下午1:23写道：
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.22.0, release candidate 2.
> >
> > Thanks to everyone who has contributed to this release.
> >  You can read the release notes here:
> > https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=e397e47b0752c0647590097584903a9561a646eb
> >
> > Its hash is e397e47b0752c0647590097584903a9561a646eb.
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc2/
> >
> > The hashes of the artifacts are as follows:
> > src.tar.gz.sha512
> >
> > 049a7650645106589b8fa65adf36ebbe90accf8359bcc05f40d7b50b00df9327b24b189c4cd1c0e12a702635cc1dc41f9e8b60238073bf83dfcff9c0ef60e66a
> > *apache-calcite-1.22.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachecalcite-1079
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/danny0405.asc
> >
> > Please vote on releasing this package as Apache Calcite 1.22.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of
> > at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.22.0
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> > Best,
> > Danny Chan
> >

[jira] [Created] (CALCITE-3826) UPDATE assigns wrong type to bind variables

[Enrico Olivelli (Jira)]

Enrico Olivelli created CALCITE-3826:


 Summary: UPDATE assigns wrong type to bind variables
 Key: CALCITE-3826
 URL: https://issues.apache.org/jira/browse/CALCITE-3826
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.22.0
Reporter: Enrico Olivelli


In 1.22.0rc1 I have found a problem about 
EnumerableTableModify#getSourceExpressionList

It looks like it is not mapping correctly the expected datatypes of
bind variables in queries like UPDATE table set a=?,b=? where pk=?.


You can see the full SQL here in this commit in my test branch here
https://github.com/diennea/herddb/pull/563/commits/157f927c9efe85cf7cac1370e1637b1c7ec46dff#diff-5d7594bc81ae0c92bbd33dee6c0d189aR2301

My case is the following:

Create a table:
CREATE TABLE t1 (
 field0 int PRIMARY KEY,
 field1 VARCHAR(10),
 field2 VARCHAR(10),
 field3 INT,
 field4 INT,
 field5 VARCHAR(10)
)

UPDATE t1 SET field3 =?, field2=?, field4=?, field5=? where field0=?

The Update maps to this Calcite plan:

EnumerableTableModify(table=[[tblspace1, ip]], operation=[UPDATE],
updateColumnList=[[field3, field2, field4, field5]],
sourceExpressionList=[[?0, ?1, ?2, ?3]], flattened=[true]): rowcount =
1.0, cumulative cost = {2.5 rows, 10.5 cpu, 0.0 io}, id = 62
  EnumerableProject(field0=[$0], field1=[$1], field2=[$2],
field3=[$3], field4=[$4], field5=[$5], EXPR$0=[?0], EXPR$1=[?1],
EXPR$2=[?2], EXPR$3=[?3]): rowcount = 1.0, cumulative cost = {1.5
rows, 10.5 cpu, 0.0 io}, id = 61
EnumerableInterpreter: rowcount = 1.0, cumulative cost = {0.5
rows, 0.5 cpu, 0.0 io}, id = 60
  BindableTableScan(table=[[tblspace1, ip]], filters=[[=($0,
?4)]]): rowcount = 1.0, cumulative cost = {0.005 rows, 0.01 cpu, 0.0
io}, id = 45

In particular the obeserved problem is:
- the updateColumnList field is: field3, field2, field4, field5
- but the bind variables have wrong type: VARCHAR, VARCHAR, INT, INT,
expecting INT VARCHAR, INT VARCHAR

even by changing the UPDATE command the types of bind variables stays the same,
and they are the in the same order as in the CREATE TABLE STATEMENT,
skipping the PK.


It may be a regression of CALCITE-3672



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 1)

[Enrico Olivelli]

-1 (non binding)

Good news: I am able to build and run tests locally

Bad news:
- I have found a regression in Bind Variable handling in "UPDATE"
statements due to
https://issues.apache.org/jira/browse/CALCITE-3672
see the separate email Thread "Problems on RC with HerdDB: was: [VOTE]
Release apache-calcite-1.22.0 (release candidate 0)"
- IMHO the "license" issue is a blocker for a release, but it is up to
the PMC to accept or not it

Thank you Danny for driving this release, it is very hard

Enrico


Il giorno mar 25 feb 2020 alle ore 14:36 Vladimir Sitnikov
 ha scritto:
>
> I have already surfaced the case in 1.20.0 release:
> https://lists.apache.org/thread.html/33694a2e754ff63e49e5fd05d52be1f72773c15f4a66adf766223b86%40%3Cdev.calcite.apache.org%3E
>
> Technically speaking, Calcite release artifacts violate ASF licensing
> policy.
> Then it is up to the release manager to decide if the release is valid or
> not.
>
> Vladimir

Re: Problems on RC with HerdDB: was: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

Il giorno mar 25 feb 2020 alle ore 14:43 Vladimir Sitnikov
 ha scritto:
>
> >Does any ring bell ?
>
> Is it related to [CALCITE-3672] Support implicit type coercion for insert
> and update ?
> https://issues.apache.org/jira/browse/CALCITE-3672
> https://github.com/apache/calcite/pull/1720


Vladimir, Danny,
yes I think that that patch can be the trigger of the regression.
I see that tests do not deal with bind variables.

I wonder if there is some way to disable the new behaviour
otherwise I feel this can be a real blocker for the release.



>
> >I am now trying to install IntelliJ, but it won't be so immediate
>
> AFAIK it should be seamless, so please follow up if you run into issues.

I am able to open the project, I just have to get used to IntelliJ, no problem.

Enrico

>
> Vladimir

Re: Problems on RC with HerdDB: was: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

I have found another problem about EnumerableTableModify#getSourceExpressionList

It looks like it is not mapping correctly the expected datatypes of
bind variables in queries like UPDATE table set a=?,b=? where pk=?.

I am sorry I am not able to create easily a test case.
I see Calcite  code switched to Gradle + Kotlin configuration, but
this is not supported by my IDE,
so I am in trouble in creating a test case on Calcite (I am now trying
to install IntelliJ, but it won't be so immediate)

You can see the full SQL here in this commit in my test branch here
https://github.com/diennea/herddb/pull/563/commits/157f927c9efe85cf7cac1370e1637b1c7ec46dff#diff-5d7594bc81ae0c92bbd33dee6c0d189aR2301

My case is the following:

Create a table:
CREATE TABLE t1 (
 field0 int PRIMARY KEY,
 field1 VARCHAR(10),
 field2 VARCHAR(10),
 field3 INT,
 field4 INT,
 field5 VARCHAR(10)
)

UPDATE t1 SET field3 =?, field2=?, field4=?, field5=? where field0=?

The Update maps to this Calcite plan:

EnumerableTableModify(table=[[tblspace1, ip]], operation=[UPDATE],
updateColumnList=[[field3, field2, field4, field5]],
sourceExpressionList=[[?0, ?1, ?2, ?3]], flattened=[true]): rowcount =
1.0, cumulative cost = {2.5 rows, 10.5 cpu, 0.0 io}, id = 62
  EnumerableProject(field0=[$0], field1=[$1], field2=[$2],
field3=[$3], field4=[$4], field5=[$5], EXPR$0=[?0], EXPR$1=[?1],
EXPR$2=[?2], EXPR$3=[?3]): rowcount = 1.0, cumulative cost = {1.5
rows, 10.5 cpu, 0.0 io}, id = 61
EnumerableInterpreter: rowcount = 1.0, cumulative cost = {0.5
rows, 0.5 cpu, 0.0 io}, id = 60
  BindableTableScan(table=[[tblspace1, ip]], filters=[[=($0,
?4)]]): rowcount = 1.0, cumulative cost = {0.005 rows, 0.01 cpu, 0.0
io}, id = 45

In particular the obeserved problem is:
- the updateColumnList field is: field3, field2, field4, field5
- but the bind variables have wrong type: VARCHAR, VARCHAR, INT, INT,
expecting INT VARCHAR, INT VARCHAR

even by changing the UPDATE command the types of bind variables stays the same,
and they are the in the same order as in the CREATE TABLE STATEMENT,
skipping the PK.

Does any ring bell ?

Enrico






Il giorno mar 25 feb 2020 alle ore 12:32 Enrico Olivelli
 ha scritto:
>
> Danny,
> We have found all of our showstoppers:
> - We were not handling JoinInfo#nonEquiConditions (maybe it has been
> added recently)
> - EnumerableDefaults#nestedLoopJoinOptimized assumes that
> AbstractEnumerable#asEnumerator returns an enumeration from the
> beginning of the stream, this is fine, but our previous code expected
> a call to "reset".
>
> We are testing downstream applications
>
> Thank you
> Enrico
>
> Il giorno mar 25 feb 2020 alle ore 10:25 Enrico Olivelli
>  ha scritto:
> >
> > The issue is indeed about JoinInfo#nonEquiConditions and non equi
> > joins in general.
> >
> > I will have an answer within a couple of days: that means all tests
> > passing in HerdDB and in a bunch of well known downstream applications
> > that use Joins extensively
> >
> > Thank you Danny !
> >
> > Enrico
> >
> >
> > Il giorno mar 25 feb 2020 alle ore 10:08 Danny Chan
> >  ha scritto:
> > >
> > > Without a plan diff of each case, it’s hard to tell whether the change is 
> > > expected or turned to be wrong.
> > >
> > > There are indeed some commits that modify the Calcite Enumerable joins.
> > >
> > > From the timeline you gave, maybe you can check there 2 commits:
> > >
> > >
> > > 1. 
> > > https://github.com/apache/calcite/commit/820f6ab4965d79946e4144db8e33aeef98c3d2ff
> > > 2. 
> > > https://github.com/apache/calcite/commit/10a5b8a89d319e6fed563e7e49518cfc960b93d6
> > >
> > > Both seem to be a result fix though ~
> > >
> > > Best,
> > > Danny Chan
> > > 在 2020年2月25日 +0800 PM2:44，Enrico Olivelli ，写道：
> > > > Danny,
> > > > This is our workbench
> > > >
> > > >
> > > > https://github.com/diennea/herddb/pull/563
> > > >
> > > > I don't have a Calcite error/stacktrace but simply join results are not
> > > > correct.
> > > >
> > > > Last know passed Travis build against Calcite SNAPSHOT is around 24th
> > > > January
> > > >
> > > > Enrico
> > > >
> > > > Il Mar 25 Feb 2020, 04:04 Danny Chan  ha scritto:
> > > >
> > > > > Thanks for the check XXX, ~
> > > > >
> > > > > You are right, in CALCITE-3769, we have deprecated the TableScanRule, 
> > > > > the
> > > > > RelBuilder#scan and sql-to-rel conversion would always invokes
> > > > > RelO

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 1)

[Enrico Olivelli]

I am building the RC
I found this warning in logs, it looks like a serious LICENSE problem

Enrico

> Task :release:license
Dependencies of license category B are not allowed for SOURCE artifacts
===

OFL-1.1
* font-awesome:font-awesome-font:4.2.0

Il giorno mar 25 feb 2020 alle ore 11:53 Danny Chan
 ha scritto:
>
> Oops thanks for the correction.
>
> Chunwei Lei 于2020年2月25日 周二下午6:35写道：
>
> > > I have created a build for Apache Calcite 1.22.0, release candidate 0.
> > Should be candidate 1? : )
> >
> >
> > +1 (non-binding)
> >
> > macOS 10.14.6, jdk 1.8.0_151, maven 3.5.4
> >  * Went over release note OK
> >  * Run Calcite tests with the RC OK
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Tue, Feb 25, 2020 at 5:04 PM Danny Chan  wrote:
> >
> > > Hi fellow, can you help to verify this release if you have time ? We plan
> > > to release at this Friday and there is no much time left.
> > >
> > > Best,
> > > Danny Chan
> > > 在 2020年2月25日 +0800 PM12:56，Danny Chan ，写道：
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Calcite 1.22.0, release candidate 0.
> > > >
> > > > Thanks to everyone who has contributed to this release.
> > > >  You can read the release notes here:
> > > >
> > >
> > https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> > > >
> > > > The commit to be voted upon:
> > > >
> > >
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=f24e1d4bd4e235d45d919b7bc59ed7eeadc80464
> > > >
> > > > Its hash is f24e1d4bd4e235d45d919b7bc59ed7eeadc80464.
> > > >
> > > > The artifacts to be voted on are located here:
> > > >
> > >
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc1/
> > > >
> > > > The hashes of the artifacts are as follows:
> > > > src.tar.gz.sha512
> > > >
> > > >
> > >
> > 3da2eaef4aedf4f7c0997594c3bc89d6ca30925be80d0376be72053b7fe371a2d77a37d410d88391059c09ce7c9f50a0cadd89daba550a0d8380811d7ef3cc9f
> > > *apache-calcite-1.22.0-src.tar.gz
> > > >
> > > > A staged Maven repository is available for review at:
> > > >
> > https://repository.apache.org/content/repositories/orgapachecalcite-1078
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/danny0405.asc
> > > >
> > > > Please vote on releasing this package as Apache Calcite 1.22.0.
> > > >
> > > > The vote is open for the next 72 hours and passes if a majority of
> > > > at least three +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache Calcite 1.22.0
> > > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Here is my vote:
> > > >
> > > > +1 (binding)
> > > >
> > > > Best,
> > > > Danny Chan
> > >
> >

Re: Problems on RC with HerdDB: was: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

Danny,
We have found all of our showstoppers:
- We were not handling JoinInfo#nonEquiConditions (maybe it has been
added recently)
- EnumerableDefaults#nestedLoopJoinOptimized assumes that
AbstractEnumerable#asEnumerator returns an enumeration from the
beginning of the stream, this is fine, but our previous code expected
a call to "reset".

We are testing downstream applications

Thank you
Enrico

Il giorno mar 25 feb 2020 alle ore 10:25 Enrico Olivelli
 ha scritto:
>
> The issue is indeed about JoinInfo#nonEquiConditions and non equi
> joins in general.
>
> I will have an answer within a couple of days: that means all tests
> passing in HerdDB and in a bunch of well known downstream applications
> that use Joins extensively
>
> Thank you Danny !
>
> Enrico
>
>
> Il giorno mar 25 feb 2020 alle ore 10:08 Danny Chan
>  ha scritto:
> >
> > Without a plan diff of each case, it’s hard to tell whether the change is 
> > expected or turned to be wrong.
> >
> > There are indeed some commits that modify the Calcite Enumerable joins.
> >
> > From the timeline you gave, maybe you can check there 2 commits:
> >
> >
> > 1. 
> > https://github.com/apache/calcite/commit/820f6ab4965d79946e4144db8e33aeef98c3d2ff
> > 2. 
> > https://github.com/apache/calcite/commit/10a5b8a89d319e6fed563e7e49518cfc960b93d6
> >
> > Both seem to be a result fix though ~
> >
> > Best,
> > Danny Chan
> > 在 2020年2月25日 +0800 PM2:44，Enrico Olivelli ，写道：
> > > Danny,
> > > This is our workbench
> > >
> > >
> > > https://github.com/diennea/herddb/pull/563
> > >
> > > I don't have a Calcite error/stacktrace but simply join results are not
> > > correct.
> > >
> > > Last know passed Travis build against Calcite SNAPSHOT is around 24th
> > > January
> > >
> > > Enrico
> > >
> > > Il Mar 25 Feb 2020, 04:04 Danny Chan  ha scritto:
> > >
> > > > Thanks for the check XXX, ~
> > > >
> > > > You are right, in CALCITE-3769, we have deprecated the TableScanRule, 
> > > > the
> > > > RelBuilder#scan and sql-to-rel conversion would always invokes
> > > > RelOptTable#toRel, so there expects to have some plan changes for the
> > > > TableScan node.
> > > >
> > > > We have moved the physical logic from RelOptTable#toRel to
> > > > EnumerableTableScanRule which would invokes RelOptTable#getExpression ->
> > > > QueryableTable#getExpression, so for your case, returns null is the 
> > > > right
> > > > way to go.
> > > >
> > > > As for the Join failure, I didn’t see the stack trace, can you give more
> > > > details ?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/CALCITE-3769
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2020年2月24日 +0800 PM9:55，Enrico Olivelli ，写道：
> > > > > Danny,
> > > > > We are testing HerdDB with 1.22.0rc0 tag and we are seeing problems 
> > > > > with
> > > > Joins.
> > > > >
> > > > > We were still on 1.19.0 and in December we created a test branch
> > > > > against current Calcite's master.
> > > > > Unfortunately during the past few weeks we stopped checking
> > > > > continuously that branch and we missed the commit id on Calcite that
> > > > > introduced these failures.
> > > > >
> > > > > All failures are about JOIN conditions that seem not to be applied
> > > > correctly.
> > > > >
> > > > > This is my test branch with the upgrade from 1.19 to 1.22.0rc0:
> > > > > https://github.com/diennea/herddb/pull/563
> > > > >
> > > > > We are investigating, hopefully is only a bug in our changes.
> > > > > Since 1.19.0 the management of JOINs has been changed a lot in Calcite
> > > > > so probably we missed something.
> > > > >
> > > > > I also had to implement QueryableTable#getExpression that wasn't
> > > > > required before, I have implemented it with a "return null"
> > > > >
> > > > > This was the error:
> > > > > java.lang.RuntimeException: Error while applying rule
> > > > > EnumerableTableScanRule(in:NONE,out:ENUMERABLE), args
> > > > > [rel#26:LogicalTableScan.NONE.[](table=[tblspace1, tsql])]
> > > > > at
> &g

Problems on RC with HerdDB: was: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

The issue is indeed about JoinInfo#nonEquiConditions and non equi
joins in general.

I will have an answer within a couple of days: that means all tests
passing in HerdDB and in a bunch of well known downstream applications
that use Joins extensively

Thank you Danny !

Enrico


Il giorno mar 25 feb 2020 alle ore 10:08 Danny Chan
 ha scritto:
>
> Without a plan diff of each case, it’s hard to tell whether the change is 
> expected or turned to be wrong.
>
> There are indeed some commits that modify the Calcite Enumerable joins.
>
> From the timeline you gave, maybe you can check there 2 commits:
>
>
> 1. 
> https://github.com/apache/calcite/commit/820f6ab4965d79946e4144db8e33aeef98c3d2ff
> 2. 
> https://github.com/apache/calcite/commit/10a5b8a89d319e6fed563e7e49518cfc960b93d6
>
> Both seem to be a result fix though ~
>
> Best,
> Danny Chan
> 在 2020年2月25日 +0800 PM2:44，Enrico Olivelli ，写道：
> > Danny,
> > This is our workbench
> >
> >
> > https://github.com/diennea/herddb/pull/563
> >
> > I don't have a Calcite error/stacktrace but simply join results are not
> > correct.
> >
> > Last know passed Travis build against Calcite SNAPSHOT is around 24th
> > January
> >
> > Enrico
> >
> > Il Mar 25 Feb 2020, 04:04 Danny Chan  ha scritto:
> >
> > > Thanks for the check XXX, ~
> > >
> > > You are right, in CALCITE-3769, we have deprecated the TableScanRule, the
> > > RelBuilder#scan and sql-to-rel conversion would always invokes
> > > RelOptTable#toRel, so there expects to have some plan changes for the
> > > TableScan node.
> > >
> > > We have moved the physical logic from RelOptTable#toRel to
> > > EnumerableTableScanRule which would invokes RelOptTable#getExpression ->
> > > QueryableTable#getExpression, so for your case, returns null is the right
> > > way to go.
> > >
> > > As for the Join failure, I didn’t see the stack trace, can you give more
> > > details ?
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-3769
> > >
> > > Best,
> > > Danny Chan
> > > 在 2020年2月24日 +0800 PM9:55，Enrico Olivelli ，写道：
> > > > Danny,
> > > > We are testing HerdDB with 1.22.0rc0 tag and we are seeing problems with
> > > Joins.
> > > >
> > > > We were still on 1.19.0 and in December we created a test branch
> > > > against current Calcite's master.
> > > > Unfortunately during the past few weeks we stopped checking
> > > > continuously that branch and we missed the commit id on Calcite that
> > > > introduced these failures.
> > > >
> > > > All failures are about JOIN conditions that seem not to be applied
> > > correctly.
> > > >
> > > > This is my test branch with the upgrade from 1.19 to 1.22.0rc0:
> > > > https://github.com/diennea/herddb/pull/563
> > > >
> > > > We are investigating, hopefully is only a bug in our changes.
> > > > Since 1.19.0 the management of JOINs has been changed a lot in Calcite
> > > > so probably we missed something.
> > > >
> > > > I also had to implement QueryableTable#getExpression that wasn't
> > > > required before, I have implemented it with a "return null"
> > > >
> > > > This was the error:
> > > > java.lang.RuntimeException: Error while applying rule
> > > > EnumerableTableScanRule(in:NONE,out:ENUMERABLE), args
> > > > [rel#26:LogicalTableScan.NONE.[](table=[tblspace1, tsql])]
> > > > at
> > > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:244)
> > > > at
> > > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:636)
> > > > at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:523)
> > > > at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:291)
> > > > at herddb.core.RawSQLTest.cacheStatement(RawSQLTest.java:96)
> > > > at 
> > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > > Method)
> > > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > at
> > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > at java.base/java.lang.reflect.Method.invoke(Method.java:567)
> > > > at
> > > org.junit.runners.model.

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

Danny,
This is our workbench


https://github.com/diennea/herddb/pull/563

I don't have a Calcite error/stacktrace but simply join results are not
correct.

Last know passed Travis build against Calcite SNAPSHOT is around 24th
January

Enrico

Il Mar 25 Feb 2020, 04:04 Danny Chan  ha scritto:

> Thanks for the check XXX, ~
>
> You are right, in CALCITE-3769, we have deprecated the TableScanRule, the
> RelBuilder#scan and sql-to-rel conversion would always invokes
> RelOptTable#toRel, so there expects to  have some plan changes for the
> TableScan node.
>
> We have moved the physical logic from RelOptTable#toRel to
> EnumerableTableScanRule which would invokes RelOptTable#getExpression ->
> QueryableTable#getExpression, so for your case, returns null is the right
> way to go.
>
> As for the Join failure, I didn’t see the stack trace, can you give more
> details ?
>
> [1] https://issues.apache.org/jira/browse/CALCITE-3769
>
> Best,
> Danny Chan
> 在 2020年2月24日 +0800 PM9:55，Enrico Olivelli ，写道：
> > Danny,
> > We are testing HerdDB with 1.22.0rc0 tag and we are seeing problems with
> Joins.
> >
> > We were still on 1.19.0 and in December we created a test branch
> > against current Calcite's master.
> > Unfortunately during the past few weeks we stopped checking
> > continuously that branch and we missed the commit id on Calcite that
> > introduced these failures.
> >
> > All failures are about JOIN conditions that seem not to be applied
> correctly.
> >
> > This is my test branch with the upgrade from 1.19 to 1.22.0rc0:
> > https://github.com/diennea/herddb/pull/563
> >
> > We are investigating, hopefully is only a bug in our changes.
> > Since 1.19.0 the management of JOINs has been changed a lot in Calcite
> > so probably we missed something.
> >
> > I also had to implement QueryableTable#getExpression that wasn't
> > required before, I have implemented it with a "return null"
> >
> > This was the error:
> > java.lang.RuntimeException: Error while applying rule
> > EnumerableTableScanRule(in:NONE,out:ENUMERABLE), args
> > [rel#26:LogicalTableScan.NONE.[](table=[tblspace1, tsql])]
> > at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:244)
> > at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:636)
> > at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:523)
> > at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:291)
> > at herddb.core.RawSQLTest.cacheStatement(RawSQLTest.java:96)
> > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:567)
> > at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> > at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> > at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> > at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> > at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> > at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> > at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.j

Re: [VOTE] Release apache-calcite-1.22.0 (release candidate 0)

[Enrico Olivelli]

Danny,
We are testing HerdDB with 1.22.0rc0 tag and we are seeing problems with Joins.

We were still on 1.19.0 and in December we created a test branch
against current Calcite's master.
Unfortunately during the past few weeks we stopped checking
continuously that branch and we missed the commit id on Calcite that
introduced these failures.

All failures are about JOIN conditions that seem not to be applied correctly.

This is my test branch with the upgrade from 1.19 to 1.22.0rc0:
https://github.com/diennea/herddb/pull/563

We are investigating, hopefully is only a bug in our changes.
Since 1.19.0 the management of JOINs has been changed a lot in Calcite
so probably we missed something.

I also had to implement QueryableTable#getExpression that wasn't
required before, I have implemented it with a "return null"

This was the error:
java.lang.RuntimeException: Error while applying rule
EnumerableTableScanRule(in:NONE,out:ENUMERABLE), args
[rel#26:LogicalTableScan.NONE.[](table=[tblspace1, tsql])]
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:244)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:636)
at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:523)
at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:291)
at herddb.core.RawSQLTest.cacheStatement(RawSQLTest.java:96)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.lang.RuntimeException: getExpression is not implemented
at herddb.sql.CalcitePlanner$TableImpl.getExpression(CalcitePlanner.java:1377)
at 
org.apache.calcite.prepare.RelOptTableImpl.lambda$getClassExpressionFunction$2(RelOptTableImpl.java:165)
at 
org.apache.calcite.prepare.RelOptTableImpl.getExpression(RelOptTableImpl.java:214)
at 
org.apache.calcite.adapter.enumerable.EnumerableTableScanRule.convert(EnumerableTableScanRule.java:63)
at org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:144)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:217)

Best regards

Enrico

Il giorno lun 24 feb 2020 alle ore 11:38 Danny Chan
 ha scritto:
>
> Just to note that, I have updated the release note though, so the history.md 
> has diff for
> tar.gz in 
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.22.0-rc0/ and
> the source tag calcite-1.22.0.
>
> Best,
> Danny Chan
> 在 2020年2月24日 +0800 PM6:34，Danny Chan ，写道：
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.22.0, release candidate 0.
> >
> > Thanks to everyone who has contributed to this release.
> >  You can read the release notes here:
> > https://github.com/apache/calcite/blob/calcite-1.22.0/site/_docs/history.md
> >
> > The commit to be voted upon:
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=c6d55f13e4a6c2f53c13474e3e86a5a21834f9ed
> >
> > Its hash is c6d55f13e4a6c2f53c13474e3e86a5a21834f9ed.
> >
> > The artifacts to be voted on are located here:
> > 

Re: [DISCUSS] Towards Calcite 1.22

[Enrico Olivelli]

Il Dom 9 Feb 2020, 08:41 Stamatis Zampetakis  ha scritto:

> Thanks for fixing the build Vladimir!
>
> I like the idea of testing against other projects on a weekly basis. It
> will certainly help detect regressions early on. It might be a bit hard to
> maintain since there could be failures quite often but I guess we will not
> know till we try.
>
> Even before adding 3rd-party projects in the CI, I would say that is worth
> running our integration tests (Druid, Cassandra, Postgres, etc.)


+1

I think that third party users should test their code against current
Calcite master and report to this list all problems as soon as possible.

In HerdDB we saw regressions too late and the code was no more compatible
with Calcite master so we are still using a very old version of Calcite.

We started to have a branch that builds and runs tests against Calcite
master.
Showstoppers in this process are when we break source/binary compatibility
in Calcite, so you have to maintain a separate branch, because you cannot
depend on SNAPSHOT release  and this is expected to happen at every release.

Having more frequent releases in Calcite would help a bit, but currently it
is not a big burden


Enrico

at a
> regular basis by CI. I am checking them now and there seems to be again
> failures.
>
> Best,
> Stamatis
>
>
>
> On Sat, Feb 8, 2020 at 11:48 AM Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
>
> > >Are current -SNAPSHOT packages on repository.apache.org up to date ?
> >
> > The snapshots were not up to date because Calcite-Snapshots Jenkins job
> was
> > using beam Jenkins node somehow.
> > I guess that was caused by misconfiguration of beam nodes.
> >
> > I've triggered the job manually, and it works now:
> > https://builds.apache.org/job/Calcite-Snapshots/
> > On top of that, I've configured mails to dev@calcite list for the
> > snapshots
> > job so we'll know if it fails again.
> >
> > >I would like to test my projects in CI against current master
> >
> > I wonder if it makes sense to add GitHub Actions CI which would validate
> > third-party projects on a weekly basis.
> > For instance, I have https://github.com/vlsi/mat-calcite-plugin , which
> is
> > not that sophisticated, however, it would be nice to see
> > if the upcoming Calcite version is going to break or not that client.
> >
> > For instance, Beam has quite interesting post-commit tests page:
> > https://github.com/apache/beam#post-commit-tests-status-on-master-branch
> >
> > WDYT?
> >
> > Vladimir
> >
>

Re: [DISCUSS] Towards Calcite 1.22

[Enrico Olivelli]

Are current -SNAPSHOT packages on repository.apache.org up to date ?
I would like to test my projects in CI against current master

Having a release would be great at this point

Enrico

Il giorno sab 8 feb 2020 alle ore 06:06 Danny Chan
 ha scritto:
>
> I can do that.
>
> Andrei Sereda 于2020年2月8日 周六上午6:18写道：
>
> > Is there anyone willing to swap a Calcite release with me ?
> >
> > On Fri, Feb 7, 2020 at 12:41 PM Andrei Sereda  wrote:
> >
> > > Hi Julian et al,
> > >
> > > Unfortunately my current work schedule is still very tense. There are
> > some
> > > internal deadlines that have passed already.
> > >
> > > I would appreciate if I can exchange 1.22 Calcite release with a Release
> > > Manager of a later version (perhaps 1.23 or 1.24?)
> > >
> > > Again, apologies to all for this situation as I didn't expect this
> > > quarter to be the way it turned out.
> > >
> > > Andrei.
> > >
> > >
> > >
> > >
> > > On Thu, Feb 6, 2020 at 7:21 PM Julian Hyde  wrote:
> > >
> > >> Andrei,
> > >>
> > >> Do you know when you might be available? Two weeks have passed since
> > >> your last note.
> > >>
> > >> We originally wanted to release in December, then it was pushed back
> > >> to mid-January. We can't wait much longer.
> > >>
> > >> Julian
> > >>
> > >> On Wed, Jan 22, 2020 at 8:07 AM Andrei Sereda  wrote:
> > >> >
> > >> > Hello,
> > >> >
> > >> > I would like to ask community if it is OK if 1.22 release gets delayed
> > >> by
> > >> > 2-3 weeks ?
> > >> >
> > >> > I have tried to book 1 week from work but, unfortunately, right now it
> > >> is
> > >> > not easy (things should be better in February).
> > >> >
> > >> > Another option is to swap with somebody doing next (1.2[345]) release.
> > >> >
> > >> > Please let me know what you think and apologies for the inconvenience.
> > >> >
> > >> > Andrei.
> > >> >
> > >> >
> > >> > On Wed, Dec 4, 2019 at 4:55 PM Julian Hyde  wrote:
> > >> >
> > >> > > I don’t mind whether the release happens in December or January, but
> > >> > > either way, let’s start burning down the backlog of PRs now.
> > >> > >
> > >> > >
> > >> > > > On Dec 2, 2019, at 11:43 PM, Enrico Olivelli  > >
> > >> > > wrote:
> > >> > > >
> > >> > > > Andrei
> > >> > > >
> > >> > > > Il mar 3 dic 2019, 08:21 Rui Wang  > >> > > amaliu...@apache.org>> ha scritto:
> > >> > > >
> > >> > > >> Thank you for this notification.
> > >> > > >>
> > >> > > >> Please try to make CALCITE-3272[1] into 1.22 (change the fix
> > >> version to
> > >> > > >> 1.22 already).
> > >> > > >>
> > >> > > >>
> > >> > > >> [1]: https://github.com/apache/calcite/pull/1587
> > >> > > >>
> > >> > > >> -Rui
> > >> > > >>
> > >> > > >> On Mon, Dec 2, 2019 at 6:35 PM Chunwei Lei <
> > chunwei.l...@gmail.com
> > >> >
> > >> > > wrote:
> > >> > > >>
> > >> > > >>> Thank you for your work, Anderi.
> > >> > > >>>
> > >> > > >>> Let's get CALCITE-1581[1] into 1.22.
> > >> > > >>>
> > >> > > >>> +1 for release at early-mid January '20 (to have more time to
> > >> review
> > >> > > >> prs).
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> [1] https://github.com/apache/calcite/pull/1138
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> Best,
> > >> > > >>> Chunwei
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> On Tue, Dec 3, 2019 at 8:02 AM Andrei Sereda 
> > >> wrote:
> > >> > > >>>
>

Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

[Enrico Olivelli]

Il dom 29 dic 2019, 20:09 Vladimir Sitnikov 
ha scritto:

> Hi,
>
> We have a 1-year old issue with an idea to sort RexNode operands so they
> are consistent.
>
> For instance, "x=5" and "5=x" have the same semantics, so it would make
> sense to stick to a single implementation.
> A discussion can be found in
> https://issues.apache.org/jira/browse/CALCITE-2450
>
> We do not normalize RexNodes, thus it results in excessive planning time,
> especially when the planner is trying to reorder joins.
> For instance, it thinks Join(A, B, $0=$1) and Join(A, B, $1=$0) are
> different joins, however, they are equivalent.
>
> The normalization does not seem to cost much, however, it enables me to
> activate more rules (e.g. EnumerabeMergeRule),
> so it is good as it enables to consider more sophisticated plans.
>
> I see two approaches:
> a) Normalize in RexNode constructor. This seems easy to implement, however,
> there's a catch
> if someone assumed that the order of operands would be the same as the one
> that was passed to the constructor.
> I don't think there are such assumptions in the wild, but there might be.
> The javadoc for the relevant methods says nothing regarding the operand
> order.
> However, the good thing would be RexNode would feel the same in the
> debugger and in its toString representation.
>
> b) Normalize at RexCall#computeDigest only.
> In other words, keep the operands unsorted, but make sure the digest is
> created as if the operands were sorted.
> This seems to be the most transparent change, however, it might surprise
> that `toString` does not match to whatever is seen in the debugger.
>
> In any case, making `RexCall#toString` print sorted representation would
> alter lots of tests.
> For :core it is like 5540 tests completed, 358 failed, 91 skipped :((
>
> WDYT?
>

I really would love this feature.

Just my 2 cents
Enrico



> Hopefully, making the RexNode representation sorted would reduce the number
> of `$1=$0` vs `$0=$1` plan diffs.
>
> Vladimir
>

Re: Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis

[Enrico Olivelli]

Congratulations Stamatis!

Enrico

Il gio 19 dic 2019, 04:40 Rui Wang  ha scritto:

> Congratulations and Thanks Stamatis!
>
>
>
> -Rui
>
> On Wed, Dec 18, 2019 at 6:52 PM XING JIN  wrote:
>
> > Congratulations Stamatis!
> >
> > -Jin
> >
> > Chunwei Lei  于2019年12月19日周四 上午10:33写道：
> >
> > > Congratulations Stamatis!
> > >
> > >
> > > Best,
> > > Chunwei
> > >
> > >
> > > On Thu, Dec 19, 2019 at 9:36 AM Danny Chan 
> wrote:
> > >
> > > > Congratulations Stamatis!
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年12月19日 +0800 AM7:37，dev@calcite.apache.org，写道：
> > > > >
> > > > > Congratulations Stamatis!
> > > >
> > >
> >
>

Re: Release managers

[Enrico Olivelli]

Hi,
AFAIK in all of the apache projects I am contributing to as committer every
committer can sign the artifacts and be release manager, the PMC has just
to VOTE or do some Apache reporter stuff or other bureaucratic stuff.

Is there any particular rule here in Calcite project?

If it is not the case I suggest to open the ability of signing artifacts to
any committer that has a GPG key valid and part of the Apache Web of Trust

Just my two sents
Enrico

Il mar 3 dic 2019, 02:11 Chunwei Lei  ha scritto:

> It's great to know that committers can be release manager.
> I volunteer to be the release manager if there is a chance.
>
> BTW, If Julian doesn't mind, I would like to take 1.24 ~~
>
>
> Best,
> Chunwei
>
>
> On Tue, Dec 3, 2019 at 5:43 AM Francis Chuang 
> wrote:
>
> > Just a quick note regarding the move to Gradle as the build tool for
> > release. Thanks to Vladimir, the release process is almost entirely
> > automated and the rc can be built and automatically uploaded to ASF's
> > servers for voting using just one command: ./gradlew prepareVote -Prc=0
> > -Pasf This builds the artifacts, signs them and uploads them to
> > dist.apache.org.
> >
> > If the RM is a committer, then I think they should do a dry-run to test
> > the build, but not upload it. The asflike-release-environment[1] should
> > be used to try uploading the release to a test environment: ./gradlew
> > prepareVote -Prc=0.
> >
> > Once everything looks good, a PMC member should build and upload the
> > signed release using ./gradlew prepareVote -Prc=0 -Pasf and forward the
> > vote email to the RM.
> >
> > In the past, maven only built the artifacts and left the uploading of
> > the files as a manual exercise to the RM, so the process has changed
> > slightly this time.
> >
> > Francis
> >
> > [1] https://github.com/vlsi/asflike-release-environment
> >
> > On 3/12/2019 6:03 am, Julian Hyde wrote:
> > > I volunteer to do 1.24.
> > >
> > > There’s one part of the release process that only a PMC member can do -
> > namely, to sign the artifacts. But that’s only a small part of the
> process,
> > and you can easily get a PMC member to do it for you. A much larger part
> of
> > the process is the herding of cats (committers, bugs, pull requests,
> > release notes). So, yes, a committer can definitely be a release manager.
> > >
> > > How does the PMC decide which committers to promote to PMC members? We
> > are looking for people who help out around the project, going above and
> > beyond the basic needs of each task to make the project a better place.
> If
> > you are a committer, helping with the release process is a good way to
> earn
> > merit.
> > >
> > > Julian
> > >
> > >
> > >> On Dec 2, 2019, at 10:48 AM, Stamatis Zampetakis 
> > wrote:
> > >>
> > >> Many thanks Haisheng and Danny for stepping up! I added you to the
> list.
> > >> There are two spots left, if nobody else comes up I will take one of
> > them!
> > >>
> > >> Release Target date Release manager
> > >> === === ===
> > >> 1.192019-03Kevin
> > >> 1.202019-06Michael
> > >> 1.212019-09Stamatis
> > >> === === ===
> > >> 1.222019-12Andrei
> > >> 1.232020-02Haisheng
> > >> 1.242020-04Julian
> > >> 1.252020-06Danny
> > >> 1.262020-08
> > >>
> > >>
> > >>
> > >> On Sat, Nov 30, 2019 at 9:52 AM Danny Chan 
> > wrote:
> > >>
> > >>> BTW,
> > >>> I can volunteer to be the release manager for v1.25 or v1.26.
> > >>>
> > >>> Best,
> > >>> Danny Chan
> > >>> 在 2019年11月30日 +0800 PM2:13，dev@calcite.apache.org，写道：
> > 
> >  I can volunteer to be the release manager for v1.23 or v1.24.
> > >>>
> > >
> >
>

Re: [DISCUSS] Towards Calcite 1.22

[Enrico Olivelli]

Andrei

Il mar 3 dic 2019, 08:21 Rui Wang  ha scritto:

> Thank you for this notification.
>
> Please try to make CALCITE-3272[1] into 1.22 (change the fix version to
> 1.22 already).
>
>
> [1]: https://github.com/apache/calcite/pull/1587
>
> -Rui
>
> On Mon, Dec 2, 2019 at 6:35 PM Chunwei Lei  wrote:
>
> > Thank you for your work, Anderi.
> >
> > Let's get CALCITE-1581[1] into 1.22.
> >
> > +1 for release at early-mid January '20 (to have more time to review
> prs).
> >
> >
> > [1] https://github.com/apache/calcite/pull/1138
> >
> >
> > Best,
> > Chunwei
> >
> >
> > On Tue, Dec 3, 2019 at 8:02 AM Andrei Sereda  wrote:
> >
> > > Hello,
> > >
> > > Calcite 1.21 was released about 3 months ago (2019-09) and it is time
> to
> > > start preparation for 1.22.
> > >
> > > Current open issues and pull requests can be seen in [1] and [2]. There
> > are
> > > many PRs left from previous releases and it would be nice to review as
> > many
> > > as possible. Please change "fix version" in JIRA to 1.22 if you would
> > like
> > > the contribution be considered for this release. It is also helpful to
> > mark
> > > PR with "LGTM-will-merge-soon" label so other contributors are aware of
> > > your review.
> > >
> > > Committers please go over existing PRs and try to prioritize / finalize
> > > them. Also let me know which changes (in your opinion) are ready or
> > should
> > > be considered for this release. Don't forget that current policy of
> > > frequent releases allows better work scheduling without blocking
> existing
> > > release plan.
> > >
> > > In terms of dates, let's agree on release time frame late December '19
> or
> > > early-mid January '20 ?
>

I (from HerdDB community) would prefer late December if possible, as we are
stuck to an older 1.19 version of Calcite.
Current Calcite master is is great shape from our point of view

Thanks for driving this
Enrico





> >
> > > Let me know if I have missed anything or if current plan is
> inconvenient.
> > >
> > > Thanks,
> > > Andrei.
> > >
> > > [1]
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > [2] https://github.com/apache/calcite/pulls
> > >
> >
>

Re: [ANNOUNCE] Danny Chan joins Calcite PMC

[Enrico Olivelli]

Congrat Danny !

Enrico

Il giorno dom 3 nov 2019 alle ore 20:29 Julian Feinauer <
j.feina...@pragmaticminds.de> ha scritto:

> Congratulations Danny! Very well deserved!
>
> Julian
>
> Am 01.11.19, 20:49 schrieb "Muhammad Gelbana" :
>
> Congratulations!
>
> Thanks,
> Gelbana
>
>
> On Fri, Nov 1, 2019 at 9:07 AM Stamatis Zampetakis 
> wrote:
>
> > Congratulations Danny!
> >
> > You are doing an amazing job. The project and the community is
> becoming
> > better every day and your help is much appreciated.
> >
> > Keep up the momentum!
> >
> > Best,
> > Stamatis
> >
> > On Thu, Oct 31, 2019 at 4:41 AM Kurt Young  wrote:
> >
> > > Congratulations Danny!
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Oct 31, 2019 at 11:18 AM Danny Chan 
> > wrote:
> > >
> > > > Thank you so much colleagues, it’s my honor to work with you!
> > > >
> > > > I have always felt respected and the harmony of the community,
> hope to
> > > > contribute more and I would give help as best as I can, thanks !
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年10月31日 +0800 AM5:22，Francis Chuang <
> francischu...@apache.org
> > >，写道：
> > > > > I'm pleased to announce that Danny has accepted an invitation
> to
> > > > > join the Calcite PMC. Danny has been a consistent and helpful
> > > > > figure in the Calcite community for which we are very
> grateful. We
> > > > > look forward to the continued contributions and support.
> > > > >
> > > > > Please join me in congratulating Danny!
> > > > >
> > > > > - Francis (on behalf of the Calcite PMC)
> > > >
> > >
> >
>
>
>

Re: ApacheCon Europe 2019 talks which are relevant to Apache Calcite

[Enrico Olivelli]

Stamatis and Julian
I am at the conference as well

Enrico

Il mer 23 ott 2019, 09:29 Julian Feinauer  ha
scritto:

> That would be really nice!
> Just ping me I will be there all days!
>
> Julian
> 
> From: Stamatis Zampetakis 
> Sent: Wednesday, October 23, 2019 8:29:11 AM
> To: dev@calcite.apache.org 
> Subject: Re: ApacheCon Europe 2019 talks which are relevant to Apache
> Calcite
>
> Most likely, I will be in Berlin on Thursday 24 for the conference!
>
> Let's try to meet!
>
> Stamatis
>
> On Tue, Oct 8, 2019 at 10:36 AM Stamatis Zampetakis 
> wrote:
>
> > https://github.com/apache/calcite/pull/1489
> >
> > On Mon, Oct 7, 2019 at 9:48 PM Julian Hyde  wrote:
> >
> >> I feel remiss in filling out
> >> https://calcite.apache.org/community/#upcoming-talks <
> >> https://calcite.apache.org/community/#upcoming-talks>. I’d be grateful
> >> if someone would remove ApacheCon NA and add ApacheCon Europe and log a
> PR.
> >>
> >> > On Oct 7, 2019, at 12:15 PM, Chris Baynes  wrote:
> >> >
> >> > Hi!
> >> >
> >> > I'll be giving a talk on "Fast federated SQL with Apache Calcite".
> >> > Would be great to meet up with any other Calciters attending!
> >> >
> >> > See you there
> >> >
> >> > Chris
> >> >
> >> > On Mon, Oct 7, 2019 at 4:01 PM Julian Feinauer <
> >> j.feina...@pragmaticminds.de>
> >> > wrote:
> >> >
> >> >> Hi all,
> >> >>
> >> >> are there any Calcite related talks in Berlin or any Calciters
> >> attending?
> >> >> I will be there.
> >> >>
> >> >> JulianF
> >> >>
> >> >> Am 04.10.19, 19:09 schrieb "my...@apache.org" :
> >> >>
> >> >>Dear Apache Calcite committers,
> >> >>
> >> >>In a little over 2 weeks time, ApacheCon Europe is taking place in
> >> >>Berlin. Join us from October 22 to 24 for an exciting program and
> >> >> lovely
> >> >>get-together of the Apache Community.
> >> >>
> >> >>We are also planning a hackathon.  If your project is interested
> in
> >> >>participating, please enter yourselves here:
> >> >>https://cwiki.apache.org/confluence/display/COMDEV/Hackathon
> >> >>
> >> >>The following talks should be especially relevant for you:
> >> >>
> >> >>  * *
> >> >>
> >> https://aceu19.apachecon.com/session/fast-federated-sql-apache-calcite*
> >> >>
> >> >>
> >> >><
> >> >>
> >>
> https://aceu19.apachecon.com/session/patterns-and-anti-patterns-running-apache-bigdata-projects-kubernetes
> >> >>>
> >> >>
> >> >>  *
> >> >>
> >> >>
> >> >>
> >>
> https://aceu19.apachecon.com/session/patterns-and-anti-patterns-running-apache-bigdata-projects-kubernetes
> >> >><
> >> >>
> >>
> https://aceu19.apachecon.com/session/open-source-big-data-tools-accelerating-physics-research-cern
> >> >>>
> >> >>
> >> >>  *
> >> >>
> >> >>
> >> >>
> >>
> https://aceu19.apachecon.com/session/open-source-big-data-tools-accelerating-physics-research-cern
> >> >><
> >> >>
> >>
> https://aceu19.apachecon.com/session/ui-dev-big-data-world-using-open-source
> >> >>>
> >> >>
> >> >>  *
> >> >>
> >> >>
> >> >>
> >>
> https://aceu19.apachecon.com/session/ui-dev-big-data-world-using-open-source
> >> >>
> >> >>Furthermore there will be a whole conference track on community
> >> >> topics:
> >> >>Learn how to motivate users to contribute patches, how the board
> of
> >> >>directors works, how to navigate the Incubator and much more:
> >> >> ApacheCon
> >> >>Europe 2019 Community track <
> >> >> https://aceu19.apachecon.com/sessions?track=42>
> >> >>
> >> >>Tickets are available here <
> >> https://aceu19.apachecon.com/registration>
> >> >> –
> >> >>for Apache Committers we offer discounted tickets.  Prices will be
> >> >> going
> >> >>up on October 7th, so book soon.
> >> >>
> >> >>Please also help spread the word and make ApacheCon Europe 2019 a
> >> >> success!
> >> >>
> >> >>We’re looking forward to welcoming you at #ACEU19!
> >> >>
> >> >>Best,
> >> >>
> >> >>Your ApacheCon team
> >> >>
> >> >>
> >> >>
> >>
> >>
>

Re: Different performance of the same code in two laptops

[Enrico Olivelli]

Can you share the differences between the two machines? RAM, CPU, OS

As far as I know internal java ergonomics make code run very differently
depending on available resources


Enrico

Il mar 15 ott 2019, 15:40 Stamatis Zampetakis  ha
scritto:

> Hi Makis,
>
> Use a profiler (e.g., VisualVM) and check where the time goes. You can also
> compare the snapshots between the two machines.
>
> Best,
> Stamatis
>
> On Tue, Oct 15, 2019 at 3:08 PM Michael Mior  wrote:
>
> > I think it's going to be hard for anyone to give you specific advice
> > without more information. How are you measuring the runtime? You may
> > need to instrument the Calcite code, but I would suggest breaking down
> > the runtime more to find out where the slowdown is happening. On a
> > related note, the optimization process is single-threaded, so this
> > would not be your problem.
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le mar. 15 oct. 2019 à 06:11, Serafeim (Makis) Papadias
> >  a écrit :
> > >
> > > Hello everyone,
> > >
> > > I am experimenting with rules and volcano optimiser lately and I am
> > facing something weird. I run my code through Intellij IDE. The databases
> > are set the same way in both machines and since I care only about query
> > rewriting for now I just read the table schemas from the databases; I do
> > not execute anything.
> > >
> > > The same code on one machine runs 10 times slower than on the other
> > machine.
> > >
> > > * I tried to rule out that the IDE is not the problem, so I installed
> > the same versions and picked the exact same vmoptions.
> > > * The database schemas are the same, thus I do not believe it is
> related
> > with the underneath sources.
> > >
> > > Is it possible that calcite uses all the threads available on one
> > machine while running only on thread the other?
> > >
> > > If you have any recommendations, they are more that welcome!
> > >
> > > Thanks in advance.
> > >
> > > Best,
> > > Makis
> >
>

Hotfix release for CALCITE-3347 ?

[Enrico Olivelli]

Hello Calciters,
in HerdDB community we are have recently upgraded Calcite to 1.21.0 but we
are getting feedback from downstream project about a bad problem with
subqueries with bind variables.

The issue is tracked as CALCITE-3347
https://issues.apache.org/jira/browse/CALCITE-3347

This is the issue on HerdDB bug tracker with the details
https://github.com/diennea/herddb/issues/479

Current master of Calcite 1.22.0-SNAPSHOT works perfectly.

Is there already a plan for 1.22.0 or a perhaps an 1.21.1 hotfix ?

Best regards
Enrico

Re: Trivial query simplification

[Enrico Olivelli]

Il mer 25 set 2019, 23:38 Stamatis Zampetakis  ha
scritto:

> Hi Enrico,
>
> The ReduceExpressionsRule.FILTER_INSTANCE is using the simplifier so if it
> works
> correctly I don't think there is anything more to be done.
>

Fine.

Cheers
Enrico

>
> Best,
> Stamatis
>
> On Wed, Sep 25, 2019 at 5:31 PM Enrico Olivelli 
> wrote:
>
> > Thank you for your feedback.
> >
> > Actually ReduceExpressionsRule.FILTER_INSTANCE fixes the problem.
> >
> > RelOptPlanner optPlanner = cluster.getPlanner();
> > optPlanner.addRule(ReduceExpressionsRule.FILTER_INSTANCE);
> >
> > This code I had was wrong:
> >  final FrameworkConfig config = Frameworks.newConfigBuilder()
> > .parserConfig(SQL_PARSER_CONFIG)
> > .defaultSchema(subSchema)
> > .traitDefs(TRAITS)
> > .programs(Programs.ofRules(Programs.RULE_SET))
> > .build();
> >
> > as the 'programs' property of FrameworkConfig is applied only if you are
> > using Planner#transform .
> >
> > @Julian do I have to create a JIRA case for RexSimplify ? Honestly I
> don't
> > have enough knowledge to explain the problem and create a meaning full
> > issue.
> >
> > Is it work to add ReduceExpressionsRule.FILTER_INSTANCE to the default
> > rules ?
> >
> > As far as I can see the "default rules" are in CalcitePrepareImpl, is
> that
> > correct?
> >
> >
> > Cheers
> > Enrico
> >
> >
> > Il giorno mar 24 set 2019 alle ore 20:01 Julian Hyde 
> ha
> > scritto:
> >
> > > A few thoughts on this.
> > >
> > >
> > > 0. I am surprised that this simplification does not happen, and I think
> > we
> > > should do it. Specifically, "filter(x = 0 and x is not null)" should
> > > simplify to “filter(x = 0)”.
> > >
> > > 1. Enrico, please log a JIRA case now, and transcribe the key points of
> > > this discussion into the JIRA case when the discussion has finished.
> > >
> > > 2. The most obvious way to achieve this is via simplification, i.e.
> > > RexSimplify, which occurs when building expressions via RelBuilder. It
> > does
> > > not require planner rules.
> > >
> > > 3. An algorithm to achieve this would be to gather the implied
> predicates
> > > as we go through a conjunction. After generating the condition “x = 0”
> we
> > > arrive at “x is not null”. We know that “x = 0” holds, therefore we
> could
> > > also deduce that “x is not null” holds (we could also deduce other
> > > conditions, such as “x < 100” holds).
> > >
> > > 4. Another way to achieve this is via
> > > ReduceExpressionsRule.FILTER_INSTANCE. The algorithm would be as
> follows.
> > > First note the condition “x = 0” and therefore constant-reduce x to 0
> in
> > > the expression “0 is not null”. This algorithm has similar problems to
> > the
> > > algorithm in 2 - it passes along the conjunctive predicates in strict
> > order
> > > and therefore only works if they are in a particular order. Also, as a
> > > planner rule, this algorithm is more expensive, so would not be applied
> > as
> > > early/often as the RexSimplify implementation.
> > >
> > > 5. We could also simplify “filter(x is not null and x = 0)” to
> “filter(x
> > =
> > > 0)”, and people would reasonably expect that we would. But that is a
> more
> > > complex algorithm because it would require, for instance, re-ordering
> > > predicates. In the past, we have discussed re-ordering predicates as
> part
> > > of simplification; I am cautious about doing it because it would
> affect a
> > > lot of existing plans (and tests). There is no perfect order for
> > > predicates, so we might come back in 6 months and want to change the
> > order
> > > again. Better to sort them during simplification but then spit them out
> > in
> > > the original order.
> > >
> > > Julian
> > >
> > >
> > >
> > > > On Sep 24, 2019, at 8:34 AM, Enrico Olivelli 
> > > wrote:
> > > >
> > > > Il giorno mar 24 set 2019 alle ore 13:45 XING JIN <
> > > jinxing.co...@gmail.com <mailto:jinxing.co...@gmail.com>>
> > > > ha scritto:
> > > >
> > > >> "v = 1 and v is null"
> > > >> cannot be simplified to "v = 1" not matter v is nullable or not
> > nu

Re: Trivial query simplification

[Enrico Olivelli]

Thank you for your feedback.

Actually ReduceExpressionsRule.FILTER_INSTANCE fixes the problem.

RelOptPlanner optPlanner = cluster.getPlanner();
optPlanner.addRule(ReduceExpressionsRule.FILTER_INSTANCE);

This code I had was wrong:
 final FrameworkConfig config = Frameworks.newConfigBuilder()
.parserConfig(SQL_PARSER_CONFIG)
.defaultSchema(subSchema)
.traitDefs(TRAITS)
.programs(Programs.ofRules(Programs.RULE_SET))
.build();

as the 'programs' property of FrameworkConfig is applied only if you are
using Planner#transform .

@Julian do I have to create a JIRA case for RexSimplify ? Honestly I don't
have enough knowledge to explain the problem and create a meaning full
issue.

Is it work to add ReduceExpressionsRule.FILTER_INSTANCE to the default
rules ?

As far as I can see the "default rules" are in CalcitePrepareImpl, is that
correct?


Cheers
Enrico


Il giorno mar 24 set 2019 alle ore 20:01 Julian Hyde  ha
scritto:

> A few thoughts on this.
>
>
> 0. I am surprised that this simplification does not happen, and I think we
> should do it. Specifically, "filter(x = 0 and x is not null)" should
> simplify to “filter(x = 0)”.
>
> 1. Enrico, please log a JIRA case now, and transcribe the key points of
> this discussion into the JIRA case when the discussion has finished.
>
> 2. The most obvious way to achieve this is via simplification, i.e.
> RexSimplify, which occurs when building expressions via RelBuilder. It does
> not require planner rules.
>
> 3. An algorithm to achieve this would be to gather the implied predicates
> as we go through a conjunction. After generating the condition “x = 0” we
> arrive at “x is not null”. We know that “x = 0” holds, therefore we could
> also deduce that “x is not null” holds (we could also deduce other
> conditions, such as “x < 100” holds).
>
> 4. Another way to achieve this is via
> ReduceExpressionsRule.FILTER_INSTANCE. The algorithm would be as follows.
> First note the condition “x = 0” and therefore constant-reduce x to 0 in
> the expression “0 is not null”. This algorithm has similar problems to the
> algorithm in 2 - it passes along the conjunctive predicates in strict order
> and therefore only works if they are in a particular order. Also, as a
> planner rule, this algorithm is more expensive, so would not be applied as
> early/often as the RexSimplify implementation.
>
> 5. We could also simplify “filter(x is not null and x = 0)” to “filter(x =
> 0)”, and people would reasonably expect that we would. But that is a more
> complex algorithm because it would require, for instance, re-ordering
> predicates. In the past, we have discussed re-ordering predicates as part
> of simplification; I am cautious about doing it because it would affect a
> lot of existing plans (and tests). There is no perfect order for
> predicates, so we might come back in 6 months and want to change the order
> again. Better to sort them during simplification but then spit them out in
> the original order.
>
> Julian
>
>
>
> > On Sep 24, 2019, at 8:34 AM, Enrico Olivelli 
> wrote:
> >
> > Il giorno mar 24 set 2019 alle ore 13:45 XING JIN <
> jinxing.co...@gmail.com <mailto:jinxing.co...@gmail.com>>
> > ha scritto:
> >
> >> "v = 1 and v is null"
> >> cannot be simplified to "v = 1" not matter v is nullable or not nullable
> >>
> >> If you really mean that "v is not null", I made below test case in
> >> RelOptRulesTest.java for illustration:
> >>
> >>
> >> // mgr is nullable
> >>  @Test public void testDEV() throws Exception {
> >>HepProgram program = new HepProgramBuilder()
> >>  .addRuleInstance(ReduceExpressionsRule.FILTER_INSTANCE)
> >>  .build();
> >>
> >>final String sql = "select deptno"
> >>  + " from emp"
> >>  + " where mgr = 10 and mgr is not null";
> >>checkPlanning(new HepPlanner(program), sql);
> >>  }
> >>
> >> The plan is
> >> LogicalProject(DEPTNO=[$7])
> >>  LogicalFilter(condition=[=($3, 10)])
> >>LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> >>
> >> Enrico ~ you may try ReduceExpressionsRule.FILTER_INSTANCE
> >>
> >
> >
> > @XING JIN
> > thank you.
> > I am with Calcite 1.19 and VolcanoPlanner
> >
> > Original query:
> >
> > select * from pippo where n1 is null AND n1 is not null
> >
> > Logical plan:
> >
> > LogicalFilter(condition=[AND(IS NULL($1), IS NOT NULL($1))]): rowcount =
> &

Re: Trivial query simplification

[Enrico Olivelli]

Il giorno mar 24 set 2019 alle ore 13:45 XING JIN 
ha scritto:

> "v = 1 and v is null"
> cannot be simplified to "v = 1" not matter v is nullable or not nullable
>
> If you really mean that "v is not null", I made below test case in
> RelOptRulesTest.java for illustration:
>
>
> // mgr is nullable
>   @Test public void testDEV() throws Exception {
> HepProgram program = new HepProgramBuilder()
>   .addRuleInstance(ReduceExpressionsRule.FILTER_INSTANCE)
>   .build();
>
> final String sql = "select deptno"
>   + " from emp"
>   + " where mgr = 10 and mgr is not null";
> checkPlanning(new HepPlanner(program), sql);
>   }
>
> The plan is
> LogicalProject(DEPTNO=[$7])
>   LogicalFilter(condition=[=($3, 10)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
>
> Enrico ~ you may try ReduceExpressionsRule.FILTER_INSTANCE
>


@XING JIN
thank you.
I am with Calcite 1.19 and VolcanoPlanner

Original query:

select * from pippo where n1 is null AND n1 is not null

Logical plan:

LogicalFilter(condition=[AND(IS NULL($1), IS NOT NULL($1))]): rowcount =
1.35, cumulative cost = {7.35 rows, 13.0 cpu, 0.0 io}, id = 48

EnumerableTableScan(table=[[herd, pippo]]): rowcount = 6.0, cumulative
cost = {6.0 rows, 7.0 cpu, 0.0 io}, id = 47

Final Plan:

BindableTableScan(table=[[herd, pippo]], filters=[[AND(IS NULL($1), IS NOT
NULL($1))]]): rowcount = 6.0, cumulative cost = {0.03 rows, 0.035 cpu, 0.0
io}, id = 59


It seems that ReduceExpressionsRule.FILTER_INSTANCE does not have any
effect.
May it be a problem of 1.19 or VolcanoPlanner ?

This is my "program" now:

 public static final ImmutableSet RULE_SET =
  ImmutableSet.of(
  EnumerableRules.ENUMERABLE_JOIN_RULE,
  EnumerableRules.ENUMERABLE_MERGE_JOIN_RULE,
  EnumerableRules.ENUMERABLE_SEMI_JOIN_RULE,
  EnumerableRules.ENUMERABLE_CORRELATE_RULE,
  EnumerableRules.ENUMERABLE_PROJECT_RULE,
  EnumerableRules.ENUMERABLE_FILTER_RULE,
  EnumerableRules.ENUMERABLE_AGGREGATE_RULE,
  EnumerableRules.ENUMERABLE_SORT_RULE,
  EnumerableRules.ENUMERABLE_LIMIT_RULE,
  EnumerableRules.ENUMERABLE_UNION_RULE,
  EnumerableRules.ENUMERABLE_INTERSECT_RULE,
  EnumerableRules.ENUMERABLE_MINUS_RULE,
  EnumerableRules.ENUMERABLE_TABLE_MODIFICATION_RULE,
  EnumerableRules.ENUMERABLE_VALUES_RULE,
  EnumerableRules.ENUMERABLE_WINDOW_RULE,
  SemiJoinRule.PROJECT,
  SemiJoinRule.JOIN,
  TableScanRule.INSTANCE,
  CalciteSystemProperty.COMMUTE.value()
  ? JoinAssociateRule.INSTANCE
  : ProjectMergeRule.INSTANCE,
  AggregateStarTableRule.INSTANCE,
  AggregateStarTableRule.INSTANCE2,
  FilterTableScanRule.INSTANCE,
  FilterProjectTransposeRule.INSTANCE,
  FilterJoinRule.FILTER_ON_JOIN,
  AggregateExpandDistinctAggregatesRule.INSTANCE,
  AggregateReduceFunctionsRule.INSTANCE,
  FilterAggregateTransposeRule.INSTANCE,
  JoinCommuteRule.INSTANCE,
  JoinPushThroughJoinRule.RIGHT,
  JoinPushThroughJoinRule.LEFT,
  SortProjectTransposeRule.INSTANCE,
  ReduceExpressionsRule.FILTER_INSTANCE);<-- HERE

Enrico



>
> Feng Zhu  于2019年9月24日周二 下午5:50写道：
>
> > Hi, Enrico,
> > I'm a little confused about your expectations. Could you clarify it?
> > Moreover, is it right for the below simplification (do you mean v is not
> > null)?
> > (v=1 and v is null) -> v=1
> > (do you mean v is not null?)
> >
> > Best regards
> >
> > Enrico Olivelli  于2019年9月24日周二 下午5:41写道：
> >
> > > Hi,
> > > I have a query like
> > > SELECT * FROM MYTABLE WHERE v = 1 and v is null
> > >
> > > I am expecting Calcite to simplify it to
> > > SELECT * FROM MYTABLE WHERE v = 1
> > >
> > > but this does not happen.
> > >
> > > Is any rule I should enable in order to make it happen ?
> > >
> > > This is the configuration of my Volcano planner:
> > >
> > >   final FrameworkConfig config = Frameworks.newConfigBuilder()
> > > .parserConfig()
> > > .defaultSchema(...)
> > > .traitDefs()
> > > .programs(Programs.ofRules(Programs.RULE_SET))
> > > .build();
> > >
> > > Best regards
> > > Enrico
> > >
> >
>

Re: Trivial query simplification

[Enrico Olivelli]

Il mar 24 set 2019, 11:50 Feng Zhu  ha scritto:

> Hi, Enrico,
> I'm a little confused about your expectations. Could you clarify it?
> Moreover, is it right for the below simplification (do you mean v is not
> null)?
> (v=1 and v is null) -> v=1
> (do you mean v is not null?)
>


Yes, sorry

Enrico

>
> Best regards
>
> Enrico Olivelli  于2019年9月24日周二 下午5:41写道：
>
> > Hi,
> > I have a query like
> > SELECT * FROM MYTABLE WHERE v = 1 and v is null
> >
> > I am expecting Calcite to simplify it to
> > SELECT * FROM MYTABLE WHERE v = 1
> >
> > but this does not happen.
> >
> > Is any rule I should enable in order to make it happen ?
> >
> > This is the configuration of my Volcano planner:
> >
> >   final FrameworkConfig config = Frameworks.newConfigBuilder()
> > .parserConfig()
> > .defaultSchema(...)
> > .traitDefs()
> > .programs(Programs.ofRules(Programs.RULE_SET))
> > .build();
> >
> > Best regards
> > Enrico
> >
>

Trivial query simplification

[Enrico Olivelli]

Hi,
I have a query like
SELECT * FROM MYTABLE WHERE v = 1 and v is null

I am expecting Calcite to simplify it to
SELECT * FROM MYTABLE WHERE v = 1

but this does not happen.

Is any rule I should enable in order to make it happen ?

This is the configuration of my Volcano planner:

  final FrameworkConfig config = Frameworks.newConfigBuilder()
.parserConfig()
.defaultSchema(...)
.traitDefs()
.programs(Programs.ofRules(Programs.RULE_SET))
.build();

Best regards
Enrico

Problems with Subqueries and simple aggregations on Calcite 1.20

[Enrico Olivelli]

Hello,
We are upgrading HerdDB from Calcite 1.19 to 1.20 but we are falling into
serious issues regarding subqueries and aggregations on very simple queries.

I am really sorry I did not have time to test 1.20 before the release this
time.

I will create JIRAs but maybe someone has already an idea of which change
could have broken these cases.

HerdDB is only using Calcite planner, just by configuring it with only
standard rules (no custom rules, no custom traits/convensions, only
EnumerableConvention), basic configuration and an implementation of the
Schema of the tables.


Problem 1: SELECT MIN(n1) as mi, MAX(n1) as ma FROM tblspace1.tsql

Result:

java.lang.IllegalArgumentException: source #1 is already mapped to target #1
at 
org.apache.calcite.util.mapping.Mappings$SurjectionWithInverse.set(Mappings.java:1326)
at org.apache.calcite.rel.core.Project.getMapping(Project.java:279)
at org.apache.calcite.rel.core.Project.getMapping(Project.java:250)
at 
org.apache.calcite.rel.rules.ProjectTableScanRule.apply(ProjectTableScanRule.java:107)
at 
org.apache.calcite.rel.rules.ProjectTableScanRule$2.onMatch(ProjectTableScanRule.java:83)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
... 30 more



Problem 2 (subquery in DML) : UPDATE tblspace1.table1 set n1=1000 WHERE k1
in (SELECT fk FROM tblspace1.table2 WHERE k2=?)


class org.apache.calcite.sql.SqlBasicCall: `K1` IN (SELECT `table2`.`fk` AS `FK`
FROM `tblspace1`.`table2` AS `TABLE2`
WHERE `table2`.`k2` = ?)
at org.apache.calcite.util.Util.needToImplement(Util.java:967)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1579)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1802)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1776)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.replaceSubQueries(SqlToRelConverter.java:1011)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertUpdate(SqlToRelConverter.java:3570)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3172)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:563)
at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:254)
at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:506)
at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:293)
at herddb.core.TestUtils.executeUpdate(TestUtils.java:43)


You can find the details in this PR
https://github.com/diennea/herddb/pull/404

Thank you in advance
Enrico

Re: [DISCUSS] Towards Calcite 1.21.0

[Enrico Olivelli]

Stamatis,
We are trying to upgrade HerdDB to 1.20 but we are stuck into a little
problem with simple aggregations [1]
I will try to give feedback as it is possible, if there is a problem in
Calcite I will be happy to try to report it before 1.21. I hope it is only
a problem on HerdDB


Cheers
Enrico

[1] https://github.com/diennea/herddb/pull/404

Il dom 21 lug 2019, 17:49 Julian Hyde  ha scritto:

> +1 for release at end of August.
>
> +1 to figure out which PRs could make it into the release, and work on
> that list in FIFO order. We tend to merge PRs that are in good shape, and
> by active contributors, and we tend to ignore old PRs that are never going
> to make it, but we tend to ignore the important PRs in the middle.
>
> Julian
>
> > On Jul 21, 2019, at 2:28 AM, Stamatis Zampetakis 
> wrote:
> >
> > Hi all,
> >
> > Approximately one month has passed from the previous release (Calcite
> > 1.20.0) and I was thinking that it would be nice to have the next release
> > out by the end of August. To do this I think we should try to have an RC
> > around the 20 of August.
> >
> > The progress towards the next release can be seen in [1]. As you can
> > observe we do not have many issues marked for fixing for 1.21.0 but we do
> > have many pull requests.
> >
> > I would like to invite committers to go over the existing PRs and for
> those
> > that we plan to review and finalize in the next month or so set the fix
> > version to 1.21.0 adding an appropriate comment.
> >
> > In term of priorities, I think we should emphasize on finalizing and
> > merging PRs that are blocking other open source projects and commercial
> > products from upgrading to the latest release. If it is not already done
> > please assign this issues the highest priority (blocker) and set the fix
> > version to 1.21.0.
> >
> > Apart from very important issues it makes sense to treat PRs in FIFO
> order.
> > Contributors who submit a PR early will certainly get discouraged to
> > contribute again if we never merge these PRs in time.
> >
> > Don't hesitate to reply to this thread if the plan above is not
> convenient
> > for you!
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
>

Re: [ANNOUNCE] New committer: Danny Chan

[Enrico Olivelli]

Congratulations!

Enrico

Il mer 15 mag 2019, 04:11 Rong Rong  ha scritto:

> Congratulations Danny!
>
> On Tue, May 14, 2019 at 2:45 PM Stamatis Zampetakis 
> wrote:
>
> > Congrats Danny, and thanks a lot for all the effort you are putting in
> > helping others and improving Calcite.
> >
> > Among so many high quality contributions it is difficult to express
> > preferences, nevertheless I think your work on
> > n implicit type casting (CALCITE-2302) is going to benefit many people.
> >
> > Welcome and keep up the good work.
> >
> > Best,
> > Stamatis
> >
> >
> > On Tue, May 14, 2019, 9:01 PM Andrei Sereda  wrote:
> >
> > > Congratulations!
> > >
> > > On Tue, May 14, 2019 at 10:14 AM Michael Mior 
> wrote:
> > >
> > > > Congratulations and thank you Danny!
> > > > --
> > > > Michael Mior
> > > > mm...@apache.org
> > > >
> > > > Le lun. 13 mai 2019 à 21:57, Yuzhao Chen  a
> > écrit
> > > :
> > > > >
> > > > > Thank you everyone for your kind messages.
> > > > >
> > > > > Currently I am working in Alibaba Blink SQL Engine team in
> Hangzhou,
> > > > Zhejiang, China. We are developing a production version of Apache
> > Flink.
> > > > Our team has done many promotions for flink-table module and recently
> > we
> > > > are merging and contributing our codebase to the Apache Flink
> > community.
> > > > >
> > > > > It is my honor to be part of Apache Calcite community, I will
> > > contribute
> > > > continuously to this great project.
> > > > >
> > > > > Best,
> > > > > Danny Chan
> > > > > 在 2019年5月14日 +0800 AM6:40，Francis Chuang  > > >，写道：
> > > > > > Apache Calcite's Project Management Committee (PMC) has invited
> > Danny
> > > > > > Chan to become a committer, and we are pleased to announce that
> he
> > > has
> > > > > > accepted.
> > > > > >
> > > > > > Danny has been a prolific contributor to Calcite, with
> CALCITE-2969
> > > > > > being one of his more complex contributions to date. He has also
> > been
> > > > > > extremely active on our mailing lists, contributing to many
> design
> > > > > > discussions.
> > > > > >
> > > > > > Danny, welcome, thank you for your contributions, and we look
> > forward
> > > > > > your further interactions with the community! If you wish, please
> > > feel
> > > > > > free to tell us more about yourself and what you are working on.
> > > > > >
> > > > > > Francis (on behalf of the Apache Calcite PMC)
> > > >
> > >
> >
>

Re: [ANNOUNCE] Stamatis Zampetakis joins Calcite PMC

[Enrico Olivelli]

Congrats !

Enrico

Il giorno dom 28 apr 2019 alle ore 00:08 Vineet Garg
 ha scritto:
>
> Congratulations Stamatis!
>
> On Fri, Apr 26, 2019 at 7:44 PM Francis Chuang 
> wrote:
>
> > I'm pleased to announce that Stamatis has accepted an invitation to
> > join the Calcite PMC. Stamatis has been a consistent and helpful
> > figure in the Calcite community for which we are very grateful. We
> > look forward to the continued contributions and support.
> >
> > Please join me in congratulating Stamatis!
> >
> > - Francis (on behalf of the Calcite PMC)
> >

Re: [VOTE] Release apache-calcite-1.19.0 (release candidate 2)

[Enrico Olivelli]

great work Kevin

+1 (non binding)
checked signatures/checksum
built from source on JDK11 (AdoptOpenJDK 11.0.2) on Linux, all tests passed.

all tests of downstream project HerdDB passed, 100% compatible,
upgrading from 1.17.0

Regards
Enrico


Il giorno mer 20 mar 2019 alle ore 22:58 Francis Chuang
 ha scritto:
>
> +1 (binding)
>
> Environment: maven:latest docker image (Maven 3.6.0, OpenJDK 11.0.2,
> Debian stretch).
>
> Verified SHA256 hash - OK
> Verified GPG signature - OK
> Ran tests using ./mvnw -DskipTests clean install and ./mvnw test - OK
> Checked history.md - OK
>
> Francis
>
> On 21/03/2019 5:05 am, Kevin Risden wrote:
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.19.0, release candidate 2.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > Since RC 1, we have fixed the following issues:
> > * [CALCITE-2929] Simplification of IS NULL checks are incorrectly assuming
> > that CAST-s are
> > possible
> > * [CALCITE-2931] Mongo Adapter- Compare Bson (not string) query
> > representation in tests
> > * [CALCITE-2932] Update stale Druid integration test cases
> >
> > You can read the release notes here:
> > https://github.com/apache/calcite/blob/branch-1.19/site/_docs/history.md
> >
> > The commit to be voted upon:
> > https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=4143176acdb2860b3a80eb18e4cb1557f5969d13
> >
> > Its hash is 4143176acdb2860b3a80eb18e4cb1557f5969d13.
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.19.0-rc2/
> >
> > The hashes of the artifacts are as follows:
> > src.tar.gz.sha256
> > 833fa3e9c97d8e89443f3b7a62fc6cce5f0e734836d82db1443425be6ac5dc65
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachecalcite-1057/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/krisden.asc
> >
> > Please vote on releasing this package as Apache Calcite 1.19.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of
> > at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.19.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> > +1 (binding)
> >
> > Kevin Risden
> >

Re: new RuntimeException(Throwable) vs SneakyThrows

[Enrico Olivelli]

Vladimir,

Il giorno mer 13 feb 2019 alle ore 21:49 Vladimir Sitnikov
 ha scritto:
>
> Hi,
>
> I've recently discovered that new RuntimeException(Throwable) results in a
> duplicated error messages.
> In fact Throwable#(Throwable) just calls cause.getMessage() and uses
> it as its own message.
>
> This makes exceptions harder to follow since the new RuntimeException(e) is
> very often used just to overcome "checked exception" rule in Java.
>
> I've recently committed a deduplicate PR to Avatica:
> https://github.com/apache/calcite-avatica/pull/84
> As you can see, it removed "RuntimeException: ..." nonsense and it made
> exceptions easier to understand. For instance, see
> https://github.com/apache/calcite-avatica/pull/84/files#diff-d499001f31481c5f7151a81380c01a60L332
>
> I think we should avoid new RuntimeException(Throwable), and I wonder if we
> should add that as a forbiddenapis rule.
>
> You can find a PR for *test* code:
> https://github.com/apache/calcite/pull/1042
>
> I'm not sure if I should just go ahead and dodge new
> RuntimeException(Throwable) from the main codebase as well.
> Do you have any negative experience with sneaky-throws pattern?
> I'm inclined to do so provided no-one objects within a week.

I think sneaky-throws works well in your case, I mean to replace new
RuntimeException(Throwable).
The only problem is that users will find checked exception thrown in
places where they aren't supposed to be thrown,
anyway using RuntimeException is weird as well.

So +1 from me

just my 2 cents
Enrico

>
> Vladimir

Re: Calcite Plan Caching

[Enrico Olivelli]

Paul,

Il giorno mer 6 feb 2019, 23:02 Paul Trepagnier  ha
scritto:

> I am using calcite right now to test some data federation scenarios.  I
> have several collections of Java objects that I am querying right now.  It
> works beautifully, but it seems like Calcite does a ton of work with each
> query and does not cache any of the work.
>
> Is there a way for me to implement a plan cache within Calcite?  Or is
> there already a mechanism for this?
>

Are you willing to cache Calcite RelNodes directly?
In one of my projects I am not caching them directly but immutable
structures created by visiting RelNodes.

Enrico


> I tested implementing my own subclass of PreparedStatementFactory and
> caching the CalciteSignature for calls to prepareSql.  This seems to work
> really well, but I am not sure if 1) this is safe to do and 2) is this the
> proper way to do it.
>
> Thanks for your help,
>
> Pau;
>

Re: [ANNOUNCE] New committer: Stamatis Zampetakis

[Enrico Olivelli]

Kudos !

Enrico

Il giorno mer 30 gen 2019, 19:27 Julian Hyde  ha scritto:

> Welcome, Stamatis! In addition to your code contributions, I have been
> appreciating your wise & moderating contributions to technical discussions,
> and answers to people’s questions on the dev list.
>
> Julian
>
>
> > On Jan 30, 2019, at 10:01 AM, Jesus Camacho Rodriguez <
> jcama...@apache.org> wrote:
> >
> > Apache Calcite's Project Management Committee (PMC) has invited
> > Stamatis Zampetakis to become a committer, and we are pleased to
> > announce that he has accepted.
> >
> > Over the past few months, Stamatis has made several contributions to
> > Calcite and he is a very active participant in discussions in issues
> > and mailing lists.
> >
> > Stamatis, welcome, thank you for your contributions, and we look
> > forward your further interactions with the community! If you wish,
> > please feel free to tell us more about yourself and what you are
> > working on.
> >
> > Jesús (on behalf of the Apache Calcite PMC)
>
>

Re: [ANNOUNCE] New committer: Zoltan Haindrich

[Enrico Olivelli]

Congrats!

Enrico

Il giorno mer 30 gen 2019, 19:25 Julian Hyde  ha scritto:

> Welcome, Zoltan! You’ve already contributed plenty of great work, and I
> look forward to further contributions!
>
> Julian
>
>
> > On Jan 30, 2019, at 10:05 AM, Jesus Camacho Rodriguez <
> jcama...@apache.org> wrote:
> >
> > Apache Calcite's Project Management Committee (PMC) has invited Zoltan
> > Haindrich to become a committer, and we are pleased to announce that
> > he has accepted.
> >
> > Over the past few months, Zoltan has contributed many improvements and
> > fixes to core parts of the project related to query optimization.
> >
> > Zoltan, welcome, thank you for your contributions, and we look forward
> > your further interactions with the community! If you wish, please feel
> > free to tell us more about yourself and what you are
> > working on.
> >
> > Jesús (on behalf of the Apache Calcite PMC)
>
>

Re: [ANNOUNCE] New Calcite PMC chair: Francis Chuang

[Enrico Olivelli]

Kudos Francis !

Enrico

Il gio 20 dic 2018, 22:24 Francis Chuang  ha
scritto:

> Thank you, Julian, Michael and the Calcite community! I look forward to
> contributing more to avatica-go, avatica and calcite itself in the next
> year.
>
> Francis
>
> On 21/12/2018 6:35 am, Julian Hyde wrote:
> > Congratulations, Francis! Well deserved.
> >
> > Michael,
> >
> > Thank you for serving as chair. I especially appreciate what you did to
> welcome newcomers, resolve disputes and make us communicate better with
> each other. The Calcite community is stronger because of your efforts.
> >
> > Julian
> >
> >> On Dec 20, 2018, at 11:25 AM, Michael Mior  wrote:
> >>
> >> Calcite community members,
> >>
> >> I am pleased to announce that we have a new PMC chair and VP. I have
> >> resigned, and Francis was duly elected by the PMC and approved
> unanimously
> >> by the Board.
> >>
> >> Please join me in congratulating Francis!
> >>
> >> -Michael
> >>
> >> PS - This should not be indicative of a lack of continued interest in
> >> Calcite on my part, but rather a desire on behalf of the entire PMC
> (myself
> >> included) to have this responsibility regularly shared.
>
>
> --


-- Enrico Olivelli

Re: [VOTE] Release apache-calcite-1.18.0 (release candidate 2)

[Enrico Olivelli]

+1 (non binding)
built from source artifacts, run tests, all passed on Linux fedora 28
run tests of HerdDB, all is ok
checked signatures and checksums, all is ok

great work
Enrico

Il giorno mer 19 dic 2018 alle ore 13:32 Stamatis Zampetakis
 ha scritto:
>
> +1 (non binding)
>
> System: Ubuntu 18.04 LTS and jdk1.8.0.66
>
> -run mvn clean install on staged sources and git repo
> -checked signatures and checksums
> -went quickly over release note
> -run tests on downstream project (no regressions detected)
>
> Best,
> Stamatis
>
> Στις Τρί, 18 Δεκ 2018 στις 9:32 μ.μ., ο/η Michael Mior 
> έγραψε:
>
> > +1 (binding) Checked signature and hashes and compiled and ran tests.
> > Thanks Julian!
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > Le mar. 18 déc. 2018 à 15:00, Julian Hyde  a écrit :
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.18.0, release candidate 2.
> > >
> > > Since the previous release candidate (RC1), issues
> > > https://issues.apache.org/jira/browse/CALCITE-2730 and
> > > https://issues.apache.org/jira/browse/CALCITE-2731 have
> > > been fixed, and Zoltan indicates that the Hive test suite now passes.
> > >
> > > Thanks to everyone who has contributed to this release.
> > > You can read the release notes here:
> > > https://github.com/apache/calcite/blob/branch-1.18/site/_docs/history.md
> > >
> > > The commit to be voted upon:
> > >
> > >
> > http://git-wip-us.apache.org/repos/asf/calcite/commit/6bca0b80859e86ac41ba1e342460db929cf72268
> > >
> > > Its hash is 6bca0b80859e86ac41ba1e342460db929cf72268.
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.18.0-rc2
> > >
> > > The hashes of the artifacts are as follows:
> > > src.tar.gz.sha256
> > > a04bfb1bac830e57475dffdbf5f0c3a797c358460d240f6203c929bac61b47ae
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachecalcite-1051
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/jhyde.asc
> > >
> > > Please vote on releasing this package as Apache Calcite 1.18.0.
> > >
> > > The vote is open for the next 72 hours (until 12 noon Pacific on Friday)
> > > and passes if a majority of at least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.18.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > > Here is my vote:
> > >
> > > +1 (binding)
> > >
> > > Julian
> > >
> >

Re: [VOTE] Release apache-calcite-1.18.0 (release candidate 1)

[Enrico Olivelli]

+1 (non binding)
- checked signature and checksum
- built from source, all tests pass on OracleJDK8 on Linux
- all tests from downstream project (HerdDB) are passing

Still great work !
Thank you Julian for driving the release

Enrico


Il giorno ven 7 dic 2018 alle ore 02:44 Francis Chuang
 ha scritto:
>
> Thanks for making this available for voting, Julian!
>
> +1 (binding)
>
> Environment: Debian Sid, OpenJDK 11.0.1, maven 3.6.0 in a docker container
> - Verified GPG signature - OK
> - Verified SHA256 - OK
> - Ran "mvn -DskipTests clean install" - OK
> - Ran "mvn test" - OK
>
> I also tested using Alpine Linux:
> Environment: Alpine Linux 3.8, OpenJDK 8u181, maven 3.6.0 in a docker
> container
>
> - Verified GPG signature - OK
> - Verified SHA256 - OK
> - Ran "mvn -DskipTests clean install" - OK
> - Ran "mvn test" - FAILED
>
> The tests failed, but since OpenJDK 8 is not supported, this isn't an issue:
>
> [ERROR] Failures:
> [ERROR]   OsAdapterTest.testFiles:146
> Expected: "type=d\ntype=f"
>   but: was ""
> [INFO]
> [ERROR] Tests run: 59, Failures: 1, Errors: 0, Skipped: 23
> [INFO]
> [INFO]
> 
> [INFO] Reactor Summary for Calcite 1.18.0:
> [INFO]
> [INFO] Calcite  SUCCESS [
> 4.064 s]
> [INFO] Calcite Linq4j . SUCCESS [
> 10.991 s]
> [INFO] Calcite Core ... SUCCESS
> [09:26 min]
> [INFO] Calcite Babel .. SUCCESS [
> 6.333 s]
> [INFO] Calcite Cassandra .. SUCCESS
> [01:56 min]
> [INFO] Calcite Druid .. SUCCESS [
> 5.862 s]
> [INFO] Calcite Elasticsearch .. SUCCESS
> [01:10 min]
> [INFO] Calcite Examples ... SUCCESS [
> 0.200 s]
> [INFO] Calcite Example CSV  SUCCESS [
> 10.818 s]
> [INFO] Calcite Example Function ... SUCCESS [
> 5.801 s]
> [INFO] Calcite File ... SUCCESS [
> 13.449 s]
> [INFO] Calcite Geode .. SUCCESS [
> 18.890 s]
> [INFO] Calcite MongoDB  SUCCESS [
> 19.340 s]
> [INFO] Calcite Pig  SUCCESS [
> 15.797 s]
> [INFO] Calcite Piglet . SUCCESS [
> 7.510 s]
> [INFO] Calcite Plus ... FAILURE [
> 15.401 s]
> [INFO] Calcite Server . SKIPPED
> [INFO] Calcite Spark .. SKIPPED
> [INFO] Calcite Splunk . SKIPPED
> [INFO] Calcite Ubenchmark . SKIPPED
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time:  14:47 min
> [INFO] Finished at: 2018-12-06T23:14:07Z
> [INFO]
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.22.0:test
> (default-test) on project calcite-plus: There are test failures.
> [ERROR]
> [ERROR] Please refer to
> /apache-calcite-1.18.0-src/plus/target/surefire-reports for the
> individual test results.
> [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
> [date].dumpstream and [date]-jvmRun[N].dumpstream.
> [ERROR] -> [Help 1]
>
> Francis
>
> On 6/12/2018 7:18 pm, Julian Hyde wrote:
> > OK, let's try again.
> >
> > I have created a build for Apache Calcite 1.18.0, release candidate 1.
> >
> > Thanks to everyone who has contributed to this release.
> > Since RC 0, we have fixed the following issues:
> >
> > * [CALCITE-2726] ReduceExpressionRule oversimplifies filter conditions
> > containing nulls
> > * [CALCITE-2670] Combine similar JSON aggregate functions in operator table
> > * [CALCITE-2468] Validator throws IndexOutOfBoundsException when
> > trying to infer operand type from STRUCT return type (Rong Rong)
> >
> > You can read the release notes here:
> > https://github.com/apache/calcite/blob/branch-1.18/site/_docs/history.md
> >
> > The commit to be voted upon:
> > http://git-wip-us.apache.org/repos/asf/calcite/commit/06d0526e9e58b88bb6722ae3e66789b9e60a70d1
> >
> > Its hash is 06d0526e9e58b88bb6722ae3e66789b9e60a70d1.
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.18.0-rc1
> >
> > The hashes of the artifacts are as follows:
> > src.tar.gz.sha256
> > 88501d9c22c5cd28c2988a1f921fa112386ac819fb1dbcc5f68504b30b8d7dd7
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachecalcite-1050
> >
> > 

Re: [VOTE] Release apache-calcite-1.18.0 (release candidate 0)

[Enrico Olivelli]

+1 (non binding)
- compiled from the staged sources and run tests on linux, all passed
- run tests of downstream project (HerdDB)
- checked signatures and checksums

great work !


Enrico
Il giorno mar 4 dic 2018 alle ore 10:00 Julian Hyde 
ha scritto:
>
> Hi all,
>
> I have created a build for Apache Calcite 1.18.0, release candidate 0.
>
> Thanks to everyone who has contributed to this release.
> With over 200 commits from 36 contributors, this is the largest
> Calcite release ever!
> You can read the release notes here:
> https://github.com/apache/calcite/blob/branch-1.18/site/_docs/history.md
>
> The commit to be voted upon:
> http://git-wip-us.apache.org/repos/asf/calcite/commit/c19e458d2bdbd42dbd450143e0f41fabbfe4ec06
>
> Its hash is c19e458d2bdbd42dbd450143e0f41fabbfe4ec06.
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.18.0-rc0
>
> The hashes of the artifacts are as follows:
> src.tar.gz.sha256
> 8931ccc08863e4fe394824ead306dc89a7a78a41f898303187a91b01a723cce6
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachecalcite-1049
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jhyde.asc
>
> Please vote on releasing this package as Apache Calcite 1.18.0.
>
> The vote is open for the next 72 hours (closing at 1am Pacific on Friday)
> and passes if a majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Calcite 1.18.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
> Julian

Re: calcite-avatica git commit: [CALCITE-2412] Add appveyor.yml to have tests on Windows against jdk1.8, jdk9, jdk10 Add Appveyor badge Add -DskipDockerCheck because of CALCITE-2385 and to make it syn

[Enrico Olivelli]

Il giorno lun 19 nov 2018 alle ore 22:41 Enrico Olivelli
 ha scritto:
>
> We can run the script in Travis, so that we will refuse any commit
> message not compliant with the "specs"
> In Travis you can run a shell script with uses directly the 'git' command

What about this github plugin ?
https://github.com/probot/semantic-pull-requests

Enrico

>
>
> Enrico
>
> Il giorno lun 19 nov 2018 alle ore 22:35 Francis Chuang
>  ha scritto:
> >
> > I am not sure about server-side hooks on ASF's git. It would require
> > execution privileges on ASF's git instance (for Github, this is not
> > allowed).
> >
> > For Windows, I use WSL (Windows Subsystem for Linux). This is a copy of
> > Ubuntu or some other supported Linux distro that runs within Windows and
> > allows the execution of Linux executables on Windows. WSL translates
> > calls to Windows native calls where needed. The downside is that WSL
> > only works on Windows 10 1607 (released august/july 2016) and later, so
> > people using Windows 8 and Windows 7 will not be able to use it.
> >
> > Francis
> >
> > On 20/11/2018 8:21 am, Julian Hyde wrote:
> > > Are we allowed to create hooks in ASF’s git?
> > >
> > > If we add a shell script, are we excluding Windows folks? (Or can we 
> > > require them to install Cygwin?)
> > >
> > >> On Nov 19, 2018, at 1:15 PM, Michael Mior  wrote:
> > >>
> > >> Sure, a shell script would work just fine as well. Really we just need to
> > >> create a symlink in the .git/hooks directory that points to a script 
> > >> which
> > >> performs whatever checks we want.
> > >> --
> > >> Michael Mior
> > >> mm...@apache.org
> > >>
> > >>
> > >> Le lun. 19 nov. 2018 à 16:11, Francis Chuang  a
> > >> écrit :
> > >>
> > >>> Will a shell script (instead of bringing in maven) running a regex check
> > >>> (our use-case seems simple enough for a regex check so far) be
> > >>> sufficient? This would avoid the need to set up a Java + maven
> > >>> environment to commit code (in my case, I prefer to run maven and Java
> > >>> in docker containers).
> > >>>
> > >>> Francis
> > >>>
> > >>> On 20/11/2018 8:00 am, Michael Mior wrote:
> > >>>> Sorry, that link should have been
> > >>>> https://github.com/phillipuniverse/githook-maven-plugin. Anyway, I 
> > >>>> don't
> > >>>> have experience with any particular plugin, but git hooks seem to be 
> > >>>> the
> > >>>> obvious way to go and that's the first one I found.
> > >>>>
> > >>>> --
> > >>>> Michael Mior
> > >>>> mm...@apache.org
> > >>>>
> > >>>>
> > >>>> Le lun. 19 nov. 2018 à 15:36, Enrico Olivelli  a
> > >>>> écrit :
> > >>>>
> > >>>>> Il lun 19 nov 2018, 21:29 Michael Mior  ha scritto:
> > >>>>>
> > >>>>>> How about making use of
> > >>>>> https://github.com/olukyrich/githook-maven-plugin?
> > >>>>>> A post-commit hooks in git seems to be an easy way to achieve this.
> > >>>>>> Unfortunately, it would require that each fresh clone of the 
> > >>>>>> repository
> > >>>>> has
> > >>>>>> a one-time command run to install the hook.
> > >>>>>>
> > >>>>> Michael,
> > >>>>> It seems this plugin is notnot available on maven central
> > >>>>> https://github.com/olukyrich/githook-maven-plugin
> > >>>>>
> > >>>>> Do you have experience with it?
> > >>>>> Enrico
> > >>>>>
> > >>>>>> --
> > >>>>>> Michael Mior
> > >>>>>> mm...@apache.org
> > >>>>>>
> > >>>>>>
> > >>>>>> Le lun. 19 nov. 2018 à 14:49, Julian Hyde  a écrit 
> > >>>>>> :
> > >>>>>>
> > >>>>>>>> On Nov 19, 2018, at 11:19 AM, Vladimir Sitnikov <
> > >>>>>>> sitnikov.vladi...@gmail.com> wrote:
> > >>>>>>>> Well, a rule of "first line should be separat

Re: calcite-avatica git commit: [CALCITE-2412] Add appveyor.yml to have tests on Windows against jdk1.8, jdk9, jdk10 Add Appveyor badge Add -DskipDockerCheck because of CALCITE-2385 and to make it syn

[Enrico Olivelli]

We can run the script in Travis, so that we will refuse any commit
message not compliant with the "specs"
In Travis you can run a shell script with uses directly the 'git' command


Enrico

Il giorno lun 19 nov 2018 alle ore 22:35 Francis Chuang
 ha scritto:
>
> I am not sure about server-side hooks on ASF's git. It would require
> execution privileges on ASF's git instance (for Github, this is not
> allowed).
>
> For Windows, I use WSL (Windows Subsystem for Linux). This is a copy of
> Ubuntu or some other supported Linux distro that runs within Windows and
> allows the execution of Linux executables on Windows. WSL translates
> calls to Windows native calls where needed. The downside is that WSL
> only works on Windows 10 1607 (released august/july 2016) and later, so
> people using Windows 8 and Windows 7 will not be able to use it.
>
> Francis
>
> On 20/11/2018 8:21 am, Julian Hyde wrote:
> > Are we allowed to create hooks in ASF’s git?
> >
> > If we add a shell script, are we excluding Windows folks? (Or can we 
> > require them to install Cygwin?)
> >
> >> On Nov 19, 2018, at 1:15 PM, Michael Mior  wrote:
> >>
> >> Sure, a shell script would work just fine as well. Really we just need to
> >> create a symlink in the .git/hooks directory that points to a script which
> >> performs whatever checks we want.
> >> --
> >> Michael Mior
> >> mm...@apache.org
> >>
> >>
> >> Le lun. 19 nov. 2018 à 16:11, Francis Chuang  a
> >> écrit :
> >>
> >>> Will a shell script (instead of bringing in maven) running a regex check
> >>> (our use-case seems simple enough for a regex check so far) be
> >>> sufficient? This would avoid the need to set up a Java + maven
> >>> environment to commit code (in my case, I prefer to run maven and Java
> >>> in docker containers).
> >>>
> >>> Francis
> >>>
> >>> On 20/11/2018 8:00 am, Michael Mior wrote:
> >>>> Sorry, that link should have been
> >>>> https://github.com/phillipuniverse/githook-maven-plugin. Anyway, I don't
> >>>> have experience with any particular plugin, but git hooks seem to be the
> >>>> obvious way to go and that's the first one I found.
> >>>>
> >>>> --
> >>>> Michael Mior
> >>>> mm...@apache.org
> >>>>
> >>>>
> >>>> Le lun. 19 nov. 2018 à 15:36, Enrico Olivelli  a
> >>>> écrit :
> >>>>
> >>>>> Il lun 19 nov 2018, 21:29 Michael Mior  ha scritto:
> >>>>>
> >>>>>> How about making use of
> >>>>> https://github.com/olukyrich/githook-maven-plugin?
> >>>>>> A post-commit hooks in git seems to be an easy way to achieve this.
> >>>>>> Unfortunately, it would require that each fresh clone of the repository
> >>>>> has
> >>>>>> a one-time command run to install the hook.
> >>>>>>
> >>>>> Michael,
> >>>>> It seems this plugin is notnot available on maven central
> >>>>> https://github.com/olukyrich/githook-maven-plugin
> >>>>>
> >>>>> Do you have experience with it?
> >>>>> Enrico
> >>>>>
> >>>>>> --
> >>>>>> Michael Mior
> >>>>>> mm...@apache.org
> >>>>>>
> >>>>>>
> >>>>>> Le lun. 19 nov. 2018 à 14:49, Julian Hyde  a écrit :
> >>>>>>
> >>>>>>>> On Nov 19, 2018, at 11:19 AM, Vladimir Sitnikov <
> >>>>>>> sitnikov.vladi...@gmail.com> wrote:
> >>>>>>>> Well, a rule of "first line should be separated by a blank line"
> >>>>> seems
> >>>>>> to
> >>>>>>>> be automatable.
> >>>>>>>> The rule of "CALCITE-XXX should be in [...]" seems to be automatable.
> >>>>>>>> And so on.
> >>>>>>> Yes, we should do that.
> >>>>>>>
> >>>>>>> However there are things that automation could never achieve, so let’s
> >>>>>>> continue to talk about those, also.
> >>>>>>>
> >>>>>>>> setDynamicParam did not look good enough to me
> >>>>>>>>
> >>> https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=blobdiff;f=core/src/main/java/org/apache/calcite/runtime/ResultSetEnumerable.java;h=52c11f9aaf752cea367ae921c04a20e5e6da5488;hp=771772f1ea39d74eee6e9f889ae8d6500f129c7f;hb=53e15af6c5e8e782b2edcd7f5bf4f5f32225d110;hpb=02ca9bc995cac5b4b97855a4d06df46e632d7c22
> >>>>>>>> I'm inclined to incline Avatica to expose
> >>>>>>>> TypedValue.setToPreparedStatement(PreparedStatement ps, int index)
> >>>>> kind
> >>>>>>> of
> >>>>>>>> API, so ResultSetEnumerable.setDynamicParam could be removed
> >>>>>> altogether.
> >>>>>>> I agree. The commit didn’t seem quite perfect to me either. However,
> >>> it
> >>>>>>> seemed to be progress. Log an Avatica JIRA if you have ideas for how
> >>> to
> >>>>>>> improve it further. Since it will be in Avatica it will take a while
> >>> to
> >>>>>>> bubble through the release cycle.
> >>>>>>>
> >>>>>>> Julian
> >>>>>>>
> >>>>>>>
> >>>>> --
> >>>>>
> >>>>>
> >>>>> -- Enrico Olivelli
> >>>>>
> >>>
>

Re: calcite-avatica git commit: [CALCITE-2412] Add appveyor.yml to have tests on Windows against jdk1.8, jdk9, jdk10 Add Appveyor badge Add -DskipDockerCheck because of CALCITE-2385 and to make it syn

[Enrico Olivelli]

Il lun 19 nov 2018, 21:29 Michael Mior  ha scritto:

> How about making use of https://github.com/olukyrich/githook-maven-plugin?
> A post-commit hooks in git seems to be an easy way to achieve this.
> Unfortunately, it would require that each fresh clone of the repository has
> a one-time command run to install the hook.
>

Michael,
It seems this plugin is notnot available on maven central
https://github.com/olukyrich/githook-maven-plugin

Do you have experience with it?
Enrico

>
> --
> Michael Mior
> mm...@apache.org
>
>
> Le lun. 19 nov. 2018 à 14:49, Julian Hyde  a écrit :
>
> >
> > > On Nov 19, 2018, at 11:19 AM, Vladimir Sitnikov <
> > sitnikov.vladi...@gmail.com> wrote:
> > >
> > > Well, a rule of "first line should be separated by a blank line" seems
> to
> > > be automatable.
> > > The rule of "CALCITE-XXX should be in [...]" seems to be automatable.
> > > And so on.
> >
> > Yes, we should do that.
> >
> > However there are things that automation could never achieve, so let’s
> > continue to talk about those, also.
> >
> > > setDynamicParam did not look good enough to me
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=blobdiff;f=core/src/main/java/org/apache/calcite/runtime/ResultSetEnumerable.java;h=52c11f9aaf752cea367ae921c04a20e5e6da5488;hp=771772f1ea39d74eee6e9f889ae8d6500f129c7f;hb=53e15af6c5e8e782b2edcd7f5bf4f5f32225d110;hpb=02ca9bc995cac5b4b97855a4d06df46e632d7c22
> > >
> > > I'm inclined to incline Avatica to expose
> > > TypedValue.setToPreparedStatement(PreparedStatement ps, int index) kind
> > of
> > > API, so ResultSetEnumerable.setDynamicParam could be removed
> altogether.
> >
> > I agree. The commit didn’t seem quite perfect to me either. However, it
> > seemed to be progress. Log an Avatica JIRA if you have ideas for how to
> > improve it further. Since it will be in Avatica it will take a while to
> > bubble through the release cycle.
> >
> > Julian
> >
> >
>
-- 


-- Enrico Olivelli

[jira] [Created] (CALCITE-2662) Planner: allow parsing directly a stream instead of a java.lang.String

[Enrico Olivelli (JIRA)]

Enrico Olivelli created CALCITE-2662:


 Summary: Planner: allow parsing directly a stream instead of a 
java.lang.String
 Key: CALCITE-2662
 URL: https://issues.apache.org/jira/browse/CALCITE-2662
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.17.0
Reporter: Enrico Olivelli
Assignee: Julian Hyde


In 1.17.0 the org.apache.calcite.tools.Planner interface only accept a 
java.lang.String as input.

In order to reduce memory allocations and copies it will be useful that the 
planner could accept the query in a more 'raw' format.

Creating a java.lang.String is very expensive, and if your "query" is coming 
from the network (think about a Netty Direct memory ByteBuf) you are forced to 
create a copy of the text of the query.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Help with EnumerableMergeJoinRule which is losing a RelCollection trait

[Enrico Olivelli]

Il sab 29 set 2018, 00:20 Stamatis Zampetakis  ha
scritto:

> Have a look in CalcitePrepareImpl (
>
> https://github.com/apache/calcite/blob/3a4fba828f3d9e48749fadd22f134891612b7072/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L1109
> )
>
> Possibly you need a few lines like below:
>
> final RelCollation collation =
>   rel instanceof Sort
>   ? ((Sort) rel).collation
>   : RelCollations.EMPTY;
>



Actually using a variation of this fix works, or at least all tests are
passing again.

Thank you very much !

Enrico





>
>
> Στις Παρ, 28 Σεπ 2018 στις 11:39 μ.μ., ο/η Enrico Olivelli <
> eolive...@gmail.com> έγραψε:
>
> > Il ven 28 set 2018, 23:00 Stamatis Zampetakis  ha
> > scritto:
> >
> > > Hi Enrico,
> > >
> > > I didn't look thoroughly but I suspect that maybe the problem is when
> you
> > > set the desiredTraits (
> > >
> > >
> >
> https://github.com/diennea/herddb/blob/0c7c01584350d57d8102511b987e5f880f3f65bd/herddb-core/src/main/java/herddb/sql/CalcitePlanner.java#L449
> > > ).
> > > Apart from the EnumerableConvention, I think you should also set the
> > > expected RelCollationTrait. Can you check what are the desiredTraits
> that
> > > you are passing to the optimizer?
> > >
> >
> > Stamatis
> > You see correctly that I am passing only EnumerableConvention as
> > desiredTraits.
> > I can't find the 'Trait' for RelCollation, there is no constant
> > like EnumerableConvention.
> > Is there anya special way to instantiate it?
> > I can't find and examplethe in Calcite tests
> >
> > Thank you so much
> > Enrico
> >
> >
> >
> > > Best,
> > > Stamatis
> > >
> > > Στις Παρ, 28 Σεπ 2018 στις 9:58 μ.μ., ο/η Enrico Olivelli <
> > > eolive...@gmail.com> έγραψε:
> > >
> > > > Sorry  in advance for this very long email
> > > > I am trying to debug, but I honestly I did not undestand clearly how
> > > > these rules work
> > > >
> > > > I have this query:
> > > > SELECT `tsql`.`k1`, `tsql`.`n1`, `tsql`.`s1`
> > > > FROM `tblspace1`.`tsql` AS `TSQL`
> > > >
> > > > This is the table:
> > > > CREATE TABLE tblspace1.tsql (k1 string primary key,n1 int,s1 string)
> > > >
> > > > Logical Plan Starts with:
> > > > LogicalSort(sort0=[$0], dir0=[ASC])
> > > >   LogicalProject(k1=[$0], n1=[$1], s1=[$2])
> > > > LogicalTableScan(table=[[tblspace1, tsql]])
> > > >
> > > > And the Best Exp eventually is:
> > > > EnumerableTableScan(table=[[tblspace1, tsql]]): rowcount = 1.0,
> > > > cumulative cost = {1.0 rows, 2.0 cpu, 0.0 io}, id = 33
> > > >
> > > > The problem is that the Sort disappears.
> > > > The Table is configured with Statistics without any Collation
> > > > Statistics.of(tableManager.getStats().getTablesize(), keys)
> > > >
> > > > This happens if I enable these traits on the Planner:
> > > > ConventionTraitDef.INSTANCE, RelCollationTraitDef.INSTANCE
> > > >
> > > > if I enable only ConventionTraitDef the sort does not disappear
> > > >
> > > > These are "TRACE" level logs of the Planner
> > > >
> > > > Maybe some expert of you can see inside the logs the anwer to my
> > problem.
> > > >
> > > > Thank you in advace, I know that your time is valuable. I appreciate
> > > > any kind of help
> > > >
> > > > Enrico
> > > >
> > > >
> > > > [main] TRACE org.apache.calcite.sql.parser - After validation: SELECT
> > > > `tsql`.`k1`, `tsql`.`n1`, `tsql`.`s1`
> > > > FROM `tblspace1`.`tsql` AS `TSQL`
> > > > ORDER BY `K1`
> > > > [main] TRACE org.apache.calcite.plan.RelOptPlanner - new
> > > > LogicalTableScan#30
> > > > [main] TRACE org.apache.calcite.plan.RelOptPlanner - new
> > > LogicalProject#31
> > > > [main] TRACE org.apache.calcite.plan.RelOptPlanner - new
> LogicalSort#32
> > > > [main] DEBUG org.apache.calcite.sql2rel - Plan after converting
> > > > SqlNode to RelNode
> > > > LogicalSort(sort0=[$0], dir0=[ASC])
> > > >   LogicalProject(k1=[$0], n1=[$1], s1=[$2])
> > > > LogicalTableScan(table=[[tblspace1, tsql]])
> > > >
> > > > [main] TRACE org.