??????[ANNOUNCE] Chunwei Lei joins Calcite PMC
ChunweiCongratulations! Xzh ---- ??: "dev"
??????[ANNOUNCE] New Calcite PMC chair: Ruben Q L
Congratulations to Ruben! Thanks for serving as Chair, Haisheng! Best regards, Zhaohui Xu ---- ??: "dev"
?????? [VOTE] Release Apache Calcite 1.28.0 (release candidate 0)
+1 1. Checked release notes - OK 2. Ran Gradle test - OK 3. Checked checksum and signature - OK Best, Zhaohui Xu ---- ??: "dev" https://github.com/apache/calcite/blob/calcite-1.28.0-rc0/site/_docs/history.md : *[CALCITE-4742] log '<' don't handle very well. * Haisheng Yuan https://github.com/apache/calcite/blob/calcite-1.28.0-rc0/site/_docs/history.md The commit to be voted upon: https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=dec167ac18272c0cd8be477d6b162d7a31a62114 Its hash is dec167ac18272c0cd8be477d6b162d7a31a62114 Tag: https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.28.0-rc0 The artifacts to be voted on are located here: https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.28.0-rc0 (revision 50450) The hashes of the artifacts are as follows: d61c4935d7d3b66425ff6e0c5e6e506b56f734d562f0fc6e69dc1d484da1d92d7d922930fd7b545abe0dc7b0178f6847ff4239ad52c52c8fa155668aa3d3fc38 *apache-calcite-1.28.0-src.tar.gz A staged Maven repository is available for review at: https://repository.apache.org/content/repositories/orgapachecalcite-1121/org/apache/calcite/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/jhyde.asc https://www.apache.org/dist/calcite/KEYS Please vote on releasing this package as Apache Calcite 1.28.0. The vote is open for the next 72 hours and passes if a majority of at least three +1 PMC votes are cast. [ ] +1 Release this package as Apache Calcite 1.28.0 [ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (binding)
??????[DISCUSS] Apply materialized recognition to streaming compute of time-window aggregate.
Hi Xurenhe Browse the design documents roughly. It seems that your materialized view scene is specialized, not universal and not applicable to Calcite. Now the materialized view recognition framework is extensible. You can customize the materialized view recognition rules and use the `RelOptMaterializations#useMaterializedViews` in your project. Best, Zhaohui Xu ---- ??: "dev" https://docs.google.com/document/d/1LtSgxhwvnpk2uAXFQiMSX-aQjnkJ6KLrXwXJX-VbsbE/edit Would love to hear your thoughts! Xurenhe
Re:Need help: Optimization rule to reduce project-filter-project-filter on one table
hi Huang 1. Reduce the interleaved project-filter to one single project-filter or filter-project,i always use `Calc Operator` to optimize `project-filter` or `filter-project`,Using the `Calc` operator, relational algebra can be simplified,This relational expression combines the functionality of `Project` and `Filter`.In Calcite, you can use `FilterToCalcRule` and `ProjectToCalcRule`, `CalcMergeRule` may also be used.These rules can be used by `HepPlanner`?? 2.I'm not sure if you want to remove `CAST` from RexNode,Maybe you can refer `org.apache.calcite.rex.RexUtil#removeCast`.In the RelNode, we can implement a `RelOptRule` to match the operator, and then remove the cast operator.you can refer `RelOptRules` in `org.apache.calcite.rel.rules.CoreRules`.In addition, you can use `RelShuttle` to rewrite RelNode, both of them can remove the `castoperator`. I hope it can help you. Best Xzh --Original-- From: "dev"
回复： [DISCUSS] Remove contributors name from commit summary
hi Julian: I think you have a good idea. In the Calcite community, people can know who has made changes by browsing GitHub, the number of PR contributions and the activity of the community.And People can know their contributions in `https://github.com/apache/calcite/graphs/contributors`. I hope the Calcite community will be better. Xzh --原始邮件-- 发件人: "dev"
??????[DISCUSS] Remove contributors name from commit summary
agree with you, +1 ---- ??: "dev"
An example of materialized view recognition
Hello everyone. I'd like to discuss an example of materialized view recognition. Query: select c1, count(distinct c2) as countd_c2, sum(c3) as sum_c3 from table group by c1 Mv1: select c1, c2 from table group by c1, c2 Mv2: select c1, c4, c5, sum(c3) as sum_c3_mv2 from table group by c1, c4, c5 If query tries to identify MV1 or MV2, neither of them can be recognized.Because the materialized view cannot express the columns required by Query. One solution as follows: select a.c1, a.countd_c2, b.sum_c3 from ( select c1, count(distinct c2) as countd_c2 from table group by c1 ) a join ( select c1, sum(c3) as sum_c3 from table group by c1 ) b on a.c1 = b.c1 Materialized view recognition: select a.c1, a.countd_c2, b.sum_c3 from ( select c1, count(distinct c2) as countd_c2 from MV1 group by c1 ) a join ( select c1, sum(sum_c3_mv2) as sum_c3 from MV2 group by c1 ) b on a.c1 = b.c1 If query is splited into Join RelNode, MV1 and MV2 can be recognized.This method can accurately identify materialized views and improve the reuse of materialized views.This example has been shown in doc. In the project, we use RelOptRule to realize it.However, Calcite does not support custom normalization rules for materialized view recognition. I have implemented the custom normalization rules before materialized view recognition, PR, which has been working for a long time in my project.Looking forward to the discussion and more feedback,thanks. DOC??https://docs.google.com/document/d/1mmAsK_uW-fBs893JERP1gspMurX2lNXcVGeesP4XUqQ/edit PR :https://github.com/apache/calcite/pull/2262
??????Tracking column's origin
Hi James: 1) I guess you want to trace the column's origin in original table. In Calcite, we can use `RelMetadatauery.getColumnOrigin()` to trace the column's origin.The method tracks the origin of columns.Here is a unit test 'org.apache.calcite.test.RelMetadataTest#testCalcColumnOriginsTable' for your reference. 2) After removing a specific operator, the column reference of the parent operator will be affected. It seems that no tool class can do this. Generally speaking, I will traverse to a specific operator pattern to modify the related column reference and generate a new RelNode. Maybe we use `RelOptRule` or `RelShuttle` to do this. I hope it can help you. Xu ---- ??: "dev"
??????Proposal to extend Calcite into a incremental query optimizer
Botong, It is very exciting. Thank you for your contribution to the community. I'm very interested in materialization and incremental computing. We know that there are many challenges in this area, such as choosing a good execution plan, materializing a good view, effectively hitting the target, materializing and identifying the view, and implementing optimization rules in specific fields. It can speed up query optimization. Thank you for your sharing and look forward to more discussion and feedback. ZhaoHui Xu ---- ??: "dev"
??????[ANNOUNCE] New Calcite PMC chair: Haisheng Yuan
Thank you for your contribution, Congratulate Haisheng! Zhaohui Xu ---- ??: "dev"
?????? Using Calcite at LinkedIn
Hi Walaa andTao: I am very happy to see your sharing. Our team is also working on SQL rewriting, analysis and optimization. Using Calcite's materialized view recognition capabilities to speed up user queries, many materialized view recognition algorithms[1] and normalization algorithms[2] have been implemented, and a lot of work has been done in preprocessing available materialized views, which has excellent performance. Welcome to discuss issues related to materialized view identification. [1]https://github.com/apache/calcite/pull/2094 [2] https://github.com/apache/calcite/pull/2262 Regards! Zhaohui Xu ---- ??: "dev" https://engineering.linkedin.com/blog/2020/coral [2] https://github.com/linkedin/coral [3] https://join.slack.com/t/coral-sql/shared_invite/zt-j9jw5idg-mkt3fjA~wgoUEMXXZqMr6g Thanks, Walaa.
??????[DISCUSS] State of the project 2020
Hi Stamatis, Thanks for your great work! Calcite is very good at semantic transformation of relational algebra. In the process of the project, users can optimize relational algebra by implementing RelOptRule or RelShuttle and so on, so as to bring relational algebra into physical query engine. Another point is the construction of materialized view recognition framework, which realizes the ability of materialized view recognition of common relational algebra. The expression of relational algebra is very flexible, and it can do better in identifying various materialized views. +1 for voting Haisheng. Best, Zhaohui Xu ---- ??: "dev" http://calcite.apache.org/news/2015/10/22/calcite-graduates/ [2] http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201509.mbox/%3CCF8D6F96-706F-4502-B41D-0689E357209D%40apache.org%3E [3] https://issues.apache.org/jira/browse/CALCITE-3923 [4] https://issues.apache.org/jira/browse/CALCITE-3724 [5] https://issues.apache.org/jira/browse/CALCITE-2157 [6] https://issues.apache.org/jira/browse/CALCITE-3916 [7] https://issues.apache.org/jira/browse/CALCITE-3896 [8] https://issues.apache.org/jira/browse/CALCITE-3753 [9] https://issues.apache.org/jira/browse/CALCITE-2970 [10] https://issues.apache.org/jira/browse/CALCITE-482 [11] https://issues.apache.org/jira/browse/CALCITE-3510 [12] https://issues.apache.org/jira/browse/CALCITE-4034
??????[ANNOUNCE] Ruben Quesada Lopez joins Calcite PMC
Congratulations,Ruben! xzh ---- ??: "dev"
回复： [VOTE] Release apache-calcite-1.25.0 (release candidate 0)
+1 (non binding) Local Calcite build with tests enabled on MacOs: Ok Calcite-based system test suite: Ok MacOs (Catalina 10.15.6) Java version(1.8.0_191) xzh --原始邮件-- 发件人: "dev" https://apache.github.io/calcite-site-preview/docs/history.html. On Mon, Aug 10, 2020 at 9:47 AM Rui Wang https://github.com/apache/calcite/blob/calcite-1.25.0-rc0/site/_docs/history.md ) - OK Spotted checked a few JARs in the Maven repository - OK Environment (OpenJDK:latest docker container): Gradle 6.3 (via gradlew) Oracle Linux Server 7.8 openjdk 14.0.2 2020-07-14 OpenJDK Runtime Environment (build 14.0.2+12-46) OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing) My vote is: +1 (binding) Francis On 10/08/2020 4:10 am, Rui Wang wrote: +1 (non-binding) platform: ubuntu16 + java8 run tests locally on: ok verify the commit hash in git tag: ok check sha512: ok verify signature by gpg: ok one thing can be fixed after vote: CALCITE-4114 does not belong to "breaking changes" in the release note. That change happened in core/test (a testing related change). -Rui On Sun, Aug 9, 2020 at 2:02 AM Enrico Olivelli < eolive...@gmail.com wrote: +1 (non binding) run tests locally on Fedora + JDK14 run tests of HerdDB just by switching from 1.24 without any change Enrico Il giorno dom 9 ago 2020 alle ore 05:22 Andrei Sereda https://apache.github.io/calcite-site-preview/docs/history.html The commit to be voted upon: https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=68b02dfd4af15bc94a91a0cd2a30655d04439555 Its hash is 68b02dfd4af15bc94a91a0cd2a30655d04439555 Tag: https://github.com/apache/calcite/tree/calcite-1.25.0-rc0 The artifacts to be voted on are located here: https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.25.0-rc0 (revision 40922) RAT report: https://apache.github.io/calcite-site-preview/rat/rat-report.txt Site preview is here: https://apache.github.io/calcite-site-preview/ JavaDoc API preview is here: https://apache.github.io/calcite-site-preview/api The hashes of the artifacts are as follows: a5e61bd93657a274ee8a1d1ecbde68e3e471fd27b85bea179991b372f094ae3cdf692672245506a08b996534f9136be26569de81af2bb7d8f026799313957e87 *apache-calcite-1.25.0-src.tar.gz A staged Maven repository is available for review at: https://repository.apache.org/content/repositories/orgapachecalcite-1097/org/apache/calcite/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/sereda.asc https://www.apache.org/dist/calcite/KEYS N.B. To create the jars and test Apache Calcite: "./gradlew build". If you do not have a Java environment available, you can run the tests using docker. To do so, install docker and docker-compose, then run "docker-compose run test" from the root of the directory. Please vote on releasing this package as Apache Calcite 1.25.0. The vote is open for the next 72 hours and passes if a majority of at least three +1 PMC votes are cast. [ ] +1 Release this package as Apache Calcite 1.25.0 [ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (non binding) -- Best regards, Anton.
??????Deploy Calcite to local Maven Repository
You can use the following command to publish jars to your local Maven repository. I hope it can help you. ./gradlew :core:publishToMavenLocal Best regards xzh ---- ??:"Enrico Olivelli"https://docs.gradle.org/current/userguide/maven_plugin.html ./gradlew install -x test I am trying to add the 'maven' plugin to Gradle configuration but without effect Is there any way to perform this task ? I would like to test current HerdDB master against current Calcite master, as probably Calcite 1.24 is going to be released soon Best regards Enrico
Re: [ANNOUNCE] New committer: Wang Yanlin
Congrats, Wang Yanlin! ---Original--- From: "Stamatis Zampetakis"
Re: [ANNOUNCE] New committer: Jin Xing
Congrats, Jin Xing! ---Original--- From: "Stamatis Zampetakis"
1
1
