RE: SQL Parsing
Maybe I will write a blog post about this (I couldn't find documentation about this with a web search), but for now this is a short summary. Hopefully nothing I say here is wrong, this is relatively new stuff for me too. Calcite uses a two-step code generation process to create the parser: - it starts with Parser.jj https://github.com/apache/calcite/blob/6f79436c178beec639e559d9152c237bbf8ec3e8/core/src/main/codegen/templates/Parser.jj. This is the base parser written in the JavaCC language, plus instructions in template language which allows for extensibility. The lines starting with "<#", e.g. https://github.com/apache/calcite/blob/6f79436c178beec639e559d9152c237bbf8ec3e8/core/src/main/codegen/templates/Parser.jj#L32 are for the FreeMarker template engine: https://freemarker.apache.org/docs/index.html - it runs FreeMarker to process Parser.jj; the template engine is controlled by config.fmpp https://github.com/apache/calcite/blob/6f79436c178beec639e559d9152c237bbf8ec3e8/core/src/main/codegen/config.fmpp and *.ftl files https://github.com/apache/calcite/blob/6f79436c178beec639e559d9152c237bbf8ec3e8/babel/src/main/codegen/includes/parserImpls.ftl The config file controls the way extensions are applied, and the ftl files contain the code for the extensions. The ftl files are really in the JavaCC language, but they are only code fragments. - the JavaCC parser generator is executed on the generated file to generate a Java file, e.g., SqlBabelParserImpl.java (the actual name comes from the configuration file). This file is not part of the repository, but you can find it after you run the Calcite build process. This file contains the real parser which gets compiled by the Java compiler and linked in the resulting Jar. That's why you need to involve the build system to customize the parser, you can't do it in pure Java. In our Calcite-based project the build system uses maven, but calcite itself uses gradle; the way you setup this workflow will depend on your tools. You can have one or multiple parsers in your executable, depending on which Calcite jars you choose to include. The server jar contains the DDL SQL language, while Babel has a very lenient parser which accepts constructs from many dialects. The core parser only has the SQL *query* language. In our project we have combined Babel + DDL in one parser: https://github.com/feldera/feldera/pull/276 In general the parser extensions are fairly small compared to the original parser, and they should be easy to read and understand if you know how parser generators work. As you can see, the original JJ file has some predefined extensibility points; if your desired extensions only need these, it should be easy to add them. I suspect adding a new syntax for generic types should be doable in the existing framework. Mihai -Original Message- From: Thomas Wang Sent: Sunday, October 1, 2023 7:37 AM To: dev@calcite.apache.org Subject: Re: SQL Parsing Hi Mihai, Could you point me to more information w.r.t modifying the parser? Also, I noticed for the ARRAY type, I can do CAST(NULL AS BIGINT ARRAY), but the grammar doesn't seem to have MAP supported, does it? Did I miss something? Thanks. Thomas On Sat, Sep 30, 2023 at 7:59 PM Mihai Budiu wrote: > Calcite is open source, so you can certainly modify the parser. The > architecture of the parser had been designed to be flexible and even > calcite has several parsers, the strandard one, server, and Babel. Let > me know if you need help figuring out how to roll your own. > > Mihai > > From: Thomas Wang > Sent: Saturday, September 30, 2023 7:42:19 PM > To: dev@calcite.apache.org > Subject: Re: SQL Parsing > > Thanks Mihai, "BIGINT ARRAY" seems to work! Just curious if I can have > flexibility to change the behavior of the parse to parse "ARRAY" > instead? > > Thomas > > On Sat, Sep 30, 2023 at 6:32 PM Mihai Budiu wrote: > > > The syntax is "bigint array" > > https://calcite.apache.org/docs/reference.html > > > > From: Thomas Wang > > Sent: Saturday, September 30, 2023 6:27:46 PM > > To: dev@calcite.apache.org > > Subject: SQL Parsing > > > > Hi Apache Calcite Community, > > > > I'm new to Apache Calcite and trying to use it for SQL parsing and > > SQL rewriting. > > > > I tried to parse a couple of SELECT statements and it works pretty well. > > However, when I tried to parse a SELECT statement that contains a > > cast to array like below, it complains it cannot recognize ARRAY. > > > > SELECT CAST(NULL AS ARRAY) FROM schema.t1 > > > > However, casting to BIGINT seems ok. The following SELECT is ok. > > > > SELECT CAST(NULL AS BIGINT) FROM schema.t1 > > > > Is there any configuration I need to enable to enable/support > > parsing ARRAY? Thanks. > > > > Thomas > > >
Re: SQL Parsing
The parser will accept literally anything if you put it in backticks (or whatever are the identifier quote symbol in the current parser config). It just thinks it is the name of a user-defined type. Not what you want. > On Oct 2, 2023, at 11:54 AM, Thomas Wang wrote: > > Thanks Jiajun. It looks like the following syntax with the backticks works > for me. > > SELECT CAST(NULL AS `MAP`) FROM schema.table > > Thomas > > On Sun, Oct 1, 2023 at 11:23 PM Jiajun Xie > wrote: > >> Hi Thomas, >> >> Cast map syntax is not supported in the current release version. >> But we will support it in the 1.36.0 version. >> >> FYI: https://issues.apache.org/jira/browse/CALCITE-5570 >> >> On Sun, 1 Oct 2023 at 22:38, Thomas Wang wrote: >> >>> Hi Mihai, >>> >>> Could you point me to more information w.r.t modifying the parser? Also, >> I >>> noticed for the ARRAY type, I can do CAST(NULL AS BIGINT ARRAY), but the >>> grammar doesn't seem to have MAP supported, does it? Did I miss >> something? >>> Thanks. >>> >>> Thomas >>> >>> On Sat, Sep 30, 2023 at 7:59 PM Mihai Budiu wrote: >>> Calcite is open source, so you can certainly modify the parser. The architecture of the parser had been designed to be flexible and even calcite has several parsers, the strandard one, server, and Babel. Let >> me know if you need help figuring out how to roll your own. Mihai From: Thomas Wang Sent: Saturday, September 30, 2023 7:42:19 PM To: dev@calcite.apache.org Subject: Re: SQL Parsing Thanks Mihai, "BIGINT ARRAY" seems to work! Just curious if I can have flexibility to change the behavior of the parse to parse >> "ARRAY" instead? Thomas On Sat, Sep 30, 2023 at 6:32 PM Mihai Budiu wrote: > The syntax is "bigint array" > https://calcite.apache.org/docs/reference.html > > From: Thomas Wang > Sent: Saturday, September 30, 2023 6:27:46 PM > To: dev@calcite.apache.org > Subject: SQL Parsing > > Hi Apache Calcite Community, > > I'm new to Apache Calcite and trying to use it for SQL parsing and >> SQL > rewriting. > > I tried to parse a couple of SELECT statements and it works pretty >>> well. > However, when I tried to parse a SELECT statement that contains a >> cast >>> to > array like below, it complains it cannot recognize ARRAY. > > SELECT CAST(NULL AS ARRAY) FROM schema.t1 > > However, casting to BIGINT seems ok. The following SELECT is ok. > > SELECT CAST(NULL AS BIGINT) FROM schema.t1 > > Is there any configuration I need to enable to enable/support parsing > ARRAY? Thanks. > > Thomas > >>> >>
[jira] [Created] (CALCITE-6032) Multilevel correlated query is failing in RelDecorrelator code path
Shiven Dvrk created CALCITE-6032:
Summary: Multilevel correlated query is failing in RelDecorrelator
code path
Key: CALCITE-6032
URL: https://issues.apache.org/jira/browse/CALCITE-6032
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Shiven Dvrk
We have following query:
{code:java}
SELECT
Sum( In0.Col_5005 * (
select
MAX(mr.Col_147077 + mr.Col_147078 )
from
Bun mr
where
mr.Col_21084 = 1
and mr.Col_21805 = 2
and mr.Col_21807 = In0.Col_5003
and mr.Col_21806 =
(
select
Max(kr.Col_21806)
from
Bun kr
where
kr.Col_21807 = mr.Col_21807
and kr.Col_21804 = mr.Col_21804
and kr.Col_21805 = mr.Col_21805
and kr.Col_21806 <= In0.Col_4085
)
) ) as mmn,
count(*) as mmnCount
FROM
Den In0
where
In0.Col_5005 < 10
and In0.Col_5005 > 0
{code}
It is failing to decorrelate with standard program:
{code:java}
cm.mapCorToCorRel.get($cor4)
java.lang.NullPointerException: cm.mapCorToCorRel.get($cor4)
at java.util.Objects.requireNonNull(Objects.java:290)
at
org.apache.calcite.sql2rel.RelDecorrelator.getCorRel(RelDecorrelator.java:928)
at
org.apache.calcite.sql2rel.RelDecorrelator.createValueGenerator(RelDecorrelator.java:820)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateInputWithValueGenerator(RelDecorrelator.java:1028)
at
org.apache.calcite.sql2rel.RelDecorrelator.maybeAddValueGenerator(RelDecorrelator.java:947)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1150)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
at
org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:749)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
at
org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:512)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
at
org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1187)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
at
org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:707)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:749)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:738)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.
Re: SQL Parsing
Thanks Jiajun. It looks like the following syntax with the backticks works for me. SELECT CAST(NULL AS `MAP`) FROM schema.table Thomas On Sun, Oct 1, 2023 at 11:23 PM Jiajun Xie wrote: > Hi Thomas, > > Cast map syntax is not supported in the current release version. > But we will support it in the 1.36.0 version. > > FYI: https://issues.apache.org/jira/browse/CALCITE-5570 > > On Sun, 1 Oct 2023 at 22:38, Thomas Wang wrote: > > > Hi Mihai, > > > > Could you point me to more information w.r.t modifying the parser? Also, > I > > noticed for the ARRAY type, I can do CAST(NULL AS BIGINT ARRAY), but the > > grammar doesn't seem to have MAP supported, does it? Did I miss > something? > > Thanks. > > > > Thomas > > > > On Sat, Sep 30, 2023 at 7:59 PM Mihai Budiu wrote: > > > > > Calcite is open source, so you can certainly modify the parser. The > > > architecture of the parser had been designed to be flexible and even > > > calcite has several parsers, the strandard one, server, and Babel. Let > me > > > know if you need help figuring out how to roll your own. > > > > > > Mihai > > > > > > From: Thomas Wang > > > Sent: Saturday, September 30, 2023 7:42:19 PM > > > To: dev@calcite.apache.org > > > Subject: Re: SQL Parsing > > > > > > Thanks Mihai, "BIGINT ARRAY" seems to work! Just curious if I can have > > > flexibility to change the behavior of the parse to parse > "ARRAY" > > > instead? > > > > > > Thomas > > > > > > On Sat, Sep 30, 2023 at 6:32 PM Mihai Budiu wrote: > > > > > > > The syntax is "bigint array" > > > > https://calcite.apache.org/docs/reference.html > > > > > > > > From: Thomas Wang > > > > Sent: Saturday, September 30, 2023 6:27:46 PM > > > > To: dev@calcite.apache.org > > > > Subject: SQL Parsing > > > > > > > > Hi Apache Calcite Community, > > > > > > > > I'm new to Apache Calcite and trying to use it for SQL parsing and > SQL > > > > rewriting. > > > > > > > > I tried to parse a couple of SELECT statements and it works pretty > > well. > > > > However, when I tried to parse a SELECT statement that contains a > cast > > to > > > > array like below, it complains it cannot recognize ARRAY. > > > > > > > > SELECT CAST(NULL AS ARRAY) FROM schema.t1 > > > > > > > > However, casting to BIGINT seems ok. The following SELECT is ok. > > > > > > > > SELECT CAST(NULL AS BIGINT) FROM schema.t1 > > > > > > > > Is there any configuration I need to enable to enable/support parsing > > > > ARRAY? Thanks. > > > > > > > > Thomas > > > > > > > > > >
Re: Draft: board report for 2023 Q3
+1, thanks Stamatis for putting this together! On Mon, 2 Oct 2023 at 11:27, LakeShen wrote: > +1, Nice work, Stamatis! > > Ruben Q L 于2023年10月2日周一 16:58写道： > > > +1, thanks Stamatis! > > > > @Sergey thanks for the info! IMO your talk would be a nice addition for > the > > next report (Q4). > > > > > > On Mon, Oct 2, 2023 at 9:48 AM Francis Chuang > > wrote: > > > > > +1, Excellent work, Stamatis! > > > > > > On 2/10/2023 6:58 pm, Stamatis Zampetakis wrote: > > > > Hello, > > > > > > > > Below you can find a draft of this quarter's board report. I plan to > > > > submit the final version next Monday (October 9, 2023). > > > > > > > > Please let me know if you have any additions or corrections. > > > > > > > > Best, > > > > Stamatis > > > > --- > > > > > > > > ## Description: > > > > Apache Calcite is a highly customizable framework for parsing and > > > planning > > > > queries on data in a wide variety of formats. It allows database-like > > > > access, > > > > and in particular a SQL interface and advanced query optimization, > for > > > data > > > > not residing in a traditional database. > > > > > > > > Avatica is a sub-project within Calcite and provides a framework for > > > > building > > > > local and remote JDBC and ODBC database drivers. Avatica has an > > > independent > > > > release schedule and its own repository. > > > > > > > > ## Project Status: > > > > Current project status: ongoing > > > > Issues for the board: none > > > > > > > > ## Membership Data: > > > > Apache Calcite was founded 2015-10-22 (8 years ago) > > > > There are currently 69 committers and 27 PMC members in this project. > > > > The Committer-to-PMC ratio is roughly 2:1. > > > > > > > > Community changes, past quarter: > > > > - No new PMC members. Last addition was Benchao Li on 2023-01-27. > > > > - TJ Banghart was added as committer on 2023-07-04 > > > > - Dan Zou was added as committer on 2023-07-04 > > > > > > > > ## Project Activity: > > > > Apache Calcite 1.35.0 was released on 2023-07-26. It contains > > > contributions > > > > from 36 contributors, and resolves 140 issues. The new release has > many > > > > improvements in the BigQuery and Spark dialect bringing in more than > 40 > > > SQL > > > > functions. Additionally, it comes with new optimizations for reducing > > > the size > > > > of generated code and more powerful expression simplifications. > > > > > > > > On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite > > > > community at the Apache Con East Asia by giving talks related with > the > > > > project. > > > > > > > > ## Community Health: > > > > The project is healthy. Previously, it was super healthy. The drop is > > > likely > > > > to the fact that the PMC has not grown in the last six months but > this > > > will > > > > very likely change in the near future since a lot of our current > > > committers > > > > are very involved with the project and hopefully they will join the > PMC > > > > shortly. > > > > > > > > The dev list had a 38% in activity in the past quarter, with busiest > > > threads > > > > been as usual those around releases and introduction of new > committers. > > > The > > > > 16% increase in activity of the JIRA also contributes to the general > > > increase > > > > in traffic of the dev list. > > > > > > > > The number of non-committer (contributor) commits per month: > > > > +-+-+-+ > > > > |year |month| contributor_commits | > > > > +-+-+-+ > > > > | 2023| 7 | 16 | > > > > | 2023| 8 | 32 | > > > > | 2023| 9 | 32 | > > > > +-+-+-+ > > > > > > > > The number of active reviewers per month: > > > > +-+-+-+ > > > > |year |month| active_reviewers | > > > > +-+-+-+ > > > > | 2023| 7 | 9 | > > > > | 2023| 8 | 9 | > > > > | 2023| 9 | 10 | > > > > +-+-+-+ > > > > > > > > Top-3 reviewers in the last quarter: > > > > +---+-+ > > > > | committer | reviews | > > > > +---+-+ > > > > | Jiajun | 15 | > > > > | Julian Hyde | 13 | > > > > | Benchao Li | 11 | > > > > +---+-+ > > > > > > > > The number of non-committer commits has increased by 9% from the last >
Re: Draft: board report for 2023 Q3
+1, Nice work, Stamatis! Ruben Q L 于2023年10月2日周一 16:58写道： > +1, thanks Stamatis! > > @Sergey thanks for the info! IMO your talk would be a nice addition for the > next report (Q4). > > > On Mon, Oct 2, 2023 at 9:48 AM Francis Chuang > wrote: > > > +1, Excellent work, Stamatis! > > > > On 2/10/2023 6:58 pm, Stamatis Zampetakis wrote: > > > Hello, > > > > > > Below you can find a draft of this quarter's board report. I plan to > > > submit the final version next Monday (October 9, 2023). > > > > > > Please let me know if you have any additions or corrections. > > > > > > Best, > > > Stamatis > > > --- > > > > > > ## Description: > > > Apache Calcite is a highly customizable framework for parsing and > > planning > > > queries on data in a wide variety of formats. It allows database-like > > > access, > > > and in particular a SQL interface and advanced query optimization, for > > data > > > not residing in a traditional database. > > > > > > Avatica is a sub-project within Calcite and provides a framework for > > > building > > > local and remote JDBC and ODBC database drivers. Avatica has an > > independent > > > release schedule and its own repository. > > > > > > ## Project Status: > > > Current project status: ongoing > > > Issues for the board: none > > > > > > ## Membership Data: > > > Apache Calcite was founded 2015-10-22 (8 years ago) > > > There are currently 69 committers and 27 PMC members in this project. > > > The Committer-to-PMC ratio is roughly 2:1. > > > > > > Community changes, past quarter: > > > - No new PMC members. Last addition was Benchao Li on 2023-01-27. > > > - TJ Banghart was added as committer on 2023-07-04 > > > - Dan Zou was added as committer on 2023-07-04 > > > > > > ## Project Activity: > > > Apache Calcite 1.35.0 was released on 2023-07-26. It contains > > contributions > > > from 36 contributors, and resolves 140 issues. The new release has many > > > improvements in the BigQuery and Spark dialect bringing in more than 40 > > SQL > > > functions. Additionally, it comes with new optimizations for reducing > > the size > > > of generated code and more powerful expression simplifications. > > > > > > On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite > > > community at the Apache Con East Asia by giving talks related with the > > > project. > > > > > > ## Community Health: > > > The project is healthy. Previously, it was super healthy. The drop is > > likely > > > to the fact that the PMC has not grown in the last six months but this > > will > > > very likely change in the near future since a lot of our current > > committers > > > are very involved with the project and hopefully they will join the PMC > > > shortly. > > > > > > The dev list had a 38% in activity in the past quarter, with busiest > > threads > > > been as usual those around releases and introduction of new committers. > > The > > > 16% increase in activity of the JIRA also contributes to the general > > increase > > > in traffic of the dev list. > > > > > > The number of non-committer (contributor) commits per month: > > > +-+-+-+ > > > |year |month| contributor_commits | > > > +-+-+-+ > > > | 2023| 7 | 16 | > > > | 2023| 8 | 32 | > > > | 2023| 9 | 32 | > > > +-+-+-+ > > > > > > The number of active reviewers per month: > > > +-+-+-+ > > > |year |month| active_reviewers | > > > +-+-+-+ > > > | 2023| 7 | 9 | > > > | 2023| 8 | 9 | > > > | 2023| 9 | 10 | > > > +-+-+-+ > > > > > > Top-3 reviewers in the last quarter: > > > +---+-+ > > > | committer | reviews | > > > +---+-+ > > > | Jiajun | 15 | > > > | Julian Hyde | 13 | > > > | Benchao Li | 11 | > > > +---+-+ > > > > > > The number of non-committer commits has increased by 9% from the last > > quarter > > > (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of > > having > > > new people contributing to the project. > > > > > > The average number of active reviewers per month has increased slightly > > from > > > the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are > > > participating in th
Re: Draft: board report for 2023 Q3
+1, thanks Stamatis! @Sergey thanks for the info! IMO your talk would be a nice addition for the next report (Q4). On Mon, Oct 2, 2023 at 9:48 AM Francis Chuang wrote: > +1, Excellent work, Stamatis! > > On 2/10/2023 6:58 pm, Stamatis Zampetakis wrote: > > Hello, > > > > Below you can find a draft of this quarter's board report. I plan to > > submit the final version next Monday (October 9, 2023). > > > > Please let me know if you have any additions or corrections. > > > > Best, > > Stamatis > > --- > > > > ## Description: > > Apache Calcite is a highly customizable framework for parsing and > planning > > queries on data in a wide variety of formats. It allows database-like > > access, > > and in particular a SQL interface and advanced query optimization, for > data > > not residing in a traditional database. > > > > Avatica is a sub-project within Calcite and provides a framework for > > building > > local and remote JDBC and ODBC database drivers. Avatica has an > independent > > release schedule and its own repository. > > > > ## Project Status: > > Current project status: ongoing > > Issues for the board: none > > > > ## Membership Data: > > Apache Calcite was founded 2015-10-22 (8 years ago) > > There are currently 69 committers and 27 PMC members in this project. > > The Committer-to-PMC ratio is roughly 2:1. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Benchao Li on 2023-01-27. > > - TJ Banghart was added as committer on 2023-07-04 > > - Dan Zou was added as committer on 2023-07-04 > > > > ## Project Activity: > > Apache Calcite 1.35.0 was released on 2023-07-26. It contains > contributions > > from 36 contributors, and resolves 140 issues. The new release has many > > improvements in the BigQuery and Spark dialect bringing in more than 40 > SQL > > functions. Additionally, it comes with new optimizations for reducing > the size > > of generated code and more powerful expression simplifications. > > > > On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite > > community at the Apache Con East Asia by giving talks related with the > > project. > > > > ## Community Health: > > The project is healthy. Previously, it was super healthy. The drop is > likely > > to the fact that the PMC has not grown in the last six months but this > will > > very likely change in the near future since a lot of our current > committers > > are very involved with the project and hopefully they will join the PMC > > shortly. > > > > The dev list had a 38% in activity in the past quarter, with busiest > threads > > been as usual those around releases and introduction of new committers. > The > > 16% increase in activity of the JIRA also contributes to the general > increase > > in traffic of the dev list. > > > > The number of non-committer (contributor) commits per month: > > +-+-+-+ > > |year |month| contributor_commits | > > +-+-+-+ > > | 2023| 7 | 16 | > > | 2023| 8 | 32 | > > | 2023| 9 | 32 | > > +-+-+-+ > > > > The number of active reviewers per month: > > +-+-+-+ > > |year |month| active_reviewers | > > +-+-+-+ > > | 2023| 7 | 9 | > > | 2023| 8 | 9 | > > | 2023| 9 | 10 | > > +-+-+-+ > > > > Top-3 reviewers in the last quarter: > > +---+-+ > > | committer | reviews | > > +---+-+ > > | Jiajun | 15 | > > | Julian Hyde | 13 | > > | Benchao Li | 11 | > > +---+-+ > > > > The number of non-committer commits has increased by 9% from the last > quarter > > (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of > having > > new people contributing to the project. > > > > The average number of active reviewers per month has increased slightly > from > > the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are > > participating in the review process which is among the main points of the > > project. > > > > In the top reviewers, we can observe that things are a bit more balanced > in > > Q3. The review count per person in the top-3 tier is lower than usual > but in > > conjunction with the increase in the number of non-committ
Re: Draft: board report for 2023 Q3
+1, Excellent work, Stamatis! On 2/10/2023 6:58 pm, Stamatis Zampetakis wrote: Hello, Below you can find a draft of this quarter's board report. I plan to submit the final version next Monday (October 9, 2023). Please let me know if you have any additions or corrections. Best, Stamatis --- ## Description: Apache Calcite is a highly customizable framework for parsing and planning queries on data in a wide variety of formats. It allows database-like access, and in particular a SQL interface and advanced query optimization, for data not residing in a traditional database. Avatica is a sub-project within Calcite and provides a framework for building local and remote JDBC and ODBC database drivers. Avatica has an independent release schedule and its own repository. ## Project Status: Current project status: ongoing Issues for the board: none ## Membership Data: Apache Calcite was founded 2015-10-22 (8 years ago) There are currently 69 committers and 27 PMC members in this project. The Committer-to-PMC ratio is roughly 2:1. Community changes, past quarter: - No new PMC members. Last addition was Benchao Li on 2023-01-27. - TJ Banghart was added as committer on 2023-07-04 - Dan Zou was added as committer on 2023-07-04 ## Project Activity: Apache Calcite 1.35.0 was released on 2023-07-26. It contains contributions from 36 contributors, and resolves 140 issues. The new release has many improvements in the BigQuery and Spark dialect bringing in more than 40 SQL functions. Additionally, it comes with new optimizations for reducing the size of generated code and more powerful expression simplifications. On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite community at the Apache Con East Asia by giving talks related with the project. ## Community Health: The project is healthy. Previously, it was super healthy. The drop is likely to the fact that the PMC has not grown in the last six months but this will very likely change in the near future since a lot of our current committers are very involved with the project and hopefully they will join the PMC shortly. The dev list had a 38% in activity in the past quarter, with busiest threads been as usual those around releases and introduction of new committers. The 16% increase in activity of the JIRA also contributes to the general increase in traffic of the dev list. The number of non-committer (contributor) commits per month: +-+-+-+ |year |month| contributor_commits | +-+-+-+ | 2023| 7 | 16 | | 2023| 8 | 32 | | 2023| 9 | 32 | +-+-+-+ The number of active reviewers per month: +-+-+-+ |year |month| active_reviewers | +-+-+-+ | 2023| 7 | 9 | | 2023| 8 | 9 | | 2023| 9 | 10 | +-+-+-+ Top-3 reviewers in the last quarter: +---+-+ | committer | reviews | +---+-+ | Jiajun | 15 | | Julian Hyde | 13 | | Benchao Li | 11 | +---+-+ The number of non-committer commits has increased by 9% from the last quarter (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of having new people contributing to the project. The average number of active reviewers per month has increased slightly from the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are participating in the review process which is among the main points of the project. In the top reviewers, we can observe that things are a bit more balanced in Q3. The review count per person in the top-3 tier is lower than usual but in conjunction with the increase in the number of non-committer commits it shows that reviews are more evenly distributed among the members of the community.
Re: Draft: board report for 2023 Q3
Hi Stamatis thanks for the draft report it looks great one thing which I currently not sure whether to add or not (because of dates) This year I will give a talk at Flink Forward 2023 (November 6-8 2023) about Calcite upgrade journey in Apache Flink On Mon, Oct 2, 2023 at 9:58 AM Stamatis Zampetakis wrote: > Hello, > > Below you can find a draft of this quarter's board report. I plan to > submit the final version next Monday (October 9, 2023). > > Please let me know if you have any additions or corrections. > > Best, > Stamatis > --- > > ## Description: > Apache Calcite is a highly customizable framework for parsing and planning > queries on data in a wide variety of formats. It allows database-like > access, > and in particular a SQL interface and advanced query optimization, for data > not residing in a traditional database. > > Avatica is a sub-project within Calcite and provides a framework for > building > local and remote JDBC and ODBC database drivers. Avatica has an independent > release schedule and its own repository. > > ## Project Status: > Current project status: ongoing > Issues for the board: none > > ## Membership Data: > Apache Calcite was founded 2015-10-22 (8 years ago) > There are currently 69 committers and 27 PMC members in this project. > The Committer-to-PMC ratio is roughly 2:1. > > Community changes, past quarter: > - No new PMC members. Last addition was Benchao Li on 2023-01-27. > - TJ Banghart was added as committer on 2023-07-04 > - Dan Zou was added as committer on 2023-07-04 > > ## Project Activity: > Apache Calcite 1.35.0 was released on 2023-07-26. It contains contributions > from 36 contributors, and resolves 140 issues. The new release has many > improvements in the BigQuery and Spark dialect bringing in more than 40 SQL > functions. Additionally, it comes with new optimizations for reducing the > size > of generated code and more powerful expression simplifications. > > On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite > community at the Apache Con East Asia by giving talks related with the > project. > > ## Community Health: > The project is healthy. Previously, it was super healthy. The drop is > likely > to the fact that the PMC has not grown in the last six months but this will > very likely change in the near future since a lot of our current committers > are very involved with the project and hopefully they will join the PMC > shortly. > > The dev list had a 38% in activity in the past quarter, with busiest > threads > been as usual those around releases and introduction of new committers. The > 16% increase in activity of the JIRA also contributes to the general > increase > in traffic of the dev list. > > The number of non-committer (contributor) commits per month: > +-+-+-+ > |year |month| contributor_commits | > +-+-+-+ > | 2023| 7 | 16 | > | 2023| 8 | 32 | > | 2023| 9 | 32 | > +-+-+-+ > > The number of active reviewers per month: > +-+-+-+ > |year |month| active_reviewers | > +-+-+-+ > | 2023| 7 | 9 | > | 2023| 8 | 9 | > | 2023| 9 | 10 | > +-+-+-+ > > Top-3 reviewers in the last quarter: > +---+-+ > | committer | reviews | > +---+-+ > | Jiajun | 15 | > | Julian Hyde | 13 | > | Benchao Li | 11 | > +---+-+ > > The number of non-committer commits has increased by 9% from the last > quarter > (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of > having > new people contributing to the project. > > The average number of active reviewers per month has increased slightly > from > the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are > participating in the review process which is among the main points of the > project. > > In the top reviewers, we can observe that things are a bit more balanced in > Q3. The review count per person in the top-3 tier is lower than usual but > in > conjunction with the increase in the number of non-committer commits > it shows that > reviews are more evenly distributed among the members of the community. > -- Best regards, Sergey
Draft: board report for 2023 Q3
Hello, Below you can find a draft of this quarter's board report. I plan to submit the final version next Monday (October 9, 2023). Please let me know if you have any additions or corrections. Best, Stamatis --- ## Description: Apache Calcite is a highly customizable framework for parsing and planning queries on data in a wide variety of formats. It allows database-like access, and in particular a SQL interface and advanced query optimization, for data not residing in a traditional database. Avatica is a sub-project within Calcite and provides a framework for building local and remote JDBC and ODBC database drivers. Avatica has an independent release schedule and its own repository. ## Project Status: Current project status: ongoing Issues for the board: none ## Membership Data: Apache Calcite was founded 2015-10-22 (8 years ago) There are currently 69 committers and 27 PMC members in this project. The Committer-to-PMC ratio is roughly 2:1. Community changes, past quarter: - No new PMC members. Last addition was Benchao Li on 2023-01-27. - TJ Banghart was added as committer on 2023-07-04 - Dan Zou was added as committer on 2023-07-04 ## Project Activity: Apache Calcite 1.35.0 was released on 2023-07-26. It contains contributions from 36 contributors, and resolves 140 issues. The new release has many improvements in the BigQuery and Spark dialect bringing in more than 40 SQL functions. Additionally, it comes with new optimizations for reducing the size of generated code and more powerful expression simplifications. On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite community at the Apache Con East Asia by giving talks related with the project. ## Community Health: The project is healthy. Previously, it was super healthy. The drop is likely to the fact that the PMC has not grown in the last six months but this will very likely change in the near future since a lot of our current committers are very involved with the project and hopefully they will join the PMC shortly. The dev list had a 38% in activity in the past quarter, with busiest threads been as usual those around releases and introduction of new committers. The 16% increase in activity of the JIRA also contributes to the general increase in traffic of the dev list. The number of non-committer (contributor) commits per month: +-+-+-+ |year |month| contributor_commits | +-+-+-+ | 2023| 7 | 16 | | 2023| 8 | 32 | | 2023| 9 | 32 | +-+-+-+ The number of active reviewers per month: +-+-+-+ |year |month| active_reviewers | +-+-+-+ | 2023| 7 | 9 | | 2023| 8 | 9 | | 2023| 9 | 10 | +-+-+-+ Top-3 reviewers in the last quarter: +---+-+ | committer | reviews | +---+-+ | Jiajun | 15 | | Julian Hyde | 13 | | Benchao Li | 11 | +---+-+ The number of non-committer commits has increased by 9% from the last quarter (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of having new people contributing to the project. The average number of active reviewers per month has increased slightly from the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are participating in the review process which is among the main points of the project. In the top reviewers, we can observe that things are a bit more balanced in Q3. The review count per person in the top-3 tier is lower than usual but in conjunction with the increase in the number of non-committer commits it shows that reviews are more evenly distributed among the members of the community.
