Re: [DISCUSS] State of the project 2024

[Francis Chuang]

Thank you, Benchao for being our chair this year, your excellent work 
with the quarterly reports and keeping the project on track with releases.

It's great to see a lot of new faces join and contribute to the project 
during the course of the year and bring new ideas to the project.

I agree that we can also do better in terms of reviewing PRs, there are 
often discussion threads started by contributors who would like to have 
their PR(s) reviewed. I think this is something that the community 
should focus on in our next year. You do not need to be a PMC member or 
Committer to review PRs, any contributor can review a PR and give their 
comments or +1s.

I would also like to see more contributors to Avatica (thanks Istvan for 
contributing a significant amount of commits to the project) and 
Avatica-Go. Although those 2 projects aren't as popular as Calcite, I 
think they are still useful projects for the community and beyond.

Finally, I would like to repeat Benchao's call for PMC chair 
nominations. Anyone can nominate a PMC member to be chair by sending an 
email to priv...@calcite.apache.org. Please note that nominations close 
on 5 November, so if you would like to make a nomination, please do so now.

Francis

On 30/10/2024 7:49 am, Julian Hyde wrote:
I feel that Calcite — the project and the community — are in good shape this 
year. There is a high velocity of contributions from a wide variety of people. 
Thank you to Benchao for steering the ship for the last twelve months.

The rate at which contributors evolve into committers and then into PMC members 
seems lower than previous years. Perhaps we are becoming better at accepting 
‘drive by’ contributions from people who do not contribute more than once; but 
perhaps not. Let’s try to improve velocity next year.

We seem to be doing better than in previous years at reviewing and merging PRs. 
This is due to the efforts of a relatively small number of committers, so a big 
thank you to those people.

A large fraction contributions (and a larger fraction than previous years) are 
related to SQL dialect translation — adding functions so that Calcite looks 
like other dialects, and adding support to the JDBC adapter for generating 
other dialects. Our testing of dialects is weak because we do not have live 
connections to SQL engines (Postgres, Snowflake, Redshift, BigQuery). (Not 
surprising, considering that we are an open source project, and these 
connections are far too expensive for GitHub to provide them for free in its CI 
environment.) We could use a radical rethink for how we write and test dialects.

Julian


On Oct 29, 2024, at 4:51 AM, Stamatis Zampetakis <zabe...@apache.org> wrote:

Thanks for the great summary Benchao and kudos for the excellent work
that you did as chair! Every board report was carefully and timely
prepared, the release cadence was kept steady, our community has
grown, and you kept us on track for every important milestone.

I have been involved in the project for a few years now and I am happy
to see that we are continuously improving almost on every aspect of
it.

 From the technical perspective the project is advancing really well
and each release makes the project better, more stable, and mature.
There are also lots of interesting and ambitious ideas that will
become the major features of the subsequent releases.

In terms of community, the project is rather welcoming to new
contributors and most of the questions in the mailing list do receive
responses. The reviewing part is holding us back a bit; if we were
able to respond better in the arriving JIRA/PRs the community would
grow much faster.

Best,
Stamatis









On Tue, Oct 22, 2024 at 12:20 PM Benchao Li <libenc...@apache.org> wrote:

Oops, another mistake was discovered. I overlooked Ruben Quesada Lopez
in the paragraph of "In terms of the community", the part about new
Apache members. He is also one of the new Apache members in 2024.

Sorry about that, Ruben, and many thanks for informing me and giving
me the opportunity to correct it myself.

Benchao Li <libenc...@apache.org> 于2024年10月22日周二 16:10写道：


The end of date for nominating Chair candidates should be 2024-11-05.
Sorry for the typo, I was copying it from last year's email.

Benchao Li <libenc...@apache.org> 于2024年10月22日周二 10:04写道：

Hello community,

Nine years ago (22 October 2015) Calcite graduated as a top-level
Apache project [1]. At that point, the community decided to have an
annual "state of the project" discussion, and we arrived at that time
of the year.

We have released two versions of Calcite (1.37.0 and 1.38.0) so far in
2024 [2]. The number is relatively lower than previous years (4 in
2023, 3 in 2022). With over 300 commits, it is evident that we have
accomplished a significant amount of work. It is worth highlighting
features such as the introduction of the Arrow Adapter, the addition
of support for lambda expressions in SQL, new grammars such as the AS
MEASURE clause, ASOF join, and the EXCLUDE clause in window
aggregations, as well as the ability to call table functions without
the TABLE() wrapper.

Regarding the sub-project Avatica, we have one release (1.25.0) so
far, and perhaps one more [3] coming before the end of the year. There
has been no release for Avatica Go in 2024, and it has been almost a
year since the last release. However, there are several new commits
that may warrant a release in the near future. The development
activity of both Avatica and Avatica Go remains relatively low, but
the project is healthy as there are various regular maintainers (such
as Francis Chuang).

In terms of the community, we have only one new Committer thus far,
Norman Jordan, which is much fewer than previous years (14 in 2023, 4
in 2022). Many contributors have made outstanding contributions to the
project, for which we are extremely grateful, and we hope to
eventually invite them as Committers once they have accumulated
sufficient merits. Our PMC team has expanded with two new PMC members,
Sergey Nuyanzin and Mihai Budiu. Calcite grows and evolves because of
(and thanks to) its community, and I would like to express my
gratitude to everyone for being a part of this family and working
together in a respectful and motivating environment. In addition to
the membership of Calcite, there are also three new Apache members[4]
from the Calcite community this year, namely Francis Chuang, Sergey
Nuyanzin, and Benchao Li, indicating that our community is strong and
stable.

It was delightful to see our community members presenting talks about
Calcite at conferences such as SIGMOD and ApacheCon Asia. Since
Calcite has numerous users and contributors from Asia, I attempted to
organize a dedicated track for Calcite earlier this year, but was
unable to gather enough talks. Nevertheless, it is pleasing to note
that we still have three talks related to Calcite.

Regarding the PR review problem that has been frequently mentioned in
past years, it persists, but it is encouraging to see that new
reviewers are constantly joining the team.

To conclude, I will repeat the questions from previous years:
1) What else are we doing well in the project?
2) What areas do we need to do better?

Please take some time to share your thoughts! Note that this
discussion is for everyone, not only for committers / PMC members;
even if you have never sent an email to the dev list before, now it is
a good time to do so.

Finally, it has been a great privilege to serve as Calcite's PMC Chair
this year, I've leart a lot during the journey and I am very grateful
for the opportunity that I was given. Following our yearly rotation
tradition, I will step down as Chair by the end of the year, and a new
one will have to be chosen. As we discussed some time ago [5], if you
have any suggestion and you would like to put someone forward as a
potential next Chair, please send an email to
priv...@calcite.apache.org. The nomination period for the new chair
will remain open for the next two weeks (till 2023-11-05). The PMC
will study all proposals, discuss, and start a vote soon after
2023-11-05. The change will be effective some time in December once
the resolution is approved by the board.

[1] https://calcite.apache.org/news/2015/10/22/calcite-graduates/
[2] https://calcite.apache.org/news/
[3] https://lists.apache.org/thread/l5qk9z5qr57qxtd2d2yyfzm5x0yp40lp
[4] 
https://news.apache.org/foundation/entry/apache-software-foundation-welcomes-59-new-members
[5] https://lists.apache.org/thread/gplfqs4snr1b6h62cngyvb65sz41z3fk

--

Best,
Benchao Li



--

Best,
Benchao Li



--

Best,
Benchao Li