[jira] [Created] (CALCITE-6507) Random functions are incorrectly considered deterministic

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6507:
--

 Summary: Random functions are incorrectly considered deterministic
 Key: CALCITE-6507
 URL: https://issues.apache.org/jira/browse/CALCITE-6507
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.38.0


{{{}RAND{}}}, {{{}RANDOM{}}}, and {{RAND_INTEGER}} don't override the 
{{isDeterministic}} method, so they get the default behavior, i.e. true, which 
is incorrect.

This can lead to undesired consequences, e.g.:
A) {{RelMetadataQuery#getPulledUpPredicates}} can consider them as constants 
(and place them inside {{{}RelOptPredicateList#constantMap{}}}) since 
{{RelMdPredicates}} calls {{{}RexUtil#isConstant{}}}, which uses a 
{{ConstantFinder}} that evaluates whether a RexCall is constant or not based on 
the operator's isDeterministic value.
B) As a consequence of A) {{SortRemoveConstantKeyRule}} can incorrectly remove 
an "ORDER BY RAND()"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Mihai Budiu joins Calcite PMC

[Ruben Q L]

Congratulations Mihai!! Thanks for your help and your great work!

Best,
Ruben



On Fri, Jul 26, 2024 at 12:20 PM Michael Mior  wrote:

> Congratulations Mihai!
> --
> Michael Mior
> mm...@apache.org
>
>
> On Thu, Jul 25, 2024 at 10:39 PM Benchao Li  wrote:
>
> > I am pleased to announce that Mihai has accepted an invitation to
> > join the Calcite PMC. Mihai has been a consistent and helpful figure
> > in the Calcite community for which we are very grateful. We look
> > forward to the continued contributions and support.
> >
> > Please join me in congratulating Mihai!
> >
> > Benchao (on behalf of the Calcite PMC)
> >
>

[jira] [Created] (CALCITE-6495) Allow ProjectSetOpTransposeRule to work with any subclass of Project

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6495:
--

 Summary: Allow ProjectSetOpTransposeRule to work with any subclass 
of Project
 Key: CALCITE-6495
 URL: https://issues.apache.org/jira/browse/CALCITE-6495
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
 Fix For: 1.38.0


Even if the default configuration of ProjectSetOpTransposeRule matches a 
LogicalProject, theoretically any downstream project could adjust the rule 
config to match a different subclass of Project, with the corresponding 
RelBuilderFactory, to obtain the rule behavior customized for their needs.

However, at this point this cannot work because 
ProjectSetOpTransposeRule#onMatch performs a
{code:java}
final LogicalProject origProject = call.rel(0);
{code}
which leads to a {{ClassCastException}} in this scenario.

Therefore, this line should be changed (and generalized) into
{code:java}
final Project origProject = call.rel(0);
{code}
(as it happens already in other rules, such as FilterSetOpTransposeRule or 
ProjectFilterTransposeRule) to improve the rule's adaptability, without 
impacting the rule's behavior.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6488) Ensure collations created by RelCollations are canonized once

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6488:
--

 Summary: Ensure collations created by RelCollations are canonized 
once
 Key: CALCITE-6488
 URL: https://issues.apache.org/jira/browse/CALCITE-6488
 Project: Calcite
  Issue Type: Task
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
 Fix For: 1.38.0


RelCollations#of canonizes internally the result before returning it; however 
some callers unnecessarily re-canonize the result on their end.
RelCollations#shift does not canonize its result, some callers canonize on 
their end while others don't.

It is proposed to align their behaviour: RelCollations#of and 
RelCollations#shift will canonize internally their results before return, so 
that their callers do not need to do it on their end.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Sortlimit and limit examples

[Ruben Q L]

Eric, if you want to use EnumerableUnion instead of implementing your own
Union, then I think you'll need to implement the appropriate Converter [1]
to transform between conventions LDAP <=> ENUMERABLE

Best,
Ruben

[1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java


On Fri, Jul 19, 2024 at 3:56 PM Eric Berryman 
wrote:

> I implement an LDAPUnion and rule to convert from logicalunion, and of
> course, the error goes away. But I didn’t implement the union correctly, so
> nothing is returned in my query. Is there a way to have my LDAPUnion
> use EnumerableUnion,
> or a way to not have LDAPUnion and take care of this with a rule?
>
> Thank you again!
> Eric
>
> On Tue, Jul 16, 2024 at 14:24 Eric Berryman 
> wrote:
>
> > Well, I was wrong. That didn’t solve my problem with :
> >
> > Missing conversion is LogicalUnion[convention: NONE -> LDAP]
> >
> > Do I have to implement my own union? Is there a way I could just use the
> > EnumerableUnion?
> > ie. NONE -> Enumerable and skip implementing one for my ldap datastore?
> >
> > Thank you!
> > Eric
> >
> > On Tue, Jul 16, 2024 at 09:00 Eric Berryman 
> > wrote:
> >
> >> When I register my tablescan object, and add my rules, I also added a
> >> removeRule for EnumerableRules.ENUMERABLE_MERGE_UNION_RULE, and now
> >> everything works with an offset also.
> >>
> >> Although, I don’t know why this is. I noticed the EnumerableMergeUnion
> >> object in the plan, and thought I would try to remove it, because it was
> >> different.
> >>
> >> Any explanation is appreciated, thank you!!!
> >> Eric
> >>
> >>
> >> On Mon, Jul 15, 2024 at 18:02 Eric Berryman 
> >> wrote:
> >>
> >>> Hello!
> >>>
> >>> I seem to have an issue with my new limit rule which pushes down to the
> >>> datastore. It works fine, unless I add an offset to the fetch. Where I
> end
> >>> up with the following error:
> >>> There are not enough rules… Missing conversion is
> >>> LogicalUnion[convention: NONE -> LDAP]
> >>>
> >>> Why would this only come up when an offset value is added?
> >>>
> >>> Thank you!
> >>> Eric
> >>>
> >>>
> >>> On Fri, Jun 21, 2024 at 09:25 Eric Berryman 
> >>> wrote:
> >>>
>  That’s perfect
> 
> 
> https://github.com/apache/calcite/blob/main/cassandra/src/main/java/org/apache/calcite/adapter/cassandra/CassandraRules.java#L401
> 
>  Thank you for such a quick response!
>  Eric
> 
> 
>  On Fri, Jun 21, 2024 at 09:14 Michael Mior  wrote:
> 
> > Eric,
> >
> > Could you give a more specific example of the failure scenario you're
> > experiencing?
> >
> > For a simple example of how limits can be pushed down, this is done
> in
> > the
> > Cassandra adapter with CassandraLimitRule. It matches an
> > EnumerableLimit on
> > top of a CassandraToEnumerableConverter and then converts that limit
> > to a
> > CassandraLimit which passes along the limit and offset information to
> > CassandraToEnumerableConverter for when the query is executed. The
> > EnumerableLimit is then replaced with a CassandraLimit effectively
> as a
> > placeholder to signal that the limit has been handled.
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > On Fri, Jun 21, 2024 at 8:44 AM Eric Berryman <
> eric.berry...@gmail.com
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > When I add limit to my relbuilder object, the planner gives up. But
> > works
> > > fine without it.
> > >
> > > I wasn’t able to find any examples of using limit and pushing the
> > limit
> > > values down to a data source.
> > >
> > > Could someone help with some links on this subject?
> > >
> > > Thank you!
> > > Eric
> > >
> >
> 
>

[jira] [Created] (CALCITE-6474) Aggregate with constant key can get a RowCount greater than its MaxRowCount

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6474:
--

 Summary: Aggregate with constant key can get a RowCount greater 
than its MaxRowCount
 Key: CALCITE-6474
 URL: https://issues.apache.org/jira/browse/CALCITE-6474
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.38.0


An Aggregate with constant key can get a RowCount greater than its MaxRowCount.

The root cause is that this logic in RelMdMaxRowCount
{code}
// Aggregate with constant GROUP BY always returns 1 row
if (rel.getGroupType() == Aggregate.Group.SIMPLE) {
  final RelOptPredicateList predicateList =
mq.getPulledUpPredicates(rel.getInput());
  if (!RelOptPredicateList.isEmpty(predicateList)
&& allGroupKeysAreConstant(rel, predicateList)) {
return 1D;
  }
}
{code}
is not applied in RelMdRowCount.

Therefore we can get an Aggregate whose MaxRowCount is 1, but its RowCount is X 
(> 1).







--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6468) RelDecorrelator throws AssertionError if correlated variable is used as Aggregate group key

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6468:
--

 Summary: RelDecorrelator throws AssertionError if correlated 
variable is used as Aggregate group key
 Key: CALCITE-6468
 URL: https://issues.apache.org/jira/browse/CALCITE-6468
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.38.0


The problem can be reproduced with this query (a "simplified" version of TPC-DS 
query1):
{code:sql}
WITH agg_sal AS
  (SELECT deptno, sum(sal) AS total FROM emp GROUP BY deptno)
SELECT 1 FROM agg_sal s1
WHERE s1.total > (SELECT avg(total) FROM agg_sal s2 WHERE s1.deptno = s2.deptno)
{code}

If we apply subquery program, FilterAggregateTransposeRule and then we call the 
RelDecorrelator, it will fail with:
{noformat}
java.lang.AssertionError
at 
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:581)
at 
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:495)
...
{noformat}

The problem appears in this assert (RelDecorrelator.java:581):
{code}
assert newPos == newInputOutput.size();
{code}

The root cause seems to be that, a few lines before, when processing the 
correlating variables from {{corDefOutputs}} a certain value is inserted in 
{{mapNewInputToProjOutputs}}:
{code}
if (!frame.corDefOutputs.isEmpty()) {
  for (Map.Entry entry : frame.corDefOutputs.entrySet()) {
RexInputRef.add2(projects, entry.getValue(), newInputOutput);
corDefOutputs.put(entry.getKey(), newPos);
mapNewInputToProjOutputs.put(entry.getValue(), newPos); // <-- HERE
newPos++;
  }
}
{code}

The problem is that this value was already in the map, as it had been inserted 
previously as part of the group key processing:
{code}
for (int i = 0; i < oldGroupKeyCount; i++) {
  final int idx = groupKeyIndices.get(i);
  ...
  // add mapping of group keys.
  outputMap.put(idx, newPos);
  int newInputPos = requireNonNull(frame.oldToNewOutputs.get(idx));
  RexInputRef.add2(projects, newInputPos, newInputOutput);
  mapNewInputToProjOutputs.put(newInputPos, newPos); // <-- HERE added firstly
  newPos++;
}
{code}

Therefore, the unnecessary insertion into {{mapNewInputToProjOutputs}} and the 
subsequent increment of {{newPos}} when the {{CorDef}}s are processed leads to 
the mismatch.

Notice how, right before the assertion, when processing the remaining fields, 
it is verified that the value is not already contained on the 
{{mapNewInputToProjOutputs}}:
{code}
// add the remaining fields
final int newGroupKeyCount = newPos;
for (int i = 0; i < newInputOutput.size(); i++) {
  if (!mapNewInputToProjOutputs.containsKey(i)) { // <-- HERE checked
RexInputRef.add2(projects, i, newInputOutput);
mapNewInputToProjOutputs.put(i, newPos);
newPos++;
  }
}
{code}

Thus, probably the solution would be to apply the same logic when the CorDef 
are processed:
{code}
if (!frame.corDefOutputs.isEmpty()) {
  for (Map.Entry entry : frame.corDefOutputs.entrySet()) {
Integer pos = mapNewInputToProjOutputs.get(entry.getValue());
if (pos == null) {
  RexInputRef.add2(projects, entry.getValue(), newInputOutput);
  corDefOutputs.put(entry.getKey(), newPos);
  mapNewInputToProjOutputs.put(entry.getValue(), newPos);
  newPos++;
} else {
  corDefOutputs.put(entry.getKey(), pos);
}
  }
}
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6460) SortRemoveConstantKeysRule fails with AssertionError due to mismatched collation on resulting Sort

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6460:
--

 Summary: SortRemoveConstantKeysRule fails with AssertionError due 
to mismatched collation on resulting Sort
 Key: CALCITE-6460
 URL: https://issues.apache.org/jira/browse/CALCITE-6460
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.37.0
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.38.0


When {{SortRemoveConstantKeysRule}} is applied, at the end of the process it 
creates the resulting Sort with the new collation (which has removed the 
constant keys from the original one):
{code}
final Sort result =
sort.copy(sort.getTraitSet(), input, RelCollations.of(collationsList));
{code}

However, if there's a collation defined inside's the Sort's traitSet, this copy 
will fail, because there will be a mismatch between the (original) collation in 
the traitSet (which is unchanged at the moment of the copy) and the new 
collation that has been computed by the rule (removing constant keys):

{noformat}
traits=NONE.[1, 2 DESC], collation=[2 DESC]
java.lang.AssertionError: traits=NONE.[1, 2 DESC], collation=[2 DESC]
at org.apache.calcite.rel.core.Sort.(Sort.java:87)
at 
org.apache.calcite.rel.logical.LogicalSort.(LogicalSort.java:48)
at org.apache.calcite.rel.logical.LogicalSort.copy(LogicalSort.java:81)
at org.apache.calcite.rel.core.Sort.copy(Sort.java:150)
at 
org.apache.calcite.rel.rules.SortRemoveConstantKeysRule.onMatch(SortRemoveConstantKeysRule.java:85)
...
{noformat}

This problem only happens if the traitSet includes 
{{RelCollationTraitDef.INSTANCE}}, so it can be unnoticed in many cases. 
However, we can reproduce it by adjusting 
{{RelOptRulesTest#testSortRemovalOneKeyConstant}}:
{code}
sql(sql)
.withVolcanoPlanner(false,  p -> {
p.addRelTraitDef(RelCollationTraitDef.INSTANCE);
RelOptUtil.registerDefaultRules(p, false, false);
})
.withRule(CoreRules.SORT_REMOVE_CONSTANT_KEYS)
.check();
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Draft: board report for 2024 Q2

[Ruben Q L]

+1
Thanks Benchao for preparing the report!


On Wed, Jul 3, 2024 at 3:31 PM Michael Mior  wrote:

> +1 Thanks Benchao!
> --
> Michael Mior
> mm...@apache.org
>
>
> On Wed, Jul 3, 2024 at 8:10 AM Benchao Li  wrote:
>
> > Hello,
> >
> > Below you can find a draft of this quarter's board report. I plan to
> > submit the final version next Tuesday (Jul 9, 2024).
> >
> > Please let me know if you have any additions or corrections.
> >
> >
> > ## Description:
> > Apache Calcite is a highly customizable framework for parsing and
> planning
> > queries on data in a wide variety of formats. It allows database-like
> > access,
> > and in particular a SQL interface and advanced query optimization, for
> data
> > not residing in a traditional database.
> >
> > Avatica is a sub-project within Calcite and provides a framework for
> > building
> > local and remote JDBC and ODBC database drivers. Avatica has an
> independent
> > release schedule and its own repository.
> >
> > ## Project Status:
> > There are no issues requiring board attention.
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-22 (9 years ago)
> > There are currently 74 committers and 28 PMC members in this project.
> > The Committer-to-PMC ratio is roughly 5:2.
> >
> > Community changes, past quarter:
> > - No new PMC members. Last addition was Sergey Nuyanzin on 2024-03-05.
> > - No new committers. Last addition was Hongyu Guo on 2023-11-03.
> >
> > ## Project Activity:
> > Apache Calcite Avatica 1.25.0 was released on 2024-04-05. It is a routine
> > release featuring support for JDK 21, Gradle 8.5 and several bug fixes.
> >
> > Apache Calcite 1.37.0 was released on 2024-05-04. It contains
> contributions
> > from 46 contributors, and resolves 138 issues. It introduces Apache Arrow
> > adapter and StarRocks dialect, adds support for lambda expressions in SQL
> > and
> > ‘Must-filter’ columns. For table function calls it is now possible to use
> > them
> > without TABLE() wrapper in FROM. Furthermore, there is support for
> optional
> > FORMAT of CAST operator from SQL:2016 and more than 15 new SQL functions
> in
> > various libraries such as BigQuery, PostgreSQL and Spark.
> >
> > Besides releases, it's worth to mention that Julian Hyde presented
> > "Measures
> > in SQL" both in SF Distributed Systems Meetup in downtown SF on Wednesday
> > May
> > 22nd, and SIGMOD 2024 conference in Santiago, Chile, which has been
> > implemented in Calcite.
> >
> > ## Community Health:
> > The community maintains a healthy status, previously it's super healthy.
> > The
> > reason is that we did not invite new committers for more than 6 months.
> > There
> > are a few new faces in the community, hopefully we'll invite new
> > committers in
> > the near future.
> >
> > Most of the statistics slightly decreased compared to last quarter (dev@
> > decreased by 39%, issues@ decreased by 22%, commits decreased by 31%,
> code
> > contributors decreased by 32%, active reviewers decreased by 36.8%). The
> > reason I can see is there is less activity in supporting new SQL
> functions
> > in
> > various libraries recently, which is much more active in previous
> quarters.
> >
> >
> > The number of non-committer (contributor) commits per month:
> > +--+---+-+
> > | year | month | contributor_commits |
> > +--+---+-+
> > | 2024 | 4 |  13 |
> > | 2024 | 5 |  11 |
> > | 2024 | 6 |  15 |
> > +--+---+-+
> >
> > The number of active reviewers per month:
> > +--+---+--+
> > | year | month | active_reviewers |
> > +--+---+--+
> > | 2024 | 4 |5 |
> > | 2024 | 5 |4 |
> > | 2024 | 6 |3 |
> > +--+---+--+
> >
> > Top reviewers in the last 3 months:
> > +--+-+
> > | committer| reviews |
> > +--+-+
> > | Mihai Budiu  |  19 |
> > | NobiGo  |   9 |
> > | Julian Hyde|   3 |
> > +--+-+
> >
> >
> > Best,
> > Benchao Li
> >
>

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 4)

[Ruben Q L]

Thanks Sergey for preparing this new RC.

- Release notes: ok (minor comment left in the PR)
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests (from both RC commit and source artifacts): ok (also
confirmed in a Windows env that the ArrowAdapter test issue CALCITE-6390 is
fixed)
- Calcite-based application test suite: ok

+0 (binding)
I would prefer to have the ASM issue fixed or at least workarounded in
1.37, but I understand this is not a new problem and strictly speaking it
is not a Calcite source issue; let's try to find a clean solution for the
next release 1.38 via CALCITE-6393.



On Wed, May 1, 2024 at 12:16 AM Julian Hyde  wrote:

> It seems that Sergey has logged
> https://issues.apache.org/jira/browse/CALCITE-6393 for this. Please add
> further discussion to that case rather than to this vote thread.
>
>
>
> > On Apr 30, 2024, at 12:44 AM, Ruben Q L  wrote:
> >
> > Thanks Guillaume for checking this!
> > When I looked at this issue on 1.36, I had the impression that assertions
> > might have a role to play in the error, but I could not confirm this
> > hypothesis.
> > IMO this smells like a JDK bug (or at least it looks like other resolved
> > issues); note that it seems we had some similar problems in the past with
> > older Java8 versions [1].
> >
> > Independently of the root cause, would it unblock the release process if
> we
> > just remove the problematic asserts, or if we substitute them with an
> > equivalent `if (...) throw new IllegalStateException("...");` ?
> >
> > Best,
> > Ruben
> >
> > [1]
> >
> https://github.com/apache/calcite/blob/1506857f404037b63dfd8a11880393b767bd1544/build.gradle.kts#L98
> >
> >
> >
> > On Mon, Apr 29, 2024 at 11:26 PM Sergey Nuyanzin 
> > wrote:
> >
> >> Hi Guilluame,
> >>
> >> I played a bit more and I realised that if from commit above I just
> remove
> >> one line with assert (assert map != null)
> >> then ArrayIndexOutOfBoundsException disappears
> >>
> >> same for current main branch, if I remove all lines from SqlFunctions
> with
> >> assert (now there are 3 such lines) then
> >> ArrayIndexOutOfBoundsException disappears
> >>
> >> Thus,  it does not look like a Calcite issue, more like a problem on ASM
> >> side
> >> please correct me if I'm wrong
> >>
> >> On Mon, Apr 29, 2024 at 10:33 PM Sergey Nuyanzin 
> >> wrote:
> >>
> >>> i follow the procedure described here
> >>> https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> >>> started
> >>> btw I played a bit with git bisect and it shows that the issue
> >>>
> >>>java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of bounds
> >> for length 297at
> >> org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2695)at
> >> org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2711)
> >>>
> >>> started appearing after this commit
> >>>
> >>>
> >>
> https://github.com/apache/calcite/commit/bcf6bd8577b25c563b1c597c70704594a18ca1a3
> >>>
> >>> On Mon, Apr 29, 2024 at 10:01 PM Guillaume Masse
> >>>  wrote:
> >>>
> >>>> Hi Sergey,
> >>>>
> >>>> thanks for trying that update
> >>>>
> >>>> I confirm I have the same issue with your release at
> >>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/
> >>>>
> >>>>
> >>>>
> >>
> https://github.com/MasseGuillaume/asm-remapper-bug/commit/852e4cd246d278db8acf5e997a54619bc4f85fc7
> >>>>
> >>>> This rules out one of my hypothesis on the java build version.
> >>>>
> >>>> Can you point me to the release procedure you are using? I saw
> >>>> https://calcite.apache.org/develop/#contributing is there more
> precise
> >>>> steps?
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Apr 29, 2024 at 3:27 PM Sergey Nuyanzin 
> >>>> wrote:
> >>>>
> >>>>> I repeated same procedure with jdk1.8u412
> >>>>> here you can find the jars
> >>>>>
> >>>>>
> >>>>
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/
> &

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 4)

[Ruben Q L]

I agree with Guillaume and lean towards a -1.
IMHO we should not produce a release with incorrect bytecode (even if this
is only detectable by other tools like ASM).

Of course, we should try to find the root cause (although it seems quite
elusive); but if there is a simple workaround to generate a RC avoiding
this issue, I think we should go for it.
Personally, I did not create a Jira ticket at the time because apparently
there is nothing wrong on our side: it seemed either an ASM bug (disproved
in [1]), or some kind of problem with the environment of the 1.36 release
manager (OS? JDK?). I honestly thought it would be a one-time-only problem,
but now we have a new release by a different RM with the same issue, so it
seems a bit more serious than I originally expected.

Best,
Ruben

[1] https://gitlab.ow2.org/asm/asm/-/issues/318008


On Tue, Apr 30, 2024 at 10:41 AM Francis Chuang 
wrote:

> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc4/site/_docs/history.md)
>
> - OK
> - Checked year and versions in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> Environments:
> Eclipse-temurin:8 docker container in WSL2 (Ubuntu 22.04.4) on Windows
> 11 23h2
>
> Eclipse-temurin:19 docker container in WSL2 (Ubuntu 22.04.4) on Windows
> 11 23h2
>
> $ docker version
> Client:
>   Cloud integration: v1.0.35+desktop.13
>   Version:   26.0.0
>   API version:   1.45
>   Go version:go1.21.8
>   Git commit:2ae903e
>   Built: Wed Mar 20 15:16:45 2024
>   OS/Arch:   linux/amd64
>   Context:   default
>
> Server: Docker Desktop
>   Engine:
>Version:  26.0.0
>API version:  1.45 (minimum version 1.24)
>Go version:   go1.21.8
>Git commit:   8b79278
>Built:Wed Mar 20 15:18:01 2024
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.28
>GitCommit:ae07eda36dd25f8a1b98dfbf587313b99c0190bb
>   runc:
>Version:  1.1.12
>GitCommit:v1.1.12-0-g51d5e94
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
> $ gradle -v
>
> 
> Gradle 7.6.1
> 
>
> Build time:   2023-02-24 13:54:42 UTC
> Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
>
> Kotlin:   1.7.10
> Groovy:   3.0.13
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  19.0.2 (Eclipse Adoptium 19.0.2+7)
> OS:   Linux 5.15.146.1-microsoft-standard-WSL2 amd64
>
> $ java -version
> openjdk version "1.8.0_402"
> OpenJDK Runtime Environment (Temurin)(build 1.8.0_402-b06)
> OpenJDK 64-Bit Server VM (Temurin)(build 25.402-b06, mixed mode)
>
> $ java -version
> openjdk version "19.0.2" 2023-01-17
> OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7)
> OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode,
> sharing)
>
> Francis
>
> On 30/04/2024 6:35 pm, Stamatis Zampetakis wrote:
> > Ubuntu 20.04.6 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 7.6.1
> >
> >   * Checked signatures and checksums OK
> >   * Checked diff between repo and artifacts OK
> >   * Checked README, NOTICE, LICENSE OK
> >   * All source files have ASF headers OK ( grep -RiL "Licensed to the
> > Apache Software Foundation")
> >   * No unexpected binary files OK (find . -type f -exec file {} \; |
> > grep -v text)
> >   * Checked structure of staged maven repo for calcite-core [1] OK
> >   * Checked LICENSE, NOTICE, signature, and checksum for
> > calcite-core-1.37.0.jar from staged maven repo [1] OK
> >   * Built from git tag and run tests (./gradlew build) OK
> >   * Built from source artifacts and run tests (gradle build) OK
> >
> > +1 (binding)
> >
> > For the ASM problem, I would suggest opening a JIRA ticket and
> > continuing the discussion there. I found the previous discussion [2]
> > but not a ticket for this. We should try to keep vote threads as clean
> > as possible and keep technical exchanges on separate threads/tickets.
> > The main deliverable of this vote is the source package distribution
> > so I don't consider this a blocker for the release especially since
> > this has popped up before and was left untreated.
> >
> > Best,
&g

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 4)

[Ruben Q L]

Thanks Guillaume for checking this!
When I looked at this issue on 1.36, I had the impression that assertions
might have a role to play in the error, but I could not confirm this
hypothesis.
IMO this smells like a JDK bug (or at least it looks like other resolved
issues); note that it seems we had some similar problems in the past with
older Java8 versions [1].

Independently of the root cause, would it unblock the release process if we
just remove the problematic asserts, or if we substitute them with an
equivalent `if (...) throw new IllegalStateException("...");` ?

Best,
Ruben

[1]
https://github.com/apache/calcite/blob/1506857f404037b63dfd8a11880393b767bd1544/build.gradle.kts#L98



On Mon, Apr 29, 2024 at 11:26 PM Sergey Nuyanzin 
wrote:

> Hi Guilluame,
>
> I played a bit more and I realised that if from commit above I just remove
> one line with assert (assert map != null)
> then ArrayIndexOutOfBoundsException disappears
>
> same for current main branch, if I remove all lines from SqlFunctions with
> assert (now there are 3 such lines) then
> ArrayIndexOutOfBoundsException disappears
>
> Thus,  it does not look like a Calcite issue, more like a problem on ASM
> side
> please correct me if I'm wrong
>
> On Mon, Apr 29, 2024 at 10:33 PM Sergey Nuyanzin 
> wrote:
>
> > i follow the procedure described here
> > https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> > started
> > btw I played a bit with git bisect and it shows that the issue
> >
> > java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of bounds
> for length 297at
> org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2695)at
> org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2711)
> >
> > started appearing after this commit
> >
> >
> https://github.com/apache/calcite/commit/bcf6bd8577b25c563b1c597c70704594a18ca1a3
> >
> > On Mon, Apr 29, 2024 at 10:01 PM Guillaume Masse
> >  wrote:
> >
> >> Hi Sergey,
> >>
> >> thanks for trying that update
> >>
> >> I confirm I have the same issue with your release at
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/
> >>
> >>
> >>
> https://github.com/MasseGuillaume/asm-remapper-bug/commit/852e4cd246d278db8acf5e997a54619bc4f85fc7
> >>
> >> This rules out one of my hypothesis on the java build version.
> >>
> >> Can you point me to the release procedure you are using? I saw
> >> https://calcite.apache.org/develop/#contributing is there more precise
> >> steps?
> >>
> >>
> >>
> >> On Mon, Apr 29, 2024 at 3:27 PM Sergey Nuyanzin 
> >> wrote:
> >>
> >> > I repeated same procedure with jdk1.8u412
> >> > here you can find the jars
> >> >
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachecalcite-1231/org/apache/calcite/
> >> > it looks like asm-remapper gives the same result...
> >> >
> >> > On Mon, Apr 29, 2024 at 8:41 PM Guillaume Masse
> >> >  wrote:
> >> >
> >> > > If you take a look at
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://www.openlogic.com/openjdk-downloads?field_java_parent_version_target_id=416_operating_system_target_id=426_architecture_target_id=391_java_package_target_id=All
> >> > >
> >> > > The most recent release is 8u412-b08
> >> > >
> >> > > If it follows more or less the openjdk schedule:
> >> > >
> >> > > https://wiki.openjdk.org/display/jdk8u/Main
> >> > >
> >> > > *Most recent and past release details:*
> >> > >
> >> > >- 8u412-b08 (GA), April 16th 2024 [Release <
> >> > https://bit.ly/openjdk8u412
> >> > > >]
> >> > >[Tag  >]
> >> [
> >> > >Binaries
> >> > ><
> >> > >
> >> https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u412-b08
> >
> >> > >]
> >> > >- 8u402-b06 (GA), January 16th 2024 [Release
> >> > >] [Tag
> >> > >]
> >> > [Binaries
> >> > ><
> >> > >
> >> https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u402-b06
> >
> >> > >]
> >> > >- 8u392-b08 (GA), October 17th 2023 [Release
> >> > >] [Tag
> >> > >]
> >> > [Binaries
> >> > ><
> >> > >
> >> https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u392-b08
> >
> >> > >]
> >> > >- 8u382-b05 (GA), July 18th 2023 [Release <
> >> > https://bit.ly/openjdk8u382
> >> > > >]
> >> > >[Tag  >]
> >> [
> >> > >Binaries
> >> > ><
> >> > >
> >> https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u382-b05
> >
> >> > >]
> >> > >- 8u372-b07 (GA), April 18th 2023 [Release <
> >> > https://bit.ly/openjdk8u372
> >> > > >]
> >> > >[Tag  >]
> >> [
> >> > >Binaries
> >> > ><
> >> > >
> >> 

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 2)

[Ruben Q L]

Thanks Sergey for preparing this new RC.

- Release notes: ok (minor comment on the PR)
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests (from both RC commit and source artifacts): ok
- Calcite-based application test suite: ok

+1 (binding)


On Fri, Apr 26, 2024 at 10:51 AM Francis Chuang 
wrote:

> Thanks for making rc2 available for voting, Sergey and thanks to
> Stamatis for fixing the bug causing tests to fail when run from the
> source distribution.
>
> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc2/site/_docs/history.md)
>
> - OK
> - Checked year and versions in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> Repeating my comment from the previous vote regarding some contributors
> being listed by their username rather than their full name: I think it
> would be better to put in their real names if they are known. These can
> usually be retrieved from their GitHub profiles. This is a minor issue
> that can be dealt with after the release.
>
> Environment:
> Eclipse-temurin:8 docker container in WSL2 (Ubuntu 22.04.4) on Windows
> 11 23h2
>
> $ docker version
> Client:
>   Cloud integration: v1.0.35+desktop.13
>   Version:   26.0.0
>   API version:   1.45
>   Go version:go1.21.8
>   Git commit:2ae903e
>   Built: Wed Mar 20 15:16:45 2024
>   OS/Arch:   linux/amd64
>   Context:   default
>
> Server: Docker Desktop
>   Engine:
>Version:  26.0.0
>API version:  1.45 (minimum version 1.24)
>Go version:   go1.21.8
>Git commit:   8b79278
>Built:Wed Mar 20 15:18:01 2024
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.28
>GitCommit:ae07eda36dd25f8a1b98dfbf587313b99c0190bb
>   runc:
>Version:  1.1.12
>GitCommit:v1.1.12-0-g51d5e94
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
> $ gradle -v
>
> 
> Gradle 7.6.1
> 
>
> Build time:   2023-02-24 13:54:42 UTC
> Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
>
> Kotlin:   1.7.10
> Groovy:   3.0.13
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  1.8.0_402 (Temurin 25.402-b06)
> OS:   Linux 5.15.146.1-microsoft-standard-WSL2 amd64
>
> $ java -version
> openjdk version "1.8.0_402"
> OpenJDK Runtime Environment (Temurin)(build 1.8.0_402-b06)
> OpenJDK 64-Bit Server VM (Temurin)(build 25.402-b06, mixed mode)
>
> Francis
> On 26/04/2024 6:28 am, Sergey Nuyanzin wrote:
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.37.0, release
> > candidate 2 (there was an issue in subject name and actual rc number, so
> > for safety it is 2).
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc2/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=6d8175111ec72f65cb12815d5d519680f62dfd08
> >
> > Its hash is 6d8175111ec72f65cb12815d5d519680f62dfd08
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.37.0-rc2
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.37.0-rc2
> > (revision 68794)
> >
> > The hashes of the artifacts are as follows:
> >
> 373665630145ac52221eeec8456c295c868dfe6a15ded4b7d92e929db769d3c13d21409478c57b93671bca703bf183a24069def42d33d078a5cdec6a2ec69368
> > *apache-calcite-1.37.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1228/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/snuyanzin.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.37.0.
> >
> > The vote is open for the next 72 hours(I think we can extend it a bit
> > given the fact that there is a weekend soon) and passes if a majority of
> at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.37.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
> >
>

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 1)

[Ruben Q L]

I think Stamatis is right. We should cancel this vote, fix CALCITE-6385 and
produce a new RC.

Best,
Ruben


On Wed, Apr 24, 2024 at 12:50 PM Sergey Nuyanzin 
wrote:

> Thanks Stamatis for signalling this issue
>
> I think we need to put Blocker priority on this jira issue
> (CALCITE-6385[1]) if we consider it as a no go for the release (please
> correct me if I'm wrong)
>
> [1] https://issues.apache.org/jira/browse/CALCITE-6385
>
> On Wed, Apr 24, 2024 at 11:59 AM Stamatis Zampetakis 
> wrote:
>
> > Ubuntu 20.04.6 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 7.6.1
> >
> >  * Checked signatures and checksums OK
> >  * Checked diff between repo and artifacts OK
> >  * Went over release note OK (Left comments in the PR)
> >  * Checked README, NOTICE, LICENSE OK
> >  * All source files have ASF headers OK (raised [1] for two files
> > where header is needed)
> >  * No unexpected binary files OK (find . -type f -exec file {} \; |
> > grep -v text)
> >  * Checked LICENSE, NOTICE, signature, and checksum for
> > calcite-core-1.37.0.jar in nexus [2] OK
> >  * Built from git tag and run tests (./gradlew build) OK
> >  * Built from source artifacts and run tests (gradle build) KO (Logged
> > CALCITE-6385 [3])
> >
> > -1 (binding)
> >
> > The negative vote is mainly due to CALCITE-6385. Although the test
> > failure is rather innocent, it makes the build fail so users building
> > from source may have a hard time figuring out what happens.
> >
> > Best,
> > Stamatis
> >
> > [1] https://issues.apache.org/jira/browse/CALCITE-6384
> > [2]
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1225/org/apache/calcite/calcite-core/1.37.0/
> > [3] https://issues.apache.org/jira/browse/CALCITE-6385
> >
> > On Wed, Apr 24, 2024 at 11:25 AM Benchao Li 
> wrote:
> > >
> > > +1 (binding)
> > >
> > > - Checked signature and checksum (OK)
> > > - Checked LICENSE and Copyright year (OK)
> > > - Checked there are no unexpected binary files (OK)
> > > - Build and test from source (testContributorsFileIsSorted and
> > > testMailmapFile in LintTest failed due to not in git repo)
> > > - Reviewed release note, left some comments which can be resolved
> > > after the release
> > >
> > > Ruben Q L  于2024年4月23日周二 16:57写道：
> > > >
> > > > Thanks Sergey for preparing this release.
> > > > Just a minor clarification: the email subject says "release candidate
> > 1",
> > > > but we are actually voting rc0.
> > > >
> > > > - Release notes: ok (with some minor comments that I have left in the
> > PR)
> > > > - Checksum: ok
> > > > - Signature: ok
> > > > - Diff source release and git repository: ok
> > > > - Build + tests: ok
> > > > - Calcite-based application test suite: ok
> > > >
> > > > +1 (binding)
> > > >
> > > > Best,
> > > > Ruben
> > > >
> > > >
> > > >
> > > > On Tue, Apr 23, 2024 at 12:55 AM Francis Chuang <
> > francischu...@apache.org>
> > > > wrote:
> > > >
> > > > > Thanks for being RM for this release, Sergey!
> > > > >
> > > > > My vote is: +1 (binding)
> > > > >
> > > > > - Verified GPG signature - OK
> > > > > - Verified SHA512 - OK
> > > > > - Diffed source release and git repository - OK
> > > > > - Checked release notes on tag
> > > > > (
> > > > >
> >
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc0/site/_docs/history.md
> > )
> > > > >
> > > > > - OK
> > > > > - Checked year and versions in NOTICE, README and HOWTO - OK
> > > > > - Ran tests (gradle check) - OK
> > > > > - Spot checked Nexus artifacts - OK
> > > > >
> > > > > Minor issue with the release notes that should be fixed after the
> > > > > release: Some of the contributors are listed as their GitHub
> > usernames,
> > > > > I think it would be better to put in their real names if they are
> > known.
> > > > > These can usually be retrieved from their GitHub profiles.
> > > > >
> > > > > Environment:
> > > > > Eclipse-temurin:8 docker container in WSL2 (Ubuntu 22.04.4) on
> > Windows
> > > > > 11 23h2
> > 

Re: [VOTE] Release Apache Calcite 1.37.0 (release candidate 1)

[Ruben Q L]

Thanks Sergey for preparing this release.
Just a minor clarification: the email subject says "release candidate 1",
but we are actually voting rc0.

- Release notes: ok (with some minor comments that I have left in the PR)
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best,
Ruben



On Tue, Apr 23, 2024 at 12:55 AM Francis Chuang 
wrote:

> Thanks for being RM for this release, Sergey!
>
> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc0/site/_docs/history.md)
>
> - OK
> - Checked year and versions in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> Minor issue with the release notes that should be fixed after the
> release: Some of the contributors are listed as their GitHub usernames,
> I think it would be better to put in their real names if they are known.
> These can usually be retrieved from their GitHub profiles.
>
> Environment:
> Eclipse-temurin:8 docker container in WSL2 (Ubuntu 22.04.4) on Windows
> 11 23h2
>
> $ docker version
> Client:
>   Cloud integration: v1.0.35+desktop.13
>   Version:   26.0.0
>   API version:   1.45
>   Go version:go1.21.8
>   Git commit:2ae903e
>   Built: Wed Mar 20 15:16:45 2024
>   OS/Arch:   linux/amd64
>   Context:   default
>
> Server: Docker Desktop
>   Engine:
>Version:  26.0.0
>API version:  1.45 (minimum version 1.24)
>Go version:   go1.21.8
>Git commit:   8b79278
>Built:Wed Mar 20 15:18:01 2024
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.28
>GitCommit:ae07eda36dd25f8a1b98dfbf587313b99c0190bb
>   runc:
>Version:  1.1.12
>GitCommit:v1.1.12-0-g51d5e94
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
> $ gradle -v
>
> 
> Gradle 7.6.1
> 
>
> Build time:   2023-02-24 13:54:42 UTC
> Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
>
> Kotlin:   1.7.10
> Groovy:   3.0.13
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  1.8.0_402 (Temurin 25.402-b06)
> OS:   Linux 5.15.146.1-microsoft-standard-WSL2 amd64
>
> $ java -version
> openjdk version "1.8.0_402"
> OpenJDK Runtime Environment (Temurin)(build 1.8.0_402-b06)
> OpenJDK 64-Bit Server VM (Temurin)(build 25.402-b06, mixed mode)
>
> Francis
>
> On 23/04/2024 9:18 am, Sergey Nuyanzin wrote:
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.37.0, release
> > candidate 0.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> https://github.com/apache/calcite/blob/calcite-1.37.0-rc0/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=53fd905eb4c730b4bedd49fa6569d5b0de47f021
> >
> > Its hash is 53fd905eb4c730b4bedd49fa6569d5b0de47f021
> >
> > Tag:
> > https://github.com/apache/calcite/tree/calcite-1.37.0-rc0
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.37.0-rc0
> > (revision 68720)
> >
> > The hashes of the artifacts are as follows:
> >
> b044fb94fd60710142edfe348ff20bbdc7e43f3d70a68b6520b65dc8fb2fe53f0ecf6b5d03b199b3212e9064a7dd04fb3995a9cd09cb8864a7bfdb5617feefbb
> > *apache-calcite-1.37.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1225/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/snuyanzin.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.37.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.37.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
>

[jira] [Created] (CALCITE-6366) Code generated by EnumUtils#convert should throw an exception if the target type is overflowed

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6366:
--

 Summary: Code generated by EnumUtils#convert should throw an 
exception if the target type is overflowed
 Key: CALCITE-6366
 URL: https://issues.apache.org/jira/browse/CALCITE-6366
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Ruben Q L


Code generated by EnumUtils#convert should throw an exception if the target 
type is overflowed (consider using Expressions#convertChecked)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release Apache Calcite Avatica 1.25.0 (release candidate 0)

[Ruben Q L]

- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Release notes: ok; one minor issue: the list item "CALCITE-6137" mentions
the upgrade to "Gradle 8.4", but both the Jira title and the initial
description of the RN say "8.5"

+1 (binding)

Best,
Ruben


On Thu, Apr 4, 2024 at 1:53 PM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> +1 (non-binding) after verifying as follows:
>
> - verified gpg signature: OK
>
> $ curl "https://downloads.apache.org/calcite/KEYS; | gpg --import
>
> $ gpg --verify apache-calcite-avatica-1.25.0-src.tar.gz.asc
> apache-calcite-avatica-1.25.0-src.tar.gz
>
> - verified package checksum: OK
>
> $ diff <(cat apache-calcite-avatica-1.25.0-src.tar.gz.sha512) <(shasum -a
> 512 apache-calcite-avatica-1.25.0-src.tar.gz)
>
> - verified gradle build from sources: OK
>
> $ cp ~/git-apache/calcite-avatica/gradlew .
>
> $ cp -r ~/git-apache/calcite-avatica/gradle/wrapper gradle
>
> $ ./gradlew build -Prelease -PskipSign
>
> $ docker-compose run test
>
> - building commit and running tests: OK
>
> $ ./gradlew build -Prelease -PskipSign
>
> $ docker-compose run test
>
> (checked CI on github for the commit)
>
> - checked release notes: OK
>
> - checked few modules in Nexus: OK
>
> - checking difference in folder: OK
>
> $ diff -qr . ~/git-apache/calcite-avatica | grep -v gradle | grep -v idea
>
> (nothing worth mentioning)
>
> -- Environment used:
>
> $ sw_vers
>
> ProductName: macOS
>
> ProductVersion: 12.6.2
>
> BuildVersion: 21G320
>
> $ ./gradlew -version
>
> 
>
> Gradle 8.5
>
> 
>
> Build time:   2023-11-29 14:08:57 UTC
>
> Revision: 28aca86a7180baa17117e0e5ba01d8ea9feca598
>
> Kotlin:   1.9.20
>
> Groovy:   3.0.17
>
> Ant:  Apache Ant(TM) version 1.10.13 compiled on January 4 2023
>
> JVM:  1.8.0_352 (Azul Systems, Inc. 25.352-b08)
>
> OS:   Mac OS X 12.6.2 aarch64
>
> $ java -version
>
> openjdk version "1.8.0_352"
>
> OpenJDK Runtime Environment (Zulu 8.66.0.15-CA-macos-aarch64) (build
> 1.8.0_352-b08)
>
> OpenJDK 64-Bit Server VM (Zulu 8.66.0.15-CA-macos-aarch64) (build
> 25.352-b08, mixed mode)
>
> Best regards,
>
> Alessandro
>
> On Thu, 4 Apr 2024 at 09:25, Stamatis Zampetakis 
> wrote:
>
> > Ubuntu 20.04.6 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 8.1.1
> >
> >  * Checked signatures and checksums OK
> >  * Checked diff between repo and artifacts OK
> >  * Went over release note OK (It mentions two Gradle versions 8.4 and
> 8.5)
> >  * Checked README, NOTICE, LICENSE OK (minor fixes in LICENSE from
> > Calcite should be ported to Avatica)
> >  * All source files have ASF headers OK (grep -RiL "Licensed to the
> > Apache Software Foundation")
> >  * No unexpected binary files OK (find . -type f -exec file {} \; |
> > grep -v text)
> >  * Checked LICENSE, NOTICE, signature, and checksum for
> > avatica-core-1.25.0.jar in nexus OK
> >  * Built from git tag and run tests (./gradlew build) OK
> >  * Built from source artifacts and run tests (gradle build) OK
> >
> > +1 (binding)
> >
> > Best,
> > Stamatis
> >
> >
> > On Wed, Apr 3, 2024 at 11:36 PM Francis Chuang  >
> > wrote:
> > >
> > > Opps, good catch! Please ignore the links to the
> > > calcite-avatica-site-preview repo. Those are generated by the
> > > asf-release plugin, but we don't use any preview repos for Calcite.
> > >
> > > On 3/04/2024 9:41 pm, Benchao Li wrote:
> > > > +1 (binding)
> > > >
> > > > - checked signature and checksum [OK]
> > > > - checked copyright year in notice [OK]
> > > > - downloaded src, make sure no unexpected files [OK]
> > > > - compiled and tested from source [OK]
> > > > - reviewed release note [OK]
> > > > - checked a few files in Nexus for signature and checksum [OK]
> > > > - downloaded javadoc jar, make sure it's in English [OK]
> > > >
> > > > One minor comment is that those gitbox urls seem not valid.
> > > >
> > > > Francis Chuang  于2024年4月2日周二 11:40写道：
> > > >>
> > > >> Hi all,
> > > >>
> > > >> I have created a build for Apache Calcite Avatica 1.25.0, release
> > > >> candidate 0.
> > > >>
> > > >> Thanks to everyone who has contributed to this release.
> > > >>
> > > >> You can read the release notes here:
> > > >>
> >
> https://github.com/apache/calcite-avatica/blob/avatica-1.25.0-rc0/site/_docs/history.md
> > > >>
> > > >> The commit to be voted upon:
> > > >>
> >
> https://gitbox.apache.org/repos/asf?p=calcite-avatica.git;a=commit;h=62b0fdd3f4e0173d8492eb0e203056f0938dd6f5
> > > >>
> > > >> Its hash is 62b0fdd3f4e0173d8492eb0e203056f0938dd6f5
> > > >>
> > > >> Tag:
> > > >> https://github.com/apache/calcite-avatica/tree/avatica-1.25.0-rc0
> > > >>
> > > >> The artifacts to be voted on are located here:
> > > >>
> >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-avatica-1.25.0-rc0
> > > >>
> > > >> RAT report:
> > > >>
> >
> 

Re: Draft: board report for 2024 Q1

[Ruben Q L]

+1

Thanks Benchao for preparing the report!


On Wed, Apr 3, 2024 at 7:34 PM Michael Mior  wrote:

> +1 Thanks Benchao!
>
> --
> Michael Mior
> mm...@apache.org
>
>
> On Wed, Apr 3, 2024 at 6:11 AM Benchao Li  wrote:
>
> > Hello,
> >
> > Below you can find a draft of this quarter's board report. I plan to
> > submit the final version next Tuesday (Apr 9, 2024).
> >
> > Please let me know if you have any additions or corrections.
> >
> > --
> >
> > Best,
> > Benchao Li
> >
> >
> > ## Description:
> > Apache Calcite is a highly customizable framework for parsing and
> planning
> > queries on data in a wide variety of formats. It allows database-like
> > access,
> > and in particular a SQL interface and advanced query optimization, for
> data
> > not residing in a traditional database.
> >
> > Avatica is a sub-project within Calcite and provides a framework for
> > building
> > local and remote JDBC and ODBC database drivers. Avatica has an
> independent
> > release schedule and its own repository.
> >
> > ## Issues:
> > There are no issues requiring board attention.
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-22 (8 years ago)
> > There are currently 74 committers and 28 PMC members in this project.
> > The Committer-to-PMC ratio is roughly 5:2.
> >
> > Community changes, past quarter:
> > - Sergey Nuyanzin was added to the PMC on 2024-03-05
> > - No new committers. Last addition was Hongyu Guo on 2023-11-03.
> >
> > It's worth mentioning that a few people in Calcite PMC have been
> > invited as ASF members this year:
> > - Benchao Li
> > - Francis Chuang
> > - Ruben Quesada Lopez
> > - Sergey Nuyanzin
> >
> > ## Project Activity:
> > There is no release rolled out in Q1 yet, but we have planned to
> > release avatica 1.25.0 and calcite 1.37.0 lately.  For now, avatica
> > 1.25.0 RC0 is in voting stage, and calcite 1.37.0 will be kicked off
> > just after avatica 1.25.0 is released since we have a few co-related
> > issues for these two versions.
> >
> > ## Community Health:
> > The community maintains a super healthy status, due to the new
> > invitation of PMC member, previously it's healthy.
> >
> > Most of the statistics are steady compared to last quarter, except a
> > slight decrease for the last three weeks. I assume the reason would be
> > that we are in the process of calcite 1.37.0 release.It's a little bit
> > longer than usual, the reason is that we have a few co-related issues
> > for calcite and avatica, and we need to release avatica before
> > calcite.
> >
> > The number of non-committer (contributor) commits per month:
> > +--+---+-+
> > | year | month | contributor_commits |
> > +--+---+-+
> > | 2024 | 1 |  20 |
> > | 2024 | 2 |  20 |
> > | 2024 | 3 |  14 |
> > +--+---+-+
> >
> > The number of active reviewers per month:
> > +--+---+--+
> > | year | month | active_reviewers |
> > +--+---+--+
> > | 2024 | 1 |6 |
> > | 2024 | 2 |7 |
> > | 2024 | 3 |6 |
> > +--+---+--+
> >
> > Top reviewers in the last 3 months:
> > +---+-+
> > | committer | reviews |
> > +---+-+
> > | hongyu guo|  15 |
> > | Mihai Budiu   |  13 |
> > | Tanner Clary  |  10 |
> > +---+-+
> >
>

[jira] [Created] (CALCITE-6338) RelMdCollation#project can return an incomplete list of collations

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6338:
--

 Summary: RelMdCollation#project can return an incomplete list of 
collations
 Key: CALCITE-6338
 URL: https://issues.apache.org/jira/browse/CALCITE-6338
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.36.0
Reporter: Ruben Q L
Assignee: Ruben Q L


{{RelMdCollation#project}} can return an incomplete list of collations.

(I'll try to produce a unit test, for now I'll just describe the situation)

Let us say we have a Project that projects the following expressions (notice 
that $2 will become $1 and $2 after the projection): $0, $2, $2, $3
The Project's input has collation [2, 3]
In order to calculate the Project's own collation, {{RelMdCollation#project}} 
will be called, and a MultiMap targets will be computed because, as in this 
case, a certain "source field" (e.g. 2) can have multiple project targets (e.g. 
1 and 2). However, when the collation is being computed, *only the first target 
will be considered* (and the rest will be discarded):
{code}
  public static @Nullable List project(RelMetadataQuery mq,
  RelNode input, List projects) {
  ...
  for (RelFieldCollation ifc : ic.getFieldCollations()) {
final Collection integers = targets.get(ifc.getFieldIndex());
if (integers.isEmpty()) {
  continue loop; // cannot do this collation
}
fieldCollations.add(ifc.withFieldIndex(integers.iterator().next()));  
// <-- HERE!!
  }
{code}
Because of this, the Project's collation will be [1 3], but there is also 
another valid one ([2 3]), so the correct (complete) result should be: [1 3] [2 
3]

This seems a minor problem, but it can be the root cause of more relevant 
issues. For instance, at the moment I have a scenario (not so easy to reproduce 
with a unit test) where a certain plan with a certain combination of rules in a 
HepPlanner results in a StackOverflow due to SortJoinTransposeRule being fired 
infinitely. The root cause is that, after the first application, the rule does 
not detect that the Join's left input is already sorted (due to the previous 
application of the rule), because there is a "problematic" Project on it (that 
shows the problem described above), which returns only one collation, whereas 
the second collation (the one being discarded) is the Sort's collation, so it 
would be one that would prevent the SortJoinTransposeRule from being re-applied 
over and over.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Sergey Nuyanzin joins Calcite PMC

[Ruben Q L]

Congratulations Sergey!


On Tue, Mar 5, 2024 at 6:48 PM Tanner Clary 
wrote:

> Congrats Sergey!
>
> Tanner
>
> On Tue, Mar 5, 2024 at 4:14 AM Benchao Li  wrote:
>
> > I am pleased to announce that Sergey has accepted an invitation to
> > join the Calcite PMC. Sergey has been a consistent and helpful figure
> > in the Calcite community for which we are very grateful. We look
> > forward to the continued contributions and support.
> >
> > Please join me in congratulating Sergey!
> >
> > Benchao (on behalf of the Calcite PMC)
> >
>

Re: calcite 1.36.0 release procedures

[Ruben Q L]

Just to clarify: on my previous email I did not mean that any of those
issues in particular were the cause of this problem, but rather that it
could be possible that a similar issue on the JDK that was used to build
the release might be the root cause, but I'm not 100% sure.

In fact, according to the tests proposed by Guillaume, it seems that
already the bytecode of SqlFunctions-1.35 was not correct, but for some
reason ASM did not fail with it (as it does with SqlFunctions-1.36). Why is
that? It's hard to tell...

As others have said, I agree that it would be a good idea to add a check to
ensure that the bytecode is correct during the release process.

Ruben



On Mon, Jan 29, 2024 at 8:27 AM Stamatis Zampetakis 
wrote:

> Having a gradle task to ensure that class files are valid bytecode
> seems like a good idea. It may not be 100% bulletproof since we yet
> rely on another tool/library to perform the verification but still
> having an extra check will not hurt if it is lightweight.
>
> It makes sense to enforce a certain JDK version/build during the
> release. In that case I would suggest also having some gradle logic
> perform this validation and not just text in the release instructions.
>
> Automating the release procedure would be great but to do that it is
> required to have reproducible builds [1]. See [2] for more background
> around this topic.
>
> Best,
> Stamatis
>
> [1]
> https://cwiki.apache.org/confluence/display/SECURITY/Reproducible+Builds
> [2] https://lists.apache.org/thread/qkstx4o9qw90fxzqcfnp69w33h7vw2yg
>
> On Fri, Jan 26, 2024 at 11:27 PM Guillaume Masse
>  wrote:
> >
> > Hi Julian,
> >
> > I don't think it's necessary to require JDK > 8. Here is what I propose:
> >
> > 1) I can add a gradle task to make sure each classfile from the release
> > jars are valid bytecode.
> >
> > 2) I think we need to make sure we compile with the latest build
> available.
> > Per https://lists.apache.org/thread/nrt4ysoc14p20sq23z744jyfqh1bznyh it
> was
> > compiled with 8u371 the 10 of November 2023. At the time the latest build
> > of java 8 was 391, so that's 2 builds behind:
> >
> > 371 > 18 April 2023
> > 381 > 18 July 2023
> > 391 > 17 October 2023
> > 401 > 16 January 2024
> >
> > I can't be sure that this was the root cause for the broken classfile.
> >
> > 3) Let's make the release procedure automatic / part of the CI. I can
> > assist on this task, but I will need help from commiters since I don't
> > have any credentials for maven central.
> >
> > Guillaume
> >
> >
> > On Fri, Jan 26, 2024 at 2:44 PM Julian Hyde 
> wrote:
> >
> > > These JDK bugs seem to be fixed in more recent java versions. Should we
> > > mandate that releases are built on a recent JDK (say 17 or 21)?
> Currently
> > > we mandate JDK 8.
> > >
> > >
> > > > On Jan 26, 2024, at 9:03 AM, Ruben Q L  wrote:
> > > >
> > > > I have the impression that this might be caused by a bug in the JDK,
> see
> > > > similar issues:
> > > > https://gitlab.ow2.org/asm/asm/-/issues/317789
> > > > https://bugs.openjdk.org/browse/JDK-8144185
> > > > https://bugs.openjdk.org/browse/JDK-8187805
> > > >
> > > >
> > > >
> > > > On Thu, Dec 21, 2023 at 8:58 PM Julian Hyde 
> wrote:
> > > >
> > > >> I think I was mistaken about Guava. I agree with Benchao's analysis
> > > >> and #2 seems to be caused by something other than Guava.
> > > >>
> > > >> On Thu, Dec 21, 2023 at 3:53 AM Benchao Li 
> > > wrote:
> > > >>>
> > > >>> There are two exceptions here:
> > > >>>
> > > >>> # 1
> > > >>> java.lang.ArrayIndexOutOfBoundsException: Index 65536 out of bounds
> > > >>> for length 297
> > > >>> at org.objectweb.asm.ClassReader.readLabel(ClassReader.java:2695)
> > > >>> at org.objectweb.asm.ClassReader.createLabel(ClassReader.java:2711)
> > > >>> at
> > > >>
> org.objectweb.asm.ClassReader.readTypeAnnotations(ClassReader.java:2777)
> > > >>> at org.objectweb.asm.ClassReader.readCode(ClassReader.java:1929)
> > > >>> at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1515)
> > > >>> at org.objectweb.asm.ClassReader.accept(ClassReader.java:745)
> > > >>> at org.objectweb.asm.ClassReader.accept(ClassReader.java:425)
> > > >>> at remap

Re: calcite 1.36.0 release procedures

[Ruben Q L]

rlier than 20 (-ish) the byte codes will be different because of
> the
> > > > presence/absence of certain varargs methods.
> > > >
> > > >
> > > > > On Dec 19, 2023, at 7:28 PM, Benchao Li 
> wrote:
> > > > >
> > > > > I tried to reproduce this using Guillaume's
> > > > > project(https://github.com/MasseGuillaume/asm-remapper-bug), and
> the
> > > > > findings are below:
> > > > >
> > > > > - compiled latest main branch
> > > > > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on
> > > > > MacOS with M2 chip: reproduced
> > > > > - compiled latest main branch
> > > > > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on
> > > > > Linux with x64 architecture: reproduced
> > > > > - compiled latest main branch
> > > > > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK11(1.11.0_21) on
> > > > > MacOS with M2 chip: cannot reproduce
> > > > > - compiled latest main branch
> > > > > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK17(1.17.0_7) on
> > > > > MacOS with M2 chip: cannot reproduce
> > > > >
> > > > > It seems unrelated to platforms, and I assume it should not either,
> > > > > since Java's bytecode should be platform independent.
> > > > >
> > > > > @Guillaume You said above that with 1.8.0_371-b11 on macos x64, you
> > > > > cannot reproduce the problem, can you confirm that? Per my testing,
> > > > > JDK8's output could all reproduce the problem
> > > > >
> > > > > Guillaume Masse 
> 于2023年12月18日周一
> > > > 23:13写道：
> > > > >>
> > > > >> If I build 1.36.0 and 1.35.0 with the same java version/build
> javap can
> > > > >> read the classfiles. Since the release process is manual it's
> really
> > > > hard
> > > > >> to know what causes the problem. My solution is to build and
> release
> > > > each
> > > > >> Calcite version myself and host it on our own S3 repository.
> > > > >>
> > > > >> On Mon, Dec 18, 2023, 8:56 AM Ruben Q L 
> wrote:
> > > > >>
> > > > >>> Hello,
> > > > >>>
> > > > >>> My project also recently upgraded to Calcite 1.36, and we are
> facing
> > > > the
> > > > >>> exact same issue when trying to shade with ASM.
> > > > >>>
> > > > >>> @Guillame , I can see that
> > > > https://gitlab.ow2.org/asm/asm/-/issues/318008
> > > > >>> has
> > > > >>> been closed, but I can't really understand why, since the
> explanation
> > > > >>> mentions SqlFunctions-1.35.0, but the issue is with 1.36, do you
> have
> > > > more
> > > > >>> info on that?
> > > > >>>
> > > > >>> Best,
> > > > >>> Ruben
> > > > >>>
> > > > >>>
> > > > >>> On Fri, Nov 24, 2023 at 10:09 PM Guillaume Masse
> > > > >>>  wrote:
> > > > >>>
> > > > >>>> We run spark in AWS EMR and it overrides the classpath, so we
> don't
> > > > have
> > > > >>>> control over it:
> > > > >>>>
> > > > >>>> Sparks will pull guava 14.0.1:
> > > > >>>>
> > > > >>>>
> > > > >>>
> > > >
> https://github.com/apache/spark/blob/master/dev/deps/spark-deps-hadoop-3-hive-2.3#L68
> > > > >>>>
> > > > >>>> That's why we need to shade guava.
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Nov 24, 2023 at 4:31 PM Julian Hyde 
> wrote:
> > > > >>>>
> > > > >>>>> The version of Guava we compile against is relevant. In 1.35 we
> > > > >>>>> compiled against Guava 19.0; in 1.36 we compiled against Guava
> > > > >>>>> 32.1.3-jre. The effect would be the same if we compiled
> against any
> > > > >>>>> version of Guava 20.0 or higher, due to the addition of
> > > > >>>>> Preconditions.checkArgument methods that in earlier Guava
> versions
> >

Re: Index Based QueryableTable Implementation

[Ruben Q L]

Hello David,

AFAIK there is no such thing.
It might be interesting for your case to take a look at this presentation
[1], which includes a demo with the initial steps to combine Calcite and
Lucene.

Best,
Ruben

[1] https://youtu.be/meI0W12f_nw


El jue, 25 ene 2024, 1:55,  escribió:

> Hello Calcite Devs,
>
> I'm working on a data mocking system that's accessible via a relational
> API: tables, columns, indexes, etc. I implemented a rudimentary query
> runner layer from scratch, so the project is already usable in MVP
> state. The next step would be the integration of Calcite.
>
> For now, I have created an initial version by implementing the
> ScannableTable interface. This was very easy, but the functionality is,
> of course, slow due to the full table scan, as it does not make use of
> the available indexes. There seems to be a big gap between
> ScannableTable and QueryableTable. Implementing the Queryable interface
> for QueryableTable manually is quite a lot of work.
>
> Is there some existing or proposed method for the (probably common) case
> where indexes are available for an otherwise scannable table? If I do
> need to implement this manually, what should I look out for?
>
> For context, here is the original storage API I defined for the MVP
> version:
>
>
> https://github.com/miniconnect/minibase/tree/master/projects/storage/src/main/java/hu/webarticum/minibase/storage/api
>
> This is my current sketchy table adapter based on ScannableTable:
>
>
> https://github.com/miniconnect/calcite-integration/blob/main/projects/minibase-calcite/src/main/java/hu/webarticum/minibase/calcite/driver/MinibaseCalciteTable.java
>
> Thanks and regards,
> Dávid Horváth

Re: FilterableTable query planner question

[Ruben Q L]

Hi Eric,

If I am not mistaken, in order to achieve that you need to use
FilterSetOpTransposeRule [1], see CoreRules#FILTER_SET_OP_TRANSPOSE.

Best regards,
Ruben

[1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/rules/FilterSetOpTransposeRule.java

On Mon, Jan 8, 2024 at 5:28 PM Eric Berryman 
wrote:

> Thank you,
>
> That makes sense. But it seems I should see a filter pushed below for a
> where on a union, no?
> ie.
> SELECT t2.field
> FROM (
> SELECT id, field FROM table1
> UNION ALL
> SELECT id, field FROM table2
> UNION ALL
> SELECT id, field FROM table3) as t2
> WHERE t2.field = 'test'
>
> Thank you again!
> Eric
>
>
> On Wed, Jan 3, 2024 at 5:43 AM Stamatis Zampetakis 
> wrote:
>
> > Hey Eric,
> >
> > When you have a disjunction in the WHERE clause it may not be safe to
> > push the condition below a join especially when it comes to outer
> > joins. I suppose that the FilterJoinRule [1] is the place that you
> > want to check to see if the filter can be pushed below the join and
> > into the scan.
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> https://github.com/apache/calcite/blob/8d9b27f1ace7f975407920cb88806715b1f0ef82/core/src/main/java/org/apache/calcite/rel/rules/FilterJoinRule.java
> >
> > On Tue, Jan 2, 2024 at 11:22 PM Eric Berryman 
> > wrote:
> > >
> > > Hello,
> > >
> > > I'm making a FilterableTable with LDAP as a backend.
> > >
> > > I noticed in the FilterableTable method:
> > > public Enumerable scan(DataContext root, List
> filters)
> > >
> > > The filters list is empty if the sql where clause is checking the same
> > > field in all tables.
> > > ie.
> > > select test1.field test2.field
> > > from test1
> > > full outer join test2 on test1.id = test2.id
> > > where test1.field = 'myval' or test2.field = 'myval';
> > >
> > > When I do an EXPLAIN PLAN FOR select ...
> > > I notice the BindableTableScan filters array is empty with both:
> > > where test1.field = 'myval' or test2.field = 'myval';
> > > and it has the filter if I remove one:
> > > where test1.field = 'myval'
> > >
> > > Is there an example I could be pointed to help understand the query
> > planner
> > > here, and hopefully write my implementation such that the filters show
> up
> > > as expected for each table?
> > >
> > > Thank you!
> > > Eric
> >
>

Re: Draft: board report for 2023 Q4

[Ruben Q L]

+1
Thanks Benchao for preparing this report!


On Fri, Jan 5, 2024 at 1:46 PM Stamatis Zampetakis 
wrote:

> Looks good thanks for putting it together Benchao!
>
> On Fri, Jan 5, 2024 at 2:41 PM Benchao Li  wrote:
> >
> > Hello,
> >
> > Below you can find a draft of this quarter's board report. I plan to
> > submit the final version next Tuesday (Jan 9, 2024).
> >
> > Please let me know if you have any additions or corrections.
> >
> > Best,
> > Benchao Li
> > ---
> >
> > ## Description:
> > Apache Calcite is a highly customizable framework for parsing and
> planning
> > queries on data in a wide variety of formats. It allows database-like
> > access,
> > and in particular a SQL interface and advanced query optimization, for
> data
> > not residing in a traditional database.
> >
> > Avatica is a sub-project within Calcite and provides a framework for
> > building
> > local and remote JDBC and ODBC database drivers. Avatica has an
> independent
> > release schedule and its own repository.
> >
> > ## Issues:
> > There are no issues requiring board attention.
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-22 (8 years ago)
> > There are currently 74 committers and 27 PMC members in this project.
> > The Committer-to-PMC ratio is roughly 5:2.
> >
> > Community changes, past quarter:
> > - No new PMC members. Last addition was Benchao Li on 2023-01-27.
> > - Hongyu Guo was added as committer on 2023-11-03
> > - Runkang He was added as committer on 2023-11-03
> > - Lei Shen was added as committer on 2023-11-01
> > - Mihai Budiu was added as committer on 2023-10-26
> > - Ran Tao was added as committer on 2023-10-28
> >
> > Following our yearly rotation tradition, the PMC has elected a new
> > chair: Benchao Li.
> >
> > ## Project Activity:
> > Apache Calcite 1.36.0 was released on 2023-11-10. It contains
> > contributions from 30 contributors, and resolves 125 issues.  The new
> > release adds 30 new SQL functions in various libraries such as
> > BigQuery and Spark, contains many improvements hardening TABLESAMPLE,
> > integrates SQL Logic Test suite, and many more core improvements such
> > as the support of recursive WITH and CREATE TABLE ... LIKE DDL.
> >
> > Apache Calcite Avatica 1.24.0 was released on 2023-12-04. It features
> > mostly dependency upgrades with some minor bug fixes and features, and
> > contains a breaking change: due to CALCITE-5678, date literals not
> > satisfying the Gregorian calendar will be rejected.
> >
> > Apache Calcite Avatica Go 5.3.0 was released on 2023-12-11. It is a
> > maintenance release of Avatica Go including dependency updates and bug
> > fixes. This release supports Go 1.20 and 1.21, which are currently the
> > versions supported and maintained by the Go team.
> >
> > ## Community Health:
> > The community maintains a healthy status.
> >
> > Mailing list and JIRA activity has slightly dropped (dev@ -4%, issues@
> > -36%, JIRA opened -47%, JIRA closed -33%) due to the holiday of
> > Chrismas and New Year. However, commit (+52%) and PRs closed(+27%) has
> > increased on the contrary, one reason is we have many commits/PRs
> > testing the unstable test problem in CALCITE-6123, another reason is
> > some new contributors are used to opening and closeing the PR
> > repeatedly.
> >
> > The number of non-committer (contributor) commits per month:
> > +--+---+-+
> > | year | month | contributor_commits |
> > +--+---+-+
> > | 2023 |10 |  35 |
> > | 2023 |11 |  21 |
> > | 2023 |12 |  13 |
> > +--+---+-+
> >
> > The number of active reviewers per month:
> > +--+---+--+
> > | year | month | active_reviewers |
> > +--+---+--+
> > | 2023 |10 |8 |
> > | 2023 |11 |   12 |
> > | 2023 |12 |6 |
> > +--+---+--+
> >
> > Top reviewers in the last 3 months:
> > +--+-+
> > | committer| reviews |
> > +--+-+
> > | Jiajun  |  11 |
> > | Julian Hyde|  11 |
> > | Mihai Budiu  |   9 |
> > +--+-+
> >
> >
> > The number of non-committer commits and active reviewers are slightly
> > decreased compared to Q3 due to the holidays, but the number is still
> > very good compared to times before 2023 Q2.
>

Re: [ANNOUNCE] New Calcite PMC chair: Benchao Li

[Ruben Q L]

Congratulations Benchao!!


On Thu, Dec 21, 2023 at 10:05 AM Sergey Nuyanzin 
wrote:

> Congratulations, Benchao!
>
> On Thu, Dec 21, 2023 at 10:47 AM Francis Chuang 
> wrote:
>
> > Congratulations, Benchao!
> >
> > On 21/12/2023 8:24 pm, Guangdong Liu wrote:
> > > Congratulations!
> > >
> > > --
> > >
> > > Best Regards
> > >
> > > 
> > >
> > > Liugddx
> > > liug...@gmail.com
> > >
> > >
> > > Stamatis Zampetakis  于2023年12月21日周四 17:18写道：
> > >
> > >> Calcite community members,
> > >>
> > >> I am pleased to announce that we have a new PMC chair and VP as per
> > >> our tradition of rotating the chair once a year. I have resigned, and
> > >> Benchao Li was duly elected by the PMC and approved unanimously by the
> > >> Board.
> > >>
> > >> Please join me in congratulating Benchao!
> > >>
> > >> Best regards,
> > >> Stamatis
> > >>
> > >
> >
>
>
> --
> Best regards,
> Sergey
>

Re: calcite 1.36.0 release procedures

[Ruben Q L]

Hello,

My project also recently upgraded to Calcite 1.36, and we are facing the
exact same issue when trying to shade with ASM.

@Guillame , I can see that https://gitlab.ow2.org/asm/asm/-/issues/318008 has
been closed, but I can't really understand why, since the explanation
mentions SqlFunctions-1.35.0, but the issue is with 1.36, do you have more
info on that?

Best,
Ruben


On Fri, Nov 24, 2023 at 10:09 PM Guillaume Masse
 wrote:

> We run spark in AWS EMR and it overrides the classpath, so we don't have
> control over it:
>
> Sparks will pull guava 14.0.1:
>
> https://github.com/apache/spark/blob/master/dev/deps/spark-deps-hadoop-3-hive-2.3#L68
>
> That's why we need to shade guava.
>
>
> On Fri, Nov 24, 2023 at 4:31 PM Julian Hyde  wrote:
>
> > The version of Guava we compile against is relevant. In 1.35 we
> > compiled against Guava 19.0; in 1.36 we compiled against Guava
> > 32.1.3-jre. The effect would be the same if we compiled against any
> > version of Guava 20.0 or higher, due to the addition of
> > Preconditions.checkArgument methods that in earlier Guava versions
> > would be handled by varargs method.
> >
> > See https://issues.apache.org/jira/browse/CALCITE-5477 (which was
> > fixed in 1.35) and https://issues.apache.org/jira/browse/CALCITE-5763
> > (which was fixed in 1.36 and essentially reverted 5477).
> >
> > Julian
> >
> > On Fri, Nov 24, 2023 at 10:20 AM Guillaume Masse
> >  wrote:
> > >
> > > Hi Benchao Li,
> > >
> > > thanks for giving me more details like the java version,
> > >
> > > I compiled with 1.8.0_371-b11 on macos x64 (emulated) and I was still
> > able
> > > to transform the classfile. The bug must come from something else. Was
> > this
> > > compiled on your personal machine? Was it on windows/linux/mac? What's
> > the
> > > processor architecture?
> > >
> > > 1.35.0 also pases the transformation and the same piece of code is
> there.
> > > How was 1.35.0 released?
> > >
> > > Thanks!
> > >
> > >
> > > On Thu, Nov 23, 2023 at 9:59 PM Benchao Li 
> wrote:
> > >
> > > > Guillaume,
> > > >
> > > > The binary release are always compiled with JDK8, this is a required
> > > > procedure[1].
> > > >
> > > > For 1.36.0, the JDK version I'm using is:
> > > > $ java -version
> > > > java version "1.8.0_371"
> > > > Java(TM) SE Runtime Environment (build 1.8.0_371-b11)
> > > > Java HotSpot(TM) 64-Bit Server VM (build 25.371-b11, mixed mode)
> > > >
> > > > [1]
> > https://calcite.apache.org/docs/howto.html#making-a-release-candidate
> > > >
> > > > Guillaume Masse 
> 于2023年11月24日周五
> > > > 07:22写道：
> > > > >
> > > > > Hi all,
> > > > >
> > > > > We recently upgraded to calcite 1.36.0 and it broke our deployment
> > > > process.
> > > > > We are using java asm   to shade (rename
> > packages
> > > > from
> > > > > the bytecode) since there are multiple dependencies clash when
> using
> > with
> > > > > Apache Spark. For example, Apache Spark is using a much older
> > version of
> > > > > guava. If I build calcite locally with Java Correto 17.0.7-amzn I
> can
> > > > > transform calcite's bytecode without any problem. Where can I find
> > the
> > > > java
> > > > > version used to compiled calcite? Is there a process in place to
> > have a
> > > > > consistent java version?
> > > > >
> > > > > https://gitlab.ow2.org/asm/asm/-/issues/318008
> > > > >
> > > > >
> > > > > --
> > > > > Guillaume Massé
> > > > > [Gee-OHM]
> > > > > (马赛卫)
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best,
> > > > Benchao Li
> > > >
> >
>

Re: [VOTE] Release Apache Calcite 1.36.0 (release candidate 0)

[Ruben Q L]

Thanks Benchao for preparing this release.

- Release notes: ok
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best,
Ruben


On Tue, Nov 7, 2023 at 9:32 PM Francis Chuang 
wrote:

> Thanks for being RM for this release, Benchao!
>
> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.36.0-rc0/site/_docs/history.md)
>
> - OK
> - Checked year and versions in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> Environment:
> Eclipse-temurin:19-jammy docker container in WSL2 (Ubuntu 22.04.3) on
> Windows 11 22h2
>
> $ docker version
> Client: Docker Engine - Community
>   Cloud integration: v1.0.35+desktop.5
>   Version:   24.0.6
>   API version:   1.43
>   Go version:go1.20.7
>   Git commit:ed223bc
>   Built: Mon Sep  4 12:32:16 2023
>   OS/Arch:   linux/amd64
>   Context:   default
>
> Server: Docker Desktop
>   Engine:
>Version:  24.0.6
>API version:  1.43 (minimum version 1.12)
>Go version:   go1.20.7
>Git commit:   1a79695
>Built:Mon Sep  4 12:32:16 2023
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.22
>GitCommit:8165feabfdfe38c65b599c4993d227328c231fca
>   runc:
>Version:  1.1.8
>GitCommit:v1.1.8-0-g82f18fe
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
> $ gradle -v
>
> 
> Gradle 7.6.1
> 
>
> Build time:   2023-02-24 13:54:42 UTC
> Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
>
> Kotlin:   1.7.10
> Groovy:   3.0.13
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  19.0.2 (Eclipse Adoptium 19.0.2+7)
> OS:   Linux 5.15.90.1-microsoft-standard-WSL2 amd64
>
> $ java --version
> openjdk 19.0.2 2023-01-17
> OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7)
> OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode,
> sharing)
>
> Francis
>
> On 8/11/2023 7:55 am, Julian Hyde wrote:
> > Oops, Ubuntu 23.10. Obviously.
> >
> > On Tue, Nov 7, 2023 at 12:02 PM Julian Hyde  wrote:
> >>
> >> +1 (binding)
> >>
> >> Downloaded; checked signatures & hashes; checked LICENSE, NOTICE,
> >> README, README.md, howto.md, history.md; ran RAT; compared to git tag.
> >> Built and ran tests using Gradle 7.4.2, OpenJDK 19.0.2, Ubuntu 24.10.
> >>
> >> Thanks Benchao!
> >>
> >> Julian
> >>
> >> On Tue, Nov 7, 2023 at 7:41 AM LakeShen 
> wrote:
> >>>
> >>> Thanks to Benchao’s contributions
> >>>
> >>> +1(binding)
> >>>
> >>> Best,
> >>> LakeShen
> >>>
> >>> Benchao Li  于2023年11月7日周二 13:20写道：
> >>>
>  Hi all,
> 
>  I have created a build for Apache Calcite 1.36.0, release
>  candidate 0.
> 
>  Thanks to everyone who has contributed to this release.
> 
>  You can read the release notes here:
> 
> 
> https://github.com/apache/calcite/blob/calcite-1.36.0-rc0/site/_docs/history.md
> 
>  The commit to be voted upon:
> 
> 
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=0be8eaebcf27afae9ecda8ab79db63c214426561
> 
>  Its hash is 0be8eaebcf27afae9ecda8ab79db63c214426561
> 
>  Tag:
>  https://github.com/apache/calcite/tree/calcite-1.36.0-rc0
> 
>  The artifacts to be voted on are located here:
> 
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.36.0-rc0
>  (revision 65079)
> 
>  The hashes of the artifacts are as follows:
> 
> 
> 3bf003a87f76f2fa85b6af9b5243cddf67c767cb5caa828532784bd380e0d929da097677a453dc8d6a0d31f8cf97085031cf66b15a390571065b666095db3b80
>  *apache-calcite-1.36.0-src.tar.gz
> 
>  A staged Maven repository is available for review at:
> 
> 
> https://repository.apache.org/content/repositories/orgapachecalcite-1219/org/apache/calcite/
> 
>  Release artifacts are signed with the following key:
>  https://people.apache.org/keys/committer/libenchao.asc
>  https://www.apache.org/dist/calcite/KEYS
> 
>  To create the jars and test Apache Calcite: "gradle build"
>  (requires an appropriate Gradle/JDK installation)
> 
>  Please vote on releasing this package as Apache Calcite 1.36.0.
> 
>  The vote is open for the next 72 hours and passes if a majority of at
>  least three +1 PMC votes are cast.
> 
>  [ ] +1 Release this package as Apache Calcite 1.36.0
>  [ ]  0 I don't feel strongly about it, but I'm okay with the release
>  [ ] -1 Do not release 

[jira] [Created] (CALCITE-6089) EnumerableSortedAggregate fails with ClassCastException: class X cannot be cast to class org.apache.calcite.runtime.FlatLists$ComparableList

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6089:
--

 Summary: EnumerableSortedAggregate fails with ClassCastException: 
class X cannot be cast to class 
org.apache.calcite.runtime.FlatLists$ComparableList
 Key: CALCITE-6089
 URL: https://issues.apache.org/jira/browse/CALCITE-6089
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.35.0
Reporter: Ruben Q L


The problem can be reproduced with this test (to be added e.g. into 
{{EnumerableSortedAggregateTest.java}}):

{code}
  @Test void sortedAggCountUnion() {
tester(false, new HrSchema())
.query(
"select count(*) as c from ( "
+ "select * from emps where deptno=10 "
+ "union all "
+ "select * from emps where deptno=20)")
.withHook(Hook.PLANNER, (Consumer) planner -> {
  planner.removeRule(EnumerableRules.ENUMERABLE_AGGREGATE_RULE);
  planner.addRule(EnumerableRules.ENUMERABLE_SORTED_AGGREGATE_RULE);
})
.explainContains(
"EnumerableSortedAggregate(group=[{}], c=[COUNT()])\n"
+ "  EnumerableUnion(all=[true])\n"
+ "EnumerableCalc(expr#0..4=[{inputs}], 
expr#5=[CAST($t1):INTEGER NOT NULL], expr#6=[10], expr#7=[=($t5, $t6)], 
commission=[$t4], $condition=[$t7])\n"
+ "  EnumerableTableScan(table=[[s, emps]])\n"
+ "EnumerableCalc(expr#0..4=[{inputs}], 
expr#5=[CAST($t1):INTEGER NOT NULL], expr#6=[20], expr#7=[=($t5, $t6)], 
commission=[$t4], $condition=[$t7])\n"
+ "  EnumerableTableScan(table=[[s, emps]])")
.returnsOrdered(
"c=4");
  }
{code}

Which fails with:
{noformat}
...
Caused by: java.lang.ClassCastException: class java.lang.Integer cannot be cast 
to class org.apache.calcite.runtime.FlatLists$ComparableList (java.lang.Integer 
is in module java.base of loader 'bootstrap'; 
org.apache.calcite.runtime.FlatLists$ComparableList is in unnamed module of 
loader 'app')
at Baz$6.compare(Unknown Source)
at 
org.apache.calcite.linq4j.EnumerableDefaults$SortedAggregateEnumerator.moveNext(EnumerableDefaults.java:938)
at 
org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:679)
at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:97)
...
{noformat}

The same test with EnumerableAggregate (instead of EnumerableSortedAggregate) 
will execute the query correctly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-6087) EnumerableSortedAggregate returns incorrect result when input is empty

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-6087:
--

 Summary: EnumerableSortedAggregate returns incorrect result when 
input is empty
 Key: CALCITE-6087
 URL: https://issues.apache.org/jira/browse/CALCITE-6087
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.35.0
Reporter: Ruben Q L


Performing a MAX on an empty table (or on a table where we apply a filter which 
does not return any value) shall return NULL, we can verify this with our 
"standard" {{EnumerableAggregate}} operator:
{code:java}
  @Test void enumerableAggregateOnEmptyInput() {
tester(false, new HrSchema())
.query("select max(deptno) as m from emps where deptno>100")
.explainContains(
"EnumerableAggregate(group=[{}], m=[MAX($1)])\n"
+ "  EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], 
expr#6=[>($t1, $t5)], proj#0..4=[{exprs}], $condition=[$t6])\n"
+ "EnumerableTableScan(table=[[s, emps]])")
.returnsOrdered("m=null");
  }
{code}
However, if we use {{Hook.PLANNER}} to force the aggregation to be implemented 
via {{EnumerableSortedAggregate}} on the same query:
{code:java}
  @Test void enumerableSortedAggregateOnEmptyInput() {
tester(false, new HrSchema())
.query("select max(deptno) as m from emps where deptno>100")
.withHook(Hook.PLANNER, (Consumer) planner -> {
  planner.removeRule(EnumerableRules.ENUMERABLE_AGGREGATE_RULE);
  planner.addRule(EnumerableRules.ENUMERABLE_SORTED_AGGREGATE_RULE);
})
.explainContains(
"EnumerableSortedAggregate(group=[{}], m=[MAX($1)])\n"
+ "  EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], 
expr#6=[>($t1, $t5)], proj#0..4=[{exprs}], $condition=[$t6])\n"
+ "EnumerableTableScan(table=[[s, emps]])")
.returnsOrdered("m=null");
  }
{code}
It fails, because the EnumerableSortedAggregate returns an empty result set 
rather than NULL:
{noformat}
java.lang.AssertionError: 
Expected: "m=null"
 but: was ""
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New committer: Lei Shen

[Ruben Q L]

Congratulations!!


On Wed, Nov 1, 2023 at 12:09 AM Zhengqiang Duan 
wrote:

> Congratulations, Lei.
>
> Best regards,
> Zhengqiang
>
>
> Benchao Li  于2023年11月1日周三 07:50写道：
>
> > Congrats, Lei! Well deserved!
> >
> > Tanner Clary  于2023年11月1日周三 05:12写道：
> > >
> > > Congratulations, Lei!
> > >
> > > Best,
> > > Tanner
> > >
> > > On Tue, Oct 31, 2023 at 1:52 PM Francis Chuang <
> francischu...@apache.org
> > >
> > > wrote:
> > >
> > > > Congrats, Lei!
> > > >
> > > > On 1/11/2023 4:34 am, Stamatis Zampetakis wrote:
> > > > > Apache Calcite's Project Management Committee (PMC) has invited Lei
> > Shen
> > > > to
> > > > > become a committer, and we are pleased to announce that they have
> > > > accepted.
> > > > >
> > > > > Lei has added many new optimizer rules to the project enhancing the
> > > > > query plans for Calcite and downstream projects and pushed various
> > > > > enhancements around the TABLESAMPLE clause. Apart from code
> > > > > contributions, Lei is very active in the dev list participating in
> > > > > discussions and helping a lot with reviews.
> > > > >
> > > > > Lei, welcome, thank you for your contributions, and we look forward
> > to
> > > > your
> > > > > further interactions with the community! If you wish, please feel
> > free
> > > > to tell
> > > > > us more about yourself and what you are working on.
> > > > >
> > > > > As your first commit, please add yourself to the contributors list
> > [1]
> > > > > and the community page will re-generate [2].
> > > > >
> > > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > > >
> > > > > [1]
> > > >
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > > > [2] https://calcite.apache.org/community/#project-members
> > > >
> >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>

Re: [ANNOUNCE] New committer: Ran Tao

[Ruben Q L]

Congrats!


On Fri, Oct 27, 2023 at 4:40 PM Tanner Clary 
wrote:

> Congrats Ran!! Have learned a lot from your pull requests!
>
> Best,
> Tanner
>
> On Fri, Oct 27, 2023 at 8:32 AM Stamatis Zampetakis 
> wrote:
>
> > Apache Calcite's Project Management Committee (PMC) has invited Ran Tao
> to
> > become a committer, and we are pleased to announce that he has accepted.
> >
> > Ran has implemented numerous improvements and fixes in the past few
> > months. He has added new SQL functions to the Spark library, addressed
> > various issues related to casts and type inference, enhanced
> > documentation, and made various changes with the aim of improving code
> > quality
> >
> > Ran, welcome, thank you for your contributions, and we look forward to
> your
> > further interactions with the community! If you wish, please feel free to
> > tell
> > us more about yourself and what you are working on.
> >
> > As your first commit, please add yourself to the contributors list [1]
> > and the community page will re-generate [2].
> >
> > Stamatis (on behalf of the Apache Calcite PMC)
> >
> > [1]
> > https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > [2] https://calcite.apache.org/community/#project-members
> >
>

Re: [ANNOUNCE] New committer: Mihai Budiu

[Ruben Q L]

Congratulations! Welcome on board!




On Thu, Oct 26, 2023 at 10:37 AM Benchao Li  wrote:

> Congrats, Mihai! Well deserved!
>
> Francis Chuang  于2023年10月26日周四 16:59写道：
> >
> > Congrats, Mihai!
> >
> > On 26/10/2023 7:56 pm, Stamatis Zampetakis wrote:
> > > Apache Calcite's Project Management Committee (PMC) has invited Mihai
> > > Budiu to become a committer, and we are pleased to announce that he
> > > has accepted.
> > >
> > > Mihai has addressed numerous compile and runtime issues in SQL
> > > functions and has made several significant improvements to the test
> > > infrastructure to enhance code coverage, quality, and stability. In
> > > addition to code contributions, Mihai is highly active on the mailing
> > > list, where he answers questions, assists others, and shares ideas for
> > > future improvements. Mihai has also been writing blogs and giving
> > > talks about Calcite at various events and conferences, showcasing
> > > Calcite in both simple and more advanced use cases, which can greatly
> > > contribute to the growth of the community.
> > >
> > > Mihai, welcome, thank you for your contributions, and we look forward
> > > to your further interactions with the community! If you wish, please
> > > feel free to tell us more about yourself and what you are working on.
> > >
> > > As your first commit, please add yourself to the contributors list [1]
> > > and the community page will re-generate [2].
> > >
> > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> > > [1]
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > [2] https://calcite.apache.org/community/#project-members
>
>
>
> --
>
> Best,
> Benchao Li
>

Re: How to handle a dynamic param as a set of values?

[Ruben Q L]

Benedek, AFAIK what you describe is simply not possible with Calcite
currently.

Best,
Ruben


On Wed, Oct 25, 2023 at 12:22 PM Benchao Li  wrote:

> Benedek,
>
> Per my understanding, dynamic parameters should be some placeholders
> of literals, which means that each dynamic parameter can only be a
> single literal. Hence, your requirement cannot be achieved by this
> definition.
>
> BTW, do you know any databases that have the ability as you described?
>
> Benedek Halasi  于2023年10月25日周三 19:12写道：
> >
> > Hi,
> >
> > We're looking for a way to parse and execute queries with Apache Calcite
> > that contain an `IN (?)` expression, where the dynamic parameter is a set
> > of values.
> >
> > As we've observed, Calcite handles dynamic parameters as single values,
> and
> > we haven't found a way to make it handle them differently. This causes
> some
> > transformations to be flawed (e.g., `IN (?)` gets transformed to `= ?`).
> >
> > Has anyone encountered this? Is there a go-to solution?
> >
> > Thanks in advance,
> > Ben
>
>
>
> --
>
> Best,
> Benchao Li
>

Re: Draft: board report for 2023 Q3

[Ruben Q L]

+1, thanks Stamatis!

@Sergey thanks for the info! IMO your talk would be a nice addition for the
next report (Q4).


On Mon, Oct 2, 2023 at 9:48 AM Francis Chuang 
wrote:

> +1, Excellent work, Stamatis!
>
> On 2/10/2023 6:58 pm, Stamatis Zampetakis wrote:
> > Hello,
> >
> > Below you can find a draft of this quarter's board report. I plan to
> > submit the final version next Monday (October 9, 2023).
> >
> > Please let me know if you have any additions or corrections.
> >
> > Best,
> > Stamatis
> > ---
> >
> > ## Description:
> > Apache Calcite is a highly customizable framework for parsing and
> planning
> > queries on data in a wide variety of formats. It allows database-like
> > access,
> > and in particular a SQL interface and advanced query optimization, for
> data
> > not residing in a traditional database.
> >
> > Avatica is a sub-project within Calcite and provides a framework for
> > building
> > local and remote JDBC and ODBC database drivers. Avatica has an
> independent
> > release schedule and its own repository.
> >
> > ## Project Status:
> > Current project status: ongoing
> > Issues for the board: none
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-22 (8 years ago)
> > There are currently 69 committers and 27 PMC members in this project.
> > The Committer-to-PMC ratio is roughly 2:1.
> >
> > Community changes, past quarter:
> > - No new PMC members. Last addition was Benchao Li on 2023-01-27.
> > - TJ Banghart was added as committer on 2023-07-04
> > - Dan Zou was added as committer on 2023-07-04
> >
> > ## Project Activity:
> > Apache Calcite 1.35.0 was released on 2023-07-26. It contains
> contributions
> > from 36 contributors, and resolves 140 issues. The new release has many
> > improvements in the BigQuery and Spark dialect bringing in more than 40
> SQL
> > functions. Additionally, it comes with new optimizations for reducing
> the size
> > of generated code and more powerful expression simplifications.
> >
> > On August 18, 2023, Benchao Li and Jiajun Xie represented the Calcite
> > community at the Apache Con East Asia by giving talks related with the
> > project.
> >
> > ## Community Health:
> > The project is healthy. Previously, it was super healthy. The drop is
> likely
> > to the fact that the PMC has not grown in the last six months but this
> will
> > very likely change in the near future since a lot of our current
> committers
> > are very involved with the project and hopefully they will join the PMC
> > shortly.
> >
> > The dev list had a 38% in activity in the past quarter, with busiest
> threads
> > been as usual those around releases and introduction of new committers.
> The
> > 16% increase in activity of the JIRA also contributes to the general
> increase
> > in traffic of the dev list.
> >
> > The number of non-committer (contributor) commits per month:
> > +-+-+-+
> > |year |month| contributor_commits |
> > +-+-+-+
> > | 2023| 7   | 16  |
> > | 2023| 8   | 32  |
> > | 2023| 9   | 32  |
> > +-+-+-+
> >
> > The number of active reviewers per month:
> > +-+-+-+
> > |year |month|  active_reviewers   |
> > +-+-+-+
> > | 2023| 7   | 9   |
> > | 2023| 8   | 9   |
> > | 2023| 9   | 10  |
> > +-+-+-+
> >
> > Top-3 reviewers in the last quarter:
> > +---+-+
> > | committer |   reviews   |
> > +---+-+
> > | Jiajun  | 15  |
> > | Julian Hyde  | 13  |
> > | Benchao Li  | 11  |
> > +---+-+
> >
> > The number of non-committer commits has increased by 9% from the last
> quarter
> > (73 commits in Q2 vs. 80 commits in Q3) keeping up the good momentum of
> having
> > new people contributing to the project.
> >
> > The average number of active reviewers per month has increased slightly
> from
> > the last quarter (7.6 in Q2 vs. 9.3 in Q3) showing that more people are
> > participating in the review process which is among the main points of the
> > project.
> >
> > In the top reviewers, we can observe that things are a bit more balanced
> in
> > Q3. The review count per person in the top-3 tier is lower than usual
> but in
> > conjunction with the increase in the number of 

[jira] [Created] (CALCITE-5980) QuidemTest are not effectively executed on Windows

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5980:
--

 Summary: QuidemTest are not effectively executed on Windows
 Key: CALCITE-5980
 URL: https://issues.apache.org/jira/browse/CALCITE-5980
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.35.0
Reporter: Ruben Q L
 Fix For: 1.36.0


Discovered by accident on my Windows+IntelliJ environment while I was trying to 
add new tests on a *.iq file. My new tests did not seem to be executed. I even 
tried adding syntax errors on purpose into different iq files to force them to 
fail, but the tests were still successful. The reason seems to be that, at 
least on my environment (Windows), the test files do not execute any of their 
statements. This seems a consequence of CALCITE-5786.


While debugging, I found this line in QuidemTest.java (that aims to create the 
inFile and outFile):
{code:java}
protected void checkRun(String path) throws Exception {
  ...
  // e.g. path = "sql/agg.iq"
  // inUrl = "file:/home/fred/calcite/core/build/resources/test/sql/agg.iq"
  // inFile = "/home/fred/calcite/core/build/resources/test/sql/agg.iq"
  // outDir = "/home/fred/calcite/core/build/quidem/test/sql"
  // outFile = "/home/fred/calcite/core/build/quidem/test/sql/agg.iq"
  inFile = Sources.of(requireNonNull(inUrl, "inUrl")).file();
  outFile = replaceDir(inFile, "resources", "quidem");
  ...
}
{code}
But it results on {*}both files being the same{*}, thus when the outFile is 
created, it actually erases all the tests that were contained inside the 
inFile, so the test runs nothing.

The reason for that is that the auxiliary method {{{}replaceDir{}}}:
{code:java}
  private static File replaceDir(File file, String target, String replacement) {
return new File(
file.getAbsolutePath().replace(n2u('/' + target + '/'),
n2u('/' + replacement + '/')));
  }
{code}
is trying to replace "/resources/" with "/quidem/" from the inFile absolute 
path, but in my case this path does not contain the pattern to be replaced, 
since it contains backslashes: 
"C:\...\calcite\core\build\resources\test\sql\agg.iq"; so the replacement 
operation does nothing.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5977) RexLiteral returns the same value for TIMESTAMP and TIMESTAMP_WITH_LOCAL_TIME_ZONE

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5977:
--

 Summary: RexLiteral returns the same value for TIMESTAMP and 
TIMESTAMP_WITH_LOCAL_TIME_ZONE
 Key: CALCITE-5977
 URL: https://issues.apache.org/jira/browse/CALCITE-5977
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L


Perhaps I'm missing something, but this seems odd to me.

If we want to get the value in Long format (i.e. milliseconds since 1970-01-01 
00:00:00) of a TIMESTAMP / TIMESTAMP_WITH_LOCAL_TIME_ZONE RexLiteral, their 
code is exactly the same (even if they are in different "case" blocks):

{code}
  public @Nullable Object getValue2() {
...
switch (typeName) {
...
case TIMESTAMP:
case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
  return getValueAs(Long.class);
...
  }

  public  @Nullable T getValueAs(Class clazz) {
...
switch (typeName) {
...
case TIMESTAMP:
  if (clazz == Long.class) {
// Milliseconds since 1970-01-01 00:00:00
return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
  }
  ...
  break;
case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
  if (clazz == Long.class) {
// Milliseconds since 1970-01-01 00:00:00
return clazz.cast(((TimestampString) value).getMillisSinceEpoch());
  }
...
{code}

In case of a TIMESTAMP_WITH_LOCAL_TIME_ZONE, shouldn't this code include some 
extra processing to "shift" the value with the proper offset / daylight savings?




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5967) UnsupportedOperationException while implementing call that requires a special collator

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5967:
--

 Summary: UnsupportedOperationException while implementing call 
that requires a special collator 
 Key: CALCITE-5967
 URL: https://issues.apache.org/jira/browse/CALCITE-5967
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.36.0


Regression introduced by a minor change within CALCITE-5914, detected while 
testing a downstream project with the latest Calcite main.

CALCITE-5914 (see 
[2a96512c|https://github.com/apache/calcite/commit/2a96512c352bda4a5d9c0c80730f5c115ac363d6])
 introduced this apparently innocuous change in 
{{RexImpTable#AbstractRexCallImplementor#unboxIfNecessary}}:
{code}
// old
argValueList.stream()
.map(AbstractRexCallImplementor::unboxExpression)
.collect(Collectors.toList());
=>
// new
Util.transform(argValueList,
AbstractRexCallImplementor::unboxExpression);
{code}

Both expressions seem equivalent, however there is a subtle difference: the old 
one returns an {{ArrayList}} (where we can add new elements); wheres the new 
one returns a {{TransformingList}} that extends {{AbstractList}} and that does 
not support {{List#add}}.

After calling {{unboxIfNecessary}}, we might need to modify the argument list 
if we need a special collator to perform the operation:
{code}
private ParameterExpression genValueStatement(...) {
  ...
  optimizedArgValueList = unboxIfNecessary(optimizedArgValueList);

  final Expression callValue =
  implementSafe(translator, call, optimizedArgValueList);
  ...
}

@Override Expression implementSafe(...) {
  ...
  final Expression fieldComparator =
  generateCollatorExpression(relDataType0.getCollation());
  if (fieldComparator != null) {
argValueList.add(fieldComparator);  // <--- 
UnsupportedOperationException!
  }
  ...
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5903) RelMdCollation does not define collations for EnumerableLimit

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5903:
--

 Summary: RelMdCollation does not define collations for 
EnumerableLimit
 Key: CALCITE-5903
 URL: https://issues.apache.org/jira/browse/CALCITE-5903
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.36.0


RelMdCollation does not define collations for EnumerableLimit; it should define 
a method that returns the EnumerableLimit's input collations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release Apache Calcite 1.35.0 (release candidate 3)

[Ruben Q L]

- Release notes: ok
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best,
Ruben


On Mon, Jul 24, 2023 at 12:55 AM Julian Hyde  wrote:

> +1
>
> Downloaded; checked signatures; checked LICENSE, NOTICE; built and ran
> tests on Linux using Gradle 7.4.2 and OpenJDK 18.0.2; ran rat.
>
> Notes:
>  * Please add your signature to KEYS. I had to import from xiong.asc
> manually.
>  * The parser generator reports a choice conflict on the "TRUNCATE"
> keyword; I presume this is a regression introduced by CALCITE-5688.
> Not a show-stopper. but it needs to be fixed.
>
> Julian
>
>
>
> On Sun, Jul 23, 2023 at 7:37 AM Benchao Li  wrote:
> >
> > +1 (binding)
> >
> > - checked signature and checksum (OK)
> > - diffed source with tag (OK)
> > - build from source (OK, but with an unstable test, I've logged
> > https://issues.apache.org/jira/browse/CALCITE-5868)
> > - checked files in Nexus staging (OK)
> > - checked copyright year in NOTICE (OK)
> >
> > Francis Chuang  于2023年7月23日周日 15:02写道：
> >
> > > My vote is: +1 (binding)
> > >
> > > - Verified GPG signature - OK
> > > - Verified SHA512 - OK
> > > - Diffed source release and git repository - OK
> > > - Checked release notes on tag
> > > (
> > >
> https://github.com/apache/calcite/blob/calcite-1.35.0-rc3/site/_docs/history.md
> )
> > >
> > > - OK
> > > - Checked year and versions in NOTICE, README and HOWTO - OK
> > > - Ran tests (gradle check) - OK
> > > - Spot checked Nexus artifacts - OK
> > >
> > > Environment:
> > > Eclipse-temurin:19-jammy docker container in WSL2 (Ubuntu 22.04.2) on
> > > Windows 11 22h2
> > >
> > > $ docker version
> > > Client: Docker Engine - Community
> > >   Cloud integration: v1.0.35
> > >   Version:   24.0.2
> > >   API version:   1.43
> > >   Go version:go1.20.4
> > >   Git commit:cb74dfc
> > >   Built: Thu May 25 21:52:17 2023
> > >   OS/Arch:   linux/amd64
> > >   Context:   default
> > >
> > > Server: Docker Desktop
> > >   Engine:
> > >Version:  24.0.2
> > >API version:  1.43 (minimum version 1.12)
> > >Go version:   go1.20.4
> > >Git commit:   659604f
> > >Built:Thu May 25 21:52:17 2023
> > >OS/Arch:  linux/amd64
> > >Experimental: false
> > >   containerd:
> > >Version:  1.6.21
> > >GitCommit:3dce8eb055cbb6872793272b4f20ed16117344f8
> > >   runc:
> > >Version:  1.1.7
> > >GitCommit:v1.1.7-0-g860f061
> > >   docker-init:
> > >Version:  0.19.0
> > >GitCommit:de40ad0
> > >
> > > $ gradle -v
> > >
> > > 
> > > Gradle 7.6.1
> > > 
> > >
> > > Build time:   2023-02-24 13:54:42 UTC
> > > Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
> > >
> > > Kotlin:   1.7.10
> > > Groovy:   3.0.13
> > > Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> > > JVM:  19.0.2 (Eclipse Adoptium 19.0.2+7)
> > > OS:   Linux 5.15.90.1-microsoft-standard-WSL2 amd64
> > >
> > > $ java --version
> > > openjdk 19.0.2 2023-01-17
> > > OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7)
> > > OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode,
> > > sharing)
> > >
> > > Francis
> > >
> > > On 22/07/2023 9:40 am, Xiong Duan wrote:
> > > > Hi all,
> > > >
> > > >
> > > > I have created a build for Apache Calcite 1.35.0, release
> > > >
> > > > candidate 3.
> > > >
> > > >
> > > > Thanks to everyone who has contributed to this release.
> > > >
> > > >
> > > > You can read the release notes here:
> > > >
> > > >
> > >
> https://github.com/apache/calcite/blob/calcite-1.35.0-rc3/site/_docs/history.md
> > > >
> > > >
> > > > The commit to be voted upon:
> > > >
> > > >
> > >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=75750b78b5ac692caa654f506fc1515d4d3991d6
> > > >
> > > >
> > > > Its hash is 75750b78b5ac692caa654f506fc1515d4d3991d6
> > > >
> > > >
> > > > Tag:
> > > >
> > > > https://github.com/apache/calcite/tree/calcite-1.35.0-rc3
> > > >
> > > >
> > > > The artifacts to be voted on are located here:
> > > >
> > > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.35.0-rc3
> > > >
> > > > (revision 63131)
> > > >
> > > >
> > > > The hashes of the artifacts are as follows:
> > > >
> > > >
> > >
> a19e0e73dfaa4d54d4d27db24c1b5674979e5461ed4c0bd1408aaec72ff82e832b3e4dee13c9d6552d3b5a9db86c1f11b9a960521e76dd65fc661565e160f63e
> > > >
> > > > *apache-calcite-1.35.0-src.tar.gz
> > > >
> > > >
> > > > A staged Maven repository is available for review at:
> > > >
> > > >
> > >
> https://repository.apache.org/content/repositories/orgapachecalcite-1216/org/apache/calcite/
> > > >
> > > >
> > > > Release artifacts are signed with the following 

Re: [VOTE] Release Apache Calcite 1.35.0 (release candidate 2)

[Ruben Q L]

Thanks Gian for signaling this issue.

Since this is a regression introduced in 1.35 (and it looks more or less
simple to fix) IMO we should cancel the rc2 vote, fix CALCITE-5865 and move
to a rc3.

Best,
Ruben



On Thu, Jul 20, 2023 at 3:29 AM Chunwei Lei  wrote:

> Thanks Xiong Duan for being RM for this release.
>
>
> - Checked release notes on tag: some comments
>
> > Some improvements in simplify an expression.
> Some improvements in simplifying an expression.
>
> > [CALCITE-5717] RelBuilder.project of literals on a single-row `Aggregate`
> should create a Values
> I think Aggregate should not be quoted since other operators are not quoted
> either.
>
>
> - Checksum: ok
> - Signature: ok
> - Diff source release and git repository: ok
> - Build + tests: ok
>
>
>
>
> Best,
> Chunwei
>
>
> On Thu, Jul 20, 2023 at 7:26 AM Gian Merlino  wrote:
>
> > When testing this with Apache Druid we found a problem: the change in
> > CALCITE-5747 broke a few of our test cases due to this piece:
> >
> https://github.com/apache/calcite/pull/3245/files#diff-e873041549333502af52ece8a1b34301ae5a059ff4719e9bddbaef48929e7047L7338-L7341
> >
> > Essentially, FLOOR and CEIL no longer work with a SqlConformance that is
> > not a SqlConformanceEnum. (In Druid we have a conformance that extends
> > SqlAbstractConformance.)
> >
> > I wrote up a Jira case here:
> > https://issues.apache.org/jira/browse/CALCITE-5865, and have a patch up
> > here: https://github.com/apache/calcite/pull/3327
> >
> > Could we include this fix in 1.35.0? Arguably, since this would be a new
> > problem in 1.35.0, it should be fixed prior to release.
> >
> > On 2023/07/17 18:24:21 xiong duan wrote:
> > > Hi all,
> > >
> > >
> > > I have created a build for Apache Calcite 1.35.0, release
> > >
> > > candidate 2.
> > >
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > >
> > > You can read the release notes here:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/calcite-1.35.0-rc2/site/_docs/history.md
> > >
> > >
> > > The commit to be voted upon:
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=2f5635f13a4eb4b89c119fabf25b1d31e0018426
> > >
> > >
> > > Its hash is 2f5635f13a4eb4b89c119fabf25b1d31e0018426
> > >
> > >
> > > Tag:
> > >
> > > https://github.com/apache/calcite/tree/calcite-1.35.0-rc2
> > >
> > >
> > > The artifacts to be voted on are located here:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.35.0-rc2
> > >
> > > (revision 63046)
> > >
> > >
> > > The hashes of the artifacts are as follows:
> > >
> > >
> >
> b5d837e3725554254bdad0e4f4a63134193a1e3f44bb9b862d9eeeb7f9a5e9bae01985e2f1857874928dc4dbf1f4ca3ae4debacc096470081c249eba699dc74e
> > >
> > > *apache-calcite-1.35.0-src.tar.gz
> > >
> > >
> > > A staged Maven repository is available for review at:
> > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1213/org/apache/calcite/
> > >
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/xiong.asc
> > >
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > >
> > > To create the jars and test Apache Calcite: "gradle build"
> > >
> > > (requires an appropriate Gradle/JDK installation)
> > >
> > >
> > > Please vote on releasing this package as Apache Calcite 1.35.0.
> > >
> > >
> > > The vote is open for the next 72 hours and passes if a majority of at
> > >
> > > least three +1 PMC votes are cast.
> > >
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.35.0
> > >
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > >
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > > Here is my vote:
> > >
> > >
> > > +1 (binding)
> > >
> >
>

Re: [VOTE] Release Apache Calcite 1.35.0 (release candidate 2)

[Ruben Q L]

Thanks again Xiong Duan for being RM for this release.

- Release notes: ok
- Checksum: ok
- Signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best,
Ruben



On Tue, Jul 18, 2023 at 3:23 AM Francis Chuang 
wrote:

> Thanks for being RM for this release, Xiong!
>
> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.35.0-rc2/site/_docs/history.md)
>
> - OK
> - Checked year and versions in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> Environment:
> Eclipse-temurin:19-jammy docker container in WSL2 (Ubuntu 22.04.2) on
> Windows 11 22h2
>
> $ docker version
> Client: Docker Engine - Community
>   Cloud integration: v1.0.35
>   Version:   24.0.2
>   API version:   1.43
>   Go version:go1.20.4
>   Git commit:cb74dfc
>   Built: Thu May 25 21:52:17 2023
>   OS/Arch:   linux/amd64
>   Context:   default
>
> Server: Docker Desktop
>   Engine:
>Version:  24.0.2
>API version:  1.43 (minimum version 1.12)
>Go version:   go1.20.4
>Git commit:   659604f
>Built:Thu May 25 21:52:17 2023
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.21
>GitCommit:3dce8eb055cbb6872793272b4f20ed16117344f8
>   runc:
>Version:  1.1.7
>GitCommit:v1.1.7-0-g860f061
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
> $ gradle -v
>
> 
> Gradle 7.6.1
> 
>
> Build time:   2023-02-24 13:54:42 UTC
> Revision: 3905fe8ac072bbd925c70ddbf4463341f4b4
>
> Kotlin:   1.7.10
> Groovy:   3.0.13
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  19.0.2 (Eclipse Adoptium 19.0.2+7)
> OS:   Linux 5.15.90.1-microsoft-standard-WSL2 amd64
>
> $ java --version
> openjdk 19.0.2 2023-01-17
> OpenJDK Runtime Environment Temurin-19.0.2+7 (build 19.0.2+7)
> OpenJDK 64-Bit Server VM Temurin-19.0.2+7 (build 19.0.2+7, mixed mode,
> sharing)
>
> Francis
>
> On 18/07/2023 4:24 am, xiong duan wrote:
> > Hi all,
> >
> >
> > I have created a build for Apache Calcite 1.35.0, release
> >
> > candidate 2.
> >
> >
> > Thanks to everyone who has contributed to this release.
> >
> >
> > You can read the release notes here:
> >
> >
> https://github.com/apache/calcite/blob/calcite-1.35.0-rc2/site/_docs/history.md
> >
> >
> > The commit to be voted upon:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=2f5635f13a4eb4b89c119fabf25b1d31e0018426
> >
> >
> > Its hash is 2f5635f13a4eb4b89c119fabf25b1d31e0018426
> >
> >
> > Tag:
> >
> > https://github.com/apache/calcite/tree/calcite-1.35.0-rc2
> >
> >
> > The artifacts to be voted on are located here:
> >
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.35.0-rc2
> >
> > (revision 63046)
> >
> >
> > The hashes of the artifacts are as follows:
> >
> >
> b5d837e3725554254bdad0e4f4a63134193a1e3f44bb9b862d9eeeb7f9a5e9bae01985e2f1857874928dc4dbf1f4ca3ae4debacc096470081c249eba699dc74e
> >
> > *apache-calcite-1.35.0-src.tar.gz
> >
> >
> > A staged Maven repository is available for review at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1213/org/apache/calcite/
> >
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/xiong.asc
> >
> > https://www.apache.org/dist/calcite/KEYS
> >
> >
> > To create the jars and test Apache Calcite: "gradle build"
> >
> > (requires an appropriate Gradle/JDK installation)
> >
> >
> > Please vote on releasing this package as Apache Calcite 1.35.0.
> >
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> >
> > least three +1 PMC votes are cast.
> >
> >
> > [ ] +1 Release this package as Apache Calcite 1.35.0
> >
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> >
> > [ ] -1 Do not release this package because...
> >
> >
> > Here is my vote:
> >
> >
> > +1 (binding)
> >
>

Re: [VOTE] Release Apache Calcite 1.35.0 (release candidate 0)

[Ruben Q L]

Thanks Xiong Duan for preparing this release!

- Release notes: ok
- Checksum: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok
- Signature: ko?
I'm getting the following "wrong key usage" error message:
$   gpg --verify apache-calcite-1.35.0-src.tar.gz.asc
apache-calcite-1.35.0-src.tar.gz
...
gpg: bad data signature from key 53B9528B541C85EC: Uso de clave
incorrecto (0x00, 0x2)

Is there something wrong with my machine, or is anybody else getting the
same error?

Best,
Ruben




On Fri, Jul 14, 2023 at 12:33 PM xiong duan  wrote:

> Hi all,
>
>
> I have created a build for Apache Calcite 1.35.0, release
>
> candidate 0.
>
>
> Thanks to everyone who has contributed to this release.
>
>
> You can read the release notes here:
>
> https://s.apache.org/p3rl9
>
>
> The commit to be voted upon:
>
> https://s.apache.org/8io6t
>
>
> Its hash is bd4cac4ee30b4c275f5a229eb1704524c3dbc376
>
>
> Tag:
>
> https://github.com/apache/calcite/tree/calcite-1.35.0-rc0
>
>
> The artifacts to be voted on are located here:
>
> https://s.apache.org/c95f1
>
> (revision 62985)
>
>
> The hashes of the artifacts are as follows:
>
>
> b2eb9b25727d1213f889e8d1a547a1f84c66af3821523ebe19a36ae5bc595daa74b4b4584914012bffca0a24be19a523c45dbaf72064d90f50feffc0de8a5e6e
>
> *apache-calcite-1.35.0-src.tar.gz
>
>
> A staged Maven repository is available for review at:
>
>
> https://repository.apache.org/content/repositories/orgapachecalcite-1206/org/apache/calcite/
>
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/xiong.asc
>
> https://www.apache.org/dist/calcite/KEYS
>
>
> To create the jars and test Apache Calcite: "gradle build"
>
> (requires an appropriate Gradle/JDK installation)
>
>
> Please vote on releasing this package as Apache Calcite 1.35.0.
>
>
> The vote is open for the next 72 hours and passes if a majority of at
>
> least three +1 PMC votes are cast.
>
>
> [ ] +1 Release this package as Apache Calcite 1.35.0
>
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
>
> +1 (binding)
>

[jira] [Created] (CALCITE-5839) EnumerableInterpretable#StaticFieldDetector can overwrite its flag and return an incorrect result

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5839:
--

 Summary: EnumerableInterpretable#StaticFieldDetector can overwrite 
its flag and return an incorrect result
 Key: CALCITE-5839
 URL: https://issues.apache.org/jira/browse/CALCITE-5839
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L
Assignee: Ruben Q L


In EnumerableInterpretable, before using the bindable cache, it is verified 
whether the class contains static fields (because in that case, the cache shall 
not be used):

{code}
  static Bindable getBindable(ClassDeclaration expr, ...) {
...
if (CalciteSystemProperty.BINDABLE_CACHE_MAX_SIZE.value() != 0) {
  StaticFieldDetector detector = new StaticFieldDetector();
  expr.accept(detector);
  if (!detector.containsStaticField) {
return BINDABLE_CACHE.get(classBody, () ->  
compileToBindable(expr.name, s, compiler));
  }
}

  

  /**
   * A visitor detecting if the Java AST contains static fields.
   */
  static class StaticFieldDetector extends VisitorImpl {
boolean containsStaticField = false;

@Override public Void visit(final FieldDeclaration fieldDeclaration) {
  containsStaticField = (fieldDeclaration.modifier & Modifier.STATIC) != 0;
  return containsStaticField ? null : super.visit(fieldDeclaration);
}
  }
{code}

However, it seem that the {{containsStaticField}} flag in 
{{{StaticFieldDetector}} can be overwritten when inspecting the field 
declarations of the class, so that, if a non-static field is inspected after a 
static field, the flag will be then (re)set to false; and the final result of 
the shuttle will be wrong (false, when it should have been true).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Draft: board report for 2023 Q2

[Ruben Q L]

+1
Thanks Stamatis for preparing it!

On Tue, Jul 4, 2023 at 10:18 AM Stamatis Zampetakis 
wrote:

> Hello,
>
> Below you can find a draft of this quarter's board report. I plan to
> submit it next Tuesday (July 11, 2023).
> Please let me know if you have any additions or corrections.
>
> Best regards,
> Stamatis
> -
>
> ## Description:
> Apache Calcite is a highly customizable framework for parsing and planning
> queries on data in a wide variety of formats. It allows database-like
> access,
> and in particular a SQL interface and advanced query optimization, for data
> not residing in a traditional database.
>
> Avatica is a sub-project within Calcite and provides a framework for
> building
> local and remote JDBC and ODBC database drivers. Avatica has an independent
> release schedule and its own repository.
>
> ## Project Status:
> Current project status: ongoing
> Issues for the board: none
>
> ## Membership Data:
> Apache Calcite was founded 2015-10-22 (8 years ago)
> There are currently 67 committers and 27 PMC members in this project.
> The Committer-to-PMC ratio is roughly 2:1.
>
> Community changes, past quarter:
> - No new PMC members. Last addition was Benchao Li on 2023-01-27.
> - Jacky Lau was added as committer on 2023-06-28
> - Oliver Lee was added as committer on 2023-06-13
> - Tanner Clary was added as committer on 2023-05-25
> - Zhe Hu was added as committer on 2023-06-28
>
> ## Project Activity:
> There were no releases during this quarter. The last release was Apache
> Calcite 1.34.0 on 2023-03-14.
>
> We are actively working towards Apache Calcite 1.35.0 and we plan to
> release
> the new version during July 2023.
>
> On 2023-06-29, the Calcite community organised a Virtual key signing party
> for
> expanding the Web of trust and empowering the cryptographic signatures of
> our
> members and future release managers. 4 PMC members and 2 committers
> attended
> the event.
>
> ## Community Health:
> The project remains super healthy.
>
> The traffic in JIRA, GitHub, issues@, commits, has increased overall,
> while
> dev@ has dropped by 18%. During this quarter there were more discussions
> and
> exchanges under specific issues/tickets shifting the traffic from dev@ to
> issues@ and other places.
>
> The number of non-committer (contributor) commits per month:
> +-+-+-+
> |year |month| contributor_commits |
> +-+-+-+
> | 2023| 4   | 17  |
> | 2023| 5   | 21  |
> | 2023| 6   | 35  |
> +-+-+-+
>
> The number of active reviewers per month:
> +-+-+-+
> |year |month|  active_reviewers   |
> +-+-+-+
> | 2023| 4   | 6   |
> | 2023| 5   | 6   |
> | 2023| 6   | 11  |
> +-+-+-+
>
> Top-3 reviewers in the last quarter:
> +---+-+
> | committer |   reviews   |
> +---+-+
> | Julian Hyde  | 28  |
> | Benchao Li  | 12  |
> | rubenada  | 8   |
> | Jiajun  | 8   |
> +---+-+
>
> The number of non-committer commits has increased roughly by 43% from the
> last
> quarter (51 commits in Q1 vs. 73 commits in Q2) which is promising for
> inviting new committers in the near term.
>
> The number of active reviewers increased slightly compared to the last
> quarter
> with June being the most active month for everyone. The preparation for the
> 1.35.0 release and the call for action did help in bringing those numbers
> up
> during June.
>
> In terms of sharing the review load, things are a bit better this quarter
> but
> still far from being completely balanced. There have been discussions
> around
> the topic in the last quarter but we obviously need more involvement from
> the
> community members.
>

Re: [DISCUSS] Towards Calcite 1.35.0

[Ruben Q L]

Hello,

Thanks for checking Xiong Duan, I have taken care of 6,7,8.

Contributors, please remember that after completing a ticket, the Jira
needs to be set to "Resolved", not "Closed" (they will be moved to Closed
by the Release Manager, once the next release is produced).

Apart from that, our dashboard [1] still shows 22 unresolved tickets with
fixVersion=1.35
Except from the blocker ones, I guess the rest would need to be resolved
shortly or otherwise moved to fixVersion=1.36 (or no fixVersion).
As a general rule, please do not set a fixVersion unless it is a blocker
issue, or you're reasonably sure that the issue will be done for the next
release.

Best,
Ruben

[1]
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950




On Wed, Jun 28, 2023 at 1:47 PM xiong duan  wrote:

> In order to release 1.35.0, I reviewed all the fixed issues in this
> version, Here are some issue statuses that need to be resolved:
>
>
>1. https://issues.apache.org/jira/browse/CALCITE-5764(The PR has
>merge.But issue is IN PROGRESS). (Assigine Julian Hyde)
>2. https://issues.apache.org/jira/browse/CALCITE-5706(The PR has
>merge.But issue is IN PROGRESS). (Assigine Julian Hyde)
>3. https://issues.apache.org/jira/browse/CALCITE-5765(The PR has
>merge.But issue is IN PROGRESS). (Assigine Julian Hyde)
>4. https://issues.apache.org/jira/browse/CALCITE-5762(The PR has
>merge.But issue is IN PROGRESS). (Assigine Julian Hyde)
>5. https://issues.apache.org/jira/browse/CALCITE-5747(The PR has
>merge.But issue is NOT Resolve). (Assigine Tanner Clary)
>6. https://issues.apache.org/jira/browse/CALCITE-5771(The Fix Version
> is
>NONE, Need set it to 1.35.0)
>7. https://issues.apache.org/jira/browse/CALCITE-5757(The Fix Version
> is
>NONE, Need set it to 1.35.0)
>8. https://issues.apache.org/jira/browse/CALCITE-4679(Resolution should
>be Fixed. But is Resolved)
>
> If you are busy, Please tell me the real status of the ISSUE, and I will
> handle it. (6,7,8 need one PMC to handle it).Thanks.
>
>
> Jacky Lau  于2023年6月26日周一 14:24写道：
>
> > hi @xiong duan:
> > thanks for your review and merged very much, and i forgot there are
> > also have one pr https://github.com/apache/calcite/pull/3262
> > do you also have time to have a look？
> >
> >
> > xiong duan  于2023年6月25日周日 19:45写道：
> >
> > > I have reviewed:
> > > https://github.com/apache/calcite/pull/3238
> > > https://github.com/apache/calcite/pull/3263
> > > If no other problem, I will merge it tomorrow.
> > >
> > > Ruben Q L  于2023年6月25日周日 17:20写道：
> > >
> > > > I'd like to include CALCITE-5789 in 1.35. I'll try to finalize it
> > > > today/tomorrow.
> > > >
> > > >
> > > >
> > > > El dom, 25 jun 2023, 6:33, Jacky Lau 
> escribió:
> > > >
> > > > > could we review this pr, so that it could be merged in 1.35. then
> the
> > > > 1.35
> > > > > will almost have the full spark  collection function.
> > > > > I would very appreciate a review. if someone has time.
> > > > >
> > > > > https://github.com/apache/calcite/pull/3238
> > > > > https://github.com/apache/calcite/pull/3263
> > > > >
> > > > > xiong duan  于2023年6月25日周日 10:31写道：
> > > > >
> > > > > > I have created CALCITE-5797 to release 1.35.0. If there are no
> > > > > objections,
> > > > > > I will create an RC in the following days.
> > > > > >
> > > > > > [1]:https://issues.apache.org/jira/browse/CALCITE-5797
> > > > > >
> > > > > > If there are other must fix for 1.35.0 please let us know so that
> > we
> > > > can
> > > > > > plan accordingly.
> > > > > >
> > > > > > Julian Hyde  于2023年6月24日周六 01:58写道：
> > > > > >
> > > > > > > We said ‘mid-June’. Time to get this release rolling?
> > > > > > >
> > > > > > > > On Jun 20, 2023, at 10:34 PM, Gian Merlino 
> > > > wrote:
> > > > > > > >
> > > > > > > > I committed the patch for CALCITE-5477. I would appreciate a
> > > review
> > > > > on
> > > > > > > CALCITE-5479, if someone has a chance.
> > > > > > > >
> > > > > > > > On 2023/06/07 09:53:08 Gian Merlino wrote:
> > > > > >

Re: Virtual key signing party

[Ruben Q L]

Hello,

I'll try to attend too.
Best,
Ruben


pub   rsa4096 2020-09-26 [SC]

  464F A4A3 D7E4 2112 E4CB  68F2 DF92 5FEB A08B 032B

uid[  absoluta ] Ruben Quesada Lopez 

sub   rsa4096 2020-09-26 [E]




On Tue, Jun 27, 2023 at 8:29 AM Stamatis Zampetakis 
wrote:

> Great let's do one on June 29, 2023, 14:30 UTC+2 [1, 2]. For those who
> can't make it in this slot feel free to propose additional ones.
>
> Best,
> Stamatis
>
> [1] https://s.apache.org/cn5xq
> [2] https://meet.google.com/vwx-mxxz-ibk
>
>
> On Tue, Jun 27, 2023 at 4:22 AM xiong duan  wrote:
> >
> > Hi Stamatis,
> > Thanks for organizing this party. I am in UTC+8. How about June 29,
> > 2023, 20:30 (UTC+8) So the UTC+2 time should be June 29, 2023, 14:30?
> > Here is my fingerprint:
> >
> > pub   rsa4096 2023-06-27 [SC]
> >
> >   EC78 877D 98B8 CDD4 9613  3DF2 B989 C696 D102 A552
> >
> > uid   [ultimate] Xiong Duan (CODE SIGNING KEY)  >
> >
> > sub   rsa4096 2023-06-27 [E]
> >
> > Michael Mior  于2023年6月27日周二 02:19写道：
> >
> > > I'll attend if possible. My key needs to be updated. (Not compromised,
> but
> > > I lost access to my previous signing key.)
> > >
> > > pub   rsa4096 2022-06-17 [SC] [expires: 2038-06-13]
> > >   EAC5 89C4 44F4 68FE 7E11  3D2D D8D7 2C13 CF2D 8DDB
> > > uid   [ultimate] Michael Mior 
> > > uid   [ultimate] Michael Mior 
> > > uid   [ultimate] Michael Mior 
> > > sub   rsa4096 2022-06-17 [E] [expires: 2038-06-13]
> > >
> > > --
> > > Michael Mior
> > > mm...@apache.org
> > >
> > >
> > > On Mon, Jun 26, 2023 at 3:27 AM Stamatis Zampetakis  >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > The last virtual key signing party [1] was about a year ago [2]. In
> > > > the upcoming releases, we have some people that are serving as
> release
> > > > managers for the first time, thus it may be a good time to hold
> > > > another party.
> > > >
> > > > Since Duan Xiong is the RM for the next release, I would suggest
> > > > holding a meeting in a timezone that is convenient for him.
> > > >
> > > > @Duan: Can you please propose a time slot that works for you?
> > > >
> > > > I am in UTC+2 and will try to attend if the slot doesn't fall in the
> > > > middle of the night (23:00 - 06:00).
> > > >
> > > > For those that would like to attend, please reply to this thread with
> > > > your public
> > > > keys' fingerprint (gpg --fingerprint) before the meeting.
> > > >
> > > > pub   rsa4096 2019-03-15 [SC] [expires: 2027-03-15]
> > > >   0474 9577 FD93 4674 B9CD  45C5 D77C 3383 F192 7570
> > > > uid   [ultimate] Stamatis Zampetakis 
> > > > uid   [ultimate] Stamatis Zampetakis 
> > > > sub   rsa4096 2019-03-15 [E] [expires: 2027-03-15]
> > > >
> > > > To verify your identity, please bring with you a government issued ID
> > > > (preferably passport).
> > > >
> > > > As a reminder of the procedure, have a look at the notes [1] taken by
> > > > Francis
> > > > during a previous party!
> > > >
> > > > Anybody can participate (not need to be a committer to the project).
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > > [1]
> > > >
> > > >
> > >
> http://www.cryptnet.net/fdp/crypto/keysigning_party/en/keysigning_party.html
> > > > [2] https://lists.apache.org/thread/4lvch7sfwszbylpzjt11jc9tokm3d0l4
> > > > [3] https://gist.github.com/F21/b0e8c62c49dfab267ff1d0c6af39ab84
> > > >
> > >
>

Re: [DISCUSS] Towards Calcite 1.35.0

[Ruben Q L]

I'd like to include CALCITE-5789 in 1.35. I'll try to finalize it
today/tomorrow.



El dom, 25 jun 2023, 6:33, Jacky Lau  escribió:

> could we review this pr, so that it could be merged in 1.35. then the 1.35
> will almost have the full spark  collection function.
> I would very appreciate a review. if someone has time.
>
> https://github.com/apache/calcite/pull/3238
> https://github.com/apache/calcite/pull/3263
>
> xiong duan  于2023年6月25日周日 10:31写道：
>
> > I have created CALCITE-5797 to release 1.35.0. If there are no
> objections,
> > I will create an RC in the following days.
> >
> > [1]:https://issues.apache.org/jira/browse/CALCITE-5797
> >
> > If there are other must fix for 1.35.0 please let us know so that we can
> > plan accordingly.
> >
> > Julian Hyde  于2023年6月24日周六 01:58写道：
> >
> > > We said ‘mid-June’. Time to get this release rolling?
> > >
> > > > On Jun 20, 2023, at 10:34 PM, Gian Merlino  wrote:
> > > >
> > > > I committed the patch for CALCITE-5477. I would appreciate a review
> on
> > > CALCITE-5479, if someone has a chance.
> > > >
> > > > On 2023/06/07 09:53:08 Gian Merlino wrote:
> > > >> Some more detail:
> > > >>
> > > >> - the PR for CALCITE-5477 is
> > > https://github.com/apache/calcite/pull/3249
> > > >> - the PR for CALCITE-5479 is
> > > https://github.com/apache/calcite/pull/3030
> > > >>
> > > >> I've just pushed up some updates to reflect the new approach we
> > > discussed for guava backwards-compat in CALCITE-5477, and to sync up
> with
> > > latest main in CALCITE-5479.
> > > >>
> > > >> We've been doing a bunch of work on the Druid side to be able to
> > update
> > > our Calcite dependency (currently, we're on 1.21.0). From what we've
> seen
> > > so far, we believe if these two patches are part of 1.35.0, then we
> could
> > > target that version.
> > > >>
> > > >> Gian
> > > >>
> > > >> On 2023/06/07 06:32:34 Abhishek Agarwal wrote:
> > > >>> We (Apache Druid) would also like
> > > >>> https://issues.apache.org/jira/browse/CALCITE-5479 and
> > > >>> https://issues.apache.org/jira/browse/CALCITE-5477 to be fixed in
> > > 1.35.0.
> > > >>> There are PRs for both of these fixes. We are working on addressing
> > > review
> > > >>> comments and just need a few days of time.
> > > >>>
> > > >>> On Thu, Jun 1, 2023 at 3:32 AM Julian Hyde 
> wrote:
> > > >>>
> > >  There's one Jira case we should fix:
> > >  https://issues.apache.org/jira/browse/CALCITE-5737 (support JDK
> 19
> > > and
> > >  20). Can someone take that on?
> > > 
> > >  Julian
> > > 
> > >  On 2023/05/30 08:33:23 xiong duan wrote:
> > > > Yes. I can release 1.35.0 when the PR is handled.
> > > >
> > > > Julian Hyde  于2023年5月30日周二 05:46写道：
> > > >
> > > >> Mid-June sounds like a good idea.
> > > >>
> > > >> I would like to see the following Jira cases finished:
> > > >> * 5701 NAMED_STRUCT (MasseGuillaume)
> > > >> * 5640 SAFE_ADD (DanZou)
> > > >> * 5625 SEARCH (olivrlee)
> > > >> * 5626 Fully-qualified names (herunkang2018)
> > > >> * 5615 SQL Logic Test (mbudiu)
> > > >> * 5607 JSON serialization (olivrlee)
> > > >> * 5564 PERCENTILE_CONT (tanclary)
> > > >> * 5526 Unparsing literals (TJ Banghart)
> > > >>
> > > >> If the authors have made changes they would like me to review,
> > > please
> > > >> add a comment to the Jira case.
> > > >>
> > > >> Any other cases, don't ask me personally, ask everyone on the
> dev
> > > >> list. My capacity to review PRs is limited.
> > > >>
> > > >> Julian
> > > >>
> > > >> On Fri, May 26, 2023 at 11:16 PM Benchao Li <
> libenc...@apache.org
> > >
> > >  wrote:
> > > >>>
> > > >>> It's been a bit more than 2 months since our last release [1]
> and
> > >  there
> > > >> are
> > > >>> currently 80+ new commits in master.
> > > >>>
> > > >>> As usual, according to our Jira dashboard [2] and Github [3],
> > there
> > >  are
> > > >>> many pending issues that could / should be part of the release.
> > I'd
> > > >> propose
> > > >>> to make a collective effort to try to clean up our 1.35 backlog
> > and
> > >  merge
> > > >>> the PRs which are in a good state. I'd propose to aim for **mid
> > >  June** to
> > > >>> release 1.35.0, this will give us about 15-20 days to clean up
> > >  pending
> > > >> PRs
> > > >>> for next version. What do you think?
> > > >>>
> > > >>> According to [4], the following release managers would be:
> > > >>> - 1.35.0 Duan Xiong
> > > >>> - 1.36.0 Benchao Li
> > > >>> - 1.37.0 Sergey Nuyanzin
> > > >>> - 1.38.0 Julian Hyde
> > > >>>
> > > >>> @Duan Xiong, are you still available for being the release
> > manager
> > >  for
> > > >>> 1.35.0?
> > > >>>
> > > >>> And contributors, if you have any work that is in good shape
> and
> > >  want to
> > > >> be
> > > >>> included in 1.35.0, please 

[jira] [Created] (CALCITE-5789) Query with two nested subqueries where the inner-most references the outer-most table returns wrong result

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5789:
--

 Summary: Query with two nested subqueries where the inner-most 
references the outer-most table returns wrong result
 Key: CALCITE-5789
 URL: https://issues.apache.org/jira/browse/CALCITE-5789
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L


tbd



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5679) HepPlanner#buildFinalplan: do not clear metadata cache if RelNode has not changed

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5679:
--

 Summary: HepPlanner#buildFinalplan: do not clear metadata cache if 
RelNode has not changed
 Key: CALCITE-5679
 URL: https://issues.apache.org/jira/browse/CALCITE-5679
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Ruben Q L
 Fix For: 1.35.0


{{HepPlanner#buildFinalplan}} clears the metadata cache info of the impacted 
RelNode after replacing its inputs, but in the cases where the RelNode does not 
change (e.g. when it has no inputs) this clear cache should not be necessary 
(unless I'm missing something).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Disable JIRA worklog for GitHub PRs

[Ruben Q L]

+1

On Wed, Apr 19, 2023 at 9:33 AM Francis Chuang 
wrote:

> +1 for this.
>
> Also noticed this and found it to be annoying.
>
> On 19/04/2023 6:22 pm, Alessandro Solimando wrote:
> > Hi Stamatis,
> > +1000 on this, thanks for starting this discussion!
> >
> > Best regards,
> > Alessandro
> >
> > On Wed 19 Apr 2023, 10:19 Stamatis Zampetakis, 
> wrote:
> >
> >> Hello,
> >>
> >> Everything that happens in a GitHub PR creates a worklog entry under
> >> the respective JIRA ticket.
> >> For every worklog entry we receive a notification from j...@apache.org
> >> when we are watching an issue. The worklog entry and email
> >> notification usually appear messy.
> >>
> >> Moreover, if we are watching the GitHub PR we are going to get a
> >> notification from notificati...@github.com which has the same content
> >> with the JIRA worklog entry and is much more readable.
> >>
> >> Finally, the PR notification is also going to
> >> comm...@calcite.apache.org so those who are subscribed to that list
> >> will get the same notification three times.
> >>
> >> Personally, I never read the JIRA worklog notifications and I largely
> >> prefer those from notificati...@github.com.
> >>
> >> How do you feel about disabling the worklog entries in JIRA coming
> >> from GitHub PRs?
> >>
> >> For archiving purposes, the notifications already go to commits@ so we
> >> don't lose anything from disabling the worklog entries. On the
> >> contrary, I find that this would reduce the noise and redundancy on
> >> our inboxes.
> >>
> >> Concretely this is what I have in mind in terms of change:
> >> https://github.com/apache/calcite/pull/3166
> >>
> >> Best,
> >> Stamatis
> >>
> >
>

Re: [DISCUSS] Sharing the load of reviewing PRs

[Ruben Q L]

Hello,

I understand Julian's frustration. We all know that reviewing PRs is a
recurring problem, and it is not the first time we discuss potential
solutions, see e.g. the discussion a year ago [1] (also started by Julian)
where several ideas were mentioned: automatic assignment, emulate the RM
process onto the reviewing process (quite similar to the current proposal),
...but in the end no change was implemented, and the problem remains.

I agree that something must be done in order to revert the situation.

In my opinion, one of the main factors is that the vast majority of PRs
(even the ones that get merged) are never assigned. This lack of assignee
can be seen as if the PR is "no-one's responsibility", so we should try
somehow to assign each PR to someone, and make that person accountable for
the PR's progression.
I think we could try Julian's idea of having a pool of reviewers and a PR
manager (taken from the pool, rotating this position every month or every
two months). Personally, I would not set hard deadlines (e.g. something
must be done within 3 days), because we are all volunteers and, even if we
are all trying to do our best here, it may happen that a certain week we
are busy with other personal or professional stuff. In the end, I think it
should be the PR manager's responsibility to ping the assigned reviewer if
a PR is not progressing after a reasonable period of time, ask them for an
update, maybe even involve a different reviewer / re-assign the PR as a
last resource.

Of course, it must remain clear that, even if we implement this approach,
anyone is still free (and encouraged) to participate in any PR review. Even
if someone is not the assigned reviewer, they can chime in and contribute
to the review nevertheless.
Also, I think another sensible rule would be: if someone from the reviewers
pool submits a PR, the PR manager will need to assign it to a different
person.

One last comment, I have the impression that with this initiative we would
be moving towards a "better done than perfect" approach. Calcite is a vast
project, with many different modules, and it could happen (it *will*
happen) that a certain reviewer gets assigned a PR concerning a part of the
project that they are not familiar with. Of course, one way of becoming
(progressively) familiar with unknown parts of the project is by reviewing
this type of PRs, but that takes time. I guess it would be possible for the
assignee to try to ping and involve other reviewers with more experience in
that area, but at the end of the day, it would be the assignee's
responsibility to review and merge some piece of code that might be a bit
obscure to them. This might lead to suboptimal or even incorrect code being
inadvertently merged into the main branch. This is not a catastrophe (it
can already happen with the current approach), and we will detect and
correct these mistakes; I'm just mentioning that they might become a bit
more frequent with the proposed approach (and we should all face them with
a constructive and positive attitude). In any case, I have the impression
that with the new idea the pros outweigh the cons, so we could give it a
try.

Best,
Ruben

[1] https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn



On Wed, Apr 12, 2023 at 3:13 AM Chunwei Lei  wrote:

> Thanks Julian for sharing the proposal. I am +1 for it. I have been busy in
> the past few months, so I have only had a quick look at the new JIRA.
> However, I will have more time in the coming months, and I would be more
> than happy to review any pull requests.
>
>
>
> Best,
> Chunwei
>
>
> On Tue, Apr 11, 2023 at 10:22 PM Jiajun Xie 
> wrote:
>
> > Thank Julian for your idea.
> >  Your plan helps to motivate new contributors.
> >
> > “If there is no response to my PR,
> > I will be disappointed or even give up on continuing to contribute.”
> >
> > I hope that every contributor will be encouraged,
> > and I also hope that the Calcite community will become stronger and
> > stronger.
> >
> > +1, I am willing to join the pool of reviews.
> >
> > On Tue, 11 Apr 2023 at 13:20, Benchao Li  wrote:
> >
> > > Thanks Julian for starting the discussion!
> > >
> > > I'm spending my spare time to contribute to Calcite, usually at
> weekends,
> > > and sometimes in the break of weekdays, hence my time would be limited
> > > because the spare time may varies. Review work is not that simple for
> me
> > > because Calcite has many complicated components and evolves many years
> > > which means we need track a lot of background. I'm still learning some
> > part
> > > while doing the review work.
> > >
> > > The complexity of PRs varies a lot, simple ones would be easier to get
> in
> > > because it only cost me minutes to hours to review. But the complex
> ones,
> > > usually I need to spend more time to understand the background, new
> > design,
> > > the effect to the whole project, and the future direction we want to
> > take.
> > > These kinds of PRs may be preempted 

Re: Draft: board report for 2023 Q1

[Ruben Q L]

Thanks Stamatis for preparing the report.
LGTM.


On Thu, Apr 6, 2023 at 1:17 PM Stamatis Zampetakis 
wrote:

> Hello,
>
> Below you can find a draft of this quarter's board report. I plan to
> submit it next Tuesday (April 11, 2023).
> Please let me know if you have any additions or corrections.
>
> Best regards,
> Stamatis
> -
>
> ## Description:
> Apache Calcite is a highly customizable framework for parsing and planning
> queries on data in a wide variety of formats. It allows database-like
> access,
> and in particular a SQL interface and advanced query optimization, for data
> not residing in a traditional database.
>
> Avatica is a sub-project within Calcite and provides a framework for
> building
> local and remote JDBC and ODBC database drivers. Avatica has an independent
> release schedule and its own repository.
>
> ## Issues:
> There are no issues requiring board attention.
>
> ## Membership Data:
> Apache Calcite was founded 2015-10-22 (7 years ago)
> There are currently 63 committers and 27 PMC members in this project.
> The Committer-to-PMC ratio is 7:3.
>
> Community changes, past quarter:
> - Benchao Li was added to the PMC on 2023-01-27
> - Alex Plehanov was added as committer on 2023-01-06
> - Jiajun Xie was added as committer on 2023-02-10
> - Istvan Toth was added as committer on 2023-01-25
>
> ## Project Activity:
> Apache Calcite 1.34.0 was released on 2023-03-14. It contains contributions
> from 18 contributors, and resolves 34 issues. It’s worth highlighting the
> introduction of QUALIFY clause (CALCITE-5268), which facilitates filtering
> the
> results of window functions. Among other improvements and fixes, it adds
> roughly 15 new functions in BigQuery library for handling dates, times, and
> timestamps.
>
> Apache Calcite 1.33.0 was released on 2023-02-06. It contains contributions
> from 33 contributors, and resolves 107 issues. It’s worth highlighting the
> support for custom time frames (CALCITE-5155), the new MEASURE type and
> AGGREGATE aggregate function (CALCITE-5105) as well as the many
> improvements
> to the BigQuery dialect (CALCITE-5180).
>
> Apache Calcite Avatica 1.23.0 was released on 2023-01-19. It fixes bugs in
> Statement.getUpdateCount(), ResultSet.getObject; and supports
> HTTP_BAD_REQUEST
> and configuring fetch size and SSL key-store type. Also, there are various
> improvements to DateTimeUtils and ByteString.
>
> On 2023-03-15, there was an online meetup of the Calcite community with
> approximately 50 participants joining the call. We had three very
> interesting
> presentations around adding measures in SQL, incremental view maintenance
> for
> streaming engines, and debugging planner issues, followed by open
> discussion.
> The videos from the meetup as well as the slides were published online
> shortly
> after the event.
>
> ## Community Health:
> The project remains super healthy and there is a general increase in
> traffic
> almost in every aspect of the project (dev@, JIRA, and GitHub).
>
> The most notable increase in traffic was for dev mailing list (46%). One
> factor that led to this increase is the addition of four new members in the
> project and the traditional welcoming emails that usually come along.
> Another
> one, would probably be the various discussion/votes around releases; we had
> more this quarter compared to our usual cadence. Last there were also some
> changes in the CI (SonarCloud intergration) that sparked some additional
> exchanges.
>
> The number of non-committer (contributor) commits per month:
> +-+-+-+
> |year |month| contributor_commits |
> +-+-+-+
> | 2023| 1   | 20  |
> | 2023| 2   | 15  |
> | 2023| 3   | 17  |
> +-+-+-+
>
> The number of active reviewers per month:
> +-+-+-+
> |year |month|  active_reviewers   |
> +-+-+-+
> | 2023| 1   | 4   |
> | 2023| 2   | 5   |
> | 2023| 3   | 4   |
> +-+-+-+
>
> Top-3 reviewers in the last quarter:
> +---+-+
> | committer |   reviews   |
> +---+-+
> | Julian Hyde  | 34  |
> | Benchao Li  | 9   |
> | Stamatis Zampetakis  | 4   |
> +---+-+
>
> The number of non-commiter commits has almost 

Re: [VOTE] Release Apache Calcite 1.34.0 (release candidate 0)

[Ruben Q L]

Thanks Stamatis for being RM.

- Release notes: ok
- Checksum and signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best regards,
Ruben


On Sat, Mar 11, 2023 at 9:38 PM Sergey Nuyanzin  wrote:

> +1 (non-binding)
>
>  * Downloaded
>  * verified signatures and hashes
>  * checked LICENSE, NOTICE, README
>  * compiled and ran tests on Fedora Linux
>  * reviewed release notes
>  * checked that there is no new issues while running against Flink tests
>
> On Sat, Mar 11, 2023 at 7:50 AM Francis Chuang 
> wrote:
>
> > Thanks for being RM for this release, Stamatis!
> >
> > My vote is: +1 (binding)
> >
> > - Verified GPG signature - OK
> > - Verified SHA512 - OK
> > - Diffed source release and git repository - OK
> > - Checked release notes on tag
> > (
> >
> https://github.com/apache/calcite/blob/calcite-1.34.0-rc0/site/_docs/history.md
> )
> >
> > - OK
> > - Checked year and versions in NOTICE, README and HOWTO - OK
> > - Ran tests (gradle check) - OK
> > - Spot checked Nexus artifacts - OK
> >
> > Environment:
> > Eclipse-temurin:17-jammy docker container in WSL2 (Ubuntu 22.04.1) on
> > Windows 11 22h2
> >
> > $ docker version
> > Client: Docker Engine - Community
> >   Cloud integration: v1.0.31
> >   Version:   20.10.23
> >   API version:   1.41
> >   Go version:go1.18.10
> >   Git commit:7155243
> >   Built: Thu Jan 19 17:34:13 2023
> >   OS/Arch:   linux/amd64
> >   Context:   default
> >   Experimental:  true
> >
> > Server: Docker Desktop
> >   Engine:
> >Version:  20.10.23
> >API version:  1.41 (minimum version 1.12)
> >Go version:   go1.18.10
> >Git commit:   6051f14
> >Built:Thu Jan 19 17:32:04 2023
> >OS/Arch:  linux/amd64
> >Experimental: false
> >   containerd:
> >Version:  1.6.18
> >GitCommit:2456e983eb9e37e47538f59ea18f2043c9a73640
> >   runc:
> >Version:  1.1.4
> >GitCommit:v1.1.4-0-g5fd4c4d
> >   docker-init:
> >Version:  0.19.0
> >GitCommit:de40ad0
> >
> > $ gradle -v
> >
> > 
> > Gradle 7.4.2
> > 
> >
> > Build time:   2022-03-31 15:25:29 UTC
> > Revision: 540473b8118064efcc264694cbcaa4b677f61041
> >
> > Kotlin:   1.5.31
> > Groovy:   3.0.9
> > Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> > JVM:  17.0.6 (Eclipse Adoptium 17.0.6+10)
> > OS:   Linux 5.15.90.1-microsoft-standard-WSL2 amd64
> >
> > $ java -version
> > openjdk version "17.0.6" 2023-01-17
> > OpenJDK Runtime Environment Temurin-17.0.6+10 (build 17.0.6+10)
> > OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (build 17.0.6+10, mixed mode,
> > sharing)
> >
> > Francis
> >
> > On 10/03/2023 10:32 pm, Stamatis Zampetakis wrote:
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.34.0, release
> > > candidate 0.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > You can read the release notes here:
> > >
> >
> https://github.com/apache/calcite/blob/calcite-1.34.0-rc0/site/_docs/history.md
> > >
> > > The commit to be voted upon:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=7dfd641baeb0e1b26dec04da5241c3999fe0ac6a
> > >
> > > Its hash is 7dfd641baeb0e1b26dec04da5241c3999fe0ac6a
> > >
> > > Tag:
> > > https://github.com/apache/calcite/tree/calcite-1.34.0-rc0
> > >
> > > The artifacts to be voted on are located here:
> > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.34.0-rc0
> > > (revision 60513)
> > >
> > > The hashes of the artifacts are as follows:
> > >
> >
> 1b4733229b67e3241329c24c52a79122e1834a4ef1db9c25856281de7e6db8a80a6ae6ac07d28dd05d3ee43c0a79587cc280ebfc6025b4ba3d974e38e804b47b
> > > *apache-calcite-1.34.0-src.tar.gz
> > >
> > > A staged Maven repository is available for review at:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1200/org/apache/calcite/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/zabetak.asc
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > > To create the jars and test Apache Calcite: "gradle build"
> > > (requires an appropriate Gradle/JDK installation)
> > >
> > > Please vote on releasing this package as Apache Calcite 1.34.0.
> > >
> > > The vote is open for the next 96 hours (due to the weekend) and passes
> > if a
> > > majority of at
> > > least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.34.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Here is my vote:
> > >
> > > +1 (binding)
> > >
> > > Stamatis
> > >
> >
>
>
> 

Re: [ANNOUNCE] New self-serve portal for projects, committers, and contributors

[Ruben Q L]

I think the list might be still relevant for people with an "old Jira
account" who need to request contributor access to Calcite project on Jira.
Wdyt?

El lun., 6 mar. 2023 12:33, Stamatis Zampetakis 
escribió:

> Thanks Francis for taking care of this!
>
> After CALCITE- [1], the jira-requests@calcite mailing list is now
> obsolete. If there are no objections I will log an INFRA ticket to delete
> it.
>
> Best,
> Stamatis
>
> [1] https://issues.apache.org/jira/browse/CALCITE-
>
> On Thu, Mar 2, 2023 at 1:38 AM Francis Chuang 
> wrote:
>
> > The ASF has published a new process for contributors to request Jira
> > accounts. We should update our documentation to direct contributors to
> > this new form and use it as part of the onboarding process.
> >
> > What do you all think?
> >
> > Francis
> >
> >  Forwarded Message 
> > Subject: [ANNOUNCE] New self-serve portal for projects, committers, and
> > contributors
> > Date: Wed, 1 Mar 2023 18:12:25 -0600
> > From: Daniel Gruno 
> > Reply-To: us...@infra.apache.org
> > To: annou...@infra.apache.org
> >
> > Hi folks,
> > During today's Infrastructure round-table session[1] we launched a new
> > infrastructure self-serve portal that replaces our old self-serve site.
> >
> > The URL remains the same, https://selfserve.apache.org/ , and while the
> > self-serve offerings are largely the same as well, I do want to call out
> > the new "Request a Jira account" feature on there.
> >
> > Prior to today, projects were tasked with managing new Jira accounts
> > themselves, including handling the initial request from users. The new
> > self-serve workflow for Jira accounts is aimed at making this a faster,
> > more streamlined experience for all projects, while also addressing data
> > privacy concerns and time constraints of projects. Any project using
> > Jira can now refer their users/contributors to
> > https://selfserve.apache.org/jira-account.html for new Jira account
> > requests, which will take care of all the needed paperwork for new
> > account requests and loop in projects for the final review and approval
> > of the accounts.
> >
> > We hope this new feature will be well received. If there are any
> > questions, concerns, or other feedback regarding the new self-serve
> > portal, please do reach out to us at: us...@infra.apache.org
> >
> > With regards,
> > Daniel on behalf of ASF Infra.
> >
> >
> > [1] https://infra.apache.org/roundtable.html
> >
>

[jira] [Created] (CALCITE-5559) Improve RepeatUnion by discarding duplicates at TableSpool level

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5559:
--

 Summary: Improve RepeatUnion by discarding duplicates at 
TableSpool level
 Key: CALCITE-5559
 URL: https://issues.apache.org/jira/browse/CALCITE-5559
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Ruben Q L


Currently, RepeatUnion operator with all=false keeps track of the elements that 
it has returned in order to discard duplicates. However, the TableSpool 
operators that are right below it do not have such control. In certain 
scenarios, duplicates are returned by the TableSpool current iteration, 
discarded by the RepeatUnion, but have been already "fed back" by the 
TableSpool into the next iteration, causing unnecessary processing.
We can optimize this scenario by keeping track of the duplicates inside the 
TableSpool too (note: we still need to keep track of duplicates at RepeatUnion 
level, because that is the only place where we can detect a potential "global 
duplicate" of an element: returned by the LHS and then also by the RHS, or by 
two different iterations of the RHS).

A PoC testing this improvement on a downstream project showed that certain 
queries can go from ~40s down to ~1s.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: RECURSIVE keyword not supported?

[Ruben Q L]

Hello Gunnar,

IIRC the idea when that doc [1] was written was providing an alternative
syntax that people would understand (since probably the RelBuilder example
would be too abstract or complex for most users).
Perhaps as it is written right now it can be misleading and we should
clarify that at the moment SQL is not supported for recursive queries, and
it is only shown for illustrative purposes.
Contributions are always welcome, including the documentation! Feel free to
open a PR to improve this part, I'll happily take a look at it.

Best,
Ruben

[1] https://github.com/apache/calcite/blob/main/site/_docs/algebra.md


On Wed, Feb 22, 2023 at 7:32 AM Gunnar Morling
 wrote:

> Hey Julien,
>
> Am Di., 21. Feb. 2023 um 23:01 Uhr schrieb Julian Hyde :
>
> > A classic problem of open source development. Should we accept a
> > contribution for recursive queries even though it doesn't have SQL
> > support? And if we choose to accept it without SQL support, do we show
> > the equivalent SQL in our documentation?
> >
>
> Yes, the struggle is real, and there's no simple answer. Being a maintainer
> of multiple open-source projects myself, I very much can relate to that.
> FWIW, I think you made the right call by accepting the feature in its
> current state. It then comes down to expectation management, i.e. being
> very transparent about what is supported and what isn't.
>
> >
> > I agree that the documentation is a bit misleading. The caveats about
> > "experimental APIs" mostly cover it, but it should say that "WITH
> > RECURSIVE" SQL is not currently supported. But you are also
> > complaining that the free lunch has french fries and baked potatoes
> > but no mashed potatoes.
> >
>
> Ugh, now that's a bit harsh, don't you think? I've asked for clarification
> about how to use this feature, and when learning that right now it isn't,
> I've offered to help clarify that fact in the doc (as much as I'd love help
> building the actual feature, it's just beyond the scope of what my time
> currently allows for, unfortunately). I think that hardly qualifies as
> "complaining".
>
> Best,
>
> --Gunnar
>
> Julian
> >
> > On Tue, Feb 21, 2023 at 1:06 PM Gunnar Morling
> >  wrote:
> > >
> > > Hey Ruben,
> > >
> > > Thanks a lot for the quick response! In that light, isn't having that
> SQL
> > > example in the docs a bit confusing then? Might be worth clarifying for
> > the
> > > time being that this is a hypothetical syntax and recursive queries are
> > > only supported in the RelBuilder API? Happy to send a PR for doing that
> > doc
> > > update, if you think it makes sense and can point me to the source of
> > that
> > > page.
> > >
> > > Thanks again,
> > >
> > > --Gunnar
> > >
> > >
> > > Am Di., 21. Feb. 2023 um 20:56 Uhr schrieb Ruben Q L <
> rube...@gmail.com
> > >:
> > >
> > > > Hello Gunnar,
> > > >
> > > > At the moment, recursive queries are only supported via RelBuilder
> > API, not
> > > > via SQL (the documentation shows the SQL equivalent to the RelBuider
> > code
> > > > just below, but Calcite currently only supports the latter, not the
> > > > former).
> > > >
> > > > There is an old ticket about supporting this feature in SQL [1] , but
> > so
> > > > far no work has been done in this regard.
> > > >
> > > > Best regards,
> > > > Ruben
> > > >
> > > > [1] https://issues.apache.org/jira/browse/CALCITE-129
> > > >
> > > >
> > > > On Tue, Feb 21, 2023 at 7:23 PM Gunnar Morling
> > > >  wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm trying to run the recursive example from the Calcite algebra
> docs
> > > > [1],
> > > > > but I'm always getting an error at the RECURSIVE keyword:
> > > > >
> > > > > WITH RECURSIVE aux(i) AS (
> > > > >   VALUES (1)
> > > > >   UNION ALL
> > > > >   SELECT i+1 FROM aux WHERE i < 10
> > > > > )
> > > > > SELECT * FROM aux
> > > > >
> > > > > "Incorrect syntax near the keyword 'RECURSIVE' at line 1,
> column
> > 6"
> > > > >
> > > > > I saw this is considered an experimental feature for the time
> being,
> > is
> > > > > there some flag/option I need to specify when creating my
> connection
> > to
> > > > > enable support for recursive queries?
> > > > >
> > > > > Thanks a lot for any help,
> > > > >
> > > > > --Gunnar
> > > > >
> > > > > [1] https://calcite.apache.org/docs/algebra.html#recursive-queries
> > > > >
> > > >
> >
>

Re: RECURSIVE keyword not supported?

[Ruben Q L]

Hello Gunnar,

At the moment, recursive queries are only supported via RelBuilder API, not
via SQL (the documentation shows the SQL equivalent to the RelBuider code
just below, but Calcite currently only supports the latter, not the former).

There is an old ticket about supporting this feature in SQL [1] , but so
far no work has been done in this regard.

Best regards,
Ruben

[1] https://issues.apache.org/jira/browse/CALCITE-129


On Tue, Feb 21, 2023 at 7:23 PM Gunnar Morling
 wrote:

> Hi,
>
> I'm trying to run the recursive example from the Calcite algebra docs [1],
> but I'm always getting an error at the RECURSIVE keyword:
>
> WITH RECURSIVE aux(i) AS (
>   VALUES (1)
>   UNION ALL
>   SELECT i+1 FROM aux WHERE i < 10
> )
> SELECT * FROM aux
>
> "Incorrect syntax near the keyword 'RECURSIVE' at line 1, column 6"
>
> I saw this is considered an experimental feature for the time being, is
> there some flag/option I need to specify when creating my connection to
> enable support for recursive queries?
>
> Thanks a lot for any help,
>
> --Gunnar
>
> [1] https://calcite.apache.org/docs/algebra.html#recursive-queries
>

Re: 1.33.0: Possible regression in rewriting COALESCE expression.

[Ruben Q L]

Hello Max,

Thanks for reporting this. At first glance it seems a regression, could you
please create a Jira ticket (and link it to CALCITE-5424)?

Best,
Ruben


On Wed, Feb 15, 2023 at 3:27 PM Maksim Zhuravkov 
wrote:

> Hello, team.
>
> Thank you for all the effort you have put into this project over the years.
>
> After updating to calcite 1.33.0, I found that validation of the following
> query fails with a ClassCastException.
>
> Query: SELECT COALESCE(DATE '2021-07-08', DATE '2020-01-01’)
>
> Error:
>
> org.apache.calcite.tools.ValidationException:
> java.lang.ClassCastException: class org.apache.calcite.sql.SqlLiteral
> cannot be cast to class org.apache.calcite.sql.SqlUnknownLiteral
> (org.apache.calcite.sql.SqlLiteral and
> org.apache.calcite.sql.SqlUnknownLiteral are in unnamed module of loader
> 'app')
>
> at org.apache.calcite.prepare.PlannerImpl.validate(PlannerImpl.java:226)
> at
> org.apache.calcite.rex.RexSqlStandardConvertletTableTest.convertSqlToRel(RexSqlStandardConvertletTableTest.java:101)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
> at
> com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
> Caused by: java.lang.ClassCastException: class
> org.apache.calcite.sql.SqlLiteral cannot be cast to class
> org.apache.calcite.sql.SqlUnknownLiteral (org.apache.calcite.sql.SqlLiteral
> and org.apache.calcite.sql.SqlUnknownLiteral are in unnamed module of
> loader 'app')
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.resolveLiteral(SqlValidatorImpl.java:6152)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:6617)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit(SqlValidatorImpl.java:6590)
>
> When I move a commit pointer prior to the commit that introduces 
> SqlUnknownLiteral ([CALCITE-5424] Customize handling of literals based on 
> type system), the test cases above passes successfully.
>
> I haven’t given a more thorough look at this error and maybe this behaviour 
> was introduced after that commit,  but I think this error is most likely a 
> bug.
>
> The reproducer (added to RexSqlStandardConvertletTableTest):
>
>   @Test void testCoalesceWithUnknownLiteral() {
> final Project project = (Project) convertSqlToRel(
> "SELECT COALESCE(DATE '2021-07-08', DATE '2020-01-01')", false);
>
> final RexNode rex = project.getProjects().get(0);
> final RexToSqlNodeConverter rexToSqlNodeConverter = 
> rexToSqlNodeConverter();
> final SqlNode convertedSql = rexToSqlNodeConverter.convertNode(rex);
>
> assertEquals(
> "CASE WHEN DATE '2021-07-08' IS NOT NULL THEN DATE '2021-07-08' ELSE 
> DATE '2020-01-01' END",
> convertedSql.toString());
>   }
>
>
> [CALCITE-5424] Customize handling of literals based on type system - ASF JIRA 
> 
> issues.apache.org 
> [image: fav-jsw.png]  
> 
>
>
> Best regards,
>
> Max Zhuravkov
>
>
>
>

Re: [ANNOUNCE] New committer: Jiajun Xie

[Ruben Q L]

Congratulations Jiajun!


On Sat, Feb 11, 2023 at 6:39 AM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> Congratulations Jiajun, very well deserved, looking forward to work more
> with you in the future!
>
> Best regards,
> Alessandro
>
> On Sat 11 Feb 2023, 03:06 Benchao Li,  wrote:
>
> > Congratulations, Jiajun. Welcome on board.
> >
> > Francis Chuang  于2023年2月11日周六 09:51写道：
> >
> > > Congrats, Jiajun!
> > >
> > > On 11/02/2023 8:09 am, Stamatis Zampetakis wrote:
> > > > Apache Calcite's Project Management Committee (PMC) has invited
> Jiajun
> > > Xie
> > > > to
> > > > become a committer, and we are pleased to announce that they have
> > > accepted.
> > > >
> > > > Jiajun has been doing some great work for the project both in terms
> of
> > > code
> > > > as
> > > > well as helping others in the mailing list! He has landed important
> > > fixes
> > > > in
> > > > RelToSqlConverter and more generally in the JDBC storage handler,
> added
> > > > new SQL functions, improved the SQL parsers and the metadata
> handlers,
> > > and
> > > > pushed many more high quality contributions in various areas.
> > > >
> > > > Jiajun, welcome, thank you for your contributions, and we look
> forward
> > to
> > > > your
> > > > further interactions with the community! If you wish, please feel
> free
> > to
> > > > tell
> > > > us more about yourself and what you are working on.
> > > >
> > > > As your first commit, please add yourself to the contributors list
> [1]
> > > > and the community page will re-generate [2].
> > > >
> > > > Stamatis (on behalf of the Apache Calcite PMC)
> > > >
> > > > [1]
> > >
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > > >
> > > > [2] https://calcite.apache.org/community/#project-members
> > > >
> > >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>

Re: withExpand behavior with ANY/ALL subqueries

[Ruben Q L]

Agree. The change makes sense, and it must be clearly documented on the
next release notes.


On Sat, Feb 11, 2023 at 12:18 AM Julian Hyde  wrote:

> Agreed. This is a breaking change.
>
> I've started work in a branch:
> https://github.com/julianhyde/calcite/tree/3870-sql2rel-expand-false
>
> On Fri, Feb 10, 2023 at 1:47 PM Stamatis Zampetakis 
> wrote:
> >
> > It totally makes sense to have expand=false since this is what we
> recommend.
> >
> > It can be a notable change though for those using SqlToRelConverter as it
> > is so we have to at least put it in a prominent place in the release
> notes.
> >
> > Best,
> > Stamatis
> >
> >
> > On Fri, Feb 10, 2023 at 10:32 PM Julian Hyde  wrote:
> >
> > > You should definitely log a bug for this. Please do so.
> > >
> > > We would prefer that new features in SqlToRelConverter are developed
> > > with expand=false. (Rationale: Keeping subqueries as RexSubQuery
> > > expressions allows us to handle them later, via a planner rule, which
> > > makes the logic more composable and therefore less buggy.) But we will
> > > accept bug fixes to SqlToRelConverter running in expand=true mode if
> > > they are clean and simple.
> > >
> > > That's quite a big 'if'. I suspect that it's quite hard to make a fix
> > > without significant changes to SqlToRelConverter. If so, you should
> > > stop, because we are not likely to accept it.
> > >
> > > Calcite committers,
> > >
> > > For a while https://issues.apache.org/jira/browse/CALCITE-3870 has
> > > been open, proposing that expand=false is the default. Should we do
> > > it? I think it will clarify our position, and make people less likely
> > > to run into bugs when they use the old expand=true behavior by
> > > accident.
> > >
> > > Julian
> > >
> > >
> > > On Fri, Feb 10, 2023 at 9:57 AM Jonathan Sternberg 
> > > wrote:
> > > >
> > > > Hi,
> > > >
> > > > I'm using Calcite for a project and am attempting to implement the
> > > behavior
> > > > for ANY/ALL such as:
> > > >
> > > > SELECT ... FROM ... WHERE a = ANY(SELECT ...)
> > > >
> > > > When I attempt to have Calcite create a plan for this query, I get
> the
> > > > runtime exception from here:
> > > >
> > >
> https://github.com/apache/calcite/blob/717eb59a73e2b456266da1af5ff9b881b6f7eeed/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5319
> > > >
> > > > withExpand = true is the default behavior and, in other cases like
> > > HAVING,
> > > > it produces a more desirable plan so I'd prefer to keep withExpand
> to its
> > > > default value rather than overriding it. Is there a way to allow
> > > subqueries
> > > > to be expanded while also allowing subqueries that can't be expanded
> to
> > > > stay as subqueries? Would Calcite be open to a change that allowed
> this
> > > > behavior?
> > > >
> > > > Thanks.
> > > >
> > > > --Jonathan Sternberg
> > >
>

Re: [VOTE] Release Apache Calcite 1.33.0 (release candidate 0)

[Ruben Q L]

Thanks Jess for producing this release.
By the way, I think your vote should be counted as "non-binding" (probably
just a copy-paste error).

- Release notes: ok (considering the comments under the PR)
- Checksum and signature: ok
- Diff source release and git repository: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best regards,
Ruben


On Fri, Feb 3, 2023 at 11:11 AM Francis Chuang 
wrote:

> Thanks for being RM for this release, Jess!
>
> My vote is: +1 (binding)
>
> - Verified GPG signature - OK
> - Verified SHA512 - OK
> - Diffed source release and git repository - OK
> - Checked release notes on tag
> (
> https://github.com/apache/calcite/blob/calcite-1.33.0-rc0/site/_docs/history.md)
>
> - OK
> - Checked year in NOTICE, README and HOWTO - OK
> - Ran tests (gradle check) - OK
> - Spot checked Nexus artifacts - OK
>
> I noticed that the commit 96b05ee12 for the release is missing from the
> main branch on GitHub, see the message on GitHub here:
>
> https://github.com/apache/calcite/commit/96b05ee12f936ed057265072ff6a2de8ea0a249e
>
> It is on the main branch in GitBox,
>
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=log;h=96b05ee12f936ed057265072ff6a2de8ea0a249e,
>
> so I think you just need to push that commit to main on GitHub.
>
> Environment:
> Eclipse-temurin:17-jammy docker container in WSL2 (Ubuntu 22.04.1) on
> Windows 11 22h2
>
>  > docker version
> Client: Docker Engine - Community
>   Cloud integration: v1.0.29
>   Version:   20.10.22
>   API version:   1.41
>   Go version:go1.18.9
>   Git commit:3a2c30b
>   Built: Thu Dec 15 22:28:22 2022
>   OS/Arch:   linux/amd64
>   Context:   default
>   Experimental:  true
>
> Server: Docker Desktop
>   Engine:
>Version:  20.10.22
>API version:  1.41 (minimum version 1.12)
>Go version:   go1.18.9
>Git commit:   42c8b31
>Built:Thu Dec 15 22:26:14 2022
>OS/Arch:  linux/amd64
>Experimental: false
>   containerd:
>Version:  1.6.14
>GitCommit:9ba4b250366a5ddde94bb7c9d1def331423aa323
>   runc:
>Version:  1.1.4
>GitCommit:v1.1.4-0-g5fd4c4d
>   docker-init:
>Version:  0.19.0
>GitCommit:de40ad0
>
>  >gradle -v
>
> 
> Gradle 7.4.2
> 
>
> Build time:   2022-03-31 15:25:29 UTC
> Revision: 540473b8118064efcc264694cbcaa4b677f61041
>
> Kotlin:   1.5.31
> Groovy:   3.0.9
> Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> JVM:  17.0.5 (Eclipse Adoptium 17.0.5+8)
> OS:   Linux 5.15.79.1-microsoft-standard-WSL2 amd64
>
>  > java -version
> openjdk version "17.0.5" 2022-10-18
> OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
> OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (build 17.0.5+8, mixed mode,
> sharing)
>
> Francis
>
>
> On 3/02/2023 3:24 pm, Jess Balint wrote:
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.33.0, release
> > candidate 0.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> https://github.com/apache/calcite/blob/calcite-1.33.0-rc0/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=96b05ee12f936ed057265072ff6a2de8ea0a249e
> >
> > Its hash is 96b05ee12f936ed057265072ff6a2de8ea0a249e
> >
> > Tag:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.33.0-rc0
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.33.0-rc0
> > (revision 59874)
> >
> > The hashes of the artifacts are as follows:
> >
> a4acc084344b48c1e9a61a70fbe8009c8093cafdb8c4f05d646681a5ea15ab79ab90b4c2fc7e410f952239a167001f4021e97c2c0be8f29edb7255015cb8a7c6
> > *apache-calcite-1.33.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1199/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/jbalint.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.33.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.33.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (binding)
> >
>

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Ruben Q L]

Congratulations Benchao!
You have become a very important and valuable figure in the community.
Thanks for your help and your contributions!

Best,
Ruben


On Fri, Jan 27, 2023 at 10:29 AM Sergey Nuyanzin 
wrote:

> Congratulations, Benchao!
>
> On Fri, Jan 27, 2023 at 11:27 AM Francis Chuang 
> wrote:
>
> > Congrats, Benchao!
> >
> > On 27/01/2023 9:26 pm, Alessandro Solimando wrote:
> > > Congratulations Benchao, very well deserved!
> > >
> > > Best regards,
> > > Alessandro
> > >
> > > On Fri, 27 Jan 2023 at 10:51, Stamatis Zampetakis 
> > wrote:
> > >
> > >> I am pleased to announce that Benchao has accepted an invitation to
> join
> > >> the Calcite PMC. Benchao has been a consistent and helpful figure in
> > >> the Calcite community for which we are very grateful. We look forward
> to
> > >> the continued contributions and support.
> > >>
> > >> Please join me in congratulating Benchao!
> > >>
> > >> Stamatis (on behalf of the Calcite PMC)
> > >>
> > >
> >
>
>
> --
> Best regards,
> Sergey
>

Re: [DISCUSS] Towards Calcite 1.33.0

[Ruben Q L]

;>>>
> > > > > >>>>> On Thu, Jan 5, 2023 at 12:24 PM Stamatis Zampetakis <
> > > > > >> zabe...@gmail.com
> > > > > >>>>
> > > > > >>>>> wrote:
> > > > > >>>>>
> > > > > >>>>>> I added https://issues.apache.org/jira/browse/CALCITE-2884
> > to the
> > > > > >>> list. I
> > > > > >>>>>> am finalizing the patch right now and will get this in a few
> > > > hours.
> > > > > >>>>>> I removed the fixVersion from
> > > > > >>>>>> https://issues.apache.org/jira/browse/CALCITE-5427 (I am
> > working
> > > > on
> > > > > >>> it)
> > > > > >>>>>> since there is no rush to get it in 1.33.0.
> > > > > >>>>>> Unfortunately, I don't have time to review other things
> today.
> > > > > >>>>>>
> > > > > >>>>>> Best,
> > > > > >>>>>> Stamatis
> > > > > >>>>>>
> > > > > >>>>>> On Wed, Jan 4, 2023 at 9:19 PM Julian Hyde <
> > > > jhyde.apa...@gmail.com>
> > > > > >>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> According to the dashboard [
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>
> > > > > >>
> > > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > > >>>>>>> <
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>
> > > > > >>
> > > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > > >>>>>>>
> > > > > >>>>>>> ] there are 6 open issues, all of which have pull requests.
> > > > > >>>>>>>
> > > > > >>>>>>> Can we have volunteers to review/merge these in the next 24
> > > > hours?
> > > > > >> As
> > > > > >>>>>> last
> > > > > >>>>>>> time, I’ll match: if 2 people take 2 issues each, I’ll take
> > the
> > > > > >> last
> > > > > >>> 2.
> > > > > >>>>>>>
> > > > > >>>>>>> Julian
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>> On Jan 4, 2023, at 11:30 AM, Jess Balint <
> jbal...@gmail.com
> > >
> > > > > >> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>> Happy New Year everyone! I am going to do a RC build this
> > week
> > > > if
> > > > > >>> there
> > > > > >>>>>>> are
> > > > > >>>>>>>> no objections.
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Thu, Dec 15, 2022 at 5:27 PM Bertil Chapuis <
> > > > > >> bchap...@gmail.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> Hello Everyone,
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> I just merged CALCITE-5417 after incorporating Julian's
> > > > > >> feedback. I
> > > > > >>>>>>>>> believe this was the final issue that needed to be
> resolved
> > > > > >> before
> > > > > >>> the
> > > > > >>>>>>>>> release.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Thanks a lot to Grigory for his help and for releasing
> new
> > > 

Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis

[Ruben Q L]

Thanks everyone for your kind words. It has been an honor to serve as PMC
Chair.
And congratulations Stamatis! You were a great Chair in the past, and
you'll be a great Chair throughout this year.
I subscribe to Stamatis' words: it is a pleasure working within this
community, with so many hardworking and motivating people collaborating (in
fact volunteering) to improve our project everyday. Let's keep up the good
work in 2023!

Best regards,
Ruben



On Fri, Jan 20, 2023 at 9:56 PM Stamatis Zampetakis 
wrote:

> Thank you everyone! Thanks Ruben for being a great chair!
>
> I feel very honored to be serving as the chair of Calcite! I will try to
> follow the great example that the previous chairs have set so far and
> fulfill at best my duties for this year.
>
> The Calcite community is not the biggest in the ASF but it is definitely
> among the strongest ones. People who join the project tend to stick with
> this community for a long period of time. I am happy to see that
> the people with whom I was collaborating when I joined the project are
> still present and active in the project, and even happier to meet and
> interact with those coming afterwards. I really like being part of this!
>
> Best,
> Stamatis
>
> On Fri, Jan 20, 2023 at 4:29 AM Benchao Li  wrote:
>
> > Thanks Ruben for being a great chair for the last year, and
> congratulations
> > to Stamatis!
> >
> > Francis Chuang  于2023年1月20日周五 05:04写道：
> >
> > > Congratulations, Stamatis, and thanks to Ruben for being a great chair!
> > >
> > > On 20/01/2023 5:47 am, Gavin Ray wrote:
> > > > Congratulations Stamatis, well-deserved!
> > > >
> > > >
> > > > On Thu, Jan 19, 2023 at 1:34 PM Julian Hyde 
> wrote:
> > > >
> > > >> As many of you remember, Stamatis has served as Chair before. This
> is
> > > >> the first time that Calcite has re-elected a previous Chair, and
> that
> > > >> speaks to Stamatis' excellent qualities. Stamatis' style is very
> > > >> similar to Ruben's style - serving the project by quietly getting
> > > >> things done, defusing arguments and personality conflicts, and
> getting
> > > >> us all to work together better.
> > > >>
> > > >> Congratulations Stamatis, and thank you for your service to the
> > project,
> > > >> Ruben.
> > > >>
> > > >> Julian
> > > >>
> > > >> On Thu, Jan 19, 2023 at 9:57 AM Michael Mior 
> > wrote:
> > > >>>
> > > >>> Congratulations Stamatis and thanks Reuben!
> > > >>>
> > > >>> --
> > > >>> Michael Mior
> > > >>> mm...@apache.org
> > > >>>
> > > >>>
> > > >>> On Thu, Jan 19, 2023 at 12:04 PM Ruben Q L 
> > wrote:
> > > >>>
> > > >>>> Calcite community members,
> > > >>>>
> > > >>>> I am pleased to announce that we have a new PMC chair and VP as
> per
> > > our
> > > >>>> tradition of rotating the chair once a year. I have resigned, and
> > > >> Stamatis
> > > >>>> Zampetakis
> > > >>>> was duly elected by the PMC and approved unanimously by the Board.
> > > >>>>
> > > >>>> Please join me in congratulating Stamatis!
> > > >>>>
> > > >>>> Best regards,
> > > >>>> Ruben
> > > >>>>
> > > >>
> > > >
> > >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>

Re: [DISCUSS] Drop unused branches from git repos

[Ruben Q L]

+1 for the cleanup.

Best,
Ruben


On Mon, Jan 23, 2023 at 7:27 AM Alessandro Solimando <
alessandro.solima...@gmail.com> wrote:

> +1 from me as well, keeping unused branches around is only confusing.
>
> Best regards,
> Alessandro
>
> On Sun 22 Jan 2023, 22:58 Julian Hyde,  wrote:
>
> > +1
> >
> > There’s probably a tag for each release but let’s make sure before we
> drop
> > the branch. If any branch just has a ‘prepare for next iteration’ commit
> > following the release tag I’m fine dropping that commit.
> >
> > Julian
> >
> >
> > > On Jan 22, 2023, at 1:05 PM, Francis Chuang 
> > wrote:
> > >
> > > +1 for dropping them. We should do this for both calcite and
> > calcite-avatica (there shouldn't be any dangling branches for
> > calcite-avatica-go).
> > >
> > > On 23/01/2023 7:13 am, Stamatis Zampetakis wrote:
> > >> Hi all,
> > >> Apart from the main, site, and possibly one or two other branches, all
> > the
> > >> rest are not used by anyone unless I am missing something.
> > >> Many of them (branch-X.Y) were generated during past releases but this
> > >> pattern stopped some time ago.
> > >> It is not a big deal keeping them around but it can be confusing for
> new
> > >> comers (.e.g., we still have the master branch around there) and tools
> > >> operating at repo level.
> > >> The most recent example is the INFRA team informing us about the use
> of
> > >> Travis in our repos I assume due to the presence of .travis.yml file
> in
> > >> branch-X.Y branches.
> > >> Are there any reasons for keeping them around? If not how about
> dropping
> > >> them.
> > >> Best,
> > >> Stamatis
> >
> >
>

[ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis

[Ruben Q L]

Calcite community members,

I am pleased to announce that we have a new PMC chair and VP as per our
tradition of rotating the chair once a year. I have resigned, and Stamatis
Zampetakis
was duly elected by the PMC and approved unanimously by the Board.

Please join me in congratulating Stamatis!

Best regards,
Ruben

Re: Contribution to Calcite

[Ruben Q L]

Hello Nikolay,

I have added you as a Calcite contributor.

Best regards,
Ruben


On Thu, Jan 19, 2023 at 1:41 PM Николай Ижиков  wrote:

> Hello, Calcite.
>
> My name is Nikolay Izhikov.
> Please, add me to the Calcite contributors so I can assign tickets to
> myself and resolve them.
>
> My jira id - nizhikov

Re: [DISCUSS] CAST DOUBLE to BOOLEAN

[Ruben Q L]

Hello,

IIRC there was a discussion some time ago about this topic, see comments in
https://issues.apache.org/jira/browse/CALCITE-4777 and related tickets (all
of them still open).

Best,
Ruben


On Mon, Jan 16, 2023 at 3:46 PM Николай Ижиков  wrote:

> Hello.
>
> Right now, Calcite allows to cast FLOAT, DOUBLE expressions to BOOLEAN.
> But, results is always `false`. Cast works as
> `java.lang.Boolean.parseBoolean` call and always return false for anything
> except «true» string(`Primitive#parse`).
>
> Simple additional test for `InterpreterTest` shows this:
>
> ```
> @Test void testDoubleCast() {
>   sql("SELECT CAST(CAST('0.0' AS DOUBLE) AS
> BOOLEAN)").returnsRows("[false]");
>   sql("SELECT CAST(CAST('0.1' AS DOUBLE) AS
> BOOLEAN)").returnsRows("[false]");
>   sql("SELECT CAST(CAST('1.0' AS DOUBLE) AS
> BOOLEAN)").returnsRows("[false]");
>   sql("SELECT CAST(CAST('42' AS DOUBLE) AS
> BOOLEAN)").returnsRows("[false]");
> }
> ```
>
> Other database, such as Postgres, disallow cast from double(float) to
> boolean:
>
> ```
> psql (14.6 (Homebrew))
> Type "help" for help.
>
> postgres=# SELECT CAST(CAST('1' AS float) AS BOOLEAN);
> ERROR:  cannot cast type double precision to boolean
> LINE 1: SELECT CAST(CAST('1' AS float) AS BOOLEAN);
>^
> postgres=#
> ```
>
> Do we really need to support this?
> I think Calcite should disallow this kind of conversion for better user
> experience.
> I can provide PR to implement proposed behavior
>
> What do you think?

Re: [DISCUSS] Code quality/coverage with SonarCloud & JaCoCo

[Ruben Q L]

Hello,

First of all, thanks Stamatis for implementing this, I think it is
something good for the project.
In the beginning things might be a bit complicated (as always) and we might
need some adjustments / clarifications, but I hope that in the long run
we'll see this as a useful feature.

Regarding the two specific issues being discussed:
- If I am not mistaken, the fact that SqlOperatorTest was moved out of
'test' was a consequence of [1], see the corresponding PR [2] "... it was
necessary to move several classes from the 'core' module to 'testkit'". I
don't know how simple (or how complex) a potential refactoring to move it
out of there might be. Alternatively, it seems that this is rather an
exceptional case, so perhaps it should qualify for an exception (e.g.
everything under /testkit/* shall not be considered for coverage).
- Regarding the instanceof, it seems that it was indeed a valid warning,
and it has recently been fixed via [3] (see discussion on its PR [4])

Best regards,
Ruben

[1] https://issues.apache.org/jira/browse/CALCITE-4885
[2] https://github.com/apache/calcite/pull/2685
[3]
https://github.com/apache/calcite/commit/4bebdb07c2f45a95c9a4fdf81e9bcfbdd11a15de
[4] https://github.com/apache/calcite/pull/2919



On Tue, Jan 10, 2023 at 3:18 PM Stamatis Zampetakis 
wrote:

> Thanks for the feedback!
>
> I would like to stretch the fact that at this point it is at the discretion
> of the reviewer/committer to enforce or ignore the information provided by
> Sonar.
>
> Sonar, as other similar systems, has limitations thus there are always
> going to be false positives. The rules/checks performed are fully
> customisable so we can enable/disable them at will.
>
> The two issues highlighted by Julian seem like true positives to me.
> * The "New code" that was introduced recently is not covered sufficiently
> by tests and that's a fact. Part of the problem seems to come from the
> recent modifications in SqlOperatorTest [1]. The class is located under
> src/main (and not under src/test) so it is considered as a production class
> and coverage checks are applied. There are ways to exclude certain paths
> from coverage but I would argue that the class shouldn't be there in the
> first place; we should probably log a CALCITE ticket for moving out of
> there.
> * The instance of warning is something that we probably don't want/cannot
> fix (for the reasons mentioned in the PR) but Sonar did well to bring this
> up; it is problematic to do comparisons by relying on the class name.
>
> I encourage others to share their thoughts/remarks as well so that we
> improve as much as possible the developer experience.
>
> Best,
> Stamatis
>
> [1]
>
> https://github.com/apache/calcite/blob/be7135cf1fd3d87e4036b2dd6e58d2f1251f8959/testkit/src/main/java/org/apache/calcite/test/SqlOperatorTest.java
>
> On Tue, Jan 10, 2023 at 1:38 AM Julian Hyde  wrote:
>
> > I see two false positives so far:
> >  * The message on be7135cf "58.1% Coverage on New Code (is less than
> 80%)"
> >  * The bug on PR 2942 "Use an "instanceof" comparison instead"
> >
> > Has Sonarcube had any true positives yet?
> >
> > Vladimir used to hate when I was skeptical of changes to the build
> > system. But I have no tolerance for a lint system that makes extra
> > work.
> >
> > Julian
> >
> >
> >
> >
> > On Mon, Jan 9, 2023 at 2:46 PM Francis Chuang 
> > wrote:
> > >
> > > Thanks for implementing this, Stamatis! Having code quality metrics on
> > > our repos is a huge win.
> > >
> > > Francis
> > >
> > > On 10/01/2023 4:22 am, Stamatis Zampetakis wrote:
> > > > I just merged the CALCITE-5427 [1] enabling Sonarcloud analysis for
> > Calcite
> > > > main branch and new PRs.
> > > >
> > > > I have left the default Sonar quality gates active so you may start
> > seeing
> > > > Sonar reporting errors in various cases.
> > > >
> > > > If you encounter problems or you would like things to work
> differently
> > > > please create JIRA tickets and ping me if you need help.
> > > >
> > > > Once we are happy with how things work for Calcite we can also port
> the
> > > > changes to Avatica.
> > > >
> > > > Note that all Calcite committers have administrative privileges to
> the
> > > > Calcite project in Sonarcloud [2].
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > > [1]
> > > >
> >
> https://github.com/apache/calcite/commit/be7135cf1fd3d87e4036b2dd6e58d2f1251f8959
> > > > [2] https://sonarcloud.io/project/roles?id=apache_calcite
> > > >
> > > > On Wed, Jan 4, 2023 at 5:44 PM Stamatis Zampetakis <
> zabe...@gmail.com>
> > > > wrote:
> > > >
> > > >> The integration is advancing well and I am hoping to merge the PR in
> > the
> > > >> coming days.
> > > >> To avoid unpleasant surprises, I am planning to create a new remote
> > branch
> > > >> in the main calcite repo to test some things out. I will delete it
> as
> > soon
> > > >> as I am done with the tests.
> > > >>
> > > >> Best.
> > > >> Stamatis
> > > >>
> > > >> On Wed, Dec 28, 2022 at 2:47 PM 

Re: How to match the arbitrary number of RelNodes

[Ruben Q L]

Hello Aitozi,

Could you please provide one or several examples of a plan that your rule
aims to match? (as it was done in
https://issues.apache.org/jira/browse/CALCITE-3761)
Maybe you could use RelRule#OperandDetailBuilder#predicate [1] and write
your (more-or-less-complex) predicate function for the rule operand(s)?

Best,
Ruben

[1]
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/plan/RelRule.java#L210



On Mon, Jan 9, 2023 at 12:40 PM Aitozi  wrote:

> Hi, community:
>   When I write a rule to match a subtree as a union followed by any
> number of RelNodes which is not an exchange node (I think this subtree
> can transpose before the union nodes), It will help reduce the shuffle
> in physical nodes. But there seems to be no suitable API to describe
> this greedy pattern match with an arbitrary number of intermediate
> nodes. If there is sth I missed, I would appreciate it if anyone could
> help point it out.
>
> BTW, I read this ticket, it seems to have similar requirements:
>
> https://issues.apache.org/jira/browse/CALCITE-3761
>
> Thanks
> Aitozi.
>

[ANNOUNCE] New committer: Alex Plehanov

[Ruben Q L]

Apache Calcite's Project Management Committee (PMC) has invited Alex
Plehanov to become a committer, and we are pleased to announce that he has
accepted the invitation.

Alex is an Apache Ignite PMC member and the main author of the
ignite-calcite module. He has done significant contributions on Calcite,
especially fixing bugs detected by Ignite.

Alex, welcome, thank you for your contributions, and we look forward to
your further interactions with the community! If you wish, please feel free
to tell us a bit more about yourself and what you are currently working on.
As your first commit, you can add yourself to the contributors list [1] and
the community page will re-generate [2].

Ruben (on behalf of the Apache Calcite PMC)

[1] https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
[2] https://calcite.apache.org/community/#project-members

Draft: board report for 2022 Q4

[Ruben Q L]

Hello,

Below these lines you can find a draft of this quarter's board report. I
plan to submit it
before Tuesday next week.
Please let me know if you have any additions or corrections.

Best regards,
Ruben

-

## Description:
Apache Calcite is a highly customizable framework for parsing and planning
queries on data in a wide variety of formats. It allows database-like
access,
and in particular a SQL interface and advanced query optimization, for data
not residing in a traditional database.

Avatica is a sub-project within Calcite and provides a framework for
building
local and remote JDBC and ODBC database drivers. Avatica has an independent
release schedule and its own repository.

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Calcite was founded 2015-10-22 (7 years ago)
There are currently 60 committers and 26 PMC members in this project.
The Committer-to-PMC ratio is roughly 2:1.

Community changes, past quarter:
- No new PMC members. Last addition was Andrei Sereda on 2022-08-08.
- Bertil Chapuis was added as committer on 2022-10-14
- Dmitry Sysolyatin was added as committer on 2022-11-07

Following our yearly rotation tradition, the PMC has elected a new chair:
Stamatis Zampetakis.

## Project Activity:
There has been no Apache Calcite releases in this quarter. The next one
(1.33.0) is currently being discussed and its scope is being finalized, so
it
 should be released soon.

Apache Calcite Avatica Go 5.2.0 was released on 2022-10-13. This is a
maintenance release of Avatica Go with some dependency updates, bug fixes
and
a new minor feature. This release supports Go 1.18 and 1.19, which are
currently the versions supported and maintained by the Go team.

## Community Health:
The community maintains a healthy status. There has been a generalized
increment in all the metrics regarding the project activity (137% increase
in
traffic in the dev mailing list, 55% increase in opened issues in Jira, 78%
increase in closed issues, 62% increase in commits, 50% increase in code
contributors, 36% increase in opened PRs in GitHub, 69% increase in closed
PRs). This is somehow expected after the slight decrease in the previous
quarter due to the summer vacation, but nevertheless it confirms the
positive
trend on Calcite community health and project contributions.

The number of non-committer (contributor) commits per month:
+-+-+-+
|year |month| contributor_commits |
+-+-+-+
| 2022| 10  | 10  |
| 2022| 11  | 11  |
| 2022| 12  | 11  |
+-+-+-+

The number of active reviewers per month:
+-+-+-+
|year |month|  active_reviewers   |
+-+-+-+
| 2022| 10  | 4   |
| 2022| 11  | 6   |
| 2022| 12  | 4   |
+-+-+-+

Top reviewers in the last 3 months:
+---+-+
|  committer| reviews |
+---+-+
| Benchao Li   | 13  |
| Julian Hyde | 6   |
| rubenada   | 4   |
| Alessandro Solimando  | 4   |
| Jess Balint| 2   |
| Dmitry Sysolyatin  | 1   |
| Volodymyr Vysotskyi | 1   |
| Stamatis Zampetakis| 1   |
+---+-+

Re: RelBuilder from SQL Query

[Ruben Q L]

Hello Christophe,

Once you have the RelRoot from the SqlToRelConverter, you can obtain the
corresponding RelNode via RelRoot#project.
After that, you could create a new RelBuilder, and then push that RelNode
inside using RelBuilder#push(RelNode) and from that point on continue using
the builder.

Maybe not relevant in your case, but just fyi another possibility to
"modify the request" (i.e. modify the RelNode) without using a RelBuilder
could be applying a RelShuttle to the RelNode (see e.g.
RelHomogeneousShuttle and subclasses if you want to check some examples).

Best regards,
Ruben



On Mon, Jan 2, 2023 at 2:25 PM Christophe Le Saëc 
wrote:

> Hello,
>
> I used RelBuilder to scan table name without any pb.
> Now, i would like to do same kind of transformation, but instead of table
> name, with a SQL query.
> Something like
> RelBuilder.scanSQL("Select x, y from ...");
>
> I try to use SqlParser / SqlToRelConcerter
>
> SqlNode sqlNode = SqlParser.create(sqlQuery).parseQuery();
> SqlToRelConverter => build RelRoot
>
> but didn't find a way to convert SqlNode nor RelRoot to RelBuilder in order
> to modify the request.
>
> Is it possible ? And if yes, how ?
>
> Best regards,
> Christophe.
>

Re: [DISCUSS] Towards Calcite 1.33.0

[Ruben Q L]

@Jess I think we should wait for [1], which is a cleaner solution for the
Proj4j issue, better than the initial quick-fix that was committed some
days ago [2].
The PR seems in a good shape (thanks Bertil and Julian for your work in
this one!), and hopefully it will be merged soon.

@Sean, no, at this moment there's no plan for Avatica 1.23 release yet (so
it will not be included in Cacite 1.33).

Best,
Ruben

[1] https://issues.apache.org/jira/browse/CALCITE-5417
[2] https://issues.apache.org/jira/browse/CALCITE-5399


On Thu, Dec 15, 2022 at 3:14 AM Sean Broeder  wrote:

> Are there plans to release a new version of Avatica(1.23)  to be included
> in Calcite 1.33.0?
>
> Thanks,
> Sean
>
> > On Dec 14, 2022, at 3:47 PM, Jess Balint  wrote:
> >
> > I have reviewed a few PRs. The Proj4j fix has been merged. Do we have any
> > more bandwidth for reviews or anybody asking for reviews? Otherwise, I
> can
> > start the RC build soon.
> >
> > On Mon, Nov 28, 2022 at 6:07 AM Ruben Q L  wrote:
> >
> >> Thanks Jess for being RM of 1.33, and thanks Benchao for volunteering as
> >> RM.
> >> The updated list for future releases will be as follows:
> >> - 1.33.0 Jess Balint
> >> - 1.34.0 Duan Xiong
> >> - 1.35.0 Benchao Li
> >>
> >> There are currently 14 unresolved issues for 1.33:
> >>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> >> Many of them have a PR available, let's try to do a collective effort
> and
> >> review them ASAP (I remind everyone that you can assign a PR to
> yourself to
> >> let others know that you are taking care of it).
> >> As I mentioned on the initial email, let's try to finalize the remaining
> >> issues during these two weeks, to see if we can produce a RC0 by
> >> mid-December.
> >>
> >> Regarding the recent Proj4j license problem (
> >> https://issues.apache.org/jira/browse/CALCITE-5399), should we consider
> >> this a blocking issue for 1.33?
> >>
> >> Best,
> >> Ruben
> >>
> >>
> >> On Fri, Nov 11, 2022 at 6:53 PM Jess Balint  wrote:
> >>
> >>> Hi Ruben,
> >>>
> >>> Thanks for the note. I'm available for RM duty. Here's the list of
> issues
> >>> currently outstanding:
> >>>
> >>>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%201.33.0%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22In%20Review%22%2C%20%22In%20Implementation%22)%20ORDER%20BY%20priority%20DESC
> >>>
> >>> I'll pick up a couple of reviews and see if we can move out some of the
> >>> issues which haven't been started yet.
> >>>
> >>> Jess
> >>>
> >>> On Fri, Nov 11, 2022 at 3:20 AM Ruben Q L  wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> It's been two months since our last release [1], if we want to keep
> our
> >>>> rhythm, I'd suggest to produce the next one before the end of the
> year.
> >>>>
> >>>> As usual, according to our Jira dashboard [2] and Github [3], there
> are
> >>>> many pending issues that could / should be part of the release. I'd
> >>> propose
> >>>> to make a collective effort to try to clean up our 1.33 backlog and
> >> merge
> >>>> the PRs which are in a good state. Shall we give ourselves around four
> >>>> weeks and aim at approximately December 11th for the first RC?
> >>>>
> >>>> According to this thread [4], the list of the last Release Managers is
> >> as
> >>>> follows:
> >>>> - 1.30.0 Liya Fan
> >>>> - 1.31.0 Andrei Sereda
> >>>> - 1.32.0 Julian Hyde (originally Jess Balint, switched with 1.33)
> >>>> ---
> >>>> - 1.33.0 Jess Balint (originally Julian Hyde, switched with 1.32)
> >>>> - 1.34.0 Duan Xiong
> >>>>
> >>>> @Jess, are you still available to be the next RM for 1.33.0?
> >>>>
> >>>> Do we have any other volunteers to be RM for the next releases?
> >>>>
> >>>> Best regards,
> >>>> Ruben
> >>>>
> >>>> [1] https://calcite.apache.org/news/2022/09/10/release-1.32.0/
> >>>> [2]
> >>>>
> >>>
> >>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> >>>> [3] https://github.com/apache/calcite/pulls
> >>>> [4] https://lists.apache.org/thread/ykbhhxmljw6wg50rxs6ypp35173hlkdv
> >>>>
> >>>
> >>
>
>

Re: [DISCUSS] Towards Calcite 1.33.0

[Ruben Q L]

Thanks Jess for being RM of 1.33, and thanks Benchao for volunteering as RM.
The updated list for future releases will be as follows:
- 1.33.0 Jess Balint
- 1.34.0 Duan Xiong
- 1.35.0 Benchao Li

There are currently 14 unresolved issues for 1.33:
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
Many of them have a PR available, let's try to do a collective effort and
review them ASAP (I remind everyone that you can assign a PR to yourself to
let others know that you are taking care of it).
As I mentioned on the initial email, let's try to finalize the remaining
issues during these two weeks, to see if we can produce a RC0 by
mid-December.

Regarding the recent Proj4j license problem (
https://issues.apache.org/jira/browse/CALCITE-5399), should we consider
this a blocking issue for 1.33?

Best,
Ruben


On Fri, Nov 11, 2022 at 6:53 PM Jess Balint  wrote:

> Hi Ruben,
>
> Thanks for the note. I'm available for RM duty. Here's the list of issues
> currently outstanding:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%201.33.0%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22In%20Review%22%2C%20%22In%20Implementation%22)%20ORDER%20BY%20priority%20DESC
>
> I'll pick up a couple of reviews and see if we can move out some of the
> issues which haven't been started yet.
>
> Jess
>
> On Fri, Nov 11, 2022 at 3:20 AM Ruben Q L  wrote:
>
> > Hello,
> >
> > It's been two months since our last release [1], if we want to keep our
> > rhythm, I'd suggest to produce the next one before the end of the year.
> >
> > As usual, according to our Jira dashboard [2] and Github [3], there are
> > many pending issues that could / should be part of the release. I'd
> propose
> > to make a collective effort to try to clean up our 1.33 backlog and merge
> > the PRs which are in a good state. Shall we give ourselves around four
> > weeks and aim at approximately December 11th for the first RC?
> >
> > According to this thread [4], the list of the last Release Managers is as
> > follows:
> > - 1.30.0 Liya Fan
> > - 1.31.0 Andrei Sereda
> > - 1.32.0 Julian Hyde (originally Jess Balint, switched with 1.33)
> > ---
> > - 1.33.0 Jess Balint (originally Julian Hyde, switched with 1.32)
> > - 1.34.0 Duan Xiong
> >
> > @Jess, are you still available to be the next RM for 1.33.0?
> >
> > Do we have any other volunteers to be RM for the next releases?
> >
> > Best regards,
> > Ruben
> >
> > [1] https://calcite.apache.org/news/2022/09/10/release-1.32.0/
> > [2]
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > [3] https://github.com/apache/calcite/pulls
> > [4] https://lists.apache.org/thread/ykbhhxmljw6wg50rxs6ypp35173hlkdv
> >
>

[jira] [Created] (CALCITE-5401) Rule fired by HepPlanner can return Volcano's RelSubset

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5401:
--

 Summary: Rule fired by HepPlanner can return Volcano's RelSubset
 Key: CALCITE-5401
 URL: https://issues.apache.org/jira/browse/CALCITE-5401
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ruben Q L


TLDR; {{CoreRules.AGGREGATE_REMOVE}} is fired by a {{HepPlanner}} but while 
removing the Aggregate, instead of returning the Aggregate's input, it returns 
a VolcanoPlanner's RelSubset with the input, which leads to unforeseeable 
consequences.

 

Details: This seems a strange issue that happens because several factors occur.

I first reproduced it on my application with the following query (on TPCH):
{code:sql}
SELECT c.c_custkey
FROM customer c
WHERE c_name IN ('271', '272', '273', '274', '275', '276', '342', '343', '344', 
'345','346', '347', '348', '349', '350', '351',  '352', '353', '354',  '355', 
'356', '357', '358', '359', '360')
AND EXISTS (SELECT 1 FROM orders o WHERE o.o_custkey = c.c_custkey)
{code}
But the issue can be reproduced also in Calcite by adding this test into 
{{{}HepPlannerTest{}}}:
{code:java}
  @Test void testAggregateRemove() {
final RelBuilder builder = RelBuilderTest.createBuilder(c -> 
c.withAggregateUnique(true));
final RelNode root =
builder
.values(new String[]{"i"}, 1, 2, 3) // important to have values 
sorted
.distinct()
.build();
final HepProgram program = new HepProgramBuilder()
.addRuleInstance(CoreRules.AGGREGATE_REMOVE)
.build();
final HepPlanner planner = new HepPlanner(program);
planner.setRoot(root);
final RelNode result = planner.findBestExp();
assertThat(result, is(instanceOf(LogicalValues.class))); // fails because 
result is a RelSubset
  }
{code}
The important elements are: firstly our {{RelOptCluster}} has a 
{{VolcanoPlanner}} as planner (so any {{relNode.getCluster().getPlanner()}} 
call that we execute will return a {{VolcanoPlanner}} instance). Nevertheless 
we also apply some rules via a {{{}HepPlanner{}}}. I think this is a quite 
common strategy in Calcite clients to obtain a better performance: first apply 
a subset of rules that are always beneficial via a {{{}HepPlanner{}}}, and then 
apply the "main" set of rules via the cost-based {{{}VolcanoPlanner{}}}.

Secondly, we have {{{}AggregateRemoveRule{}}}, which we use in the 
{{HepPlanner}} phase.
This rule contains the following code:
{code:java}
  @Override public void onMatch(RelOptRuleCall call) {
final Aggregate aggregate = call.rel(0);
final RelNode input = aggregate.getInput();
...
final RelNode newInput = convert(input, 
aggregate.getTraitSet().simplify()); // <-- *** [1]
relBuilder.push(newInput);
...
call.getPlanner().prune(aggregate);  // <-- *** [2]
call.transformTo(relBuilder.build());
  }
{code}
Notice the line [2] which uses {{call.getPlanner()}} to call the {{prune}} 
method. By using {{call.getPlanner()}} we get the correct planner of the rule 
that is being fired, in this case a {{{}HepPlanner{}}}, so we end up calling 
{{{}HepPlanner#prune{}}}, which is fine.
However, the line [1] calls the {{RelOptRule#convert}} static method:
{code:java}
  public static RelNode convert(RelNode rel, RelTraitSet toTraits) {
RelOptPlanner planner = rel.getCluster().getPlanner(); // <-- *** [3]
RelTraitSet outTraits = rel.getTraitSet();
for (int i = 0; i < toTraits.size(); i++) {
  RelTrait toTrait = toTraits.getTrait(i);
  if (toTrait != null)
outTraits = outTraits.replace(i, toTrait);
}
if (rel.getTraitSet().matches(outTraits))
  return rel;
return planner.changeTraits(rel, outTraits); // <-- *** [4]
  }
{code}
Notice how in this case, the planner is obtained from the relNode's cluster 
[3], in our case that would be the {{{}VolcanoPlanner{}}}, which is potentially 
problematic. Further down, if the relNode matches the {{{}outTraits{}}}, no 
action is done and the same relNode is returned, no problem here. But, if it 
does not match them, then {{RelOptPlanner#changeTraits}} will be called, i.e. 
{{VolcanoPlanner#changeTraits}} [4], and this is where the problem will 
originate: in our scenario {{VolcanoPlanner#changeTraits}} will return a 
Volcano's {{{}RelSubset{}}}, which is completely unhandable by the 
{{HepPlanner}} that triggered the rule, and that leads to the incorrect plan 
returned by the {{{}HepPlanner{}}}.

In this case, what happens with our original query ({{{}LogicalValues{}}} with 
sorted values), we get to {{RelOptRule#convert}} with the RelNode being a 
{{LogicalValues}} with {{Convention.NONE}} + {{{}Collation[0]{}}}, and the 
{{toTraits}} are the ones from the {{LogicalAggregate}} that we are removing: 
{{Convention.NONE}} + {{Collation[]}} . Since the traits from the 
{{LogicalValues}} do not match the Log

[DISCUSS] Towards Calcite 1.33.0

[Ruben Q L]

Hello,

It's been two months since our last release [1], if we want to keep our
rhythm, I'd suggest to produce the next one before the end of the year.

As usual, according to our Jira dashboard [2] and Github [3], there are
many pending issues that could / should be part of the release. I'd propose
to make a collective effort to try to clean up our 1.33 backlog and merge
the PRs which are in a good state. Shall we give ourselves around four
weeks and aim at approximately December 11th for the first RC?

According to this thread [4], the list of the last Release Managers is as
follows:
- 1.30.0 Liya Fan
- 1.31.0 Andrei Sereda
- 1.32.0 Julian Hyde (originally Jess Balint, switched with 1.33)
---
- 1.33.0 Jess Balint (originally Julian Hyde, switched with 1.32)
- 1.34.0 Duan Xiong

@Jess, are you still available to be the next RM for 1.33.0?

Do we have any other volunteers to be RM for the next releases?

Best regards,
Ruben

[1] https://calcite.apache.org/news/2022/09/10/release-1.32.0/
[2]
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
[3] https://github.com/apache/calcite/pulls
[4] https://lists.apache.org/thread/ykbhhxmljw6wg50rxs6ypp35173hlkdv

[DISCUSS] State of the project 2022

[Ruben Q L]

Hello community,

A bit more than seven years ago (22 October 2015) Calcite graduated as a
top-level Apache project [1]. At that point, the community decided to have
an annual “state of the project” discussion, and we have arrived to that
time of the year.

We have had three Calcite releases so far in 2022 [2] (many thanks to each
Release Manager), with probably one more coming before the end of the year.
We have seen great evolutions such as new spatial functions, polymorphic
table functions, Firebolt dialect, parsing parenthesized joins (kudos to
Julian Hyde for dealing with this very old issue), improvements on SQL
hints, and many more.

Regarding the sub-project Avatica, we have seen two releases this year, and
two more releases for Avatica Go [3] (special thanks to Francis Chuang for
being the Release Manager for all of them). These were all basically
maintenance releases. It is clear that Avatica has fewer users,
contributors and evolutions than Calcite. More of them will be very
welcome, although I am not sure about how we could promote it.

We have also faced several vulnerabilities for both Calcite and Avatica
during the last year, including the famous log4j issues, and we were able
to deal with them in a swiftly and collaboratively manner.

In terms of community, I think this has been a great year. Calcite
continues to increase steadily its contributors. We see more and more
people participating in email discussions, Jira tickets and Github PRs. Our
list of committers has grown with Jing Zhang, Benchao Li, Bertil Chapuis
and Dmitry Sysolyatin; and so has our PMC with Chunwei Lei, Vladimir Ozerov
and Andrei Sereda. Calcite grows and evolves because of (and thanks to) its
community, so I would like to thank everyone for being part of this family
and working together in a respectful and motivating environment.

We started the year with an online meetup, which was a great opportunity
for the community to virtually meet and share some interesting
presentations. Perhaps we could begin the discussions to try to organize
another one in the near future (beginning of 2023?), in order to try to
hold at least one every year.

In a less positive aspect, it is probably worth mentioning that we continue
to struggle on PR reviewing. Our pull-requests list continues to increase
and frequently it is hard to have people reviewing / merging them, and this
can be frustrating for non-committers. This is an old problem of our
project. We have tried several things to improve it (like metrics and
credit on our quarterly reports; or sporadic "reviewing campaigns",
especially before a release). Any ideas on improving this situation will be
highly appreciated.

To conclude, I will repeat the questions from previous years:
1) What else are we doing well in the project?
2) What areas do we need to do better?

Please take some time to share your thoughts!
Note that this discussion is for everyone, not only for committers / PMC
members; even if you have never sent an email to the dev list before, now
it is a good time to do so :)

Finally, it has been a privilege to serve as Calcite's PMC Chair this year.
I have learnt a lot and I am very grateful for the opportunity that I was
given. Following our yearly rotation tradition, I will step down as Chair
by the end of the year, and a new one will have to be chosen. As we
discussed some time ago [4], if you have any suggestion and you would like
to put someone forward as a potential next Chair, please send an email to
priv...@calcite.apache.org (the PMC will study all proposals, discuss and
vote).

Best regards,
Ruben

[1] http://calcite.apache.org/news/2015/10/22/calcite-graduates/
[2] https://calcite.apache.org/news/
[3] https://calcite.apache.org/avatica/news/
[4] https://lists.apache.org/thread/gplfqs4snr1b6h62cngyvb65sz41z3fk

[ANNOUNCE] New committer: Dmitry Sysolyatin

[Ruben Q L]

Apache Calcite's Project Management Committee (PMC) has invited Dmitry
Sysolyatin to become a committer, and we are pleased to announce he has
accepted the invitation.

During the last months Dmitry has pushed a lot of high quality patches,
fixing and improving code around several places in Calcite core. Apart from
code contributions, he has been regularly involved in discussions regarding
multiple topics in the mailing list, always with a hardworking and
courteous spirit.

Dmitry, welcome, thank you for your contributions, and we look forward to
your further interactions with the community! If you wish, please feel free
to tell us a bit more about yourself and what you are currently working on.
As your first commit, you can add yourself to the contributors list [1] and
the community page will re-generate [2].

Ruben (on behalf of the Apache Calcite PMC)

[1] https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
[2] https://calcite.apache.org/community/#project-members

Re: [DISCUSS] JIRA account signups no longer available after 6th November 2022

[Ruben Q L]

Thanks for the feedback.
We will proceed with the proposed plan. Details will follow in [1] (thanks
Francis for driving this).

Best regards,
Ruben

[1] https://issues.apache.org/jira/browse/CALCITE-5353


On Fri, Oct 28, 2022 at 6:10 AM Benchao Li  wrote:

> Sounds good to me.
>
> Requesting from the private mailing list would ease the concerns that
> users' privacy (email address) would be public.
>
> Alessandro Solimando  于2022年10月26日周三
> 17:03写道：
>
> > It's understandable from the ASF side, and the proposal you drafted LGTM.
> >
> > I'd probably also add some instructions under
> > https://calcite.apache.org/develop/
> >
> > Best regards,
> > Alessandro
> >
> > On Wed, 26 Oct 2022 at 10:58, Francis Chuang 
> > wrote:
> >
> > > Hello Community Members,
> > >
> > > ASF Infra has announced that contributors will no longer be able to
> sign
> > > up for a JIRA account from 6th November 2022 onward. This is due to the
> > > amount of spam issues and users being created in JIRA, causing a lot of
> > > overhead for the infra team.
> > >
> > >  From the 6th of November, contributors who wish to have a JIRA account
> > > (to create, assign, watch, etc issues) will need to request an account
> > > through an ASF PMC. Note that if you already have a JIRA account, or if
> > > you requested an account from the PMC of a different ASF project after
> > > the 6th of November, you can use that account to open, watch, comment,
> > > etc on issues from other projects, in other words, you only need to
> > > request an account once from a PMC to access JIRA for all ASF projects.
> > >
> > > The Calcite PMC has had some discussion over the last few days to
> design
> > > a process for requesting a JIRA account through the Calcite project. We
> > > are currently proposing the following process for Calcite:
> > >
> > > 1. Contributors who wish to have a JIRA account should email a
> dedicated
> > > private mailing list (for example, jira-account@calcite.a.o) with the
> > > following:
> > > -- Proposed JIRA username [a-z0-9] only
> > > -- Full display name
> > > -- Email address
> > > -- Reasons why you are requesting an account
> > >
> > > This list is private and will not be accessible via any public archives
> > > on the internet.
> > >
> > > Once a PMC member looks at the request, they will request the account
> to
> > > be created via ASF's system. In the event that the chosen username is
> > > not available, they will get back to the requestor and ask for an
> > > alternative username.
> > >
> > > 2. We will document the process in simple and clear steps on our
> > > website. This page will be linked prominently in the README.md files of
> > > our 3 code repositories: calcite, calcite-avatica and
> calcite-avatica-go.
> > >
> > > Please let us know what you think!
> > >
> > > Francis
> > >
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: Migrating away from Travis-CI

[Ruben Q L]

+1


On Tue, Oct 25, 2022 at 2:31 AM Benchao Li  wrote:

> +1, thanks Francis for driving this.
>
> Julian Hyde  于2022年10月25日周二 07:57写道：
>
> > +1
> >
> > > On Oct 24, 2022, at 3:49 PM, Francis Chuang 
> > wrote:
> > >
> > > Travis will no longer be available for ASF projects at the end of 2022.
> > >
> > > For Calcite, only the calcite and calcite-avatica repos use travis and
> > those projects are also using Github Actions already. Perhaps in our case
> > we will just need to remove the Travis configs from those repos.
> > >
> > > Francis
> > >
> > >  Forwarded Message 
> > > Subject:  Migrating away from Travis-CI
> > > Date: Mon, 24 Oct 2022 15:46:16 -0500
> > > From: Drew Foulks 
> > > Reply-To: us...@infra.apache.org
> > > To:   annou...@infra.apache.org
> > >
> > >
> > >
> > > Greetings PMC Members!
> > >
> > > Infrastructure is moving away from Travis-CI at the end of 2022.
> > >
> > > If your project is not currently using Travis-CI, this email is not
> > actionable.
> > >
> > > Projects using Travis-CI:
> > >
> > > Please take a look at
> > https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations <
> > https://cwiki.apache.org/confluence/display/INFRA/Travis+Migrations>
> > >
> > > and join the builds meetings and mailing list for project updates.
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Drew Foulks
> > > ASF Infra
> > >
> > >
> >
> >
>
> --
>
> Best,
> Benchao Li
>

Re: Using Calcite in a multi-tenant application

[Ruben Q L]

> If you wanted to, you could write a merge sort algorithm that uses sort
runs on disk. But that’s not in Calcite currently.
Just to complete this info, there is a ticket logged for that feature [1],
but AFAIK there is nobody working on it at the moment.

[1] https://issues.apache.org/jira/browse/CALCITE-4193


On Thu, Oct 20, 2022 at 9:01 PM Julian Hyde  wrote:

> First, remember that the Calcite "interface Schema" is quite low level. It
> is basically a namespace that contains a set of tables and a set of
> sub-schemas. You can, and probably should, make it immutable.
>
> A Schema instance doesn’t know its own name or path. That allows you to
> “mount” it (as you would mount a filesystem) under several paths.
>
> This gives you a clue for how you could serve multiple tenants, each
> possibly seeing the same schema and tables (or possibly seeing different
> subsets). Just set a different root schema for each tenant’s connection. I
> believe that’s analogous to how a container like Docker works.
>
> Calcite does not, by default, write anything to disk. The built-in
> adapters read from foreign wrappers (e.g. connecting to another JDBC
> database or Cassandra or MongoDB or CSV files) but then process data in
> memory.
>
> If you wanted to, you could write a merge sort algorithm that uses sort
> runs on disk. But that’s not in Calcite currently.
>
> Julian
>
>
> > On Oct 19, 2022, at 8:19 PM, JiaTao Tao  wrote:
> >
> > In fact, you can change calcite to use like HMS to manager schema
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > Mike Albritton  于2022年10月20日周四 07:16写道：
> >
> >> Hi all,
> >>
> >>
> >> I am considering using Calcite in a multi-tenant application that can
> run
> >> queries from different customers in the same process and have a few
> >> questions for those familiar with the code about this approach.
> >>
> >>
> >> Specifically, I would like to know if metadata is completely separated
> >> per-schema that is loaded or if it is ever shared between schemas?
> Also, I
> >> read in the docs somewhere that Calcite manages memory for cross schema
> >> joins which seemed to suggest it may persist data to disk temporarily.
> Is
> >> this the case or is there somewhere I can learn more about how data
> >> persistence works?  Are there any other concerns that I should have when
> >> running Calcite in a multi-tenant application?
> >>
> >>
> >> Finally, if you or someone you know may be interested in a short term
> >> contract to assist with a Calcite POC, let me know.
> >>
> >>
> >> Regards,
> >>
> >> Mike
> >>
>
>

Re: Avatica Pull Requests

[Ruben Q L]

Hello,

I'll take care of 2 or 3 PRs too.

Best regards,
Ruben


On Mon, Oct 17, 2022 at 8:40 AM Stamatis Zampetakis 
wrote:

> Hello,
>
> I assigned 3 PRs to myself. I will try to finalize (merge or close) them
> till the next release.
>
> Best,
> Stamatis
>
> On Sun, Oct 16, 2022 at 10:15 PM Francis Chuang 
> wrote:
>
> > Thanks Julian! Would other community members be able to help us here?
> >
> > Francis
> >
> > On 15/10/2022 5:54 am, Julian Hyde wrote:
> > > I’ll review a couple of PRs if two or three people volunteer to review
> a
> > couple.
> > >
> > > Julian
> > >
> > >> On Oct 13, 2022, at 2:45 PM, Francis Chuang  >
> > wrote:
> > >>
> > >> Hey everyone,
> > >>
> > >> There are currently 13 open pull requests for Avatica[1]. Since we're
> > not in the middle of a Calcite release, would it be possible for
> community
> > members to review and perhaps merge those PRs?
> > >>
> > >> There are a few that look ready to be merged and there are a few that
> > are a few years old that could perhaps be closed out. It would be nice if
> > we could action on these PRs as there are a few that a few years old that
> > are still open.
> > >>
> > >> Francis
> > >>
> > >> [1] https://github.com/apache/calcite-avatica/pulls
> > >
> >
>

Draft: board report for 2022 Q3

[Ruben Q L]

Hello,

Below these lines you can find a draft of this quarter's board report. I
plan to submit it
before Friday next week.
Please let me know if you have any additions or corrections.

Best regards,
Ruben

-


## Description:
Apache Calcite is a highly customizable framework for parsing and planning
queries on data in a wide variety of formats. It allows database-like
access,
and in particular a SQL interface and advanced query optimization, for data
not residing in a traditional database.

Avatica is a sub-project within Calcite and provides a framework for
building
local and remote JDBC and ODBC database drivers. Avatica has an independent
release schedule and its own repository.

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Calcite was founded 2015-10-22 (7 years ago)
There are currently 58 committers and 26 PMC members in this project.
The Committer-to-PMC ratio is roughly 2:1.

Community changes, past quarter:
- Andrei Sereda was added to the PMC on 2022-08-08
- Jing Zhang was added as committer on 2022-07-04
- Benchao Li was added as committer on 2022-07-04

## Project Activity:
Apache Calcite 1.31.0 was released on 2022-08-02. It contains contributions
from 28 contributors, and resolves 81 issues. Among others, it is worth
highlighting the following improvements: Allow table functions to be
polymorphic; Support SQL hint for Filter, SetOp, Sort, Window, Values;
Support
parsing parenthesized joins; Derive IS NOT NULL filter for the inputs of
inner
join; and Firebolt dialect implementation.

Apache Calcite 1.32.0 was released on 2022-09-10, with 19 issues fixed by 17
contributors. This release fixes CVE-2022-39135, an XML External Entity
(XEE)
vulnerability that allows a SQL query to read the contents of files via the
SQL functions EXISTS_NODE, EXTRACT_XML, XML_TRANSFORM or EXTRACT_VALUE.
Apart
from that, it is worth highlighting that this release also replaces the ESRI
spatial engine with JTS and proj4j, adds 65 spatial SQL functions including
ST_Centroid, ST_Covers and ST_GeomFromGeoJSON, adds the CHAR SQL function,
and
improves the return type of the ARRAY and MULTISET functions.

Apache Calcite Avatica 1.22.0 was released on 2022-07-28. This is a
maintenance release to resolve CVE-2022-36364: Apache Calcite Avatica JDBC
driver httpclient_impl connection property can be used as an RCE vector.

## Community Health:
The community maintains a healthy status. There has been a generalized
decrease in the Jira & Github activity in the last quarter (-14% issues
opened, -27% issues closed, -25% commits, -21% code contributions, -11% PRs
opened, -40% PRs closed). This is expected since many members of the
community
were probably on vacation during this summertime in the Northern Hemisphere.
Nevertheless, the development mailing list has kept a big activity, even
increasing its traffic 137% in this quarter.

Is is worth mentioning that several vulnerabilities were reported in both
Calcite and Avatica, and the PMC showed a great reactivity to deal with them
swiftly.

The number of non-committer (contributor) commits per month:
+-+-+-+
|year |month| contributor_commits |
+-+-+-+
| 2022| 7   | 0   |
| 2022| 8   | 5   |
| 2022| 9   | 15  |
+-+-+-+

The number of active reviewers per month:
+-+-+-+
|year |month|  active_reviewers   |
+-+-+-+
| 2022| 7   | 0   |
| 2022| 8   | 4   |
| 2022| 9   | 3   |
+-+-+-+

Top reviewers in the last 3 months:
+-+-+
|   committer | reviews |
+-+-+
| Julian Hyde   | 9   |
| Benchao Li | 7   |
| Stamatis Zampetakis  | 2   |
| rubenada | 1   |
| chunwei.lcw | 1   |
+-+-+

Re: Requesting Information Regarding Enumerable

[Ruben Q L]

Hello Pranav,

Inside YourRelMdNonCumulativeCost class, you'd need to create one method
per operator (for those whose RelOptCost you want to "override").
You can take a look at the code of the existing RelMd* classes in Calcite
to have an idea [1].


Once you have your metadata provider, i.e. something like:
 RelMetadataProvider YOUR_INSTANCE =
ChainedRelMetadataProvider.of(ImmutableList.of(YourRelMdNonCumulativeCost.INSTANCE,
DefaultRelMetadataProvider.INSTANCE));

Then you just need to use it in the RelOptCluster:
  RelOptCluster cluster = ...
  cluster.setMetadataProvider(YOUR_INSTANCE);

Best,
Ruben

[1]
https://github.com/apache/calcite/tree/main/core/src/main/java/org/apache/calcite/rel/metadata


On Wed, Sep 21, 2022 at 4:26 PM Pranav Deshpande <
deshpande.v.pra...@gmail.com> wrote:

> Hi Ruben,
> Thank you very much for your reply!
>
> Will I have to create a different Metadata Handler per operator (eg.
> Project, Correlate etc.) or can I make a common one?
>
> Also, I am having some trouble finding where to put this statement:
>
>
> ChainedRelMetadataProvider.of(ImmutableList.of(YourRelMdNonCumulativeCost.INSTANCE,
> DefaultRelMetadataProvider.INSTANCE));
>
> Could you please help me out by pointing me to a code example?
>
> I viewed this tutorial: https://www.youtube.com/watch?v=p1O3E33FIs8, but
> nowhere in the query processing code do I see where to put this (it is
> mentioned in the architecture though).
>
> Thanks & Regards,
> Pranav
>
> On Sat, Sep 17, 2022 at 6:07 AM Ruben Q L  wrote:
>
> > Hello Pranav,
> >
> > If you just need to "override" the cost function of certain operators,
> > there is no need to create new operators (and rules), you can do that by
> > extending the Metadata.
> > Let's say you want to override the cost of the Correlate, you should be
> > able to do that by creating your own class:
> >
> > public final class YourRelMdNonCumulativeCost implements
> > MetadataHandler
> > {
> >   static final RelMetadataProvider INSTANCE =
> > ReflectiveRelMetadataProvider.reflectiveSource(new
> > YourRelMdNonCumulativeCost(),
> > BuiltInMetadata.NonCumulativeCost.Handler.class);
> >
> >   private YourRelMdNonCumulativeCost() { }
> >
> >   @Override
> >   public MetadataDef getDef() { return
> > NonCumulativeCost.DEF; }
> >
> >   public RelOptCost getNonCumulativeCost(Correlate correlate,
> > RelMetadataQuery mq)
> >   {
> > // your code here...
> >   }
> > }
> >
> > And then as MetadataProvider, instead of using just Calcite's default
> one,
> > you would use a chain of yours + Calcite's default:
> >
> >
> >
> ChainedRelMetadataProvider.of(ImmutableList.of(YourRelMdNonCumulativeCost.INSTANCE,
> > DefaultRelMetadataProvider.INSTANCE));
> >
> >
> > Having said that, if you really need to create your own operators, the
> > easiest way, as you mention, is to create your operator (which would
> extend
> > the Calcite operator) and your rule (which could extend the Calcite rule,
> > but not necessarily). As you say, it is not possible to use the
> > Immutable*Rule Config, but you can use the rule's default Config or
> create
> > your own default Config. You can even do all that without Immutables in
> > your project, this thread might be of interest [1].
> >
> > Best,
> > Ruben
> >
> > [1] https://lists.apache.org/thread/2h0mcdqs4q1psgjy9sxlk8wxzo3w1zq9
> >
> >
> >
> > On Fri, Sep 16, 2022 at 10:10 PM Pranav Deshpande <
> > deshpande.v.pra...@gmail.com> wrote:
> >
> > > Hi Julian/Apache Calcite Dev Team,
> > > Thank you very much for your reply
> > > I see that's a good point and it makes a lot of sense. I think I should
> > > exactly tell you what my problem is: I am only trying to override the
> > cost
> > > function so that I can group operators in a certain way in the
> Relational
> > > Tree.
> > >
> > > That is the only functionality that is required (for my example) right
> > now.
> > > I was extending the rules and the nodes to override this 1 function
> only.
> > > Is there another way to do this? If not, what is the recommended
> approach
> > > (is it to duplicate the enumerable rels and rules)?
> > >
> > > Please let me know.
> > >
> > > Thanks & Regards,
> > > Pranav
> > >
> > > On Fri, Sep 16, 2022 at 3:13 PM Julian Hyde 
> > > wrote:
> > >
> > > > We don’t necessary want you to extend existing classes. If we change
&

Re: Requesting Information Regarding Enumerable

[Ruben Q L]

Hello Pranav,

If you just need to "override" the cost function of certain operators,
there is no need to create new operators (and rules), you can do that by
extending the Metadata.
Let's say you want to override the cost of the Correlate, you should be
able to do that by creating your own class:

public final class YourRelMdNonCumulativeCost implements
MetadataHandler
{
  static final RelMetadataProvider INSTANCE =
ReflectiveRelMetadataProvider.reflectiveSource(new
YourRelMdNonCumulativeCost(),
BuiltInMetadata.NonCumulativeCost.Handler.class);

  private YourRelMdNonCumulativeCost() { }

  @Override
  public MetadataDef getDef() { return
NonCumulativeCost.DEF; }

  public RelOptCost getNonCumulativeCost(Correlate correlate,
RelMetadataQuery mq)
  {
// your code here...
  }
}

And then as MetadataProvider, instead of using just Calcite's default one,
you would use a chain of yours + Calcite's default:

ChainedRelMetadataProvider.of(ImmutableList.of(YourRelMdNonCumulativeCost.INSTANCE,
DefaultRelMetadataProvider.INSTANCE));


Having said that, if you really need to create your own operators, the
easiest way, as you mention, is to create your operator (which would extend
the Calcite operator) and your rule (which could extend the Calcite rule,
but not necessarily). As you say, it is not possible to use the
Immutable*Rule Config, but you can use the rule's default Config or create
your own default Config. You can even do all that without Immutables in
your project, this thread might be of interest [1].

Best,
Ruben

[1] https://lists.apache.org/thread/2h0mcdqs4q1psgjy9sxlk8wxzo3w1zq9



On Fri, Sep 16, 2022 at 10:10 PM Pranav Deshpande <
deshpande.v.pra...@gmail.com> wrote:

> Hi Julian/Apache Calcite Dev Team,
> Thank you very much for your reply
> I see that's a good point and it makes a lot of sense. I think I should
> exactly tell you what my problem is: I am only trying to override the cost
> function so that I can group operators in a certain way in the Relational
> Tree.
>
> That is the only functionality that is required (for my example) right now.
> I was extending the rules and the nodes to override this 1 function only.
> Is there another way to do this? If not, what is the recommended approach
> (is it to duplicate the enumerable rels and rules)?
>
> Please let me know.
>
> Thanks & Regards,
> Pranav
>
> On Fri, Sep 16, 2022 at 3:13 PM Julian Hyde 
> wrote:
>
> > We don’t necessary want you to extend existing classes. If we change the
> > base class in future, your code breaks, and you complain that we have
> > broken semantic versioning. Making things private is, in that sense, a
> > feature.
> >
> > If what you are doing is a feature that would benefit other Calcite
> users,
> > you should propose that feature.
> >
> > Julian
> >
> >
> > > On Sep 16, 2022, at 11:41 AM, Pranav Deshpande <
> > deshpande.v.pra...@gmail.com> wrote:
> > >
> > > Dear Apache Calcite Team,
> > > I am trying to modify some parts of the RelNode Tree in Calcite for my
> > own
> > > custom logic. For this, I am extending existing nodes (eg.
> > > EnumerableAggregate etc.) and then also extending the respective rule
> > (eg.
> > > EnumerableAggregateRule) and overriding the respective functions in
> both
> > so
> > > that my custom node is used in the tree (logical -> custom node which
> is
> > > basically an extended enumerable) instead of logical -> enumerable.
> > >
> > > Some of the rules here (eg. EnumerableLimit) have references to
> Immutable
> > > classes which are package private and I hence I am unable to extend
> these
> > > (EnumerableLimitRule is itself one such example:
> > >
> >
> https://github.com/apache/calcite/blob/b9c2099ea92a575084b55a206efc5dd341c0df62/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableLimitRule.java#L72
> > > )
> > >
> > > Any advice on how I can solve this problem?
> > >
> > > Thanks & Regards,
> > > Pranav
> >
> >
>

CVE-2022-39135: Apache Calcite: potential XEE attacks

[Ruben Q L]

Description:

In Apache Calcite prior to version 1.32.0 the SQL operators EXISTS_NODE, 
EXTRACT_XML, XML_TRANSFORM and EXTRACT_VALUE do not restrict XML External 
Entity references in their configuration, which makes them vulnerable to a 
potential XML External Entity (XXE) attack. Therefore any client exposing these 
operators, typically by using Oracle dialect (the first three) or MySQL dialect 
(the last one), is affected by this vulnerability (the extent of it will depend 
on the user under which the application is running).

>From Apache Calcite 1.32.0 onwards, Document Type Declarations and XML 
>External Entity resolution are disabled on the impacted operators.

Credit:

Apache Calcite would like to thank David Handermann for reporting this issue

Re: [VOTE] Release Apache Calcite 1.32.0 (release candidate 0)

[Ruben Q L]

Thanks Julian for producing this release.

- Checksum and signature: ok
- Build + tests: ok
- Calcite-based application test suite: ok

+1 (binding)

Best,
Ruben


On Fri, Sep 9, 2022 at 10:03 PM Stamatis Zampetakis 
wrote:

> Ubuntu 20.04.5 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 7.4.2
>
>  * Checked signatures and checksums OK
>  * Went over release note OK (left minor comments under [1] and [2])
>  * Built from git tag and run tests (./gradlew clean build) OK
>  * Built from source artifacts and run unit tests + slow tests OK
>  * Checked diff between git repo and release sources OK
>
> +1 (binding)
>
> Best,
> Stamatis
>
> [1]
>
> https://github.com/julianhyde/calcite/commit/141e6842c5c85e90aeeb3f49776e731208170b1a
> [2]
>
> https://github.com/julianhyde/calcite/commit/11047a4e45e6cc371779364c62883a57f7693f4c
>
> On Fri, Sep 9, 2022 at 11:13 AM Francis Chuang 
> wrote:
>
> > Thanks for being RM for this release, Julian!
> >
> > My vote is: +1 (binding)
> >
> > - Verified GPG signature - OK
> > - Verified SHA512 - OK
> > - Diffed source release and git repository - OK
> > - Checked release notes on tag
> > (
> >
> https://github.com/apache/calcite/blob/calcite-1.32.0-rc0/site/_docs/history.md
> )
> >
> > - OK
> > - Ran tests (gradle check) - OK
> > - Spot checked Nexus artifacts - OK
> >
> > Environment:
> > Eclipse-temurin:17-jammy docker container in WSL2 (Ubuntu 22.04) on
> > Windows 10 21h2
> >
> >  > docker version
> > Client: Docker Engine - Community
> >   Cloud integration: v1.0.29
> >   Version:   20.10.17
> >   API version:   1.41
> >   Go version:go1.17.11
> >   Git commit:100c701
> >   Built: Mon Jun  6 23:03:17 2022
> >   OS/Arch:   linux/amd64
> >   Context:   default
> >   Experimental:  true
> >
> > Server: Docker Desktop
> >   Engine:
> >Version:  20.10.17
> >API version:  1.41 (minimum version 1.12)
> >Go version:   go1.17.11
> >Git commit:   a89b842
> >Built:Mon Jun  6 23:01:23 2022
> >OS/Arch:  linux/amd64
> >Experimental: false
> >   containerd:
> >Version:  1.6.8
> >GitCommit:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
> >   runc:
> >Version:  1.1.4
> >GitCommit:v1.1.4-0-g5fd4c4d
> >   docker-init:
> >Version:  0.19.0
> >GitCommit:de40ad0
> >
> >  > gradle -v
> >
> > 
> > Gradle 7.3.3
> > 
> >
> > Build time:   2021-12-22 12:37:54 UTC
> > Revision: 6f556c80f945dc54b50e0be633da6c62dbe8dc71
> >
> > Kotlin:   1.5.31
> > Groovy:   3.0.9
> > Ant:  Apache Ant(TM) version 1.10.11 compiled on July 10 2021
> > JVM:  17.0.4.1 (Eclipse Adoptium 17.0.4.1+1)
> > OS:   Linux 5.10.102.1-microsoft-standard-WSL2 amd64
> >
> >  > java -version
> > openjdk version "17.0.4.1" 2022-08-12
> > OpenJDK Runtime Environment Temurin-17.0.4.1+1 (build 17.0.4.1+1)
> > OpenJDK 64-Bit Server VM Temurin-17.0.4.1+1 (build 17.0.4.1+1, mixed
> > mode, sharing)
> >
> > Francis
> >
> > On 9/09/2022 5:41 pm, Julian Hyde wrote:
> > > Hi all,
> > >
> > > I have created a build for Apache Calcite 1.32.0, release
> > > candidate 0.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > You can read the release notes here:
> > >
> >
> https://github.com/apache/calcite/blob/calcite-1.32.0-rc0/site/_docs/history.md
> > >
> > > The commit to be voted upon:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=413eded693a9087402cc1a6eefeca7a29445d337
> > >
> > > Its hash is 413eded693a9087402cc1a6eefeca7a29445d337
> > >
> > > Tag:
> > >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.32.0-rc0
> > >
> > > The artifacts to be voted on are located here:
> > >
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.32.0-rc0
> > > (revision 56753)
> > >
> > > The hashes of the artifacts are as follows:
> > >
> >
> 2e9df199891cbb111e109e2d1f72d867430875b9e265deb29eb5a1fc36daff85725cc7d3160cdc397a83ca04986653de828825408a33f9fc0c48d206208c240b
> > > *apache-calcite-1.32.0-src.tar.gz
> > >
> > > A staged Maven repository is available for review at:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1174/org/apache/calcite/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/jhyde.asc
> > > https://www.apache.org/dist/calcite/KEYS
> > >
> > > Please vote on releasing this package as Apache Calcite 1.32.0.
> > >
> > > The vote is open for the next 24 hours and passes if a majority of at
> > > least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Calcite 1.32.0
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package 

Re: [DISCUSS] Towards Calcite 1.32.0

[Ruben Q L]

Thanks for the comment, Gavin.
It seems there are still some open points in that PR, so it looks like a
bit rushed for this one, let's aim for the next release (1.33).
We try to keep a rhythm of one release every two months approximately, so
in around two months we shall have a new opportunity to release this
feature.

Ruben


On Thu, Sep 8, 2022 at 8:09 PM Gavin Ray  wrote:

> This may not be the appropriate place to ask, but is there any chance of
> merging the below for this release, or maybe next release?
> It's been around a few years and I think it is one of the few blockers in
> place for taking advantage of MULTISET functionality, much-appreciated if
> you want to return nested hierarchical results.
>
> - [CALCITE-4188] Support EnumerableBatchNestedLoopJoin inJdbcToEnumerab… by
> angelzouxin · Pull Request #2116 · apache/calcite (github.com)
> <https://github.com/apache/calcite/pull/2116>
>
> On Thu, Sep 8, 2022 at 2:53 PM Julian Hyde  wrote:
>
> > I would like to make a release candidate and start a vote this evening
> > (i.e. 11 hours from now). In three hours I shall close the main
> > branch.
> >
> > If you are a committer and there is a PR that you would like to merge,
> > please either do it in the next three hours or reply to this email
> > asking for permission.
> >
> > There is a PR with draft release notes [
> > https://github.com/apache/calcite/pull/2902 ]. Please review. I won't
> > merge the PR, but I will rebase it onto the release branch after
> > fixing any issues.
> >
> > Julian
> >
> > On Thu, Sep 8, 2022 at 10:07 AM Ruben Q L  wrote:
> > >
> > > Hello,
> > >
> > > It has been more than one month since our last release. It is a good
> time
> > > to keep momentum and try to move on with the next one.
> > >
> > > On a separate thread, Julian Hyde volunteered to be Release Manager for
> > > this one (thanks Julian!).
> > >
> > > As usual, there are several unresolved tickets [1], let's try to sort
> > them
> > > out as soon as possible: merge the PRs if ready, or move the tickets
> to a
> > > future version.
> > >
> > > Best regards,
> > > Ruben
> > >
> > > [1]
> > >
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> >
>

[DISCUSS] Towards Calcite 1.32.0

[Ruben Q L]

Hello,

It has been more than one month since our last release. It is a good time
to keep momentum and try to move on with the next one.

On a separate thread, Julian Hyde volunteered to be Release Manager for
this one (thanks Julian!).

As usual, there are several unresolved tickets [1], let's try to sort them
out as soon as possible: merge the PRs if ready, or move the tickets to a
future version.

Best regards,
Ruben

[1]
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950

[jira] [Created] (CALCITE-5277) Make EnumerableRelImplementor stashedParameters order deterministic to increase BINDABLE_CACHE hit rate

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5277:
--

 Summary: Make EnumerableRelImplementor stashedParameters order 
deterministic to increase BINDABLE_CACHE hit rate
 Key: CALCITE-5277
 URL: https://issues.apache.org/jira/browse/CALCITE-5277
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Ruben Q L
 Fix For: 1.32.0


Make EnumerableRelImplementor stashedParameters order deterministic to increase 
BINDABLE_CACHE hit rate



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (CALCITE-5263) Improve XmlFunctions by using an XML DocumentBuilder

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5263:
--

 Summary: Improve XmlFunctions by using an XML DocumentBuilder
 Key: CALCITE-5263
 URL: https://issues.apache.org/jira/browse/CALCITE-5263
 Project: Calcite
  Issue Type: Task
  Components: core
Reporter: Ruben Q L
Assignee: Ruben Q L
 Fix For: 1.32.0


Improve XmlFunctions by using an XML DocumentBuilder



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[ANNOUNCE] Andrei Sereda joins Calcite PMC

[Ruben Q L]

I am pleased to announce that Andrei has accepted an invitation to join
the Calcite PMC. Andrei has been a consistent and helpful figure in
the Calcite community for which we are very grateful. We look forward to
the continued contributions and support.

Please join me in congratulating Andrei!

Ruben (on behalf of the Calcite PMC)

Re: Problem with char type after upgrading from 1.30.0 to 1.31.0

[Ruben Q L]

Following Julian's comment, may I add: if your organization is using
Calcite in an unconventional way, it is really helpful for the community to
participate in the release vote process, so that we can spot any issue with
a RC as soon as possible, in order to avoid rolling out releases with
potential regressions.

Best,
Ruben


On Fri, Aug 12, 2022 at 5:40 PM Julian Hyde  wrote:

> I don’t know whether we ever officially supported the Java ‘char’ type.
> It’s worth checking whether there are any tests for it.
>
> More generally: if your organization is using Calcite in an unconventional
> way, consider writing some tests for that area of functionality and
> contributing them. You will be helping yourself and Calcite.
>
> Julian
>
>
> > On Aug 12, 2022, at 7:06 AM, Ruben Q L  wrote:
> >
> > Hello Dmitry,
> >
> > At first glance, it looks like a regression. Could you please create a
> Jira
> > ticket (ideally with a unit test that runs fine 1.30 but fails in 1.31)?
> >
> > Best,
> > Ruben
> >
> >
> > On Fri, Aug 12, 2022 at 2:48 PM Dmitry Sysolyatin <
> dm.sysolya...@gmail.com>
> > wrote:
> >
> >> Hi!
> >> I have a problem with char data type after upgrading from 1.30 to 1.31.
> I
> >> tried to execute a simple query "SELECT c.relkind FROM
> pg_catalog.pg_class
> >> c" (relkind is JavaType(char) NOT NULL) and got an exception:
> >>
> >> Unable to implement EnumerableCalc(expr#0..32=[{inputs}],
> relkind=[$t16]):
> >> rowcount = 100.0, cumulative cost = {200.0 rows, 3501.0 cpu, 0.0 io},
> id =
> >> 28
> >>  EnumerableTableScan(table=[[default, pg_catalog, pg_class]]): rowcount
> =
> >> 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 19
> >>
> >> Suppressed: java.lang.RuntimeException: while resolving method
> >> 'toChar[class java.lang.Object]' in class class
> >> org.apache.calcite.runtime.SqlFunctions
> >> at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:318)
> >> at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:448)
> >> at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:460)
> >>
> >> Queries with char literal work OK: "SELECT 'r';"
> >> Does someone have an idea what can be wrong ?
> >>
>
>

Re: Problem with char type after upgrading from 1.30.0 to 1.31.0

[Ruben Q L]

Hello Dmitry,

At first glance, it looks like a regression. Could you please create a Jira
ticket (ideally with a unit test that runs fine 1.30 but fails in 1.31)?

Best,
Ruben


On Fri, Aug 12, 2022 at 2:48 PM Dmitry Sysolyatin 
wrote:

> Hi!
> I have a problem with char data type after upgrading from 1.30 to 1.31. I
> tried to execute a simple query "SELECT c.relkind FROM pg_catalog.pg_class
> c" (relkind is JavaType(char) NOT NULL) and got an exception:
>
> Unable to implement EnumerableCalc(expr#0..32=[{inputs}], relkind=[$t16]):
> rowcount = 100.0, cumulative cost = {200.0 rows, 3501.0 cpu, 0.0 io}, id =
> 28
>   EnumerableTableScan(table=[[default, pg_catalog, pg_class]]): rowcount =
> 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 19
>
> Suppressed: java.lang.RuntimeException: while resolving method
> 'toChar[class java.lang.Object]' in class class
> org.apache.calcite.runtime.SqlFunctions
> at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:318)
> at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:448)
> at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:460)
>
> Queries with char literal work OK: "SELECT 'r';"
> Does someone have an idea what can be wrong ?
>

Question: null policy on set / array operators

[Ruben Q L]

Hello community,

I was playing with set / array operators and I found something a bit
surprising.
Let us say I have a table "MyTable" that contains one of such fields, with
potentially null values for certain rows.
Executing the following query will run just fine:
SELECT * FROM MyTable t WHERE CARDINALITY(t.field) = 0

However, a "similar" query with IS EMPTY operator, will result in a NPE in
the Calcite generated code:
SELECT * FROM MyTable t WHERE t.field IS EMPTY

The reason for that seems to be that CARDINALITY defines NullPolicy.STRICT,
whereas IS EMPTY defines NullPolicy.NONE, see RexImpTable [1].
Looking a bit more in detail, it seems there is a mix of null policies in
these operators. For instance CARDINALITY was added a long time ago (as
STRICT); then a bunch of operators were added via [2], all of them as NONE;
and a few others were added recently in [3] as STRICT.
Is this intended? Is there a rule in place to explain this apparent
inconsistency regarding null policies in these operators?

Best,
Ruben

[1]
https://github.com/apache/calcite/blob/7e0057e8de93930f1b2952a1cbcee8ad7a6bfb4b/core/src/main/java/org/apache/calcite/adapter/enumerable/RexImpTable.java#L513
[2]
https://github.com/apache/calcite/commit/d35c11c73cc78cf379d7e886cbbfb62b91d75fad
[3]
https://github.com/apache/calcite/commit/f61541d633cfde53a4b0de0c23a010250c93274e

Re: [VOTE] Release Apache Calcite 1.31.0 (release candidate 3)

[Ruben Q L]

Thanks again Andrei for working on this.

- Checksum and signature: ok
- Gradle test: ok
- Calcite-based application test suite: ok
- Diff between git repo and release sources: ok

My vote is: +1 (binding)

Best,
Ruben


On Fri, Jul 29, 2022 at 1:50 PM Enrico Olivelli  wrote:

> +1 (non binding)
>
> All tests of HerdDB pass without any code changes
> CI results on this PR:  https://github.com/diennea/herddb/pull/789
>
>
> Thanks for driving the release
> Enrico
>
> Il giorno ven 29 lug 2022 alle ore 14:07 Andrei Sereda
>  ha scritto:
> >
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.31.0, release
> > candidate 3.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> https://github.com/apache/calcite/blob/calcite-1.31.0-rc3/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=86d1bf40e4799665951065fd5f4ee4a771e7b655
> >
> > Its hash is 86d1bf40e4799665951065fd5f4ee4a771e7b655
> >
> > Tag:
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.31.0-rc3
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc3
> > (revision 56034)
> >
> > The hashes of the artifacts are as follows:
> >
> 477848037465e883bc7f45d71a59be5932ddfc37d4fd4571f17d7463afe6cc3df1ab5bd20f38fb28fd6468d5b23f559eba4ff3adde9d4f689622e95abd736260
> > *apache-calcite-1.31.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1172/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/sereda.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.31.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.31.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (non-binding)
>

Re: [VOTE] Release Apache Calcite 1.31.0 (release candidate 1)

[Ruben Q L]

I concur, there seems to be an issue with the checksum (I get the same
result as Stamatis).
Other checks ok.

-1 (binding)



On Fri, Jul 29, 2022 at 9:58 AM Stamatis Zampetakis 
wrote:

> Ubuntu 20.04.4 LTS, jdk1.8.0_261, Gradle wrapper, Gradle 7.4.2
>
>  * Checked checksums KO
>  * Went over release note OK (left minor comments in the draft PR)
>
> $ sha512sum -c apache-calcite-1.31.0-src.tar.gz.sha512
> apache-calcite-1.31.0-src.tar.gz: FAILED
> sha512sum: WARNING: 1 computed checksum did NOT match
>
> $ sha512sum apache-calcite-1.31.0-src.tar.gz
>
> 9495f28e2c16ce6d5905a0b19b75e856e9a81e31c0252e8f6afed79c40ec88f71ecd708d90a63a1e5815a95dedfab56ebfa7e81851b3153833736447100a8f47
>  apache-calcite-1.31.0-src.tar.gz
>
> -1 (binding)
>
> On Thu, Jul 28, 2022 at 6:40 PM Andrei Sereda  wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Calcite 1.31.0, release
> > candidate 1.
> >
> > Thanks to everyone who has contributed to this release.
> >
> > You can read the release notes here:
> >
> >
> https://github.com/apache/calcite/blob/calcite-1.31.0-rc1/site/_docs/history.md
> >
> > The commit to be voted upon:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=commit;h=4912aabe66fa06a30ea688c4bc2bf06dc05ee677
> >
> > Its hash is 4912aabe66fa06a30ea688c4bc2bf06dc05ee677
> >
> > Tag:
> >
> >
> https://gitbox.apache.org/repos/asf?p=calcite.git;a=tag;h=refs/tags/calcite-1.31.0-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.31.0-rc1
> > (revision 56015)
> >
> > The hashes of the artifacts are as follows:
> >
> >
> 527e699857958b9d98e733059bd67ca6fc42e75238074dec65cc86caa8f5ae176e269d9ac478754ad206ea2336289a8f40f37ee86818130b37b53a53ee79bb2c
> > *apache-calcite-1.31.0-src.tar.gz
> >
> > A staged Maven repository is available for review at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecalcite-1168/org/apache/calcite/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/sereda.asc
> > https://www.apache.org/dist/calcite/KEYS
> >
> > To create the jars and test Apache Calcite: "gradle build"
> > (requires an appropriate Gradle/JDK installation)
> >
> > Please vote on releasing this package as Apache Calcite 1.31.0.
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Calcite 1.31.0
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Here is my vote:
> >
> > +1 (non-binding)
> >
>

[jira] [Created] (CALCITE-5221) Upgrade Calcite to Avatica 1.22

[Ruben Q L (Jira)]

Ruben Q L created CALCITE-5221:
--

 Summary: Upgrade Calcite to Avatica 1.22
 Key: CALCITE-5221
 URL: https://issues.apache.org/jira/browse/CALCITE-5221
 Project: Calcite
  Issue Type: Task
Reporter: Ruben Q L
 Fix For: 1.31.0


Upgrade Calcite to Avatica 1.22, which includes a vulnerability fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)