[DISCUSS] Draft board report for Jul 2021

[Haisheng Yuan]

Attached below is a draft of this month's board report. I plan to submit it on
Jul 13. Please let me know if you have any additions or corrections.

## Description:
The mission of Calcite is the creation and maintenance of software related to 
Dynamic data management framework

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Calcite was founded 2015-10-21 (6 years ago)
There are currently 53 committers and 23 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:3.

Community changes, past quarter:
- No new PMC members. Last addition was Ruben Q L on 2020-08-09.
- Vladimir Ozerov was added as committer on 2021-06-23.

## Project Activity:
Calcite Avatica 1.18.0 was released on 2021-05-18. Calcite 1.27.0 was
released on 2021-06-03, with important features such as InnoDB adapter,
Three-valued logic for SEARCH operator and MergeUnion operator in
Enumerable convention.

On Apr 26th, we had online meeting to discuss the proposal of extending 
Calcite to an incremental query optimizer.

## Community Health:
The overall activity in the community has slightly decreased in the past
few months without this being worrisome. Specifically we have less active
reviewers in the past 3 months. We need more committers to spend some
time to review open pull requests.

the number of non-committer (contributor) commits per month
+-+-+-+
|year |month| contributor_commits |
+-+-+-+
| 2021| 4   | 14  |
| 2021| 5   | 4   |
| 2021| 6   | 5   |
+-+-+-+

top reviewers in the last 3 months
+---+-+
| committer |   reviews   |
+---+-+
| Julian Hyde  | 13  |
| Stamatis Zampetakis  | 5   |
| rubenada  | 1   |
| amaliujia  | 1   |
| ForwardXu  | 1   |
| yuzhao.cyz  | 1   |
| liyafan82  | 1   |
+---+-+

Thanks,
Haisheng Yuan

Re: Request to be a contributor in JIRA

[Francis Chuang]

Hey Wenjuan,

I've added you as a contributor in jira.

Francis

On 6/07/2021 11:55 am, Wenjuan BI wrote:

Hi,

I am currently a graduate student working on a project using apache calcite
to parse SQL, and I would love to be a contributor to help the community
with some simple issues. Thank you in advance for your time and
consideration.

My information are below:
Username:ClaudineFull name:wenjuan bi

Best regards,

Wenjuan BI

Re: Support for constraints in jdbc adapter and optional decorrelation step in PlannerImpl.rel?

[Julian Hyde]

Re 1. It seems that the Server parser goes further than the Server 
implementation. You can specify PRIMARY KEY in the DDL, and we can parse it, 
but Server cannot create tables with primary keys. Please log a bug. 
Contributions welcome.

One workaround is for you to parse but not execute. The information you need is 
in the parse tree (SqlCreateTable, SqlKeyConstraint).

Re 2. Can you log a bug? Related to 
https://issues.apache.org/jira/browse/CALCITE-3582 
 and 
https://issues.apache.org/jira/browse/CALCITE-4642 
.

Julian




> On Jul 4, 2021, at 8:14 AM, Arthur Pan  wrote:
> 
> Greetings to dev@calcite:
> 
> I am currently working on extracting RelNodes from DML queries with schemas
> derived from DDL statements, and I have encountered a few issues:
> 
> 1. My current method to derive schemas from DDL statements is
>(a) connect to "jdbc:calcite" with parser factory set to
> ServerDdlExecutor.ParserFactory
>(b) execute the DDL statements
>(c) extract schema with connection.getRootSchem()
>  Currently simple create statements can be correctly executed. However,
> the execution (at
> org.apache.calcite.server.ServerDdlExecutor.execute(ServerDdlExecutor.java:472))
> will fail when constraints (e.g. PRIMARY KEY) are involved. The CREATE
> statement I used is pasted below:
>CREATE TABLE EMP (EMP_ID INTEGER NOT NULL,
> EMP_NAME VARCHAR,
> DEPT_ID INTEGER,
> PRIMARY KEY (DEPT_ID))
>  Is there a way to fix this? Or is there a better way to derive schema
> from DDL statements?
> 
> 2. It seems that currently PlannerImpl.rel(SqlNode) decorrelate (at
> PlannerImpl.java:265) the RelRoot converted from SqlNode, regardless what
> the RelBuilderConfig was passed in to the planner. Is there a way to bypass
> such a transformation to RelNode and fetch the raw RelRoot converted from
> SqlNode by SqlToRelConverter in PlannerImpl?
> 
> I would appreciate any insights into anything related. Thanks in advance!
> 
> Sincerely,
> Arthur Pan

Request to be a contributor in JIRA

[Wenjuan BI]

Hi,

I am currently a graduate student working on a project using apache calcite
to parse SQL, and I would love to be a contributor to help the community
with some simple issues. Thank you in advance for your time and
consideration.

My information are below:
Username:ClaudineFull name:wenjuan bi

Best regards,

Wenjuan BI

[jira] [Created] (CALCITE-4680) AssertionError: contains $cor1, while converting SQL query with UNNEST to logical plan

[Thomas Rebele (Jira)]

Thomas Rebele created CALCITE-4680:
--

 Summary: AssertionError: contains $cor1, while converting SQL 
query with UNNEST to logical plan
 Key: CALCITE-4680
 URL: https://issues.apache.org/jira/browse/CALCITE-4680
 Project: Calcite
  Issue Type: Bug
Affects Versions: 1.27.0
Reporter: Thomas Rebele


The exception happens with the following SQL query
{code:java}
SELECT field1 FROM tab t1
WHERE t1.field1 NOT IN (
SELECT l.subfield
FROM tab t2, UNNEST(t2.field2) AS l
WHERE t1.field1 = t2.field1)
{code}
and the schema
{code:java}
RecordType(VARCHAR NOT NULL field1, RecordType(VARCHAR subfield) ARRAY field2)
{code}
(where field2.subfield is nullable). Previous versions of Calcite threw a 
ClassCastException: LogicalCorrelate cannot be cast to Join.

However, it passes after removing the WHERE clause of the subquery.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4679) Search/sarg simplification leaves is not null(literal) unsimplified

[Vladimir Sitnikov (Jira)]

Vladimir Sitnikov created CALCITE-4679:
--

 Summary: Search/sarg simplification leaves is not null(literal) 
unsimplified
 Key: CALCITE-4679
 URL: https://issues.apache.org/jira/browse/CALCITE-4679
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Vladimir Sitnikov


Case: {{SEARCH(100500, Sarg[=])}} (Sarg[=] is "all values match, unknown as 
unknown").

While the issue does not look severe, it results in rex fuzzer false positives, 
so it might mask true bugs.


Expected: {{true}}
Actual:
{noformat}
unknown as unknown: OR(IS NOT NULL(0), null)
unknown as false: IS NOT NULL(0)
unknown as true: true <-- finally got the right answer
{noformat}

Test:

{code:java}
checkSimplify(
rexBuilder.makeCall(
SqlStdOperatorTable.SEARCH,
literal(BigDecimal.ZERO),
rexBuilder.makeSearchArgumentLiteral(
Sarg.of(RexUnknownAs.UNKNOWN, 
ImmutableRangeSet.of(Range.all())),
tInt())),
"true"
);
{code}

---

fuzzer:

{code:java}
  @Test void singleFuzzyTest() {
Random r = new Random();
r.setSeed(-8889103384303613092L);
RexFuzzer fuzzer = new RexFuzzer(rexBuilder, typeFactory);
generateRexAndCheckTrueFalse(fuzzer, r);
  }{code}

{noformat}
AssertionFailedError: SEARCH(100500, Sarg[=])
rexBuilder.makeCall(SqlStdOperatorTable.SEARCH, literal(100500), 
literal(Sarg[=])) isAlwaysTrue, so it should simplify to TRUE unknownAsFalse 
==> expected:  but was: 
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
at 
org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:69)
at 
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:188)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1146)
at 
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:251)
at 
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAsAndShrink(RexProgramFuzzyTest.java:202)
at 
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.checkUnknownAs(RexProgramFuzzyTest.java:165)
at 
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.generateRexAndCheckTrueFalse(RexProgramFuzzyTest.java:454)
at 
org.apache.calcite.test.fuzzer.RexProgramFuzzyTest.singleFuzzyTest(RexProgramFuzzyTest.java:463)
{noformat}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4678) AssertionError: result mismatch when simplifying case+search+isdistinctfrom+isnottrue

[Vladimir Sitnikov (Jira)]

Vladimir Sitnikov created CALCITE-4678:
--

 Summary: AssertionError: result mismatch when simplifying 
case+search+isdistinctfrom+isnottrue
 Key: CALCITE-4678
 URL: https://issues.apache.org/jira/browse/CALCITE-4678
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Vladimir Sitnikov


Frankly speaking, I do not know what triggers the issue. so the issue summary 
is abstract for now.

I do not know the expected outcome, however, I expect that Calcite should not 
fail with AssertionErrors.

Test case:

{code:java}
checkSimplifyAs(
isNotTrue(
case_(
rexBuilder.makeCall(
SqlStdOperatorTable.SEARCH,
vInt(1),
rexBuilder.makeSearchArgumentLiteral(
Sarg.of(
RexUnknownAs.TRUE,

ImmutableRangeSet.of(Range.lessThan(BigDecimal.ZERO))),
tInt())),
trueLiteral,
le(trueLiteral, isDistinctFrom(literal(0), literal(0),
RexUnknownAs.TRUE,
is("no idea what is expected"));
{code}

Error:

{noformat}
result mismatch (unknown as TRUE): when applied to {?0.int1=NULL},
IS NOT TRUE(CASE(SEARCH(?0.int1, Sarg[(-∞..0); NULL AS TRUE]), true, <=(true, 
IS DISTINCT FROM(0, 0 yielded false;
AND(IS NOT DISTINCT FROM(0, 0), >=(?0.int1, 0)) yielded true
java.lang.AssertionError: result mismatch (unknown as TRUE): when applied to 
{?0.int1=NULL},
IS NOT TRUE(CASE(SEARCH(?0.int1, Sarg[(-∞..0); NULL AS TRUE]), true, <=(true, 
IS DISTINCT FROM(0, 0 yielded false;
AND(IS NOT DISTINCT FROM(0, 0), >=(?0.int1, 0)) yielded true
at org.apache.calcite.rex.RexSimplify.verify(RexSimplify.java:2098)
at 
org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250)
at 
org.apache.calcite.rex.RexProgramTestBase.checkSimplifyAs(RexProgramTestBase.java:152){noformat}


The relevant fuzzer-driven case is
{code:java}
  @Test void singleFuzzyTest() {
Random r = new Random();
r.setSeed(6192825841324574146L);
RexFuzzer fuzzer = new RexFuzzer(rexBuilder, typeFactory);
generateRexAndCheckTrueFalse(fuzzer, r);
  }{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)