[jira] [Created] (CALCITE-4395) Add an interface in RelOptMaterializations to allow registering normalization rules
xzh_dz created CALCITE-4395: --- Summary: Add an interface in RelOptMaterializations to allow registering normalization rules Key: CALCITE-4395 URL: https://issues.apache.org/jira/browse/CALCITE-4395 Project: Calcite Issue Type: Wish Reporter: xzh_dz In the current framework of materialized recognition, the commonly used relational algebra materialized view recognition algorithm has been implemented, but in specific scenarios, users need to customize the algorithm of materialized view recognition to enhance the ability of materialized view recognition. The algorithm of user-defined materialized view recognition has been implemented here[1]. In addition, the ability of normalization is a very important point before materialized recognition. Normalization can simplify the difficulty of materialized view recognition. Currently, the ability of normalization based on relation algebra can not well support the needs of users. Users need to be able to customize normalization rules, which may be equivalent transformation of some relational algebra. Therefore, we should allow users to customize some normalization algorithms, enhance the normalization ability before the recognition of materialized views. [1][Add an interface in RelOptMaterializations to allow registering UnifyRule|https://github.com/apache/calcite/pull/2094] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4364) Simplify "a IN (1, 2) AND a = 1" to "a = 1"
[
https://issues.apache.org/jira/browse/CALCITE-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230376#comment-17230376
]
Danny Chen commented on CALCITE-4364:
-
I'm going to merge within 24 hours if there are no more review comments ~
> Simplify "a IN (1, 2) AND a = 1" to "a = 1"
> ---
>
> Key: CALCITE-4364
> URL: https://issues.apache.org/jira/browse/CALCITE-4364
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Calcite should simplify "\{{a IN (1, 2) AND a = 1}}" to "\{{a = 1}}". Check
> this test in {{RexProgramTest}}:
> {code:java}
> @Test void testSimplifyInAnd() {
> // deptno in (20, 10) and deptno = 10
> // ==>
> // deptno = 10
> final RexNode e =
> and(
> in(vInt(), literal(20), literal(10)),
> eq(vInt(), literal(10)));
> checkSimplify(e, "=(?0.int0, 10)");
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-4394) When generating code for a function call, take the inferred operand type into account
[
https://issues.apache.org/jira/browse/CALCITE-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reassigned CALCITE-4394:
Assignee: Julian Hyde
> When generating code for a function call, take the inferred operand type into
> account
> -
>
> Key: CALCITE-4394
> URL: https://issues.apache.org/jira/browse/CALCITE-4394
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
>
> When generating code for a function call, take the inferred operand type into
> account. If we don't do this, the Java code we generate for
> {{CONCAT(CAST(NULL AS ANY), 'x')}} does not compile, because the first
> parameter is of type Object; it needs to be String (based on the inferred
> operand type of VARCHAR).
> We achieve this by using {{SqlOperator.operandTypeInference}} at validate
> time, storing the list of inferred operand types for the call, then using
> that list (if available) during code generation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-4364) Simplify "a IN (1, 2) AND a = 1" to "a = 1"
[
https://issues.apache.org/jira/browse/CALCITE-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen reassigned CALCITE-4364:
---
Assignee: Danny Chen
> Simplify "a IN (1, 2) AND a = 1" to "a = 1"
> ---
>
> Key: CALCITE-4364
> URL: https://issues.apache.org/jira/browse/CALCITE-4364
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Calcite should simplify "\{{a IN (1, 2) AND a = 1}}" to "\{{a = 1}}". Check
> this test in {{RexProgramTest}}:
> {code:java}
> @Test void testSimplifyInAnd() {
> // deptno in (20, 10) and deptno = 10
> // ==>
> // deptno = 10
> final RexNode e =
> and(
> in(vInt(), literal(20), literal(10)),
> eq(vInt(), literal(10)));
> checkSimplify(e, "=(?0.int0, 10)");
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
[
https://issues.apache.org/jira/browse/CALCITE-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4392:
Labels: pull-request-available (was: )
> The operation of checking types equal ignoring null can be more efficient
> -
>
> Key: CALCITE-4392
> URL: https://issues.apache.org/jira/browse/CALCITE-4392
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
> RelDataType) is on the hot path for many scenarios, as it compare types with
> nullability ignored.
> In the implementation, the expensive operations are
> {{RelDataType#equals(Object)}} and
> {{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
> multiple sub-fields.
> For the current implementation, the {{RelDataType#equals(Object)}} is called
> at least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
> whenever the nullability values are different.
> We can improve the implementation so that {{RelDataType#equals(Object)}} is
> called exactly once, and the call to
> {{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
> are totally different (with different type names, and nullability values)
> {noformat}
> if (type1.isNullable() == type2.isNullable()) {
> return type1.equals(type2);
> } else {
> return type1.equals(
> factory.createTypeWithNullability(type2, type1.isNullable()));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4394) When generating code for a function call, take the inferred operand type into account
Julian Hyde created CALCITE-4394:
Summary: When generating code for a function call, take the
inferred operand type into account
Key: CALCITE-4394
URL: https://issues.apache.org/jira/browse/CALCITE-4394
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
When generating code for a function call, take the inferred operand type into
account. If we don't do this, the Java code we generate for {{CONCAT(CAST(NULL
AS ANY), 'x')}} does not compile, because the first parameter is of type
Object; it needs to be String (based on the inferred operand type of VARCHAR).
We achieve this by using {{SqlOperator.operandTypeInference}} at validate time,
storing the list of inferred operand types for the call, then using that list
(if available) during code generation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4374) Support materialized view recognition when query distinct aggregate on target GROUP BY column
[
https://issues.apache.org/jira/browse/CALCITE-4374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230322#comment-17230322
]
Wang Yanlin commented on CALCITE-4374:
--
Fixed in Fixed in
https://github.com/apache/calcite/commit/1e6b7425ec20e2bddc6331a21016475a43f9d995,
thx [~xzh_dz]
> Support materialized view recognition when query distinct aggregate on target
> GROUP BY column
> -
>
> Key: CALCITE-4374
> URL: https://issues.apache.org/jira/browse/CALCITE-4374
> Project: Calcite
> Issue Type: Improvement
>Reporter: Wang Yanlin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When query distinct aggregate on the column, which is in the group by list of
> target.
> Materialized view recognition should success.
> See this case.
> {noformat}
> @Test void testQueryDistinctColumnInTargetGroupByList() {
> final String mv = ""
> + "select \"name\", \"deptno\" "
> + "from \"emps\" group by \"name\", \"deptno\"";
> final String query = ""
> + "select \"name\", count(distinct \"deptno\")\n"
> + "from \"emps\" group by \"name\"";
> sql(mv, query).ok();
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4374) Support materialized view recognition when query distinct aggregate on target GROUP BY column
[
https://issues.apache.org/jira/browse/CALCITE-4374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang Yanlin resolved CALCITE-4374.
--
Resolution: Fixed
> Support materialized view recognition when query distinct aggregate on target
> GROUP BY column
> -
>
> Key: CALCITE-4374
> URL: https://issues.apache.org/jira/browse/CALCITE-4374
> Project: Calcite
> Issue Type: Improvement
>Reporter: Wang Yanlin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When query distinct aggregate on the column, which is in the group by list of
> target.
> Materialized view recognition should success.
> See this case.
> {noformat}
> @Test void testQueryDistinctColumnInTargetGroupByList() {
> final String mv = ""
> + "select \"name\", \"deptno\" "
> + "from \"emps\" group by \"name\", \"deptno\"";
> final String query = ""
> + "select \"name\", count(distinct \"deptno\")\n"
> + "from \"emps\" group by \"name\"";
> sql(mv, query).ok();
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
[
https://issues.apache.org/jira/browse/CALCITE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4393:
---
Description:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
{{ExceptionInInitializerError}} on WebSphere 9 (on Ubuntu) when trying to start
an application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 74 more
** Caused by: java.lang.ExceptionInInitializerError
** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
**
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 85 more
** Caused by: java.lang.NullPointerException
**
com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
** ... 92 more
** 1 errors, 5 infos.
{noformat}
As we can see there is a NPE in
[SqlCallBinding:60|https://github.com/apache/calcite/blob/2e9edae7fc57ab9c9c7c097008724ac99a1791a3/core/src/main/java/org/apache/calcite/sql/SqlCallBinding.java#L60],
when it tries to call a method on {{SqlStdOperatorTable.DEFAULT}}, which seems
to be null.
According to the logs, what I think is happening is a circular dependency issue
when classes get loaded (more details in first comment).
However, so far I have only seen this error in WebSphere, in other environments
the same application is correctly initialized and launched, and this exception
does not happen.
Since WebSphere uses IBM's own JVM, my hypothesis is that its class loader
follows a different (and unlucky) order to load Calcite classes.
So far I have not been able to reproduce the issue with a unit test, neither on
[Github CI|https://github.com/apache/calcite/pull/2251], nor in a local
environment.
was:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java
[jira] [Updated] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
[
https://issues.apache.org/jira/browse/CALCITE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4393:
Labels: pull-request-available (was: )
> ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular
> dependency
> --
>
> Key: CALCITE-4393
> URL: https://issues.apache.org/jira/browse/CALCITE-4393
> Project: Calcite
> Issue Type: Bug
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> See discussion here:
> https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
> ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
> application that uses Calcite (shaded) library. The exception message reads:
> {noformat}
> ** error [-]: Unexpected error.
> ** ...
> ** Caused by: java.lang.BootstrapMethodError:
> java.lang.ExceptionInInitializerError
> **
> com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
> **
> com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
> **
> com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
> **
> com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
> ** java.lang.Class.forNameImpl(Native Method)
> ** java.lang.Class.forName(Class.java:348)
> **
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> **
> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> **
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> ** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> Method)
> **
> java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
> ** ... 74 more
> ** Caused by: java.lang.ExceptionInInitializerError
> ** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
> **
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
> ** java.lang.Class.forNameImpl(Native Method)
> ** java.lang.Class.forName(Class.java:348)
> **
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> **
> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> **
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> ** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> Method)
> **
> java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
> ** ... 85 more
> ** Caused by: java.lang.NullPointerException
> **
> com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
> ** ... 92 more
> ** 1 errors, 5 infos.
> {noformat}
> As we can see there is a NPE in
> [SqlCallBinding:60|https://github.com/apache/calcite/blob/2e9edae7fc57ab9c9c7c097008724ac99a1791a3/core/src/main/java/org/apache/calcite/sql/SqlCallBinding.java#L60],
> when it tries to call a method on {{SqlStdOperatorTable.DEFAULT}}, which
> seems to be null.
> According to the logs, what I think is happening is a circular dependency
> issue when classes get loaded (more details in first comment).
> However, so far I have only seen this error in WebSphere, in other
> environments the same application is correctly initialized and launched, and
> this exception does not happen.
> Since WebSphere uses IBM's own JVM, my hypothesis is that its class loader
> follows a different (and unlucky) order to load Calcite classes.
> So far I have not been able to reproduce the issue with a unit test, neither
> on [Github CI|https://github.com/apache/calcite/pull/2251], nor in a local
> environment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
[
https://issues.apache.org/jira/browse/CALCITE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229998#comment-17229998
]
Ruben Q L commented on CALCITE-4393:
{code:title=1) RexToLixTranslator.java}
/*90*/ public class RexToLixTranslator implements
RexVisitor {
/*91*/ public static final Map JAVA_TO_SQL_METHOD_MAP =
/*92*/ Util.mapOf(
/*93*/ findMethod(String.class, "toUpperCase"),
SqlStdOperatorTable.UPPER,
// --> LOAD SqlStdOperatorTable
{code}
{code:title=2) SqlStdOperatorTable.java}
/*118*/ public static final SqlMultisetSetOperator MULTISET_UNION_DISTINCT =
/*119*/ new SqlMultisetSetOperator("MULTISET UNION DISTINCT", 14, false);
// --> LOAD SqlMultisetSetOperator
{code}
{code:title=3) SqlMultisetSetOperator.java}
/*42*/ public SqlMultisetSetOperator(String name, int prec, boolean all) {
/*43*/super(
/*44*/name,
/*45*/SqlKind.OTHER,
/*46*/prec,
/*47*/true,
/*48*/ReturnTypes.MULTISET_NULLABLE,
/*49*/InferTypes.FIRST_KNOWN, // --> LOAD InferTypes
/*50*/OperandTypes.MULTISET_MULTISET);
/*51*/this.all = all;
/*52*/ }
{code}
{code:title=4) InferTypes.java}
/*41*/ public static final SqlOperandTypeInference FIRST_KNOWN =
/*42*/ (callBinding, returnType, operandTypes) -> {
/*43*/final RelDataType unknownType =
/*44*/callBinding.getValidator().getUnknownType();
/*45*/RelDataType knownType = unknownType;
/*46*/for (SqlNode operand : callBinding.operands()) {
/*47*/ knownType = SqlTypeUtil.deriveType(callBinding, operand);
/*48*/ if (!knownType.equals(unknownType)) {
/*49*/break;
/*50*/ }
/*51*/}
// --> LOAD SqlCallBinding
{code}
{code:title=5) SqlCallBinding.java}
/*59*/ private static final SqlCall DEFAULT_CALL =
/*60*/ SqlStdOperatorTable.DEFAULT.createCall(SqlParserPos.ZERO);
// SqlStdOperatorTable is still loading in 2)! SqlStdOperatorTable.DEFAULT in
SqlStdOperatorTable:185 is null => NPE!
{code}
> ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular
> dependency
> --
>
> Key: CALCITE-4393
> URL: https://issues.apache.org/jira/browse/CALCITE-4393
> Project: Calcite
> Issue Type: Bug
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.27.0
>
>
> See discussion here:
> https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
> ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
> application that uses Calcite (shaded) library. The exception message reads:
> {noformat}
> ** error [-]: Unexpected error.
> ** ...
> ** Caused by: java.lang.BootstrapMethodError:
> java.lang.ExceptionInInitializerError
> **
> com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
> **
> com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
> **
> com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
> **
> com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
> ** java.lang.Class.forNameImpl(Native Method)
> ** java.lang.Class.forName(Class.java:348)
> **
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> **
> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> **
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> ** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> Method)
> **
> java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
> ** ... 74 more
> ** Caused by: java.lang.ExceptionInInitializerError
> ** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
> **
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
> ** java.lang.Class.forNameImpl(Native Method)
> ** java.lang.Class.forName(Class.java:348)
> **
> java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
> **
> java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
> **
> java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
> ** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native
> Method)
> **
> java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
>
[jira] [Updated] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
[
https://issues.apache.org/jira/browse/CALCITE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4393:
---
Description:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 74 more
** Caused by: java.lang.ExceptionInInitializerError
** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
**
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 85 more
** Caused by: java.lang.NullPointerException
**
com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
** ... 92 more
** 1 errors, 5 infos.
{noformat}
As we can see there is a NPE in
[SqlCallBinding:60|https://github.com/apache/calcite/blob/2e9edae7fc57ab9c9c7c097008724ac99a1791a3/core/src/main/java/org/apache/calcite/sql/SqlCallBinding.java#L60],
when it tries to call a method on {{SqlStdOperatorTable.DEFAULT}}, which seems
to be null.
According to the logs, what I think is happening is a circular dependency issue
when classes get loaded (more details in first comment).
However, so far I have only seen this error in WebSphere, in other environments
the same application is correctly initialized and launched, and this exception
does not happen.
Since WebSphere uses IBM's own JVM, my hypothesis is that its class loader
follows a different (and unlucky) order to load Calcite classes.
So far I have not been able to reproduce the issue with a unit test, neither on
[Github CI|https://github.com/apache/calcite/pull/2251], nor in a local
environment.
was:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
[jira] [Updated] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
[
https://issues.apache.org/jira/browse/CALCITE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4393:
---
Description:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 74 more
** Caused by: java.lang.ExceptionInInitializerError
** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
**
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 85 more
** Caused by: java.lang.NullPointerException
**
com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
** ... 92 more
** 1 errors, 5 infos.
{noformat}
As we can see there is a NPE in
[SqlCallBinding:60|https://github.com/apache/calcite/blob/2e9edae7fc57ab9c9c7c097008724ac99a1791a3/core/src/main/java/org/apache/calcite/sql/SqlCallBinding.java#L60],
when it tries to call a method on {{SqlStdOperatorTable.DEFAULT}}, which seems
to be null.
According to the logs, what I think is happening is a circular dependency issue
when classes get loaded (more details in first comment).
However, so far I have only seen this error in WebSphere, in other environments
the same application is correctly initialized and launched, and this exception
does not happen.
Since WebSphere uses IBM's own JVM, my hypothesis is that its class loader
follows a different (and unlucky) order to load Calcite classes.
was:
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.jav
[jira] [Created] (CALCITE-4393) ExceptionInInitializerError due to NPE in SqlCallBinding caused by circular dependency
Ruben Q L created CALCITE-4393:
--
Summary: ExceptionInInitializerError due to NPE in SqlCallBinding
caused by circular dependency
Key: CALCITE-4393
URL: https://issues.apache.org/jira/browse/CALCITE-4393
Project: Calcite
Issue Type: Bug
Reporter: Ruben Q L
Assignee: Ruben Q L
Fix For: 1.27.0
See discussion here:
https://lists.apache.org/thread.html/red0fc146f78a196af6fdaa5393b2c0906e0a49361a1734a850185777%40%3Cdev.calcite.apache.org%3E
ExceptionInInitializerErroron WebSphere 9 (on Ubuntu) when trying to start an
application that uses Calcite (shaded) library. The exception message reads:
{noformat}
** error [-]: Unexpected error.
** ...
** Caused by: java.lang.BootstrapMethodError:
java.lang.ExceptionInInitializerError
**
com.onwbp.org.apache.calcite.sql.type.InferTypes.(InferTypes.java:41)
**
com.onwbp.org.apache.calcite.sql.fun.SqlMultisetSetOperator.(SqlMultisetSetOperator.java:43)
**
com.onwbp.org.apache.calcite.sql.fun.SqlStdOperatorTable.(SqlStdOperatorTable.java:118)
**
com.onwbp.org.apache.calcite.adapter.enumerable.RexToLixTranslator.(RexToLixTranslator.java:93)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 74 more
** Caused by: java.lang.ExceptionInInitializerError
** java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
**
java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
** java.lang.Class.forNameImpl(Native Method)
** java.lang.Class.forName(Class.java:348)
**
java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:311)
**
java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:373)
**
java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:286)
** java.lang.invoke.MethodHandle.getCPMethodTypeAt(Native Method)
**
java.lang.invoke.MethodHandle.resolveInvokeDynamic(MethodHandle.java:849)
** ... 85 more
** Caused by: java.lang.NullPointerException
**
com.onwbp.org.apache.calcite.sql.SqlCallBinding.(SqlCallBinding.java:60)
** ... 92 more
** 1 errors, 5 infos.
{noformat}
As we can see there is a NPE in
[SqlCallBinding:60|https://github.com/apache/calcite/blob/2e9edae7fc57ab9c9c7c097008724ac99a1791a3/core/src/main/java/org/apache/calcite/sql/SqlCallBinding.java#L60],
when it tries to call a method on SqlStdOperatorTable.DEFAULT, which seems to
be null.
According to the logs, what I think is happening is a circular dependency issue
when classes get loaded (more details in first comment).
However, so far I have only seen this error in WebSphere, in other environments
the same application is correctly initialized and launched, and this exception
does not happen.
Since WebSphere uses IBM's own JVM, my hypothesis is that its class loader
follows a different (and unlucky) order to load Calcite classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
[
https://issues.apache.org/jira/browse/CALCITE-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liya Fan updated CALCITE-4392:
--
Description:
Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
RelDataType) is on the hot path for many scenarios, as it compare types with
nullability ignored.
In the implementation, the expensive operations are
{{RelDataType#equals(Object)}} and
{{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
multiple sub-fields.
For the current implementation, the {{RelDataType#equals(Object)}} is called at
least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
whenever the nullability values are different.
We can improve the implementation so that {{RelDataType#equals(Object)}} is
called exactly once, and the call to
{{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
are totally different (with different type names, and nullability values)
{noformat}
if (type1.isNullable() == type2.isNullable()) {
return type1.equals(type2);
} else {
return type1.equals(
factory.createTypeWithNullability(type2, type1.isNullable()));
}
{noformat}
was:
Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
RelDataType) is on the hot path for many scenarios, as it compare types with
nullability ignored.
In the implementation, the expensive operations are
{{RelDataType#equals(Object)}} and
{{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
multiple sub-fields.
For the current implementation, the {{RelDataType#equals(Object)}} is called at
least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
whenever the nullability values are different.
We can improve the implementation so that {{RelDataType#equals(Object)}} is
called at most once, and the call to
{{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
are totally different (with different type names, and nullability values)
{noformat}
if (type1.isNullable() == type2.isNullable()) {
return type1.equals(type2);
} else {
return type1.equals(
factory.createTypeWithNullability(type2, type1.isNullable()));
}
{noformat}
> The operation of checking types equal ignoring null can be more efficient
> -
>
> Key: CALCITE-4392
> URL: https://issues.apache.org/jira/browse/CALCITE-4392
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
>
> Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
> RelDataType) is on the hot path for many scenarios, as it compare types with
> nullability ignored.
> In the implementation, the expensive operations are
> {{RelDataType#equals(Object)}} and
> {{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
> multiple sub-fields.
> For the current implementation, the {{RelDataType#equals(Object)}} is called
> at least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
> whenever the nullability values are different.
> We can improve the implementation so that {{RelDataType#equals(Object)}} is
> called exactly once, and the call to
> {{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
> are totally different (with different type names, and nullability values)
> {noformat}
> if (type1.isNullable() == type2.isNullable()) {
> return type1.equals(type2);
> } else {
> return type1.equals(
> factory.createTypeWithNullability(type2, type1.isNullable()));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
[
https://issues.apache.org/jira/browse/CALCITE-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229943#comment-17229943
]
Liya Fan commented on CALCITE-4392:
---
Sounds reasonable. Thanks for your feedback.
> The operation of checking types equal ignoring null can be more efficient
> -
>
> Key: CALCITE-4392
> URL: https://issues.apache.org/jira/browse/CALCITE-4392
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
>
> Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
> RelDataType) is on the hot path for many scenarios, as it compare types with
> nullability ignored.
> In the implementation, the expensive operations are
> {{RelDataType#equals(Object)}} and
> {{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
> multiple sub-fields.
> For the current implementation, the {{RelDataType#equals(Object)}} is called
> at least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
> whenever the nullability values are different.
> We can improve the implementation so that {{RelDataType#equals(Object)}} is
> called at most once, and the call to
> {{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
> are totally different (with different type names, and nullability values)
> {noformat}
> if (type1.isNullable() == type2.isNullable()) {
> return type1.equals(type2);
> } else {
> return type1.equals(
> factory.createTypeWithNullability(type2, type1.isNullable()));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
[
https://issues.apache.org/jira/browse/CALCITE-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229938#comment-17229938
]
Vladimir Sitnikov commented on CALCITE-4392:
Makes sense, however, I would suggest avoiding {{else}} after {{return}}.
> The operation of checking types equal ignoring null can be more efficient
> -
>
> Key: CALCITE-4392
> URL: https://issues.apache.org/jira/browse/CALCITE-4392
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
>
> Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
> RelDataType) is on the hot path for many scenarios, as it compare types with
> nullability ignored.
> In the implementation, the expensive operations are
> {{RelDataType#equals(Object)}} and
> {{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
> multiple sub-fields.
> For the current implementation, the {{RelDataType#equals(Object)}} is called
> at least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
> whenever the nullability values are different.
> We can improve the implementation so that {{RelDataType#equals(Object)}} is
> called at most once, and the call to
> {{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
> are totally different (with different type names, and nullability values)
> {noformat}
> if (type1.isNullable() == type2.isNullable()) {
> return type1.equals(type2);
> } else {
> return type1.equals(
> factory.createTypeWithNullability(type2, type1.isNullable()));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
[
https://issues.apache.org/jira/browse/CALCITE-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liya Fan updated CALCITE-4392:
--
Description:
Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
RelDataType) is on the hot path for many scenarios, as it compare types with
nullability ignored.
In the implementation, the expensive operations are
{{RelDataType#equals(Object)}} and
{{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
multiple sub-fields.
For the current implementation, the {{RelDataType#equals(Object)}} is called at
least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
whenever the nullability values are different.
We can improve the implementation so that {{RelDataType#equals(Object)}} is
called at most once, and the call to
{{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
are totally different (with different type names, and nullability values)
{noformat}
if (type1.isNullable() == type2.isNullable()) {
return type1.equals(type2);
} else {
return type1.equals(
factory.createTypeWithNullability(type2, type1.isNullable()));
}
{noformat}
was:
Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
RelDataType) is on the hot path for many scenarios, as it compare types with
nullability ignored.
In the implementation, the expensive operations are
{{RelDataType#equals(Object)}} and
{{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
multiple sub-fields.
For the current implementation, the {{RelDataType#equals(Object)}} is called at
least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
whenever the nullability values are different.
We can improve the implementation so that {{RelDataType#equals(Object)}} is
called at most once, and the call to
{{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
are totally different (with different type names, and nullability values)
{{noformat}}
if (type1.isNullable() == type2.isNullable()) {
return type1.equals(type2);
} else {
return type1.equals(
factory.createTypeWithNullability(type2, type1.isNullable()));
}
{{noformat}}
> The operation of checking types equal ignoring null can be more efficient
> -
>
> Key: CALCITE-4392
> URL: https://issues.apache.org/jira/browse/CALCITE-4392
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
>
> Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
> RelDataType) is on the hot path for many scenarios, as it compare types with
> nullability ignored.
> In the implementation, the expensive operations are
> {{RelDataType#equals(Object)}} and
> {{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
> multiple sub-fields.
> For the current implementation, the {{RelDataType#equals(Object)}} is called
> at least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
> whenever the nullability values are different.
> We can improve the implementation so that {{RelDataType#equals(Object)}} is
> called at most once, and the call to
> {{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
> are totally different (with different type names, and nullability values)
> {noformat}
> if (type1.isNullable() == type2.isNullable()) {
> return type1.equals(type2);
> } else {
> return type1.equals(
> factory.createTypeWithNullability(type2, type1.isNullable()));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4392) The operation of checking types equal ignoring null can be more efficient
Liya Fan created CALCITE-4392:
-
Summary: The operation of checking types equal ignoring null can
be more efficient
Key: CALCITE-4392
URL: https://issues.apache.org/jira/browse/CALCITE-4392
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Liya Fan
Assignee: Liya Fan
Method {{SqlTypeUtil#equalSansNullability(RelDataTypeFactory, RelDataType,
RelDataType) is on the hot path for many scenarios, as it compare types with
nullability ignored.
In the implementation, the expensive operations are
{{RelDataType#equals(Object)}} and
{{RelDataTypeFactory#createTypeWithNullability}}, especially for types with
multiple sub-fields.
For the current implementation, the {{RelDataType#equals(Object)}} is called at
least once, and {{RelDataTypeFactory#createTypeWithNullability}} is called
whenever the nullability values are different.
We can improve the implementation so that {{RelDataType#equals(Object)}} is
called at most once, and the call to
{{RelDataTypeFactory#createTypeWithNullability}} can be avoided if the types
are totally different (with different type names, and nullability values)
{{noformat}}
if (type1.isNullable() == type2.isNullable()) {
return type1.equals(type2);
} else {
return type1.equals(
factory.createTypeWithNullability(type2, type1.isNullable()));
}
{{noformat}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4386) Support RelShuttle visit specific logical operators
[
https://issues.apache.org/jira/browse/CALCITE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229858#comment-17229858
]
xzh_dz commented on CALCITE-4386:
-
A test case is shown below:
{code:java}
// code placeholder
org.apache.calcite.test.SqlToRelConverterTest#testRelShuttleForLogicalCalc
@Test void testRelShuttleForLogicalCalc() {
final String sql = "select ename from emp";
final RelNode rel = tester.convertSqlToRel(sql).rel;
final HepProgramBuilder programBuilder = HepProgram.builder();
programBuilder.addRuleInstance(CoreRules.PROJECT_TO_CALC);
final HepPlanner planner = new HepPlanner(programBuilder.build());
planner.setRoot(rel);
// Not LogicalCalc
final RelNode calc = planner.findBestExp();
final List rels = new ArrayList<>();
final RelShuttleImpl visitor = new RelShuttleImpl() {
@Override public RelNode visit(LogicalCalc calc) {
RelNode visitedRel = super.visit(calc);
rels.add(visitedRel);
return visitedRel;
}
};
visitor.visit(calc);
assertThat(rels.size(), is(1));
assertThat(rels.get(0), isA(LogicalCalc.class));
}
{code}
result:
{code:java}
// code placeholder
java.lang.AssertionError:
Expected: is <1>
but: was <0>
Expected :is <1>
Actual :<0>
{code}
> Support RelShuttle visit specific logical operators
> ---
>
> Key: CALCITE-4386
> URL: https://issues.apache.org/jira/browse/CALCITE-4386
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the current implementation, some operators, such as `LogicalProject` and
> `LogicalFilter`, can be directly accessed by `Relshuttle`, while others, such
> as `LogicalCalc` and `LogicalWindow`, cannot be accessed directly.
> `RelShuttle` is a very important tool class. Usually, we will do equivalent
> transformation of relational algebra in the process of relational algebra
> optimization. This PR will cover the method of the operator visit
> `RelShuttle`, and the specific operator can be accessed directly by visiting
> `RelNode`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4390) SqlMatchRecognize returns wrong operand list
[
https://issues.apache.org/jira/browse/CALCITE-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4390:
Labels: pull-request-available (was: )
> SqlMatchRecognize returns wrong operand list
>
>
> Key: CALCITE-4390
> URL: https://issues.apache.org/jira/browse/CALCITE-4390
> Project: Calcite
> Issue Type: Bug
>Reporter: Dawid Wysakowicz
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{SqlMatchRecognize}} does not return full list of its operands. The
> method {{SqlMatchRecognizeOperator#createCall}} expects 12 operands whereas
> {{SqlMatchRecognize#getOperandList}} returns only 10. It misses
> {{rowsPerMatch}} and {{interval}}.
> It results that the {{SqlShuttle}} and more concretely the
> {{CallCopyingArgHandler}} does not work with {{SqlMatchRecognize}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
