[jira] [Created] (CALCITE-3379) Support expand atom expression in table to relational node conversion
Danny Chen created CALCITE-3379: --- Summary: Support expand atom expression in table to relational node conversion Key: CALCITE-3379 URL: https://issues.apache.org/jira/browse/CALCITE-3379 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Now there are 2 ways to convert a RelOptTable to LogicalTableScan: 1. One way is tp open the Config#sConvertTableAccess[1] flag and the SqlToRelConverter would invoke the #toRel method which transforms the table to a node returned by the user(Usually a table scan). 2. Another way it to use the LogicalTableScan rule, this rule would invoke RelOptTable#toRel and wrap the returned node with a LogicalTableScan. The different between 1 and 2 is that, 2 happens in the planning rule but 1 happens in sql-to-rel conversion, 1 also supports to expand the table columns based on the defined default values expressions, see InitializerExpressionFactory#newColumnDefaultValue. The problem with the InitializerExpressionFactory#newColumnDefaultValue is that it use InitializerContext#convertExpression to convert a SqlNode, if the SqlNode is not validated, we always got a RexCall with SqlUnresolvedFunction. We should give the user change to validate their SqlNode or even we can support pure string expressions because they can be used to persist. Another problem with #toRel is that after the expressions applied as a projection, use has no change to apply any other rel nodes if they want, we can actually support this, the same way as we support the column expressions. [1]https://github.com/apache/calcite/blob/2dc97e6723e1b5bf762540f87b5cd1a848a1/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5605 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3392) Column expression in DDL should be validated before converting to RexNode
Danny Chen created CALCITE-3392: --- Summary: Column expression in DDL should be validated before converting to RexNode Key: CALCITE-3392 URL: https://issues.apache.org/jira/browse/CALCITE-3392 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 As described in CALCITE-3379, we should validate the column expression so that the expression can be converted correctly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3404) Promote plan for AggregateExpandDistinctAggregatesRule when all the agg expressions are distinct and have same arguments
Danny Chen created CALCITE-3404: --- Summary: Promote plan for AggregateExpandDistinctAggregatesRule when all the agg expressions are distinct and have same arguments Key: CALCITE-3404 URL: https://issues.apache.org/jira/browse/CALCITE-3404 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 After CALCITE-3159, the distinct constraint for MAX/MIN/BIT_OR/BIT_AND are removed automatically, so if we have a query: {code:sql} select sum(distinct deptno), count(distinct deptno), max(deptno) from emp {code} Than plan has regression from {code:xml} LogicalAggregate(group=[{}], EXPR$0=[SUM($0)], EXPR$1=[COUNT($0)], EXPR$2=[MAX($0)]) LogicalAggregate(group=[{0}]) LogicalProject(DEPTNO=[$7]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code} to {code:xml} LogicalAggregate(group=[{}], EXPR$0=[SUM($0) FILTER $2], EXPR$1=[COUNT($0) FILTER $2], EXPR$2=[MIN($1) FILTER $3]) LogicalProject(DEPTNO=[$0], EXPR$2=[$1], $g_0=[=($2, 0)], $g_1=[=($2, 1)]) LogicalAggregate(group=[{0}], groups=[[{0}, {}]], EXPR$2=[MAX($0)], $g=[GROUPING($0)]) LogicalProject(DEPTNO=[$7]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code} The distinct trait actually can be reused. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3410) Simplify RelOptRulesTest and HepPlannerTest by making test methods fluent
Danny Chen created CALCITE-3410: --- Summary: Simplify RelOptRulesTest and HepPlannerTest by making test methods fluent Key: CALCITE-3410 URL: https://issues.apache.org/jira/browse/CALCITE-3410 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3420) NullPointerException throws for implicit type coercion of nested SET operations
Danny Chen created CALCITE-3420: --- Summary: NullPointerException throws for implicit type coercion of nested SET operations Key: CALCITE-3420 URL: https://issues.apache.org/jira/browse/CALCITE-3420 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Check this sql in TypeCoercionTest: {code:sql} select t1_varchar20 from t1 union select t2_int from t2 union select t1_int from t1 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3441) Remove SqlTypeExplicitPrecedenceList.COMPACT_NUMERIC_TYPES because the NULL delimiters are useless
Danny Chen created CALCITE-3441: --- Summary: Remove SqlTypeExplicitPrecedenceList.COMPACT_NUMERIC_TYPES because the NULL delimiters are useless Key: CALCITE-3441 URL: https://issues.apache.org/jira/browse/CALCITE-3441 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 We do not need the "null" value as delimiter for the equivalence class because method Util.skip already did that. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3446) Make RelMetadataQuery extendible
Danny Chen created CALCITE-3446: --- Summary: Make RelMetadataQuery extendible Key: CALCITE-3446 URL: https://issues.apache.org/jira/browse/CALCITE-3446 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3456) AssertionError throws when aggregation has non-aggregate calls
Danny Chen created CALCITE-3456: --- Summary: AssertionError throws when aggregation has non-aggregate calls Key: CALCITE-3456 URL: https://issues.apache.org/jira/browse/CALCITE-3456 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Check this sql in SqlToRelConverterTest: {code:java} @Test public void testAggregateWithCaseWhen() { final String sql = "select\n" + " CASE WHEN job IN ('81', '82') THEN job\n" + " ELSE 'error'\n" + " END AS id,\n" + " count(empno)\n" + "FROM emp\n" + "where job <> '' or job in ('81', '82')\n" + "GROUP by deptno, job"; sql(sql).ok(); } {code} I tested PostgreSQL 9.6 and MySQL 5.6, this is a valid SQL and they both outputs the right result. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3518) Use List instead of BitSet to keep inheritPath in RelHint
Danny Chen created CALCITE-3518: --- Summary: Use List instead of BitSet to keep inheritPath in RelHint Key: CALCITE-3518 URL: https://issues.apache.org/jira/browse/CALCITE-3518 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Use List instead of BitSet, because the BitSet would just eliminate the tailing zeros from the original path integers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3523) Strengthen the java doc and code of class RelHint and SqlHint
Danny Chen created CALCITE-3523: --- Summary: Strengthen the java doc and code of class RelHint and SqlHint Key: CALCITE-3523 URL: https://issues.apache.org/jira/browse/CALCITE-3523 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 * Add more doc the RelHint and SqlHint * Add validation for SqlHint -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
Danny Chen created CALCITE-3531: --- Summary: AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic Key: CALCITE-3531 URL: https://issues.apache.org/jira/browse/CALCITE-3531 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Fix For: 1.22.0 Now AggregateProjectPullUpConstantsRule simplify the query: {code:sql} select hiredate from sales.emp where sal is null and hiredate = current_timestamp group by sal, hiredate having count(*) > 3 {code} from plan: {code:xml} LogicalProject(HIREDATE=[$1]) LogicalFilter(condition=[>($2, 3)]) LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()]) LogicalProject(SAL=[$5], HIREDATE=[$4]) LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code} to plan: {code:xml} LogicalProject(HIREDATE=[$1]) LogicalFilter(condition=[>($2, 3)]) LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1]) LogicalAggregate(group=[{0}], agg#0=[COUNT()]) LogicalProject(SAL=[$5], HIREDATE=[$4]) LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) {code} which is unsafe, because for stream sql, we need to group data by day, also the result is wrong is a batch job runs across days. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3534) Support parse(unparse) alien system non-standard data type
Danny Chen created CALCITE-3534: --- Summary: Support parse(unparse) alien system non-standard data type Key: CALCITE-3534 URL: https://issues.apache.org/jira/browse/CALCITE-3534 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 For some systems, they have customized SQL type that does not belong to standard SQL, for example, UNSIGNED is a built-in type in MySQL which is synonym of INTEGER. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3550) Make SqlTypeAssignmentRules conversion mapping pluggable
Danny Chen created CALCITE-3550: --- Summary: Make SqlTypeAssignmentRules conversion mapping pluggable Key: CALCITE-3550 URL: https://issues.apache.org/jira/browse/CALCITE-3550 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Fix For: 1.22.0 In CALCITE-2302, we introduce implicit type coercion for Calcite builtin conversion behaviors, we also expand the conversion matrix to support more type conversion compared to the old matrix [1], that means, we implement a conversion matrix of our Calcite style. As a library of other sql engines, i think we should make the conversion matrix pluggable, so that each engine can customize if the conversion is valid from a type to another. We have made the TypeCoercion pluggable/customizable, so if this matrix is also pluggable, then, the whole conversion behavior can be customized. That would give the type conversion of Calcite a pretty good flexibility. [1] https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3584) Support SQL hints propagation for decorrelation
Danny Chen created CALCITE-3584: --- Summary: Support SQL hints propagation for decorrelation Key: CALCITE-3584 URL: https://issues.apache.org/jira/browse/CALCITE-3584 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 In CALCITE-482, we have support a basic SQL hints framework to let other engines to extend and have their own hints implementations. But for the decorrelation, the hints would just lost. We should support that because most of the sub-queries needs decorrelation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3608) Promote RelOptUtil.createCastRel to not create new projection if the input rel is already a project
Danny Chen created CALCITE-3608: --- Summary: Promote RelOptUtil.createCastRel to not create new projection if the input rel is already a project Key: CALCITE-3608 URL: https://issues.apache.org/jira/browse/CALCITE-3608 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3620) Remove implicit lateral operator for temporal table join
Danny Chen created CALCITE-3620: --- Summary: Remove implicit lateral operator for temporal table join Key: CALCITE-3620 URL: https://issues.apache.org/jira/browse/CALCITE-3620 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Just as what CALCITE-3603 comment addressed, LATERAL and temporal table does not have any direct relational-ship in standard SQL, we should remove that binding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3635) Supports hint option as string or numeric literal
Danny Chen created CALCITE-3635: --- Summary: Supports hint option as string or numeric literal Key: CALCITE-3635 URL: https://issues.apache.org/jira/browse/CALCITE-3635 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Support hint item syntax as: {code:sql} select /*+ time_zone(1, 1.23, 'a bc', -1.0) */ * from emp {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3649) Hints should be propagated correctly in planner rules if original node is transformed to different kind
Danny Chen created CALCITE-3649: --- Summary: Hints should be propagated correctly in planner rules if original node is transformed to different kind Key: CALCITE-3649 URL: https://issues.apache.org/jira/browse/CALCITE-3649 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 If the AGG was transformed to PROJECT + AGG, the hints of AGG should be propagated to the AGG node. In current implementation, the hints would lost. The perfect solution is to identify the replaced sub-tree, and for this tree, we should check which node is the right one to attach hints. But for this patch, we only consider the new transformed node with pattern of PROJECT + "node of same kind of original", this solves most of the cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3672) Support implicit type coercion for insert and update
Danny Chen created CALCITE-3672: --- Summary: Support implicit type coercion for insert and update Key: CALCITE-3672 URL: https://issues.apache.org/jira/browse/CALCITE-3672 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3715) Add a interface to pass the table hints to RelOptTable
Danny Chen created CALCITE-3715: --- Summary: Add a interface to pass the table hints to RelOptTable Key: CALCITE-3715 URL: https://issues.apache.org/jira/browse/CALCITE-3715 Project: Calcite Issue Type: Sub-task Components: core, csv-adapter Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 In sql to rel converter, we pass a ToRelContext to the AbstractPreparingTable, in order to translate the table to a relational expression. We should also pass the hints to the table so that other engines can use the hints to create their source/reader. I'm so sorry to say that i have no idea how to write tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3719) Add customizable hint error handler
Danny Chen created CALCITE-3719: --- Summary: Add customizable hint error handler Key: CALCITE-3719 URL: https://issues.apache.org/jira/browse/CALCITE-3719 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Based on the fact that most of the vendors does not throws for hints error, we should keep sync with them. This patch also make the error behaviors customizable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3728) Eliminate the warnings for Gradle build task
Danny Chen created CALCITE-3728: --- Summary: Eliminate the warnings for Gradle build task Key: CALCITE-3728 URL: https://issues.apache.org/jira/browse/CALCITE-3728 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Vladimir Sitnikov Fix For: 1.22.0 Here is the terminal outputs when executing "gradle build -x test" under /core {code:xml} > Task :core:test Gradle Test Executor 1 STANDARD_ERROR 一月 13, 2020 11:04:01 上午 org.junit.jupiter.engine.config.EnumConfigurationParameterConverter get 信息: Using parallel execution mode 'CONCURRENT' set via the 'junit.jupiter.execution.parallel.mode.default' configuratn parameter. org.apache.calcite.jdbc.CalciteRemoteDriverTest STANDARD_ERROR 一月 13, 2020 11:04:01 上午 org.junit.jupiter.engine.config.EnumConfigurationParameterConverter get 信息: Using parallel execution mode 'CONCURRENT' set via the 'junit.jupiter.execution.parallel.mode.default' configuratn parameter. org.apache.calcite.test.SqlToRelConverterTest > testGroupingSetsWithNestedColumn() STANDARD_OUT 2020-01-13 11:04:02,368 [ForkJoinPool-1-worker-1] INFO - Logging initialized @2657ms to org.eclipse.jetty.util.log.Slf4jLog 2020-01-13 11:04:02,782 [ForkJoinPool-1-worker-1] INFO - jetty-9.4.15.v20190215; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826fa0b; jvm 1.8.0_161-b12 2020-01-13 11:04:02,834 [ForkJoinPool-1-worker-1] INFO - Started ServerConnector@3dd3565c{HTTP/1.1,[http/1.1]}{0.0.0.0:61478} 2020-01-13 11:04:02,835 [ForkJoinPool-1-worker-1] INFO - Started @3127ms 2020-01-13 11:04:02,835 [ForkJoinPool-1-worker-1] INFO - Service listening on port 61478. org.apache.calcite.jdbc.CalciteRemoteDriverTest > testRemotePreparedStatementFetch() STANDARD_OUT 2020-01-13 11:04:03,084 [ForkJoinPool-1-worker-7] INFO - open start - state modified 1.6sec,2 completed, 0 failed, 0 skipped, org.apache.calcite.test.UdtTest org.apache.calcite.jdbc.CalciteRemoteDriverTest STANDARD_OUT 2020-01-13 11:04:05,626 [ForkJoinPool-1-worker-1] INFO - Service terminating. 2020-01-13 11:04:05,637 [ForkJoinPool-1-worker-1] INFO - Stopped ServerConnector@3dd3565c{HTTP/1.1,[http/1.1]}{0.0.0.0:0} 3.7sec, 28 completed, 0 failed, 0 skipped, org.apache.calcite.jdbc.CalciteRemoteDriverTest 0.0sec,1 completed, 0 failed, 0 skipped, org.apache.calcite.TestKtTest 0.1sec, 16 completed, 0 failed, 0 skipped, org.apache.calcite.test.SqlValidatorMatchTest WARNING 3.8sec, 19 completed, 0 failed, 1 skipped, org.apache.calcite.test.JdbcFrontLinqBackTest 0.0sec,5 completed, 0 failed, 0 skipped, org.apache.calcite.test.SqlValidatorDynamicTest 0.0sec,7 completed, 0 failed, 0 skipped, org.apache.calcite.sql.type.RelDataTypeSystemTest 0.0sec,7 completed, 0 failed, 0 skipped, org.apache.calcite.sql.type.SqlTypeUtilTest 0.0sec,9 completed, 0 failed, 0 skipped, org.apache.calcite.sql.type.SqlTypeFactoryTest WARNING 2.1sec, 373 completed, 0 failed, 4 skipped, org.apache.calcite.sql.parser.SqlParserTest 3.1sec, org.apache.calcite.test.CoreQuidemTest > test(String)[2], [2] sql/winagg.iq WARNING 2.6sec, 373 completed, 0 failed, 6 skipped, org.apache.calcite.sql.parser.SqlUnParserTest WARNING 0.9sec, 377 completed, 0 failed, 5 skipped, org.apache.calcite.sql.parser.parserextensiontesting.ExtensionSqlParserTest WARNING 6.0sec, 266 completed, 0 failed, 1 skipped, org.apache.calcite.sql.test.SqlOperatorTest 0.5sec,2 completed, 0 failed, 0 skipped, org.apache.calcite.sql.test.DocumentationTest WARNING 0.1sec, 40 completed, 0 failed, 1 skipped, org.apache.calcite.sql.test.SqlPrettyWriterTest WARNING 0.4sec, 37 completed, 0 failed, 3 skipped, org.apache.calcite.sql.test.SqlAdvisorTest 0.0sec, 36 completed, 0 failed, 0 skipped, org.apache.calcite.sql.test.SqlTypeNameTest 0.0sec,4 completed, 0 failed, 0 skipped, org.apache.calcite.sql.SqlSetOptionOperatorTest 0.0sec,8 completed, 0 failed, 0 skipped, org.apache.calcite.sql.validate.SqlValidatorUtilTest 0.0sec,5 completed, 0 failed, 0 skipped, org.apache.calcite.sql.validate.LexEscapeTest 0.3sec, 15 completed, 0 failed, 0 skipped, org.apache.calcite.sql.validate.LexCaseSensitiveTest org.apache.calcite.materialize.LatticeSuggesterTest > testAggregateExpression() STANDARD_OUT 2020-01-13 11:04:18,980 [ForkJoinPool-1-worker-7] INFO - Checkpoint start 2020-01-13 11:04:18,980 [ForkJoinPool-1-worker-7] INFO - Checkpoint end - txts: 279 org.apache.calcite.test.SqlToRelConverterTest > testJoinUsing() S
[jira] [Created] (CALCITE-3730) Add hints to RelBuilder
Danny Chen created CALCITE-3730: --- Summary: Add hints to RelBuilder Key: CALCITE-3730 URL: https://issues.apache.org/jira/browse/CALCITE-3730 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3736) Add an interface in RelOptRuleCall to customize the propagation of hints before registering into planner rule
Danny Chen created CALCITE-3736: --- Summary: Add an interface in RelOptRuleCall to customize the propagation of hints before registering into planner rule Key: CALCITE-3736 URL: https://issues.apache.org/jira/browse/CALCITE-3736 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3747) Constructing BETWEEN with RelBuilder throws class cast exception
Danny Chen created CALCITE-3747: --- Summary: Constructing BETWEEN with RelBuilder throws class cast exception Key: CALCITE-3747 URL: https://issues.apache.org/jira/browse/CALCITE-3747 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Fix the return type inference of BETWEEN operator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3765) Returns early when there is an existing operand when assigning operands solve order
Danny Chen created CALCITE-3765: --- Summary: Returns early when there is an existing operand when assigning operands solve order Key: CALCITE-3765 URL: https://issues.apache.org/jira/browse/CALCITE-3765 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 I found this when i read the planner code. Hmm ~ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3766) Add a Builder to RelHint
Danny Chen created CALCITE-3766: --- Summary: Add a Builder to RelHint Key: CALCITE-3766 URL: https://issues.apache.org/jira/browse/CALCITE-3766 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Add a builder to RelHint to constructor the it conveniently. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3769) Deprecate TableScanRule
Danny Chen created CALCITE-3769: --- Summary: Deprecate TableScanRule Key: CALCITE-3769 URL: https://issues.apache.org/jira/browse/CALCITE-3769 Project: Calcite Issue Type: Wish Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 The TableScanRule is the only planner rule that for a logical node(e.g. the table scan), it’s functionality is to pass along the cluster object and invoke the RelOptTable#toRel. It’s function is very trivial because it supplies only a ToRelContext that does not support expanding view/passing table hints. For rels that come from the sql-to-rel conversion, there is already a table conversion logic[1]. This code gives a powerful ToRelContext that has the complete functionality. The only reason that I saw the meaning of existing TableScanRule is for the TableScan that comes from the RelBuilder#scan. So I would suggest to deprecate the TableScanRule, instead, we support translating the table directly in RelBuilder#scan, We also add a new interface RelBuilder#scan(Iterable tableNames, ToRelContext context), so that we can pass in a more powerful ToRelContext explicitly. [1] https://github.com/apache/calcite/blob/d6fa25cd11625ad7b4b74dafbd0211c701b38d49/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L3498 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3781) HintStrategy can specify excluded rules for planner
Danny Chen created CALCITE-3781: --- Summary: HintStrategy can specify excluded rules for planner Key: CALCITE-3781 URL: https://issues.apache.org/jira/browse/CALCITE-3781 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 It is efficient to support exclude rules explicitly for some hints, i.e. USE_MERGE, NO_USE_HASH. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3792) Remove the generic type declaration of method RelOptCluster#getMetadataQuery
Danny Chen created CALCITE-3792: --- Summary: Remove the generic type declaration of method RelOptCluster#getMetadataQuery Key: CALCITE-3792 URL: https://issues.apache.org/jira/browse/CALCITE-3792 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 The type inference of Scala code 'val mq = cluster.getMetadataQuery' is failed if we have that generic type declaration for the Java method -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3793) AssertionError throws for planner digest cache key when the rel type is not struct
Danny Chen created CALCITE-3793: --- Summary: AssertionError throws for planner digest cache key when the rel type is not struct Key: CALCITE-3793 URL: https://issues.apache.org/jira/browse/CALCITE-3793 Project: Calcite Issue Type: Improvement Reporter: Danny Chen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3795) Promote the Project digest to only print "0" instead of "0..0" when there is only one trivial field
Danny Chen created CALCITE-3795: --- Summary: Promote the Project digest to only print "0" instead of "0..0" when there is only one trivial field Key: CALCITE-3795 URL: https://issues.apache.org/jira/browse/CALCITE-3795 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3797) RelMdPredicates should not assume the RexExecutor instance is RexExecutorImpl
Danny Chen created CALCITE-3797: --- Summary: RelMdPredicates should not assume the RexExecutor instance is RexExecutorImpl Key: CALCITE-3797 URL: https://issues.apache.org/jira/browse/CALCITE-3797 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 The RexExecutor is actually pluggable, we should do such an assumption. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3798) Make RelBuilder view expander pluggable
Danny Chen created CALCITE-3798: --- Summary: Make RelBuilder view expander pluggable Key: CALCITE-3798 URL: https://issues.apache.org/jira/browse/CALCITE-3798 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 User can config a view expander directly when constructing the RelBuilder. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3799) Following CALCITE-3769, fix the slow tests
Danny Chen created CALCITE-3799: --- Summary: Following CALCITE-3769, fix the slow tests Key: CALCITE-3799 URL: https://issues.apache.org/jira/browse/CALCITE-3799 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3800) FileReaderTest#testFileReaderUrlNoPath() timeout for AppVeyor env
Danny Chen created CALCITE-3800: --- Summary: FileReaderTest#testFileReaderUrlNoPath() timeout for AppVeyor env Key: CALCITE-3800 URL: https://issues.apache.org/jira/browse/CALCITE-3800 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 The timeout is annoying, so i would disable it until we find a solution ~ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3801) Deprecate SqlToRelConverter.Config#isConvertTableAccess
Danny Chen created CALCITE-3801: --- Summary: Deprecate SqlToRelConverter.Config#isConvertTableAccess Key: CALCITE-3801 URL: https://issues.apache.org/jira/browse/CALCITE-3801 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 Because of CALCITE-3769, this config option is actually useless now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3805) Add a new method to control the agg input prune with explicit flag
Danny Chen created CALCITE-3805: --- Summary: Add a new method to control the agg input prune with explicit flag Key: CALCITE-3805 URL: https://issues.apache.org/jira/browse/CALCITE-3805 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 This feature is introduced by CALCITE-3763, which is cool for normal group aggregations. But in Flink, we have window group aggregation, we invoke the normal aggregate first then construct our LogicalWindowAggregate, the window may have some attributes that reference the pruned columns. I though about how i can control the prune flexibility, but this behavior is configured by the whole RelBuilder.Config, what i what is only forbidden this behavior when i construct the window aggregate, i still want this feature for normal aggregations. So, i propose to add a new method: {code:java} RelBuilder aggregate( GroupKey groupKey, Iterable aggCalls, boolean pruneInputOfAggregate) {code}. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3808) Release Calcite 1.22.0
Danny Chen created CALCITE-3808: --- Summary: Release Calcite 1.22.0 Key: CALCITE-3808 URL: https://issues.apache.org/jira/browse/CALCITE-3808 Project: Calcite Issue Type: Task Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3812) Refactor HintStrategy to HintPredicate
Danny Chen created CALCITE-3812: --- Summary: Refactor HintStrategy to HintPredicate Key: CALCITE-3812 URL: https://issues.apache.org/jira/browse/CALCITE-3812 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 The HintStrategy was introduced in commit https://github.com/apache/calcite/commit/52bc246d5e07510689635f407f4d9dc494b0af4d, at that moment, we can only define the hint match rules for propagation, and the class name is okey. But with more components added to the HintStrategyTable(i.e. the HintOptionChecker, the error hander, the excluded rules), i found the "HintStrategy" is mis-leading. So i propose to rename HintStrategy to HintPredicate and HintStrategyTable.Entry to HintStrategy. I want this change to be included in 1.22.0. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3813) Support hint option key as string literal
Danny Chen created CALCITE-3813: --- Summary: Support hint option key as string literal Key: CALCITE-3813 URL: https://issues.apache.org/jira/browse/CALCITE-3813 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3818) Upgrade Avatica version to 1.16.0
Danny Chen created CALCITE-3818: --- Summary: Upgrade Avatica version to 1.16.0 Key: CALCITE-3818 URL: https://issues.apache.org/jira/browse/CALCITE-3818 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.22.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3847) Decorrelation for join with lateral table outputs wrong plan if the join condition contains correlation variables
Danny Chen created CALCITE-3847: --- Summary: Decorrelation for join with lateral table outputs wrong plan if the join condition contains correlation variables Key: CALCITE-3847 URL: https://issues.apache.org/jira/browse/CALCITE-3847 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 After CALCITE-2004, we can decorrelate plan for query: {code:sql} select deptno, r.num from dept join lateral table(ramp(dept.deptno)) as r(num) on deptno=num {code} to plan like: {code:xml} LogicalProject(DEPTNO=[$0], NUM=[$2]) LogicalJoin(condition=[=($0, $2)], joinType=[inner]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) LogicalTableFunctionScan(invocation=[RAMP($cor0.DEPTNO)], rowType=[RecordType(INTEGER I)]) {code} See there was a correlate variable left. To fix this, i would suggest to stop decorrelating when we encounter the correlate variables in LogicalTableFunctionScan. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3855) Supports snapshot on table with virtual columns during sql-to-rel conversion
Danny Chen created CALCITE-3855: --- Summary: Supports snapshot on table with virtual columns during sql-to-rel conversion Key: CALCITE-3855 URL: https://issues.apache.org/jira/browse/CALCITE-3855 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3856) Remove code to be removed before 1.23
Danny Chen created CALCITE-3856: --- Summary: Remove code to be removed before 1.23 Key: CALCITE-3856 URL: https://issues.apache.org/jira/browse/CALCITE-3856 Project: Calcite Issue Type: Task Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3871) Remove dependency of org.apiguardian:apiguardian-api
Danny Chen created CALCITE-3871: --- Summary: Remove dependency of org.apiguardian:apiguardian-api Key: CALCITE-3871 URL: https://issues.apache.org/jira/browse/CALCITE-3871 Project: Calcite Issue Type: Improvement Components: core, linq4j Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 The org.apiguardian:apiguardian-api is introduced in CALCITE-3652 in order to mark the new introduced API status. Remove the dependency and copy the class into Calcite because the org.apiguardian:apiguardian-api jar has only a single API.java class and it is not necessary to add a dependency for that(All the downstream projects that have calcite-core as a dependency would see this jar which is annoying). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3900) Add Config for SqlValidator
Danny Chen created CALCITE-3900: --- Summary: Add Config for SqlValidator Key: CALCITE-3900 URL: https://issues.apache.org/jira/browse/CALCITE-3900 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 The SqlValidator now has 7 setXXX methods to all kinds of control flags, which is hard to code evolving. There is also no way to config these things through the FrameworkConfig. Add a SqlValidator.Config to solves these problems. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3931) Add LOOKAHEAD(2) for methods defined in createStatementParserMethods
Danny Chen created CALCITE-3931: --- Summary: Add LOOKAHEAD(2) for methods defined in createStatementParserMethods Key: CALCITE-3931 URL: https://issues.apache.org/jira/browse/CALCITE-3931 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 The default LOOKAHEAD is 1 which is very probably to conflict, especially for custom parse block like SqlCreate. Sets the LOOKAHEAD(2) to reduce conflict. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3934) Supports implicit type coercion for operators with repeated type families
Danny Chen created CALCITE-3934: --- Summary: Supports implicit type coercion for operators with repeated type families Key: CALCITE-3934 URL: https://issues.apache.org/jira/browse/CALCITE-3934 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 Supports implicit type coercion for operators with repeated type families, such as: {code:sql} CONCAT(123, 'abc', DATE '2020-04-17') {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3940) Hint item can not parse correctly if the name is right after token "/*+"
Danny Chen created CALCITE-3940: --- Summary: Hint item can not parse correctly if the name is right after token "/*+" Key: CALCITE-3940 URL: https://issues.apache.org/jira/browse/CALCITE-3940 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 After parsing /*+OPTIONS('k1'='v1', 'k2'='v2')*/, the parsed hint name is "PTIONS", while it is expected to be "OPTIONS". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3942) Move type-coercion configurations into SqlValidator.Config
Danny Chen created CALCITE-3942: --- Summary: Move type-coercion configurations into SqlValidator.Config Key: CALCITE-3942 URL: https://issues.apache.org/jira/browse/CALCITE-3942 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 SqlValidator.Config is the new role to config all kinds of configurations of SqlValidator. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3943) Remove the JSON functions keyword from parser
Danny Chen created CALCITE-3943: --- Summary: Remove the JSON functions keyword from parser Key: CALCITE-3943 URL: https://issues.apache.org/jira/browse/CALCITE-3943 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 The JSON functions was introduced in CALCITE-2266, we coded the JSON keywords into the parser and generates the operator directly, so there is no chance for downstream projects to change the builtin operator and do some override. This issue tries to remove these keywords from the parser, we should always generates SqlUnresolvedFunction instead specific operators in the SQL parser. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3954) Always compare types using equals
Danny Chen created CALCITE-3954: --- Summary: Always compare types using equals Key: CALCITE-3954 URL: https://issues.apache.org/jira/browse/CALCITE-3954 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3955) Remove the first operand of RexCall from SqlWindowTableFunction
Danny Chen created CALCITE-3955: --- Summary: Remove the first operand of RexCall from SqlWindowTableFunction Key: CALCITE-3955 URL: https://issues.apache.org/jira/browse/CALCITE-3955 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.22.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 In CALCITE-3382, we introduced TUMBLE window function to replace the deprecated group tumble window. But for query {code:sql} select * from table(tumble(table Shipments, descriptor(rowtime), INTERVAL '1' MINUTE)) {code} the outputs plan is {code:xml} LogicalProject(ORDERID=[$0], ROWTIME=[$1], window_start=[$2], window_end=[$3]) LogicalTableFunctionScan(invocation=[TUMBLE($1, DESCRIPTOR($1), 6:INTERVAL MINUTE)], rowType=[RecordType(INTEGER ORDERID, TIMESTAMP(0) ROWTIME, TIMESTAMP(0) window_start, TIMESTAMP(0) window_end)]) LogicalProject(ORDERID=[$0], ROWTIME=[$1]) LogicalTableScan(table=[[CATALOG, SALES, SHIPMENTS]]) {code} The first operand of TUMBLE rex call is always the last input field, but actually it represents the source table which is the input rel node. This issue remove the first operand from the RexCall because it is useless and confusing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3962) Make JSON_VALUE operands varadic
Danny Chen created CALCITE-3962: --- Summary: Make JSON_VALUE operands varadic Key: CALCITE-3962 URL: https://issues.apache.org/jira/browse/CALCITE-3962 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.22.0, 1.21.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.23.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4031) Remove code to be removed before 1.24
Danny Chen created CALCITE-4031: --- Summary: Remove code to be removed before 1.24 Key: CALCITE-4031 URL: https://issues.apache.org/jira/browse/CALCITE-4031 Project: Calcite Issue Type: Task Components: core Affects Versions: 1.23.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.24.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4070) Add benchmark test for new Digest interface
Danny Chen created CALCITE-4070: --- Summary: Add benchmark test for new Digest interface Key: CALCITE-4070 URL: https://issues.apache.org/jira/browse/CALCITE-4070 Project: Calcite Issue Type: Sub-task Components: core Affects Versions: 1.24.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.24.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4073) Add a new component RexNormalize for more effect rex nodes normalization
Danny Chen created CALCITE-4073: --- Summary: Add a new component RexNormalize for more effect rex nodes normalization Key: CALCITE-4073 URL: https://issues.apache.org/jira/browse/CALCITE-4073 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.24.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.24.0 Currently, we only have simple normalization such as: $2 = $1 => $1 = $2 $2 > $1 => $1 < $2 we can extend the logic to more like: - put the complex predicates behind, like OR(OR(udf($1), $2), $3) => OR($3, OR(udf($1), $2)) - put the udf operand behind, like OR(udf($1), $2) => OR($2, udf($1)) - put the local variables that appears more frequently in the front, which is more user-friendly to code generation because the variable can be cached and reused. Also note that, we should allows flexible configuration for switch this function on/off(not a system calcite property now) for downstream users and projects(because the normalization may have some performance trade-off). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4115) Improve the prompt of using SQL keywords for sql parser
Danny Chen created CALCITE-4115: --- Summary: Improve the prompt of using SQL keywords for sql parser Key: CALCITE-4115 URL: https://issues.apache.org/jira/browse/CALCITE-4115 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.23.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.24.0 Current when a user uses a reserved-keyword in the SQL, for example: {code:sql} select DEFAULT from emp; {code} the parser would report with error message: {noformat} Encountered "DEFAULT" at line 1, column 8. Was expecting one of: "ABS" ... "ALL" ... "ARRAY" ... "AVG" ... "CARDINALITY" ... "CASE" ... "CAST" ... "CEIL" ... "CEILING" ... "CHAR_LENGTH" ... "CHARACTER_LENGTH" ... {noformat} It is hard to infer that the DEFAULT is actually a reserved-keyword, we can promote the message to make it more clear. For example, in SQL-SERVER, if you use such a keyword, the server reports "Incorrect syntax near the keyword 'DEFAULT'", which is very clear that the DEFAULT is a keyword. Based on the SQL-SERVER message, i would suggest to also report the position with a suggested fix solution, the template is like this: {code:java} Incorrect syntax near the keyword {keyword} at line {line_number}, column {column_number}.\n Either uses a non-reserved keyword or quotes the keyword with character: {quoting_character}. {code} We would tell user that the mistake is a keyword and its position, also how to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4145) Exception when query from UDF field with structured type
Danny Chen created CALCITE-4145: --- Summary: Exception when query from UDF field with structured type Key: CALCITE-4145 URL: https://issues.apache.org/jira/browse/CALCITE-4145 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.24.0, 1.23.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.25.0 Check this sql in {{SqlToRelConverterTest}} {code:java} @Test void testUdfWithStructuredReturnType() { final String sql = "SELECT deptno, tmp.r.not_null_field, tmp.r.nullable_field FROM\n" + "(SELECT deptno, ROW_FUNC() AS r from dept)tmp"; sql(sql).ok(); } {code} {{ArrayIndexOutOfBoundsException}} throws because the nested field got a -1 index when querying the field by simple name. To solve the problem, for a nested field, return its parent field's monotonicity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4167) Group by COALESCE IN throws NullPointerException
Danny Chen created CALCITE-4167: --- Summary: Group by COALESCE IN throws NullPointerException Key: CALCITE-4167 URL: https://issues.apache.org/jira/browse/CALCITE-4167 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.24.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.25.0 Added this test case in SqlToRelConverterTest: {code:java} @Test void testGroupByCoalesceIn() { final String sql = "select case when coalesce(ename, 'a') in ('1', '2')\n" + "then 'CKA' else 'QT' END, count(distinct deptno) from emp\n" + "group by case when coalesce(ename, 'a') in ('1', '2') then 'CKA' else 'QT' END"; sql(sql).ok(); } {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4171) Support named parameters for table window functions
Danny Chen created CALCITE-4171: --- Summary: Support named parameters for table window functions Key: CALCITE-4171 URL: https://issues.apache.org/jira/browse/CALCITE-4171 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.24.0, 1.23.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.25.0 Supports window functions like named params style: ```sql select * from table( session( data => table Shipments, timecol => descriptor(rowtime), key => descriptor(orderId), size => INTERVAL '10' MINUTE)) ``` -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4206) RelDecorrelator outputs wong plan for correlate sort with fetch limit
Danny Chen created CALCITE-4206: --- Summary: RelDecorrelator outputs wong plan for correlate sort with fetch limit Key: CALCITE-4206 URL: https://issues.apache.org/jira/browse/CALCITE-4206 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.25.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.26.0 For query {code:sql} -- create cities table CREATE TABLE cities ( name STRING NOT NULL, state STRING NOT NULL, pop INT NOT NULL ) WITH ( 'connector' = 'kafka', 'topic' = 'cities', 'properties.bootstrap.servers' = 'kafka:9092', 'properties.group.id' = 'mygroup', 'scan.startup.mode' = 'earliest-offset', 'format' = 'json' ); -- fill cities table INSERT INTO cities VALUES ('Los_Angeles', 'CA', 3979576), ('Phoenix', 'AZ', 1680992), ('Houston', 'TX', 2320268), ('San_Diego', 'CA', 1423851), ('San_Francisco', 'CA', 881549), ('New_York', 'NY', 8336817), ('Dallas', 'TX', 1343573), ('San_Antonio', 'TX', 1547253), ('San_Jose', 'CA', 1021795), ('Chicago', 'IL', 2695598), ('Austin', 'TX', 978908); -- execute query SELECT state, name FROM (SELECT DISTINCT state FROM cities) states, LATERAL ( SELECT name, pop FROM cities WHERE state = states.state ORDER BY pop DESC LIMIT 3 ); -- result state name CA Los_Angeles NY New_York IL Chicago -- expected result state | name --+- TX| Dallas AZ| Phoenix IL| Chicago TX| Houston CA| San_Jose NY| New_York CA| San_Diego CA| Los_Angeles TX| San_Antonio {code} The current plan after decorrelation is {code:xml} // {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4225) Make RelDecorrelator pluggable
Danny Chen created CALCITE-4225: --- Summary: Make RelDecorrelator pluggable Key: CALCITE-4225 URL: https://issues.apache.org/jira/browse/CALCITE-4225 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.25.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.26.0 {{ RelDecorrelator }} is out core component that decorrelates the queries. But actually, the pattern it can decorrelates successfully is very limited. And it often causes bug because it decorrelates into a wrong plan (with non-correct sementics). When there are bugs there, the downstream project can only wait for the version upgrade (if it is fixed in Calcite master), or they copy the whole Java class and override the Calcite one which is hard to maintain. It would be nice if we can make the decorrelation logic pluggable. e.g. we can control how a RelNode was decorrelated(not a coarse-grained flag to turn-on/off the decorrelation). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4282) Promote the window table functions window attribute data type with precision 3
Danny Chen created CALCITE-4282: --- Summary: Promote the window table functions window attribute data type with precision 3 Key: CALCITE-4282 URL: https://issues.apache.org/jira/browse/CALCITE-4282 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.25.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.26.0 Now the {{window_start}} and {{window_end}} has a type of a Timestamp with default system precision. But, actually, many DB vendors has default precision as 6 (which is also defined in the SQL standard). We better promote the precision to 3 because: 1. For windowing, time unit as millisecond is enough 2. Make the precision deterministic instead of overriding by each engine's default one -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4283) Promote the error message when creating table function scan with non table functions
Danny Chen created CALCITE-4283: --- Summary: Promote the error message when creating table function scan with non table functions Key: CALCITE-4283 URL: https://issues.apache.org/jira/browse/CALCITE-4283 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.25.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.26.0 Throw a more meaningful error message instead of a AssertionError by the explicit type cast. {code:java} final SqlTableFunction operator = (SqlTableFunction) call.getOperator(); {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4333) The Sort rel should be decorrelated even though it has fetch or limit when its parent is not a Correlate
Danny Chen created CALCITE-4333: --- Summary: The Sort rel should be decorrelated even though it has fetch or limit when its parent is not a Correlate Key: CALCITE-4333 URL: https://issues.apache.org/jira/browse/CALCITE-4333 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4337) Supports PARTITION BY clause for table function table argument
Danny Chen created CALCITE-4337: --- Summary: Supports PARTITION BY clause for table function table argument Key: CALCITE-4337 URL: https://issues.apache.org/jira/browse/CALCITE-4337 Project: Calcite Issue Type: New Feature Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 An example from the SQL standard 2016 Polymorphic Table Functions: {code:sql} SELECT W.wstart, W.wend, OI.customer, SUM(OI.price) FROM TABLE(SESSION( data => TABLE(order_item) AS OI PARTITION BY customer, timecol => DESCRIPTOR(order_time), timeout => INTERVAL '10' MINUTE)) W GROUP BY 1, 2, 3 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4364) "a in (1, 2) and a = 1" should be simplified to "a=1"
Danny Chen created CALCITE-4364: --- Summary: "a in (1, 2) and a = 1" should be simplified to "a=1" Key: CALCITE-4364 URL: https://issues.apache.org/jira/browse/CALCITE-4364 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Fix For: 1.27.0 Check this test in {{RexProgramTest}}: ```java @Test void testSimplifyInAnd() { // deptno in (20, 10) and deptno = 10 // ==> // deptno = 10 final RexNode e = and( in(vInt(), literal(20), literal(10)), eq(vInt(), literal(10))); checkSimplify(e, "=(?0.int0, 10)"); } ``` -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4377) Fix the return nullability inference during rex simplification
Danny Chen created CALCITE-4377: --- Summary: Fix the return nullability inference during rex simplification Key: CALCITE-4377 URL: https://issues.apache.org/jira/browse/CALCITE-4377 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 Check test of {{RexProgramTest.reproducerFor3457}}. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4396) Allows table valued functions of FROM and JOIN context without TABLE clause
Danny Chen created CALCITE-4396: --- Summary: Allows table valued functions of FROM and JOIN context without TABLE clause Key: CALCITE-4396 URL: https://issues.apache.org/jira/browse/CALCITE-4396 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 In current code base, user can use a table valued function mainly for 2 kinds of SQL statements: {code:sql} -- select from the table valued function directly SELECT ... FROM TABLE(my_tvf(...)); -- join the table valued function with LATERAL keyword ... t1 JOIN LATERAL TABLE(my_tvf(...)) ON ... {code} While since version 12.1+, Oracle supports syntax without TABLE keyword, thus you can write SQL as following: {code:sql} -- select from the table valued function directly SELECT ... FROM my_tvf(...); -- join the table valued function with LATERAL keyword ... t1 JOIN my_tvf(...) ON ... {code} SQL-SERVER also allows select from the table valued function directly without the explicit TABLE keyword, but they need the CROSS APPLY clause for table and function join. [2] We already had a discussion on the DEV mailing list, see [3]. I propose to support the Oracle style table valued function syntax: to omit the TABLE keyword because the syntax is straight-forward and concise. Of course, this syntax is only valid under Oracle SQL conformance. [1] https://livesql.oracle.com/apex/livesql/file/tutorial_GSOTSK8FWLYZOG5CJJ9KPX7RX.html [2] https://docs.microsoft.com/en-us/sql/relational-databases/user-defined-functions/create-user-defined-functions-database-engine?view=sql-server-ver15#TVF [3] https://lists.apache.org/x/thread.html/ra98db08e280ddd9adeef62f456f61aedfdf7756e215cb4d66e2a52c9@%3Cdev.calcite.apache.org%3E -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4406) SqlTableRef OPERATOR should create a SqlTableRef as the call
Danny Chen created CALCITE-4406: --- Summary: SqlTableRef OPERATOR should create a SqlTableRef as the call Key: CALCITE-4406 URL: https://issues.apache.org/jira/browse/CALCITE-4406 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 {{SqlTableRef}} OPERATOR should create a SqlTableRef as the call, so that after substitution of {{SqlShuttle}}, it is still a {{SqlTableRef}} instead of {{SqlBasicCall}}. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4456) Allows all the value expressions for explicit row value constructor
Danny Chen created CALCITE-4456: --- Summary: Allows all the value expressions for explicit row value constructor Key: CALCITE-4456 URL: https://issues.apache.org/jira/browse/CALCITE-4456 Project: Calcite Issue Type: Task Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 In SQL standard 2011 part 7 value expression, it defines the SQL syntax for explicit ROW value constructor to allows all the valid value expreesion: {code:sql} ::= | | ::= | | | | | | ::= | ROW | ::= {code} I also tried the PostgreSQL 9.6 for the query: {code:sql} create table t1( f0 int, f1 varchar(20) ); insert into t1 values(1, 'abc'); -- select row(t1.f0 + 1, t1.f1) from t1; -- outputs (2, abc) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4473) Deprecate the config option TARGET_SCHEMA_FILE_PROP for HoodieFlinkStreamer
Danny Chen created CALCITE-4473: --- Summary: Deprecate the config option TARGET_SCHEMA_FILE_PROP for HoodieFlinkStreamer Key: CALCITE-4473 URL: https://issues.apache.org/jira/browse/CALCITE-4473 Project: Calcite Issue Type: Improvement Reporter: Danny Chen The option {{hoodie.deltastreamer.schemaprovider.source.schema.file}} is confusing and copied from the {{HoodieDeltaStreamer}}, we better give it a more readable name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4479) "vFloat in (1.0, 2.0)" throws UnsupportedOperationException
Danny Chen created CALCITE-4479: --- Summary: "vFloat in (1.0, 2.0)" throws UnsupportedOperationException Key: CALCITE-4479 URL: https://issues.apache.org/jira/browse/CALCITE-4479 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: Danny Chen Assignee: Danny Chen Fix For: 1.27.0 Check this test in {{RexBuilderTest}}: {code:java} @Test void testMakeIn() { final RelDataTypeFactory typeFactory = new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT); final RexBuilder rexBuilder = new RexBuilder(typeFactory); final RelDataType floatType = typeFactory.createSqlType(SqlTypeName.FLOAT); RexNode left = rexBuilder.makeInputRef(floatType, 0); final RexNode literal1 = rexBuilder.makeLiteral(1.0f, floatType); final RexNode literal2 = rexBuilder.makeLiteral(2.0f, floatType); RexNode inCall = rexBuilder.makeIn(left, ImmutableList.of(literal1, literal2)); assertThat(inCall.getKind(), is(SqlKind.SEARCH)); } {code} The stacktrace is: {noformat} class org.apache.calcite.sql.type.SqlTypeName: FLOAT java.lang.UnsupportedOperationException: class org.apache.calcite.sql.type.SqlTypeName: FLOAT at org.apache.calcite.util.Util.needToImplement(Util.java:1085) at org.apache.calcite.rex.RexLiteral.appendAsJava(RexLiteral.java:726) at org.apache.calcite.rex.RexLiteral.toJavaString(RexLiteral.java:427) at org.apache.calcite.rex.RexLiteral.computeDigest(RexLiteral.java:289) at org.apache.calcite.rex.RexLiteral.(RexLiteral.java:233) at org.apache.calcite.rex.RexLiteral.toLiteral(RexLiteral.java:762) at org.apache.calcite.rex.RexLiteral.lambda$printSarg$4(RexLiteral.java:733) at org.apache.calcite.util.RangeSets$Printer.singleton(RangeSets.java:409) at org.apache.calcite.util.RangeSets.forEach(RangeSets.java:249) at org.apache.calcite.util.Sarg.lambda$printTo$0(Sarg.java:119) at org.apache.calcite.linq4j.Ord.forEach(Ord.java:157) at org.apache.calcite.util.Sarg.printTo(Sarg.java:115) at org.apache.calcite.rex.RexLiteral.printSarg(RexLiteral.java:732) at org.apache.calcite.rex.RexLiteral.lambda$appendAsJava$1(RexLiteral.java:673) at org.apache.calcite.util.Util.asStringBuilder(Util.java:2525) at org.apache.calcite.rex.RexLiteral.appendAsJava(RexLiteral.java:672) at org.apache.calcite.rex.RexLiteral.toJavaString(RexLiteral.java:427) at org.apache.calcite.rex.RexLiteral.computeDigest(RexLiteral.java:289) at org.apache.calcite.rex.RexLiteral.(RexLiteral.java:233) at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:990) at org.apache.calcite.rex.RexBuilder.makeSearchArgumentLiteral(RexBuilder.java:1085) at org.apache.calcite.rex.RexBuilder.makeIn(RexBuilder.java:1335) at org.apache.calcite.rex.RexBuilderTest.testMakeIn(RexBuilderTest.java:621) {noformat} The root cause is that {{RexLiteral#strictTypeName}} has different type name strategies with what {{RexBuilder.makeLiteral}} follows, the best way to fix is to make the rules synced, but here i only give a simple fix because the code path only used for Sarg digest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4507) 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
Danny Chen created CALCITE-4507: --- Summary: 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)' Key: CALCITE-4507 URL: https://issues.apache.org/jira/browse/CALCITE-4507 Project: Calcite Issue Type: Bug Reporter: Danny Chen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4565) UpperBound and lowerBound are silently re-written incorrectly during sql to rel conversion
Danny Chen created CALCITE-4565: --- Summary: UpperBound and lowerBound are silently re-written incorrectly during sql to rel conversion Key: CALCITE-4565 URL: https://issues.apache.org/jira/browse/CALCITE-4565 Project: Calcite Issue Type: Bug Components: core Affects Versions: next Reporter: Danny Chen -- This message was sent by Atlassian Jira (v8.3.4#803005)