[jira] [Commented] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813013#comment-16813013
]
Haisheng Yuan commented on CALCITE-1338:
Good idea, that seems much simpler.
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813008#comment-16813008
]
Hongze Zhang commented on CALCITE-1338:
---
It sounds to be a little tricky if we add an {{Operator#isStrict()}} method and
use that method to check a whole Rex tree's strictness. Do you have a plan?
[~Chunwei Lei]
Or maybe we can just use {{RexSimplify}} to check if a Rex tree could be
simplified to null literal within all input references replaced by nulls? I
think that should be safe in most cases.
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang reassigned CALCITE-1338:
-
Assignee: Chunwei Lei (was: Hongze Zhang)
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongze Zhang reassigned CALCITE-1338:
-
Assignee: Hongze Zhang (was: Chunwei Lei)
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Hongze Zhang
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812998#comment-16812998
]
Chunwei Lei commented on CALCITE-1338:
--
{quote}We may need a function in RexUtil to check the strictness of a RexNode.
Simply check literal or rexcall is not enough, e.g. 1+2. If there is no column
reference, do not pull up. We need to check the whole rex node tree.
{quote}
Make sense. Checking the whole rex node tree is necessary.
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812998#comment-16812998
]
Chunwei Lei edited comment on CALCITE-1338 at 4/9/19 4:11 AM:
--
{quote}We may need a function in RexUtil to check the strictness of a RexNode.
Simply check literal or rexcall is not enough, e.g. 1+2. If there is no column
reference, do not pull up. We need to check the whole rex node tree.
{quote}
Make sense. Checking the whole rex node tree is necessary.
was (Author: chunwei lei):
{quote}We may need a function in RexUtil to check the strictness of a RexNode.
Simply check literal or rexcall is not enough, e.g. 1+2. If there is no column
reference, do not pull up. We need to check the whole rex node tree.
{quote}
Make sense. Checking the whole rex node tree is necessary.
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812997#comment-16812997
]
Haisheng Yuan commented on CALCITE-1338:
I don't know if there is any better solution than adding strict field to
SqlOperator. So I agree.
We may need a function in RexUtil to check the strictness of a RexNode. Simply
check literal or rexcall is not enough, e.g. 1+2. If there is no column
reference, do not pull up. We need to check the whole rex node tree.
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2872) Support for specifying encoding in JSON representation clause
[
https://issues.apache.org/jira/browse/CALCITE-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812990#comment-16812990
]
Hongze Zhang commented on CALCITE-2872:
---
I am not sure currently but SQL:2016 std does not introduce keywords such as
{{UTF16BE}} / {{UTF16LE}} or something.
> Support for specifying encoding in JSON representation clause
> -
>
> Key: CALCITE-2872
> URL: https://issues.apache.org/jira/browse/CALCITE-2872
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Hongze Zhang
>Priority: Major
>
> Syntax of {{JSON representation}} clause:
> {code:java}
> ::= JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] |
>
> {code}
> From ISO/IEC 9075-6[1]:
> {quote}
> A may have an optional . This
> indicates that the values expression should be parsed as JSON. The
> standardized option is FORMAT JSON; implementations may also support syntax
> such as FORMAT AVRO or FORMAT BSON. When using the , the
> may be either a character string or a binary string.
> {quote}
> Currently we have already added base syntax support of UTF8/UTF16/UTF32
> encoded JSON to parser[2]. However the flags won't appear in parse tree for
> now.
> [1]
> http://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip
> [2]
> https://github.com/apache/calcite/blob/master/core/src/main/codegen/templates/Parser.jj#L4875
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[
https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812948#comment-16812948
]
Hongze Zhang commented on CALCITE-2985:
---
Thanks, [~x1q1j1]. And we should reorganize built-in / MySQL's JSON functions
in the fix of CALCITE-2846 or just after it get resolved. Currently the size of
related classes such as {{StdOperatorTable}}, {{SqlFunctions}}, and test
classes are expanding due to the adding of these functions. We should anyway do
something to keep them easy to maintain (in future).
> Add the JSON_STORAGE_SIZE function
> --
>
> Key: CALCITE-2985
> URL: https://issues.apache.org/jira/browse/CALCITE-2985
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
>
> JSON_STORAGE_SIZE(json_val)
> This function returns the number of bytes used to store the binary
> representation of a JSON document. When the argument is a JSON column, this
> is the space used to store the JSON document. json_val must be a valid JSON
> document or a string which can be parsed as one. In the case where it is
> string, the function returns the amount of storage space in the JSON binary
> representation that is created by parsing the string as JSON and converting
> it to binary. It returns NULL if the argument is NULL.
> An error results when json_val is not NULL, and is not—or cannot be
> successfully parsed as—a JSON document.
> To illustrate this function's behavior when used with a JSON column as its
> argument, we create a table named jtable containing a JSON column jcol,
> insert a JSON value into the table, then obtain the storage space used by
> this column with JSON_STORAGE_SIZE(), as shown here:
> {code:java}
> SELECT
> JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
> JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
> {code}
> | A | B | C | D |
> | 29 | 37 | 40 | 36 |
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2974) Timestamp conversion performance can be improved
[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812942#comment-16812942
]
Chunwei Lei edited comment on CALCITE-2974 at 4/9/19 2:31 AM:
--
[~ValTrofFuture], Thanks for raising this.
Though your proposal greatly improves performance, I think we should not use
implementation in SqlFunctions in RexSimplify which may be not user's expect
behavior. Besides, it may cause different behavior between
RexSimplify.simplifyCast and ReduceExpressionRule if users has provided an own
executor.
was (Author: chunwei lei):
[~ValTrofFuture], Thanks for raising this.
Though your proposal greatly improves performance, I think we should not use
implementation in SqlFunctions in RexSimplify which may be not user's expect
behavior. Besides, it may cause different behavior between
RexSimplify.simplifyCast and ReduceExpressionRule if users has provide an own
executor.
> Timestamp conversion performance can be improved
>
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Valeriy Trofimov
>Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2974) Timestamp conversion performance can be improved
[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812942#comment-16812942
]
Chunwei Lei commented on CALCITE-2974:
--
[~ValTrofFuture], Thanks for raising this.
Though your proposal greatly improves performance, I think we should not use
implementation in SqlFunctions in RexSimplify which may be not user's expect
behavior. Besides, it may cause different behavior between
RexSimplify.simplifyCast and ReduceExpressionRule if users has provide an own
executor.
> Timestamp conversion performance can be improved
>
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Valeriy Trofimov
>Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[
https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Forward Xu updated CALCITE-2985:
Description:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
{code:java}
SELECT
JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
{code}
| A | B | C | D |
| 29 | 37 | 40 | 36 |
was:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
SELECT
JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
| A | B | C | D |
| 29 | 37 | 40 | 36 |
> Add the JSON_STORAGE_SIZE function
> --
>
> Key: CALCITE-2985
> URL: https://issues.apache.org/jira/browse/CALCITE-2985
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
>
> JSON_STORAGE_SIZE(json_val)
> This function returns the number of bytes used to store the binary
> representation of a JSON document. When the argument is a JSON column, this
> is the space used to store the JSON document. json_val must be a valid JSON
> document or a string which can be parsed as one. In the case where it is
> string, the function returns the amount of storage space in the JSON binary
> representation that is created by parsing the string as JSON and converting
> it to binary. It returns NULL if the argument is NULL.
> An error results when json_val is not NULL, and is not—or cannot be
> successfully parsed as—a JSON document.
> To illustrate this function's behavior when used with a JSON column as its
> argument, we create a table named jtable containing a JSON column jcol,
> insert a JSON value into the table, then obtain the storage space used by
> this column with JSON_STORAGE_SIZE(), as shown here:
> {code:java}
> SELECT
> JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
> JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
> {code}
> | A | B | C | D |
> | 29 | 37 | 40 | 36 |
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[
https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Forward Xu updated CALCITE-2985:
Description:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
SELECT
JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
| A | B | C | D |
| 29 | 37 | 40 | 36 |
was:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
SELECT
JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
| A | B | C | D |
| 29 | 37 | 40 | 36 |
+++++
> Add the JSON_STORAGE_SIZE function
> --
>
> Key: CALCITE-2985
> URL: https://issues.apache.org/jira/browse/CALCITE-2985
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
>
> JSON_STORAGE_SIZE(json_val)
> This function returns the number of bytes used to store the binary
> representation of a JSON document. When the argument is a JSON column, this
> is the space used to store the JSON document. json_val must be a valid JSON
> document or a string which can be parsed as one. In the case where it is
> string, the function returns the amount of storage space in the JSON binary
> representation that is created by parsing the string as JSON and converting
> it to binary. It returns NULL if the argument is NULL.
> An error results when json_val is not NULL, and is not—or cannot be
> successfully parsed as—a JSON document.
> To illustrate this function's behavior when used with a JSON column as its
> argument, we create a table named jtable containing a JSON column jcol,
> insert a JSON value into the table, then obtain the storage space used by
> this column with JSON_STORAGE_SIZE(), as shown here:
> SELECT
> JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
> JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
> | A | B | C | D |
> | 29 | 37 | 40 | 36 |
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[
https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Forward Xu updated CALCITE-2985:
Description:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
SELECT
JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
+++++
| A | B | C | D |
+++++
| 29 | 37 | 40 | 36 |
+++++
was:
JSON_STORAGE_SIZE(json_val)
This function returns the number of bytes used to store the binary
representation of a JSON document. When the argument is a JSON column, this is
the space used to store the JSON document. json_val must be a valid JSON
document or a string which can be parsed as one. In the case where it is
string, the function returns the amount of storage space in the JSON binary
representation that is created by parsing the string as JSON and converting it
to binary. It returns NULL if the argument is NULL.
An error results when json_val is not NULL, and is not—or cannot be
successfully parsed as—a JSON document.
To illustrate this function's behavior when used with a JSON column as its
argument, we create a table named jtable containing a JSON column jcol, insert
a JSON value into the table, then obtain the storage space used by this column
with JSON_STORAGE_SIZE(), as shown here:
> Add the JSON_STORAGE_SIZE function
> --
>
> Key: CALCITE-2985
> URL: https://issues.apache.org/jira/browse/CALCITE-2985
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
>
> JSON_STORAGE_SIZE(json_val)
> This function returns the number of bytes used to store the binary
> representation of a JSON document. When the argument is a JSON column, this
> is the space used to store the JSON document. json_val must be a valid JSON
> document or a string which can be parsed as one. In the case where it is
> string, the function returns the amount of storage space in the JSON binary
> representation that is created by parsing the string as JSON and converting
> it to binary. It returns NULL if the argument is NULL.
> An error results when json_val is not NULL, and is not—or cannot be
> successfully parsed as—a JSON document.
> To illustrate this function's behavior when used with a JSON column as its
> argument, we create a table named jtable containing a JSON column jcol,
> insert a JSON value into the table, then obtain the storage space used by
> this column with JSON_STORAGE_SIZE(), as shown here:
> SELECT
> JSON_STORAGE_SIZE('[100, "sakila", [1, 3, 5], 425.05]') AS A,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "a", "c": "[1, 3, 5, 7]"}') AS B,
> JSON_STORAGE_SIZE('{"a": 1000, "b": "wxyz", "c": "[1, 3, 5, 7]"}') AS C,
> JSON_STORAGE_SIZE('[100, "json", [[10, 20, 30], 3, 5], 425.05]') AS D;
> +++++
> | A | B | C | D |
> +++++
> | 29 | 37 | 40 | 36 |
> +++++
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2974) Timestamp conversion performance can be improved
[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812871#comment-16812871
]
Valeriy Trofimov edited comment on CALCITE-2974 at 4/9/19 12:47 AM:
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question - call to
reduce() and the improved code where we manually convert string to timestamp or
int.
was (Author: valtroffuture):
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question - call to
reduce()) and the improved code where we manually convert string to timestamp
or int.
> Timestamp conversion performance can be improved
>
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Valeriy Trofimov
>Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2974) Timestamp conversion performance can be improved
[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812871#comment-16812871
]
Valeriy Trofimov edited comment on CALCITE-2974 at 4/9/19 12:46 AM:
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question - call to
reduce()) and the improved code where we manually convert string to timestamp
or int.
was (Author: valtroffuture):
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question (call to call to
reduce()) and the improved code where we manually convert string to timestamp
or int.
> Timestamp conversion performance can be improved
>
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Valeriy Trofimov
>Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812867#comment-16812867
]
Vineet Garg edited comment on CALCITE-2986 at 4/8/19 11:23 PM:
---
[~hyuan] I have given it some thought and as you noticed <>ANY is not
equivalent to NOT IN.
Problem with semi join transformation is that it will evaluate to only TRUE or
FALSE (SJ will output rows where ON condition is true and will not have rows
where ON condition is not true). But <>ANY requires it to be evaluated to
TRUE, FALSE or UNKNOWN.
<>ANY should probably be transformed in same way as IN is transformed in
SELECT. (e.g. select p_partkey, ( p_partkey IN (select p_partkey from part))
from part)
was (Author: vgarg):
[~hyuan] I have given it some thought and as you noticed <>ANY is not
equivalent to NOT IN.
Problem with semi join transformation is that it will evaluate to only TRUE or
FALSE (SJ with have rows where ON condition is true and will not have rows
where ON condition is not true). But <>ANY requires it to be evaluated to
TRUE, FALSE or UNKNOWN.
<>ANY should probably be transformed in same way as IN is transformed in
SELECT. (e.g. select p_partkey, ( p_partkey IN (select p_partkey from part))
from part)
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2974) Timestamp conversion performance can be improved
[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812871#comment-16812871
]
Valeriy Trofimov commented on CALCITE-2974:
---
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question (call to call to
reduce()) and the improved code where we manually convert string to timestamp
or int.
> Timestamp conversion performance can be improved
>
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Valeriy Trofimov
>Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812867#comment-16812867
]
Vineet Garg commented on CALCITE-2986:
--
[~hyuan] I have given it some thought and as you noticed <>ANY is not
equivalent to NOT IN.
Problem with semi join transformation is that it will evaluate to only TRUE or
FALSE (SJ with have rows where ON condition is true and will not have rows
where ON condition is not true). But <>ANY requires it to be evaluated to
TRUE, FALSE or UNKNOWN.
<>ANY should probably be transformed in same way as IN is transformed in
SELECT. (e.g. select p_partkey, ( p_partkey IN (select p_partkey from part))
from part)
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (CALCITE-2984) Add the JSON_STORAGE_SIZE function
[ https://issues.apache.org/jira/browse/CALCITE-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis closed CALCITE-2984. Resolution: Duplicate > Add the JSON_STORAGE_SIZE function > -- > > Key: CALCITE-2984 > URL: https://issues.apache.org/jira/browse/CALCITE-2984 > Project: Calcite > Issue Type: New Feature >Reporter: Forward Xu >Priority: Major > > JSON_STORAGE_SIZE(json_val) > This function returns the number of bytes used to store the binary > representation of a JSON document. When the argument is a JSON column, this > is the space used to store the JSON document. json_val must be a valid JSON > document or a string which can be parsed as one. In the case where it is > string, the function returns the amount of storage space in the JSON binary > representation that is created by parsing the string as JSON and converting > it to binary. It returns NULL if the argument is NULL. > An error results when json_val is not NULL, and is not—or cannot be > successfully parsed as—a JSON document. > To illustrate this function's behavior when used with a JSON column as its > argument, we create a table named jtable containing a JSON column jcol, > insert a JSON value into the table, then obtain the storage space used by > this column with JSON_STORAGE_SIZE(), as shown here: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2882) ConnectionProperties lose effectiveness when connection reopen after expired
[
https://issues.apache.org/jira/browse/CALCITE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812853#comment-16812853
]
Francis Chuang edited comment on CALCITE-2882 at 4/8/19 10:44 PM:
--
[~shining] There are a few comments on your PR. Can you please have a look and
address them? We are planning to make Avatica 1.14.0 available for voting soon,
and I'd like to include this PR if possible.
was (Author: francischuang):
[~shining] There are a few comments on your PR. Can you please have a look and
address them?
> ConnectionProperties lose effectiveness when connection reopen after expired
>
>
> Key: CALCITE-2882
> URL: https://issues.apache.org/jira/browse/CALCITE-2882
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.12.0
> Environment: Phoenix 5.1
> avatca 1.12
>Reporter: shining
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2019-02-28-17-25-39-478.png,
> image-2019-02-28-17-28-31-926.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When use avatica connect Phoenix QueryServer, I create an AvaticaConnection:
> {code:java}
> Connection conntion = DriverManage.getConnection(url);
> connection.setAutoCommit(true);
> {code}
> Avatica keep PhoenixConnection alive in the Cache, which will be expired
> after 10min by default.
> I still use the older AvaticaConnection , it will reopen an
> PhoenixConnection, but the ConnectionProperties is loss, such as AutoCommit.
> I use sqlline-thin.py to reappear the problem:
> 1) sqlline-thin.py http://localhost:8765
> 2) upsert one row and select
> !image-2019-02-28-17-25-39-478.png!
> 3) after 10 min, upsert again, the connection will be recreate, but select
> null
> !image-2019-02-28-17-28-31-926.png!
> [link title|https://github.com/apache/calcite-avatica/pull/86]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2882) ConnectionProperties lose effectiveness when connection reopen after expired
[
https://issues.apache.org/jira/browse/CALCITE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812853#comment-16812853
]
Francis Chuang commented on CALCITE-2882:
-
[~shining] There are a few comments on your PR. Can you please have a look and
address them?
> ConnectionProperties lose effectiveness when connection reopen after expired
>
>
> Key: CALCITE-2882
> URL: https://issues.apache.org/jira/browse/CALCITE-2882
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.12.0
> Environment: Phoenix 5.1
> avatca 1.12
>Reporter: shining
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2019-02-28-17-25-39-478.png,
> image-2019-02-28-17-28-31-926.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When use avatica connect Phoenix QueryServer, I create an AvaticaConnection:
> {code:java}
> Connection conntion = DriverManage.getConnection(url);
> connection.setAutoCommit(true);
> {code}
> Avatica keep PhoenixConnection alive in the Cache, which will be expired
> after 10min by default.
> I still use the older AvaticaConnection , it will reopen an
> PhoenixConnection, but the ConnectionProperties is loss, such as AutoCommit.
> I use sqlline-thin.py to reappear the problem:
> 1) sqlline-thin.py http://localhost:8765
> 2) upsert one row and select
> !image-2019-02-28-17-25-39-478.png!
> 3) after 10 min, upsert again, the connection will be recreate, but select
> null
> !image-2019-02-28-17-28-31-926.png!
> [link title|https://github.com/apache/calcite-avatica/pull/86]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812830#comment-16812830
]
Haisheng Yuan commented on CALCITE-2986:
Sorry, I mean <> ALL is equivalent to NOT IN. for <> ANY, <> should be kept as
the join filter. like
foo left semi join bar on foo.a <> bar.a
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812824#comment-16812824
]
Haisheng Yuan commented on CALCITE-2986:
You are right, <> ANY is equivalent to NOT IN.
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812799#comment-16812799
]
Vineet Garg commented on CALCITE-2986:
--
{{=ANY}} is equivalent to IN therefore such queries should be rewritten into
IN. I am not what is the correct transformation for {{<>ANY}}.
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812799#comment-16812799
]
Vineet Garg edited comment on CALCITE-2986 at 4/8/19 9:22 PM:
--
{{=ANY}} is equivalent to IN therefore such queries should be rewritten into
IN. I am not sure what is the correct transformation for {{<>ANY}}.
was (Author: vgarg):
{{=ANY}} is equivalent to IN therefore such queries should be rewritten into
IN. I am not what is the correct transformation for {{<>ANY}}.
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated CALCITE-2986:
-
Labels: sub-query (was: )
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2986) Wrong results with =ANY subquery
[
https://issues.apache.org/jira/browse/CALCITE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated CALCITE-2986:
-
Description:
ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
{{<>ALL}}).
Query
{code:sql}
select * from "scott".emp where empno = any (select empno from "scott".emp);
{code}
Expected output for above query is all rows from {{scott.emp}} but actual is
only one row
Test case: e.g.
https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
was:ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is
wrong transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
{{<>ALL}}).
> Wrong results with =ANY subquery
>
>
> Key: CALCITE-2986
> URL: https://issues.apache.org/jira/browse/CALCITE-2986
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
> transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
> {{<>ALL}}).
> Query
> {code:sql}
> select * from "scott".emp where empno = any (select empno from "scott".emp);
> {code}
> Expected output for above query is all rows from {{scott.emp}} but actual is
> only one row
> Test case: e.g.
> https://github.com/apache/calcite/compare/master...vineetgarg02:CALCITE-2986
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2986) Wrong results with =ANY subquery
Vineet Garg created CALCITE-2986:
Summary: Wrong results with =ANY subquery
Key: CALCITE-2986
URL: https://issues.apache.org/jira/browse/CALCITE-2986
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Vineet Garg
Assignee: Vineet Garg
ANY/SOME subqueries are rewritten using MAX/MIN and cross-join. This is wrong
transformation for {{=ANY}} and {{<>ANY}} (and therefore {{=ALL}} and
{{<>ALL}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1037) Column uniqueness is calculated incorrectly for 'Correlate' expression
[
https://issues.apache.org/jira/browse/CALCITE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812633#comment-16812633
]
Vladimir Sitnikov commented on CALCITE-1037:
Just in case, the commit was
https://github.com/apache/calcite/commit/1ca15524992927eb88b41951a8a6ff3f45dfd4f7
PS [~julianhyde]: the funny thing is GitHub links are much more stable than
*.a.o ones.
Could we ask Infra to redirect "old" links to new ones?
> Column uniqueness is calculated incorrectly for 'Correlate' expression
> --
>
> Key: CALCITE-1037
> URL: https://issues.apache.org/jira/browse/CALCITE-1037
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.5.0
>Reporter: Alexey Makhmutov
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.11.0
>
>
> Column uniqueness is calculated incorrectly for 'Correlate' expression -- and
> in some cases this leads to java.lang.IndexOutOfBoundsException. Example of
> such code:
> {code}select
> x.v
> from
> (
> select
>t1.v
> from
>(values (1,1),(1,2)) as t1(k,v)
>join (values (1)) as t2(k) on t1.k=t2.k
> ) x,
> lateral
> (
> select
>t.v
> from
>unnest(multiset[x.v]) as t(v)
> ) y
> group by x.v,y.v{code}
> The problems seems to be related to the
> org.apache.calcite.rel.metadata.RelMdColumnUniqueness.areColumnsUnique(Correlate
> rel, ImmutableBitSet columns, boolean ignoreNulls) method -- it just
> delegates uniqueness check to left input without changing columns list, which
> leads to Exception if this list references columns from right input.
> It seems, that right behavior should be following:
> * For Anti/Semi join type keep the current behavior (as resulting rows
> contains fields only from left input).
> * For Left/Inner join type columns set for correlate is unique only if it
> includes unique sets from both sides.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812531#comment-16812531
]
Chunwei Lei edited comment on CALCITE-1338 at 4/8/19 3:54 PM:
--
To resolve this, I would like to proposal:
* add a new method named isStrict in SqlOperator to indicate whether the
operator is strict or not
* add a check in JoinProjectTransposeRule to assure not transform when the
project has literal or non-strict call
The code looks like as follows:
{code:java}
if (includeOuter) {
Project project = null;
if (rightProj != null
&& joinType.generatesNullsOnRight()) {
project = rightProj;
} else if (leftProj != null && joinType.generatesNullsOnLeft()) {
project = leftProj;
}
if (project != null) {
for (RexNode node: project.getProjects()) {
if (node instanceof RexLiteral) {
return;
} else if (node instanceof RexCall) {
RexCall rexCall = (RexCall) node;
if (!rexCall.getOperator().isStrict()) {
return;
}
}
}
}
}
{code}
Can someone check whether it is a right way? Appreciate it.
was (Author: chunwei lei):
To resolve this, I would like to proposal:
* add a new method named isStrict in SqlOperator to indicate whether the
operator is strict or not
* add a check in JoinProjectTransposeRule to assure not transform when the
project has literal or non-strict call
The code looks like as follows:
{code:java}
if (includeOuter) {
Project project = null;
if (rightProj != null
&& joinType.generatesNullsOnRight()) {
project = rightProj;
} else if (leftProj != null && joinType.generatesNullsOnLeft()) {
project = leftProj;
}
if (project != null) {
for (RexNode node: project.getProjects()) {
if (node instanceof RexLiteral) {
return;
} else if (node instanceof RexCall) {
RexCall rexCall = (RexCall) node;
if (!rexCall.getOperator().isStrict()) {
return;
}
}
}
}
}
{code}
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-1338) JoinProjectTransposeRule makes wrong transformation when the right child of left outer join has a RexLiteral project expression.
[
https://issues.apache.org/jira/browse/CALCITE-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812531#comment-16812531
]
Chunwei Lei commented on CALCITE-1338:
--
To resolve this, I would like to proposal:
* add a new method named isStrict in SqlOperator to indicate whether the
operator is strict or not
* add a check in JoinProjectTransposeRule to assure not transform when the
project has literal or non-strict call
The code looks like as follows:
{code:java}
if (includeOuter) {
Project project = null;
if (rightProj != null
&& joinType.generatesNullsOnRight()) {
project = rightProj;
} else if (leftProj != null && joinType.generatesNullsOnLeft()) {
project = leftProj;
}
if (project != null) {
for (RexNode node: project.getProjects()) {
if (node instanceof RexLiteral) {
return;
} else if (node instanceof RexCall) {
RexCall rexCall = (RexCall) node;
if (!rexCall.getOperator().isStrict()) {
return;
}
}
}
}
}
{code}
> JoinProjectTransposeRule makes wrong transformation when the right child of
> left outer join has a RexLiteral project expression.
>
>
> Key: CALCITE-1338
> URL: https://issues.apache.org/jira/browse/CALCITE-1338
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Yi Xinglu
>Assignee: Chunwei Lei
>Priority: Major
>
> h3. SQL:
> {code:borderStyle=solid}
> SELECT *
> FROM dept a
> LEFT JOIN (
> SELECT b.name, 1 FROM dept b
> ) AS b
> ON a.name=b.name
> {code}
> h3. Selected rule set:
> {code:borderStyle=solid}
> SubQueryRemoveRule.JOIN
> JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER
> {code}
> h3. Optimized logical plan:
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2],
> EXPR$1=[CAST($3):INTEGER])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$3], EXPR$1=[1])
> LogicalJoin(condition=[=($3, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. The right logical plan should be
> {code:borderStyle=solid}
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[$2], EXPR$1=[$3])
> LogicalJoin(condition=[=($2, $1)], joinType=[left])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(NAME=[$1], EXPR$1=[1])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> h3. Summary
> The RexLiteral project expression will make logical plan get different
> results when it's right child or parent node of left outer join.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[ https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated CALCITE-2985: Issue Type: Sub-task (was: New Feature) Parent: CALCITE-2867 > Add the JSON_STORAGE_SIZE function > -- > > Key: CALCITE-2985 > URL: https://issues.apache.org/jira/browse/CALCITE-2985 > Project: Calcite > Issue Type: Sub-task >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > JSON_STORAGE_SIZE(json_val) > This function returns the number of bytes used to store the binary > representation of a JSON document. When the argument is a JSON column, this > is the space used to store the JSON document. json_val must be a valid JSON > document or a string which can be parsed as one. In the case where it is > string, the function returns the amount of storage space in the JSON binary > representation that is created by parsing the string as JSON and converting > it to binary. It returns NULL if the argument is NULL. > An error results when json_val is not NULL, and is not—or cannot be > successfully parsed as—a JSON document. > To illustrate this function's behavior when used with a JSON column as its > argument, we create a table named jtable containing a JSON column jcol, > insert a JSON value into the table, then obtain the storage space used by > this column with JSON_STORAGE_SIZE(), as shown here: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
[ https://issues.apache.org/jira/browse/CALCITE-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu reassigned CALCITE-2985: --- Assignee: Forward Xu > Add the JSON_STORAGE_SIZE function > -- > > Key: CALCITE-2985 > URL: https://issues.apache.org/jira/browse/CALCITE-2985 > Project: Calcite > Issue Type: New Feature >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > JSON_STORAGE_SIZE(json_val) > This function returns the number of bytes used to store the binary > representation of a JSON document. When the argument is a JSON column, this > is the space used to store the JSON document. json_val must be a valid JSON > document or a string which can be parsed as one. In the case where it is > string, the function returns the amount of storage space in the JSON binary > representation that is created by parsing the string as JSON and converting > it to binary. It returns NULL if the argument is NULL. > An error results when json_val is not NULL, and is not—or cannot be > successfully parsed as—a JSON document. > To illustrate this function's behavior when used with a JSON column as its > argument, we create a table named jtable containing a JSON column jcol, > insert a JSON value into the table, then obtain the storage space used by > this column with JSON_STORAGE_SIZE(), as shown here: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2985) Add the JSON_STORAGE_SIZE function
Forward Xu created CALCITE-2985: --- Summary: Add the JSON_STORAGE_SIZE function Key: CALCITE-2985 URL: https://issues.apache.org/jira/browse/CALCITE-2985 Project: Calcite Issue Type: New Feature Reporter: Forward Xu JSON_STORAGE_SIZE(json_val) This function returns the number of bytes used to store the binary representation of a JSON document. When the argument is a JSON column, this is the space used to store the JSON document. json_val must be a valid JSON document or a string which can be parsed as one. In the case where it is string, the function returns the amount of storage space in the JSON binary representation that is created by parsing the string as JSON and converting it to binary. It returns NULL if the argument is NULL. An error results when json_val is not NULL, and is not—or cannot be successfully parsed as—a JSON document. To illustrate this function's behavior when used with a JSON column as its argument, we create a table named jtable containing a JSON column jcol, insert a JSON value into the table, then obtain the storage space used by this column with JSON_STORAGE_SIZE(), as shown here: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2984) Add the JSON_STORAGE_SIZE function
Forward Xu created CALCITE-2984: --- Summary: Add the JSON_STORAGE_SIZE function Key: CALCITE-2984 URL: https://issues.apache.org/jira/browse/CALCITE-2984 Project: Calcite Issue Type: New Feature Reporter: Forward Xu JSON_STORAGE_SIZE(json_val) This function returns the number of bytes used to store the binary representation of a JSON document. When the argument is a JSON column, this is the space used to store the JSON document. json_val must be a valid JSON document or a string which can be parsed as one. In the case where it is string, the function returns the amount of storage space in the JSON binary representation that is created by parsing the string as JSON and converting it to binary. It returns NULL if the argument is NULL. An error results when json_val is not NULL, and is not—or cannot be successfully parsed as—a JSON document. To illustrate this function's behavior when used with a JSON column as its argument, we create a table named jtable containing a JSON column jcol, insert a JSON value into the table, then obtain the storage space used by this column with JSON_STORAGE_SIZE(), as shown here: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2983) Add Avatica compatibility page for TLS and IBM Java
[ https://issues.apache.org/jira/browse/CALCITE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated CALCITE-2983: -- Fix Version/s: avatica-1.14.0 > Add Avatica compatibility page for TLS and IBM Java > --- > > Key: CALCITE-2983 > URL: https://issues.apache.org/jira/browse/CALCITE-2983 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Kevin Risden >Priority: Major > Fix For: avatica-1.14.0 > > > With the Jetty upgrade in CALCITE-2972, there are some compatibility issues > between TLS support and IBM Java. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2983) Add Avatica compatibility page for TLS and IBM Java
[ https://issues.apache.org/jira/browse/CALCITE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812434#comment-16812434 ] Kevin Risden commented on CALCITE-2983: --- cc [~elserj] > Add Avatica compatibility page for TLS and IBM Java > --- > > Key: CALCITE-2983 > URL: https://issues.apache.org/jira/browse/CALCITE-2983 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Kevin Risden >Priority: Major > Fix For: avatica-1.14.0 > > > With the Jetty upgrade in CALCITE-2972, there are some compatibility issues > between TLS support and IBM Java. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2972) Upgrade jetty to 9.4.15.v20190215
[ https://issues.apache.org/jira/browse/CALCITE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden resolved CALCITE-2972. --- Resolution: Fixed > Upgrade jetty to 9.4.15.v20190215 > - > > Key: CALCITE-2972 > URL: https://issues.apache.org/jira/browse/CALCITE-2972 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.14.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Avatica should be upgraded to the latest Jetty 9.4.15.v20190215 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2983) Add Avatica compatibility page for TLS and IBM Java
Kevin Risden created CALCITE-2983: - Summary: Add Avatica compatibility page for TLS and IBM Java Key: CALCITE-2983 URL: https://issues.apache.org/jira/browse/CALCITE-2983 Project: Calcite Issue Type: Task Components: avatica Reporter: Kevin Risden With the Jetty upgrade in CALCITE-2972, there are some compatibility issues between TLS support and IBM Java. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2972) Upgrade jetty to 9.4.15.v20190215
[ https://issues.apache.org/jira/browse/CALCITE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812433#comment-16812433 ] Kevin Risden commented on CALCITE-2972: --- Created CALCITE-2983 > Upgrade jetty to 9.4.15.v20190215 > - > > Key: CALCITE-2972 > URL: https://issues.apache.org/jira/browse/CALCITE-2972 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.14.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Avatica should be upgraded to the latest Jetty 9.4.15.v20190215 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2972) Upgrade jetty to 9.4.15.v20190215
[
https://issues.apache.org/jira/browse/CALCITE-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812427#comment-16812427
]
Josh Elser commented on CALCITE-2972:
-
{quote}We should make change to the docs/website with a compatibility change.
Do you want to do that as part of this change or separate JIRA?
{quote}
Either way. Doesn't really matter to me.
> Upgrade jetty to 9.4.15.v20190215
> -
>
> Key: CALCITE-2972
> URL: https://issues.apache.org/jira/browse/CALCITE-2972
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>Priority: Major
> Labels: pull-request-available
> Fix For: avatica-1.14.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Avatica should be upgraded to the latest Jetty 9.4.15.v20190215
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2981) Syntax error with simple nested select query
[ https://issues.apache.org/jira/browse/CALCITE-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812426#comment-16812426 ] Danny Chan commented on CALCITE-2981: - [~vamshi.v.kris...@gmail.com] Seems should not happen, i will try to fix it is a bug. > Syntax error with simple nested select query > > > Key: CALCITE-2981 > URL: https://issues.apache.org/jira/browse/CALCITE-2981 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Vamshi >Assignee: Danny Chan >Priority: Critical > > Below standard sql-92 query reports syntax error in calcite: > create table test1(id integer, name integer); > create table test2(id integer, name integer); > select * from ( > ( select id,name from test1 ) test3 > inner join > ( select id,name from test2) test4 on test4.id = test3.id > ); > Caused by: org.apache.calcite.parser.ParseException: Encountered "test3" at > line 4, column 31. > Was expecting one of: > ")" ... > "ORDER" ... > "LIMIT" ... > "OFFSET" ... > "FETCH" ... > "UNION" ... > "INTERSECT" ... > "EXCEPT" ... > "MINUS" ... > "." ... > "NOT" ... > "IN" ... > "<" ... > "<=" ... > ">" ... > ">=" ... > "=" ... > "<>" ... > "!=" ... > "BETWEEN" ... > "LIKE" ... > "SIMILAR" ... > "+" ... > "-" ... > "*" ... > "/" ... > "%" ... > "||" ... > "AND" ... > "OR" ... > "IS" ... > "MEMBER" ... > "SUBMULTISET" ... > "CONTAINS" ... > "OVERLAPS" ... > "EQUALS" ... > "PRECEDES" ... > "SUCCEEDS" ... > "IMMEDIATELY" ... > "MULTISET" ... > "[" ... > "YEAR" ... > "MONTH" ... > "DAY" ... > "HOUR" ... > "MINUTE" ... > "SECOND" ... > > The above query works in standard databases like postgres/oracle/sql server. > This is a commonly generated query pattern by popular BI tools. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2981) Syntax error with simple nested select query
[ https://issues.apache.org/jira/browse/CALCITE-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan reassigned CALCITE-2981: --- Assignee: Danny Chan > Syntax error with simple nested select query > > > Key: CALCITE-2981 > URL: https://issues.apache.org/jira/browse/CALCITE-2981 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Vamshi >Assignee: Danny Chan >Priority: Critical > > Below standard sql-92 query reports syntax error in calcite: > create table test1(id integer, name integer); > create table test2(id integer, name integer); > select * from ( > ( select id,name from test1 ) test3 > inner join > ( select id,name from test2) test4 on test4.id = test3.id > ); > Caused by: org.apache.calcite.parser.ParseException: Encountered "test3" at > line 4, column 31. > Was expecting one of: > ")" ... > "ORDER" ... > "LIMIT" ... > "OFFSET" ... > "FETCH" ... > "UNION" ... > "INTERSECT" ... > "EXCEPT" ... > "MINUS" ... > "." ... > "NOT" ... > "IN" ... > "<" ... > "<=" ... > ">" ... > ">=" ... > "=" ... > "<>" ... > "!=" ... > "BETWEEN" ... > "LIKE" ... > "SIMILAR" ... > "+" ... > "-" ... > "*" ... > "/" ... > "%" ... > "||" ... > "AND" ... > "OR" ... > "IS" ... > "MEMBER" ... > "SUBMULTISET" ... > "CONTAINS" ... > "OVERLAPS" ... > "EQUALS" ... > "PRECEDES" ... > "SUCCEEDS" ... > "IMMEDIATELY" ... > "MULTISET" ... > "[" ... > "YEAR" ... > "MONTH" ... > "DAY" ... > "HOUR" ... > "MINUTE" ... > "SECOND" ... > > The above query works in standard databases like postgres/oracle/sql server. > This is a commonly generated query pattern by popular BI tools. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei reassigned CALCITE-1581:
---
Assignee: pengzhiwei (was: Sihua Zhou)
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available, udtf
> Fix For: 1.20.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2921) nullif(null,y) throws exception in verification
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804546#comment-16804546
]
pengzhiwei edited comment on CALCITE-2921 at 4/8/19 11:40 AM:
--
Hi [~julianhyde], the pr has been updated, here is the mainly change list:
* Forbid to return "NULL" type for SqlOperator. As it may cause uncertain
semantics.For example, "Not(null) + 1" can pass the validate, if the
"Not(null)"'s return type is "NULL". I have added a default type for most
operator which may return "NULL" type.
* add "allowNakedNull()" in SqlConformance.
* fix other issue you mentioned in last review.
Please have a look again,Thanks!
was (Author: pzw2018):
Hi [~julianhyde], the pr has been updated, here is the mainly change list:
* The "NULL" return type of SqlOperator is forbidden. As it may cause
uncertain semantics.For example, "Not(null) + 1" can pass the validate, if the
"Not(null)"'s return type is "NULL". I have added a default type for most
operator which may return "NULL" type.
* add "allowNakedNull()" in SqlConformance.
* fix other issue you mentioned in last review.
Please have a look again,Thanks!
> nullif(null,y) throws exception in verification
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently calcite will translate "nullif(x,y)" to a case-when expression,just
> like "case when x = y then null else x".So when "x" is null literal,a
> exception throws out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2982) SqlItemOperator should throw understandable exception message for incorrect field type
[
https://issues.apache.org/jira/browse/CALCITE-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei updated CALCITE-2982:
Description:
In the following sql:
{code:java}
select name[0] from dept
{code}
_name_ is a field of _VARCHAR_ type, the follow exception message is thrown out:
{code:java}
java.lang.AssertionError: VARCHAR
at
org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
at
org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
{code}
The exception message is hard to understand for user. I think a message like
"Cannot apply item opeator to VARCHAR type for filed "name" at line xx column
xx " is more friendly to the user.
was:
In the follow sql:
{code:java}
select name[0] from dept
{code}
_name_ is a field of _VARCHAR_ type, the follow exception message is thrown out:
{code:java}
java.lang.AssertionError: VARCHAR
at
org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
at
org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
{code}
The exception message is hard to understand for user. I think a message like
"Cannot apply item opeator to VARCHAR type for filed "name" at line xx column
xx " is more friendly to the user.
> SqlItemOperator should throw understandable exception message for incorrect
> field type
> --
>
> Key: CALCITE-2982
> URL: https://issues.apache.org/jira/browse/CALCITE-2982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the following sql:
> {code:java}
> select name[0] from dept
> {code}
> _name_ is a field of _VARCHAR_ type, the follow exception message is thrown
> out:
> {code:java}
> java.lang.AssertionError: VARCHAR
> at
> org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
> at
> org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> {code}
> The exception message is hard to understand for user. I think a message like
> "Cannot apply item opeator to VARCHAR type for filed "name" at line xx
> column xx " is more friendly to the user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2872) Support for specifying encoding in JSON representation clause
[
https://issues.apache.org/jira/browse/CALCITE-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812352#comment-16812352
]
Forward Xu commented on CALCITE-2872:
-
hi [~zhztheplayer] UTF8, UTF16, UTF32 do you need to distinguish between bom
and no bom?
> Support for specifying encoding in JSON representation clause
> -
>
> Key: CALCITE-2872
> URL: https://issues.apache.org/jira/browse/CALCITE-2872
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Hongze Zhang
>Priority: Major
>
> Syntax of {{JSON representation}} clause:
> {code:java}
> ::= JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] |
>
> {code}
> From ISO/IEC 9075-6[1]:
> {quote}
> A may have an optional . This
> indicates that the values expression should be parsed as JSON. The
> standardized option is FORMAT JSON; implementations may also support syntax
> such as FORMAT AVRO or FORMAT BSON. When using the , the
> may be either a character string or a binary string.
> {quote}
> Currently we have already added base syntax support of UTF8/UTF16/UTF32
> encoded JSON to parser[2]. However the flags won't appear in parse tree for
> now.
> [1]
> http://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip
> [2]
> https://github.com/apache/calcite/blob/master/core/src/main/codegen/templates/Parser.jj#L4875
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (CALCITE-2981) Syntax error with simple nested select query
[ https://issues.apache.org/jira/browse/CALCITE-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pingle wang updated CALCITE-2981: - Comment: was deleted (was: ```sql select * from ( (select * from test1) test3 ) ``` Occurring Exception is not parsed sub-query test3 table reference correctly, I can try this issue, thanks.) > Syntax error with simple nested select query > > > Key: CALCITE-2981 > URL: https://issues.apache.org/jira/browse/CALCITE-2981 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Vamshi >Priority: Critical > > Below standard sql-92 query reports syntax error in calcite: > create table test1(id integer, name integer); > create table test2(id integer, name integer); > select * from ( > ( select id,name from test1 ) test3 > inner join > ( select id,name from test2) test4 on test4.id = test3.id > ); > Caused by: org.apache.calcite.parser.ParseException: Encountered "test3" at > line 4, column 31. > Was expecting one of: > ")" ... > "ORDER" ... > "LIMIT" ... > "OFFSET" ... > "FETCH" ... > "UNION" ... > "INTERSECT" ... > "EXCEPT" ... > "MINUS" ... > "." ... > "NOT" ... > "IN" ... > "<" ... > "<=" ... > ">" ... > ">=" ... > "=" ... > "<>" ... > "!=" ... > "BETWEEN" ... > "LIKE" ... > "SIMILAR" ... > "+" ... > "-" ... > "*" ... > "/" ... > "%" ... > "||" ... > "AND" ... > "OR" ... > "IS" ... > "MEMBER" ... > "SUBMULTISET" ... > "CONTAINS" ... > "OVERLAPS" ... > "EQUALS" ... > "PRECEDES" ... > "SUCCEEDS" ... > "IMMEDIATELY" ... > "MULTISET" ... > "[" ... > "YEAR" ... > "MONTH" ... > "DAY" ... > "HOUR" ... > "MINUTE" ... > "SECOND" ... > > The above query works in standard databases like postgres/oracle/sql server. > This is a commonly generated query pattern by popular BI tools. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2981) Syntax error with simple nested select query
[ https://issues.apache.org/jira/browse/CALCITE-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812202#comment-16812202 ] pingle wang commented on CALCITE-2981: -- ```sql select * from ( (select * from test1) test3 ) ``` Occurring Exception is not parsed sub-query test3 table reference correctly, I can try this issue, thanks. > Syntax error with simple nested select query > > > Key: CALCITE-2981 > URL: https://issues.apache.org/jira/browse/CALCITE-2981 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Vamshi >Priority: Critical > > Below standard sql-92 query reports syntax error in calcite: > create table test1(id integer, name integer); > create table test2(id integer, name integer); > select * from ( > ( select id,name from test1 ) test3 > inner join > ( select id,name from test2) test4 on test4.id = test3.id > ); > Caused by: org.apache.calcite.parser.ParseException: Encountered "test3" at > line 4, column 31. > Was expecting one of: > ")" ... > "ORDER" ... > "LIMIT" ... > "OFFSET" ... > "FETCH" ... > "UNION" ... > "INTERSECT" ... > "EXCEPT" ... > "MINUS" ... > "." ... > "NOT" ... > "IN" ... > "<" ... > "<=" ... > ">" ... > ">=" ... > "=" ... > "<>" ... > "!=" ... > "BETWEEN" ... > "LIKE" ... > "SIMILAR" ... > "+" ... > "-" ... > "*" ... > "/" ... > "%" ... > "||" ... > "AND" ... > "OR" ... > "IS" ... > "MEMBER" ... > "SUBMULTISET" ... > "CONTAINS" ... > "OVERLAPS" ... > "EQUALS" ... > "PRECEDES" ... > "SUCCEEDS" ... > "IMMEDIATELY" ... > "MULTISET" ... > "[" ... > "YEAR" ... > "MONTH" ... > "DAY" ... > "HOUR" ... > "MINUTE" ... > "SECOND" ... > > The above query works in standard databases like postgres/oracle/sql server. > This is a commonly generated query pattern by popular BI tools. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (CALCITE-2981) Syntax error with simple nested select query
[ https://issues.apache.org/jira/browse/CALCITE-2981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pingle wang updated CALCITE-2981: - Comment: was deleted (was: ```sql select * from ( (select * from test1) test3 ) ``` Occurring Exception is not parsed sub-query test3 table reference correctly, I can try this issue, thanks.) > Syntax error with simple nested select query > > > Key: CALCITE-2981 > URL: https://issues.apache.org/jira/browse/CALCITE-2981 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.18.0 >Reporter: Vamshi >Priority: Critical > > Below standard sql-92 query reports syntax error in calcite: > create table test1(id integer, name integer); > create table test2(id integer, name integer); > select * from ( > ( select id,name from test1 ) test3 > inner join > ( select id,name from test2) test4 on test4.id = test3.id > ); > Caused by: org.apache.calcite.parser.ParseException: Encountered "test3" at > line 4, column 31. > Was expecting one of: > ")" ... > "ORDER" ... > "LIMIT" ... > "OFFSET" ... > "FETCH" ... > "UNION" ... > "INTERSECT" ... > "EXCEPT" ... > "MINUS" ... > "." ... > "NOT" ... > "IN" ... > "<" ... > "<=" ... > ">" ... > ">=" ... > "=" ... > "<>" ... > "!=" ... > "BETWEEN" ... > "LIKE" ... > "SIMILAR" ... > "+" ... > "-" ... > "*" ... > "/" ... > "%" ... > "||" ... > "AND" ... > "OR" ... > "IS" ... > "MEMBER" ... > "SUBMULTISET" ... > "CONTAINS" ... > "OVERLAPS" ... > "EQUALS" ... > "PRECEDES" ... > "SUCCEEDS" ... > "IMMEDIATELY" ... > "MULTISET" ... > "[" ... > "YEAR" ... > "MONTH" ... > "DAY" ... > "HOUR" ... > "MINUTE" ... > "SECOND" ... > > The above query works in standard databases like postgres/oracle/sql server. > This is a commonly generated query pattern by popular BI tools. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2982) SqlItemOperator should throw understandable exception message for incorrect field type
[
https://issues.apache.org/jira/browse/CALCITE-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-2982:
Labels: pull-request-available (was: )
> SqlItemOperator should throw understandable exception message for incorrect
> field type
> --
>
> Key: CALCITE-2982
> URL: https://issues.apache.org/jira/browse/CALCITE-2982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> In the follow sql:
> {code:java}
> select name[0] from dept
> {code}
> _name_ is a field of _VARCHAR_ type, the follow exception message is thrown
> out:
> {code:java}
> java.lang.AssertionError: VARCHAR
> at
> org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
> at
> org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> {code}
> The exception message is hard to understand for user. I think a message like
> "Cannot apply item opeator to VARCHAR type for filed "name" at line xx
> column xx " is more friendly to the user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2982) SqlItemOperator should throw understandable exception message for incorrect field type
[
https://issues.apache.org/jira/browse/CALCITE-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei updated CALCITE-2982:
Summary: SqlItemOperator should throw understandable exception message for
incorrect field type (was: SqlItemOpeator should throw understandable
exception message for incorrect field type)
> SqlItemOperator should throw understandable exception message for incorrect
> field type
> --
>
> Key: CALCITE-2982
> URL: https://issues.apache.org/jira/browse/CALCITE-2982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Fix For: 1.20.0
>
>
> In the follow sql:
> {code:java}
> select name[0] from dept
> {code}
> _name_ is a field of _VARCHAR_ type, the follow exception message is thrown
> out:
> {code:java}
> java.lang.AssertionError: VARCHAR
> at
> org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
> at
> org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> {code}
> The exception message is hard to understand for user. I think a message like
> "Cannot apply item opeator to VARCHAR type for filed "name" at line xx
> column xx " is more friendly to the user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2982) SqlItemOpeator should throw understandable exception message for incorrect field type
[
https://issues.apache.org/jira/browse/CALCITE-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
pengzhiwei updated CALCITE-2982:
Description:
In the follow sql:
{code:java}
select name[0] from dept
{code}
_name_ is a field of _VARCHAR_ type, the follow exception message is thrown out:
{code:java}
java.lang.AssertionError: VARCHAR
at
org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
at
org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
{code}
The exception message is hard to understand for user. I think a message like
"Cannot apply item opeator to VARCHAR type for filed "name" at line xx column
xx " is more friendly to the user.
was:
In the follow sql:
{code:java}
sql("select name[0] from dept").ok();
{code}
_name_ is a field of _VARCHAR_ type, the follow exception message is thrown out:
{code:java}
java.lang.AssertionError: VARCHAR
at
org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
at
org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
{code}
The exception message is hard to understand for user. I think a message like
"Cannot apply item opeator to VARCHAR type for filed "name" at line xx column
xx " is more friendly to the user.
> SqlItemOpeator should throw understandable exception message for incorrect
> field type
> -
>
> Key: CALCITE-2982
> URL: https://issues.apache.org/jira/browse/CALCITE-2982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Fix For: 1.20.0
>
>
> In the follow sql:
> {code:java}
> select name[0] from dept
> {code}
> _name_ is a field of _VARCHAR_ type, the follow exception message is thrown
> out:
> {code:java}
> java.lang.AssertionError: VARCHAR
> at
> org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
> at
> org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> {code}
> The exception message is hard to understand for user. I think a message like
> "Cannot apply item opeator to VARCHAR type for filed "name" at line xx
> column xx " is more friendly to the user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2982) SqlItemOpeator should throw understandable exception message for incorrect field type
pengzhiwei created CALCITE-2982:
---
Summary: SqlItemOpeator should throw understandable exception
message for incorrect field type
Key: CALCITE-2982
URL: https://issues.apache.org/jira/browse/CALCITE-2982
Project: Calcite
Issue Type: Bug
Components: core
Reporter: pengzhiwei
Assignee: pengzhiwei
Fix For: 1.20.0
In the follow sql:
{code:java}
sql("select name[0] from dept").ok();
{code}
_name_ is a field of _VARCHAR_ type, the follow exception message is thrown out:
{code:java}
java.lang.AssertionError: VARCHAR
at
org.apache.calcite.sql.fun.SqlItemOperator.getChecker(SqlItemOperator.java:109)
at
org.apache.calcite.sql.fun.SqlItemOperator.checkOperandTypes(SqlItemOperator.java:91)
at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:516)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5607)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5594)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
{code}
The exception message is hard to understand for user. I think a message like
"Cannot apply item opeator to VARCHAR type for filed "name" at line xx column
xx " is more friendly to the user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
