[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Description: 
When a Sort represents 'Order By x Limit n',and its input is guaranteed that is 
at most one row,we could remove this Sort.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

This JIRA is enhancement for  {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
restrict Sort must be pure Order(Offset and Limit is null),after this jira,this 
optimization could be applied to 'Order By x' Or 'Order By x Limit n'.

 

  was:
When a Sort represents 'Order By x Limit n',and its input is guaranteed that is 
at most one row,we could remove this Sort.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:




 


> Remove 'Order By x Limit n' when the input is guaranteed  that is at most one 
> row
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort represents 'Order By x Limit n',and its input is guaranteed that 
> is at most one row,we could remove this Sort.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
> This JIRA is enhancement for  {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
> restrict Sort must be pure Order(Offset and Limit is null),after this 
> jira,this optimization could be applied to 'Order By x' Or 'Order By x Limit 
> n'.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-6038) Remove 'Order By x Limit n' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774309#comment-17774309
 ] 

LakeShen edited comment on CALCITE-6038 at 10/12/23 5:43 AM:
-

Hi [~julianhyde] ,thank you for reminding me.You are right, using 'TopN' is not 
appropriate to describe this JIRA.

I think that 'TopN' is closer to the business layer,so it has multiple 
understandings,such as 'Order By x Limit n', ' ROW_NUMBER() OVER .. + where 
rownum <= N' and so on.

In this jira,I just want to represent 'Order By x Limit n',so using 'Order By x 
Limit n' directly may be more accurate.I want to change the title and 
description of this Jira for using 'Order By x Limit n'.

WDYT?

In CALCITE-6009, if a RelNode is pure Limit which means that is 'Limit N',and 
it doesn't have the 'Offset' and 'Order By' semantics,it could be removed when 
its input RelNode's max row count is less than or equal to the Limit's fetch.

In {-}CALCITE-5994{-}, if a RelNode is pure Order which means that is 'Order 
By',and it doesn't have the 'Offset' and 'Limit' semantics,it could be removed 
when its input RelNode's max row count is less than or equal to one.

This JIRA is more relevant to {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
restrict Sort must be pure Order(Offset and Limit is null),after this jira,this 
optimization could be applied to 'Order By x' Or 'Order By x Limit n',this JIRA 
is an enhancement for {-}CALCITE-5994{-}.


was (Author: shenlang):
Hi [~julianhyde] ,thank you for reminding me.You are right, using 'TopN' is not 
appropriate to describe this JIRA.

I think that 'TopN' is closer to the business layer,so it has multiple 
understandings,such as 'Order By x Limit n', ' ROW_NUMBER() OVER .. + where 
rownum <= N' and so on.

In this jira,I just want to represent 'Order By x Limit n',so using 'Order By x 
Limit n' directly may be more accurate.I want to change the title and 
description of this Jira for using 'Order By x Limit n'.

WDYT?

In CALCITE-6009, if a RelNode is pure Limit which means that is 'Limit N',and 
it doesn't have the 'Offset' and 'Order By' semantics,it could be removed when 
its input RelNode's max row count is less than or equal to the Limit's fetch.

In {-}CALCITE-5994{-}, if a RelNode is pure Order which means that is 'Order 
By',and it doesn't have the 'Offset' and 'Limit' semantics,it could be removed 
when its input RelNode's max row count is less than or equal to one.

This JIRA is more relevant to {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
restrict Sort must be pure Order(Offset and Limit is null),after this jira,this 
optimization could be applied to 'Order By x' Or 'Order By x Limit n',this JIRA 
is an enhancement.

> Remove 'Order By x Limit n' when the input is guaranteed  that is at most one 
> row
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort represents 'Order By x Limit n',and its input is guaranteed that 
> is at most one row,we could remove this Sort.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
> This JIRA is enhancement for  {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
> restrict Sort must be pure Order(Offset and Limit is null),after this 
> jira,this optimization could be applied to 'Order By x' Or 'Order By x Limit 
> n'.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Description: 
When a Sort represents 'Order By x Limit n',and its input is guaranteed that is 
at most one row,we could remove this Sort.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:




 

  was:
When a Sort represents 'Order By x Limit n',and its input source row count is 
provably at most one row,then we could remove the redundant TopN.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

 


> Remove 'Order By x Limit n' when the input is guaranteed  that is at most one 
> row
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort represents 'Order By x Limit n',and its input is guaranteed that 
> is at most one row,we could remove this Sort.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Summary: Remove 'Order By x Limit n' when the input is guaranteed  that is 
at most one row  (was: Remove 'Order By x Limit n'' when the input is 
guaranteed  that is at most one row)

> Remove 'Order By x Limit n' when the input is guaranteed  that is at most one 
> row
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort is ''TopN's input source row count is provably at most one 
> row,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Description: 
When a Sort represents 'Order By x Limit n',and its input source row count is 
provably at most one row,then we could remove the redundant TopN.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

 

  was:
When a Sort is ''TopN's input source row count is provably at most one row,then 
we could remove the redundant TopN.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

 


> Remove 'Order By x Limit n' when the input is guaranteed  that is at most one 
> row
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort represents 'Order By x Limit n',and its input source row count is 
> provably at most one row,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n'' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Description: 
When a Sort is ''TopN's input source row count is provably at most one row,then 
we could remove the redundant TopN.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

 

  was:
In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
count is provably at most one row

,then we could remove the redundant TopN.

For example,the sql:
{code:java}
SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the sql 
is :
{code:java}
SELECT count(*) FROM orders  {code}
Above logic are same as Presto/Trino's 
[RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
 rule:

 


> Remove 'Order By x Limit n'' when the input is guaranteed  that is at most 
> one row
> --
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> When a Sort is ''TopN's input source row count is provably at most one 
> row,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Order By x Limit n'' when the input is guaranteed that is at most one row

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Summary: Remove 'Order By x Limit n'' when the input is guaranteed  that is 
at most one row  (was: Remove 'Limit n' when the input is guaranteed to never 
return more than 'n' rows)

> Remove 'Order By x Limit n'' when the input is guaranteed  that is at most 
> one row
> --
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6042) Add test cases for ARRAY-related functions by using spark array function(Spark Library)

2023-10-11 Thread Ran Tao (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ran Tao updated CALCITE-6042:
-
Summary: Add test cases for ARRAY-related functions by using spark array 
function(Spark Library)  (was: Add tests for ARRAY functions(Spark Library))

> Add test cases for ARRAY-related functions by using spark array 
> function(Spark Library)
> ---
>
> Key: CALCITE-6042
> URL: https://issues.apache.org/jira/browse/CALCITE-6042
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.35.0
>Reporter: Ran Tao
>Assignee: Ran Tao
>Priority: Major
>
> Currently, we support a serials of Spark ARRAY functions such as 
> ARRAY/ARRAY_DISTINCT/ARRAY_EXCEPT/ARRAY_POSITION/ARRAY_CONTAINS...
> however, the test cases of these functions use calcite array constructor such 
> as
> {code:java}
> array_except(array[2], array[2, 3]) {code}
> We don't have any testcase related to the Spark's array function, however 
> this is actually important. Because these array functions belong to Spark 
> library, and the Spark's array has some different behaviors with calcite's 
> array constructor.
>  # use map(k, v, ...) rather than map[k, v, ...]
>  # allows empty map such as map(); however calcite map constructor not 
> support it.
> we expect to add some test cases like:
> {code:java}
> array_except(array(2), array(2, 3))  
> array_except(array(2), array())  {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6046) quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will cause the SqlLiteral

2023-10-11 Thread xiaogang zhou (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774311#comment-17774311
 ] 

xiaogang zhou commented on CALCITE-6046:


I understood, the problem is the parser should not parse the comment as a  

[  

and it should be parsed by 

StringLiteral() :
 

> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral
> --
>
> Key: CALCITE-6046
> URL: https://issues.apache.org/jira/browse/CALCITE-6046
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.35.0
>Reporter: xiaogang zhou
>Priority: Major
> Fix For: 1.36.0
>
>
> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral 
>  
> for example with a SQL
>  
> {code:java}
> // code placeholder
> CREATE TABLE source (
>     a BIGINT
> ) comment '测试test'
> WITH (
>   'connector' = 'test'
> ); {code}
> with a parsed Sqlnode, the toString will create a SQL like below, which is 
> not parsable again.
>  
> {code:java}
> // code placeholder
> CREATE TABLE `source` (
>   `a` BIGINT
> )
> COMMENT u&'\5218\51eftest' WITH (
>   'connector' = 'test'
> ) {code}
> I think this is caused by 
> {code:java}
> // code placeholder
> public void quoteStringLiteralUnicode(StringBuilder buf, String val) {
>   buf.append("u&'"); {code}
> not sure if I misconfigured something. Is it possiable to remove the 
> buf.append("u&'"); ?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Limit n' when the input is guaranteed to never return more than 'n' rows

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Issue Type: Improvement  (was: New Feature)

> Remove 'Limit n' when the input is guaranteed to never return more than 'n' 
> rows
> 
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6038) Remove 'Limit n' when the input is guaranteed to never return more than 'n' rows

2023-10-11 Thread LakeShen (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774309#comment-17774309
 ] 

LakeShen commented on CALCITE-6038:
---

Hi [~julianhyde] ,thank you for reminding me.You are right, using 'TopN' is not 
appropriate to describe this JIRA.

I think that 'TopN' is closer to the business layer,so it has multiple 
understandings,such as 'Order By x Limit n', ' ROW_NUMBER() OVER .. + where 
rownum <= N' and so on.

In this jira,I just want to represent 'Order By x Limit n',so using 'Order By x 
Limit n' directly may be more accurate.I want to change the title and 
description of this Jira for using 'Order By x Limit n'.

WDYT?

In CALCITE-6009, if a RelNode is pure Limit which means that is 'Limit N',and 
it doesn't have the 'Offset' and 'Order By' semantics,it could be removed when 
its input RelNode's max row count is less than or equal to the Limit's fetch.

In {-}CALCITE-5994{-}, if a RelNode is pure Order which means that is 'Order 
By',and it doesn't have the 'Offset' and 'Limit' semantics,it could be removed 
when its input RelNode's max row count is less than or equal to one.

This JIRA is more relevant to {-}CALCITE-5994{-}, in -CALCITE-5994- ,we 
restrict Sort must be pure Order(Offset and Limit is null),after this jira,this 
optimization could be applied to 'Order By x' Or 'Order By x Limit n',this JIRA 
is an enhancement.

> Remove 'Limit n' when the input is guaranteed to never return more than 'n' 
> rows
> 
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6046) quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will cause the SqlLiteral

2023-10-11 Thread xiaogang zhou (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774304#comment-17774304
 ] 

xiaogang zhou commented on CALCITE-6046:


[~julianhyde] 

Hi, I found this problem when I used below code to split SQL statements. the 
process is SQL string -> SqlNode -> SQL String
{code:java}
// code placeholder
SqlParser.Config parserConfig = getCurrentSqlParserConfig(sqlDialect);
SqlParser sqlParser = SqlParser.create(sqlContent, parserConfig);
SqlNodeList sqlNodeList = sqlParser.parseStmtList(); 

sqlParser.parse(sqlNodeList.get(0));{code}
the Dialect/ SqlConformance is a costumed one:

[https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/main/java/org/apache/flink/sql/parser/validate/FlinkSqlConformance.java]

 

 

then I found below SQL
{code:java}

// code placeholder
CREATE TABLE source (
    a BIGINT
) comment '测试test'
WITH (
  'connector' = 'test'
);  {code}
transformed to
{code:java}
// code placeholder
CREATE TABLE `source` (
  `a` BIGINT
)
COMMENT u&'\5218\51eftest' WITH (
  'connector' = 'test'
)  {code}
 

and the SQL parser template is like
{code:java}
// code placeholder
SqlCreate SqlCreateTable(Span s, boolean replace, boolean isTemporary) :
{
final SqlParserPos startPos = s.pos();
boolean ifNotExists = false;
SqlIdentifier tableName;
List constraints = new ArrayList();
SqlWatermark watermark = null;
SqlNodeList columnList = SqlNodeList.EMPTY;
   SqlCharStringLiteral comment = null;
   SqlTableLike tableLike = null;
SqlNode asQuery = null;

SqlNodeList propertyList = SqlNodeList.EMPTY;
SqlNodeList partitionColumns = SqlNodeList.EMPTY;
SqlParserPos pos = startPos;
}
{


ifNotExists = IfNotExistsOpt()

tableName = CompoundIdentifier()
[
 { pos = getPos(); TableCreationContext ctx = new 
TableCreationContext();}
TableColumn(ctx)
(
 TableColumn(ctx)
)*
{
pos = pos.plus(getPos());
columnList = new SqlNodeList(ctx.columnList, pos);
constraints = ctx.constraints;
watermark = ctx.watermark;
}

]
[   {
String p = SqlParserUtil.parseString(token.image);
comment = SqlLiteral.createCharString(p, getPos());
}]
[
 
partitionColumns = ParenthesizedSimpleIdentifierList()
]
[

propertyList = TableProperties()
]
[

tableLike = SqlTableLike(getPos())
{
return new SqlCreateTableLike(startPos.plus(getPos()),
tableName,
columnList,
constraints,
propertyList,
partitionColumns,
watermark,
comment,
tableLike,
isTemporary,
ifNotExists);
}
|

asQuery = OrderedQueryOrExpr(ExprContext.ACCEPT_QUERY)
{
return new SqlCreateTableAs(startPos.plus(getPos()),
tableName,
columnList,
constraints,
propertyList,
partitionColumns,
watermark,
comment,
asQuery,
isTemporary,
ifNotExists);
}
]
{
return new SqlCreateTable(startPos.plus(getPos()),
tableName,
columnList,
constraints,
propertyList,
partitionColumns,
watermark,
comment,
isTemporary,
ifNotExists);
}
} {code}
will give a exception :

Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered 
"u&\'\\5218\\51eftest\'" at line 4, column 9.
Was expecting:
     ...

 

> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral
> --
>
> Key: CALCITE-6046
> URL: https://issues.apache.org/jira/browse/CALCITE-6046
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.35.0
>Reporter: xiaogang zhou
>Priority: Major
> Fix For: 1.36.0
>
>
> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral 
>  
> for example with a SQL
>  
> {code:java}
> // code placeholder
> CREATE TABLE source (
>     a BIGINT
> ) comment '测试test'
> WITH (
>   'connector' = 'test'
> ); {code}
> with a parsed Sqlnode, the toString will create a SQL like below, which is 
> not parsable again.
>  
> {code:java}
> // code placeholder
> CREATE TABLE `source` (
>   `a` BIGINT
> )
> COMMENT u&'\5218\51eftest' WITH (
>   'connector' = 'test'
> ) {code}
> I think this is caused by 
> 

[jira] [Created] (CALCITE-6047) Release Calcite 1.36.0

2023-10-11 Thread Benchao Li (Jira)
Benchao Li created CALCITE-6047:
---

 Summary: Release Calcite 1.36.0
 Key: CALCITE-6047
 URL: https://issues.apache.org/jira/browse/CALCITE-6047
 Project: Calcite
  Issue Type: Task
Reporter: Benchao Li
Assignee: Benchao Li
 Fix For: 1.36.0


Releasing Calcite 1.36.0



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6044) Column uniqueness for single-row relations

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774298#comment-17774298
 ] 

Julian Hyde commented on CALCITE-6044:
--

I see; every other combination of columns is a superset of that, and therefore 
a key. If you do that, you should make sure the javadoc is clear.

And while you are fixing the javadoc, change 'Handler.classinitely' to 
'definitely'; there was an over-zealous search-and-replace in CALCITE-4928.

> Column uniqueness for single-row relations
> --
>
> Key: CALCITE-6044
> URL: https://issues.apache.org/jira/browse/CALCITE-6044
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.35.0
>Reporter: Paul Jackson
>Priority: Major
>
> A single-row relation can result from a {{LIMIT 1}} or an aggregation without 
> {{{}GROUP BY{}}}. Every column in one of these relations should be unique by 
> virtue of having a max row count of 1.
> When joining with a single-row relation on a key field, the join result 
> should no longer require that key field for uniqueness. For example, suppose 
> the {{emp}} table had a composite key {{{}(empno,hiredate){}}}. If we join on 
> {{hiredate=max(hiredate)}} then {{empno}} alone should be a unique column:
>  
> {code:java}
> SELECT * FROM emp A
> JOIN ( SELECT MAX(hiredate) last_hired  FROM emp) B
> ON A.hiredate = B.last_hired
> {code}
> {{join(A,B).empno}} should be unique because {{(A.empno,A.hiredate)}} is 
> unique and {{A.hiredate}} is effectively constant because it is equal to 
> {{{}B.last_hired{}}}.
> Here are some {{RelMetadataTests}} cases that I think should pass.
> {code:java}
> @Test void testColumnUniquenessForLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp\n"
>  + "limit 1";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select * from emp\n"
>  + "  limit 1) B\n"
>  + "on A.empno = B.empno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(10), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnAggregation() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select max(empno) AS maxno from emp) B\n"
>  + "on A.empno = B.maxno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForCorrelatedSubquery() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "where empno = (\n"
>  + "  select max(empno) from emp)";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> } {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6044) Column uniqueness for single-row relations

2023-10-11 Thread Paul Jackson (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774297#comment-17774297
 ] 

Paul Jackson commented on CALCITE-6044:
---

It would make sense to me if {{getUniqueKeys}} returned a set containing a 
single, empty set.

> Column uniqueness for single-row relations
> --
>
> Key: CALCITE-6044
> URL: https://issues.apache.org/jira/browse/CALCITE-6044
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.35.0
>Reporter: Paul Jackson
>Priority: Major
>
> A single-row relation can result from a {{LIMIT 1}} or an aggregation without 
> {{{}GROUP BY{}}}. Every column in one of these relations should be unique by 
> virtue of having a max row count of 1.
> When joining with a single-row relation on a key field, the join result 
> should no longer require that key field for uniqueness. For example, suppose 
> the {{emp}} table had a composite key {{{}(empno,hiredate){}}}. If we join on 
> {{hiredate=max(hiredate)}} then {{empno}} alone should be a unique column:
>  
> {code:java}
> SELECT * FROM emp A
> JOIN ( SELECT MAX(hiredate) last_hired  FROM emp) B
> ON A.hiredate = B.last_hired
> {code}
> {{join(A,B).empno}} should be unique because {{(A.empno,A.hiredate)}} is 
> unique and {{A.hiredate}} is effectively constant because it is equal to 
> {{{}B.last_hired{}}}.
> Here are some {{RelMetadataTests}} cases that I think should pass.
> {code:java}
> @Test void testColumnUniquenessForLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp\n"
>  + "limit 1";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select * from emp\n"
>  + "  limit 1) B\n"
>  + "on A.empno = B.empno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(10), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnAggregation() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select max(empno) AS maxno from emp) B\n"
>  + "on A.empno = B.maxno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForCorrelatedSubquery() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "where empno = (\n"
>  + "  select max(empno) from emp)";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> } {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6021) Add CURRENT_DATETIME function (enabled in BigQuery library)

2023-10-11 Thread Julian Hyde (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Hyde updated CALCITE-6021:
-
Fix Version/s: 1.36.0

> Add CURRENT_DATETIME function (enabled in BigQuery library)
> ---
>
> Key: CALCITE-6021
> URL: https://issues.apache.org/jira/browse/CALCITE-6021
> Project: Calcite
>  Issue Type: Bug
>Reporter: Tanner Clary
>Assignee: Tanner Clary
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> Calcite supports parsing and validation for the CURRENT_DATETIME function 
> according to CALCITE-4297 . However a native implementation was never added.
> The docs for the function may be found here: 
>  
> https://cloud.google.com/bigquery/docs/reference/standard-sql/datetime_functions#current_datetime
> An example:
> {{SELECT CURRENT_DATETIME([timezone])}} should return the current time as a 
> {{DATETIME}} (a Calcite TIMESTAMP). There is an optional time zone argument.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6044) Column uniqueness for single-row relations

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774267#comment-17774267
 ] 

Julian Hyde commented on CALCITE-6044:
--

This sounds right.

However, beware trying to make {{getUniqueKeys}} consistent. If a table has N 
columns and 1 row, then any permutation/combination of columns is a key. There 
are 2^N of those, and that would take too much memory.

> Column uniqueness for single-row relations
> --
>
> Key: CALCITE-6044
> URL: https://issues.apache.org/jira/browse/CALCITE-6044
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.35.0
>Reporter: Paul Jackson
>Priority: Major
>
> A single-row relation can result from a {{LIMIT 1}} or an aggregation without 
> {{{}GROUP BY{}}}. Every column in one of these relations should be unique by 
> virtue of having a max row count of 1.
> When joining with a single-row relation on a key field, the join result 
> should no longer require that key field for uniqueness. For example, suppose 
> the {{emp}} table had a composite key {{{}(empno,hiredate){}}}. If we join on 
> {{hiredate=max(hiredate)}} then {{empno}} alone should be a unique column:
>  
> {code:java}
> SELECT * FROM emp A
> JOIN ( SELECT MAX(hiredate) last_hired  FROM emp) B
> ON A.hiredate = B.last_hired
> {code}
> {{join(A,B).empno}} should be unique because {{(A.empno,A.hiredate)}} is 
> unique and {{A.hiredate}} is effectively constant because it is equal to 
> {{{}B.last_hired{}}}.
> Here are some {{RelMetadataTests}} cases that I think should pass.
> {code:java}
> @Test void testColumnUniquenessForLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp\n"
>  + "limit 1";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnLimit1() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select * from emp\n"
>  + "  limit 1) B\n"
>  + "on A.empno = B.empno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(10), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForJoinOnAggregation() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "join (\n"
>  + "  select max(empno) AS maxno from emp) B\n"
>  + "on A.empno = B.maxno";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(9), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> }
> @Test void testColumnUniquenessForCorrelatedSubquery() {
>   final String sql = ""
>  + "select *\n"
>  + "from emp A\n"
>  + "where empno = (\n"
>  + "  select max(empno) from emp)";
>   sql(sql)
>   .assertThatAreColumnsUnique(bitSetOf(0), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(1), is(true))
>   .assertThatAreColumnsUnique(bitSetOf(), is(true));
> } {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6045) CURRENT_TIMESTAMP has incorrect return type

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774265#comment-17774265
 ] 

Julian Hyde commented on CALCITE-6045:
--

This all sounds right. I have been feeling guilty for several years that 
{{CURRENT_TIMESTAMP}} returns a {{TIMESTAMP}} rather than the {{TIMESTAMP WITH 
TIME ZONE}} as required by the SQL standard.

We should fix it, but it will be a breaking change. Also, do you think the 
{{TIMESTAMP WITH TIME ZONE}} data type is in good enough shape to support it?

Note that there is also {{LOCAL_TIMESTAMP}}, which I believe is of type 
{{TIMESTAMP}} and is in the session time zone. I believe that its type and 
value are correct.

Should we add a new function that returns a {{TIMESTAMP WITH LOCAL TIME ZONE}}?

> CURRENT_TIMESTAMP has incorrect return type
> ---
>
> Key: CALCITE-6045
> URL: https://issues.apache.org/jira/browse/CALCITE-6045
> Project: Calcite
>  Issue Type: Bug
>Reporter: Tanner Clary
>Priority: Major
>
> When trying to work on CALCITE-6021, I noticed that {{CURRENT_TIMESTAMP}} 
> currently returns type {{TIMESTAMP}} when it should be 
> {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}}.
> After modifying it, I noticed function was returning the time from (UTC - 
> System TZ) hours ago. For example, I am in {{America/Los_Angeles}} and if I 
> called the function at {{2023-10-10 13:28:00 America/Los_Angeles}}, it would 
> return {{2023-10-10 06:28:00 America/Los_Angeles}}. 
> I think this is because the DataContext {{CURRENT_TIMESTAMP}} variable, which 
> is meant to represent milliseconds since epoch UTC, actually has the timezone 
> offset applied in {{CalciteConnectionImpl#DataContextImpl}} 
> [here|https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/jdbc/CalciteConnectionImpl.java#L442].
>  To be clear: it is meant to represent millis since epoch UTC, but instead it 
> is millis since epoch [system tz], as I understand it. 
> Additionally, I believe the {{getString()}} method for timestamps in 
> AvaticaResultSet should behave similarly to 
> [{{SqlFunctions#timestampWithLocalTimezoneToString()}}|https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java#L4021]
>  when dealing with a {{TIMESTAMP WITH LOCAL TIME ZONE}}. Right now, it does 
> not take the timezone into consideration so although it represents the 
> accurate instant in time, it displays differently than 
> {{CAST(CURRENT_TIMESTAMP AS VARCHAR)}}.
> For example, {{SELECT CURRENT_TIMESTAMP, CAST(CURRENT_TIMESTAMP AS 
> VARCHAR)}}, with the correct return type, returns something like:
> {{2023-10-10 13:28:00 |  2023-10-10 06:28:00.000 America/Los_Angeles}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6009) Add optimization to remove redundant LIMIT that is more than input row count

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774237#comment-17774237
 ] 

Julian Hyde commented on CALCITE-6009:
--

The way that this was combined with SORT_REMOVE_REDUNDANT (added in 
CALCITE-5994) is confusing. That case removes Sort (when there is at most one 
row); this case removes fetch (but only when there is no sort).

The documentation you added SORT_REMOVE_REDUNDANT doesn't explain the 
situation. In CALCITE-6038 you need to improve the documentation significantly.

> Add optimization to remove redundant LIMIT that is more than input row count
> 
>
> Key: CALCITE-6009
> URL: https://issues.apache.org/jira/browse/CALCITE-6009
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In calcite,Limit would be represented by Sort,such as `LogicalSort[fetch = 
> 5]`.
> When the Limit' source row number is less than the Limit's fetch,we could 
> remove the the redundant Limit.
> For example:
> {code:java}
> SELECT * FROM (VALUES 1,2,3,4,5,6) AS t1 LIMIT 10 {code}
> The plan tree is :
> {code:java}
>  LogicalSort(fetch=[10])
>   LogicalProject(t1=[$0])
>     LogicalValues(tuples=[[{ 1 }, { 2 }, { 3 }, { 4 }, { 5 }, { 6 }]]) {code}
> Because the Limit's source max row number is 6,the Limit's fetch is 10,so we 
> could remove the redundant Limit.
> Another example is :
> {code:java}
> SELECT count(*) FROM orders LIMIT 2 {code}
> The plan tree is :
> {code:java}
>  LogicalSort(fetch=[2])
>   LogicalAggregate(group=[{}], EXPR$0=[COUNT()])
>     LogicalTableScan(table=[[tpch, ORDERS]]) {code}
> Because Limit's source max row number is 1,the Limit's fetch is 2, so we 
> could remove the redundant Limit.
> The logic is same as presto's RemoveRedundantLimit 
> rule:https://github.com/prestodb/presto/blob/50fbc07111ecca60a1a5e62755f095fa204120d0/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantLimit.java#L27



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6038) Remove 'Limit n' when the input is guaranteed to never return more than 'n' rows

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774234#comment-17774234
 ] 

Julian Hyde commented on CALCITE-6038:
--

I am thoroughly confused. Does 'TopN' mean Sort+Limit?

I just discovered CALCITE-6009. It is highly relevant to this case and yet you 
don't mention it.

> Remove 'Limit n' when the input is guaranteed to never return more than 'n' 
> rows
> 
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6046) quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will cause the SqlLiteral

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774227#comment-17774227
 ] 

Julian Hyde commented on CALCITE-6046:
--

Can you talk about the behavior rather than the code. What SQL does it 
generate? On which dialect(s) is that SQL invalid? The summary in particular 
should talk about behavior.

Only when you have defined the problem, you can start talking about the fix.

> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral
> --
>
> Key: CALCITE-6046
> URL: https://issues.apache.org/jira/browse/CALCITE-6046
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.35.0
>Reporter: xiaogang zhou
>Priority: Major
> Fix For: 1.36.0
>
>
> quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
> cause the SqlLiteral 
>  
> for example with a SQL
>  
> {code:java}
> // code placeholder
> CREATE TABLE source (
>     a BIGINT
> ) comment '测试test'
> WITH (
>   'connector' = 'test'
> ); {code}
> with a parsed Sqlnode, the toString will create a SQL like below, which is 
> not parsable again.
>  
> {code:java}
> // code placeholder
> CREATE TABLE `source` (
>   `a` BIGINT
> )
> COMMENT u&'\5218\51eftest' WITH (
>   'connector' = 'test'
> ) {code}
> I think this is caused by 
> {code:java}
> // code placeholder
> public void quoteStringLiteralUnicode(StringBuilder buf, String val) {
>   buf.append("u&'"); {code}
> not sure if I misconfigured something. Is it possiable to remove the 
> buf.append("u&'"); ?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove 'Limit n' when the input is guaranteed to never return more than 'n' rows

2023-10-11 Thread Julian Hyde (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Hyde updated CALCITE-6038:
-
Summary: Remove 'Limit n' when the input is guaranteed to never return more 
than 'n' rows  (was: Remove redundant TopN when its input's row count is 
provably at most one row.)

> Remove 'Limit n' when the input is guaranteed to never return more than 'n' 
> rows
> 
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-6038) Remove redundant TopN when its input's row count is provably at most one row.

2023-10-11 Thread Julian Hyde (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774225#comment-17774225
 ] 

Julian Hyde commented on CALCITE-6038:
--

I changed the summary from "Remove redundant TopN when its input's row count is 
provably at most one row." to "Remove 'Limit n' when the input is guaranteed to 
never return more than 'n' rows".

('TopN' is only used by a few dialects. Summaries should not end in '.'. 
Generalize 'one' to 'N'.)

> Remove redundant TopN when its input's row count is provably at most one row.
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-5658) Collection types are un-parsed incorrectly for Apache Spark

2023-10-11 Thread Jira


 [ 
https://issues.apache.org/jira/browse/CALCITE-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guillaume Massé updated CALCITE-5658:
-
Description: 
See
[https://spark.apache.org/docs/latest/sql-ref-datatypes.html]

Complex types

Struct / Row:
{code:java}
SELECT CAST(NULL AS ROW(a INT, b INT)) t    -- Obtained
SELECT CAST(NULL AS struct) t -- Expected{code}
Array:
{code:java}
SELECT CAST(NULL AS int array) t  -- Obtained
SELECT CAST(NULL AS array) t -- Expected{code}
-Map-
{code:java}
SELECT CAST(NULL AS (INT, INT) MAP) -- This is unsupported in Calcite
SELECT CAST(NULL AS map)  -- Expected{code}
-This is probably another bug by itself:-
{code:java}
java.lang.UnsupportedOperationException: Unsupported type when 
convertTypeToSpec: MAP
at 
org.apache.calcite.sql.type.SqlTypeUtil.convertTypeToSpec(SqlTypeUtil.java:1120)
at org.apache.calcite.sql.SqlDialect.getCastSpec(SqlDialect.java:848)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.castNullType(RelToSqlConverter2.java:472)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.visit(RelToSqlConverter2.java:454)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.dispatch(RelToSqlConverter2.java:138){code}
 

Update: Map was resolved in https://issues.apache.org/jira/browse/CALCITE-5570

  was:
See
https://spark.apache.org/docs/latest/sql-ref-datatypes.html

Complex types

Struct / Row:
{code:java}
SELECT CAST(NULL AS ROW(a INT, b INT)) t    -- Obtained
SELECT CAST(NULL AS struct) t -- Expected{code}
Array:
{code:java}
SELECT CAST(NULL AS int array) t  -- Obtained
SELECT CAST(NULL AS array) t -- Expected{code}
Map
{code:java}
SELECT CAST(NULL AS (INT, INT) MAP) -- This is unsupported in Calcite
SELECT CAST(NULL AS map)  -- Expected{code}

This is probably another bug by itself:
{code:java}

java.lang.UnsupportedOperationException: Unsupported type when 
convertTypeToSpec: MAP
at 
org.apache.calcite.sql.type.SqlTypeUtil.convertTypeToSpec(SqlTypeUtil.java:1120)
at org.apache.calcite.sql.SqlDialect.getCastSpec(SqlDialect.java:848)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.castNullType(RelToSqlConverter2.java:472)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.visit(RelToSqlConverter2.java:454)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:532)
at 
org.apache.calcite.rel.rel2sql.RelToSqlConverter2.dispatch(RelToSqlConverter2.java:138){code}


> Collection types are un-parsed incorrectly for Apache Spark
> ---
>
> Key: CALCITE-5658
> URL: https://issues.apache.org/jira/browse/CALCITE-5658
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.34.0
>Reporter: Guillaume Massé
>Priority: Minor
>
> See
> [https://spark.apache.org/docs/latest/sql-ref-datatypes.html]
> Complex types
> Struct / Row:
> {code:java}
> SELECT CAST(NULL AS ROW(a INT, b INT)) t    -- Obtained
> SELECT CAST(NULL AS struct) t -- Expected{code}
> Array:
> {code:java}
> SELECT CAST(NULL AS int array) t  -- Obtained
> SELECT CAST(NULL AS array) t -- Expected{code}
> -Map-
> {code:java}
> SELECT CAST(NULL AS (INT, INT) MAP) -- This is unsupported in Calcite
> SELECT CAST(NULL AS map)  -- Expected{code}
> -This is probably another bug by itself:-
> {code:java}
> java.lang.UnsupportedOperationException: Unsupported type when 
> convertTypeToSpec: MAP
> at 
> org.apache.calcite.sql.type.SqlTypeUtil.convertTypeToSpec(SqlTypeUtil.java:1120)
> at org.apache.calcite.sql.SqlDialect.getCastSpec(SqlDialect.java:848)
> at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverter2.castNullType(RelToSqlConverter2.java:472)
> at 
> org.apache.calcite.rel.rel2sql.RelToSqlConverter2.visit(RelToSqlConverter2.java:454)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
> at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at 

[jira] [Resolved] (CALCITE-5993) Add CODE_POINTS_TO_STRING, TO_CODE_POINTS function (enabled in BigQuery library)

2023-10-11 Thread Tanner Clary (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tanner Clary resolved CALCITE-5993.
---
Resolution: Fixed

Merged via 
[77b3689|https://github.com/apache/calcite/commit/77b3689e907e188d25d76e5d82e5f19a3323dba8],
 thanks for the fix, [~hongyuguo]!

> Add CODE_POINTS_TO_STRING, TO_CODE_POINTS function (enabled in BigQuery 
> library)
> 
>
> Key: CALCITE-5993
> URL: https://issues.apache.org/jira/browse/CALCITE-5993
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Affects Versions: 1.35.0
>Reporter: hongyu guo
>Assignee: hongyu guo
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> CODE_POINTS_TO_STRING: Takes an array of Unicode code points as ARRAY 
> and returns a STRING.
> TO_CODE_POINTS: Takes a STRING or BYTES value and returns an array of INT64 
> values that represent code points or extended ASCII character values.
> If value is a STRING, each element in the returned array represents a code 
> point. Each code point falls within the range of [0, 0xD7FF] and [0xE000, 
> 0x10].
> If value is BYTES, each element in the array is an extended ASCII character 
> value in the range of [0, 255].
> See more in 
> [code_points_to_string|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#code_points_to_string]
>  and 
> [to_code_points|https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#to_code_points]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove redundant TopN when its input's row count is provably at most one row.

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Fix Version/s: 1.36.0

> Remove redundant TopN when its input's row count is provably at most one row.
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.36.0
>
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove redundant TopN when its input's row count is provably at most one row.

2023-10-11 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-6038:

Labels: pull-request-available  (was: )

> Remove redundant TopN when its input's row count is provably at most one row.
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>  Labels: pull-request-available
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (CALCITE-6046) quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will cause the SqlLiteral

2023-10-11 Thread xiaogang zhou (Jira)
xiaogang zhou created CALCITE-6046:
--

 Summary: quoteStringLiteralUnicode returns unparsed string with 
u&' prefix, which will cause the SqlLiteral
 Key: CALCITE-6046
 URL: https://issues.apache.org/jira/browse/CALCITE-6046
 Project: Calcite
  Issue Type: Improvement
  Components: core
Affects Versions: 1.35.0
Reporter: xiaogang zhou
 Fix For: 1.36.0


quoteStringLiteralUnicode returns unparsed string with u&' prefix, which will 
cause the SqlLiteral 

 

for example with a SQL

 
{code:java}
// code placeholder
CREATE TABLE source (
    a BIGINT
) comment '测试test'
WITH (
  'connector' = 'test'
); {code}
with a parsed Sqlnode, the toString will create a SQL like below, which is not 
parsable again.

 
{code:java}
// code placeholder
CREATE TABLE `source` (
  `a` BIGINT
)
COMMENT u&'\5218\51eftest' WITH (
  'connector' = 'test'
) {code}
I think this is caused by 
{code:java}
// code placeholder
public void quoteStringLiteralUnicode(StringBuilder buf, String val) {
  buf.append("u&'"); {code}
not sure if I misconfigured something. Is it possiable to remove the 
buf.append("u&'"); ?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (CALCITE-6038) Remove redundant TopN when its input's row count is provably at most one row.

2023-10-11 Thread LakeShen (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LakeShen updated CALCITE-6038:
--
Component/s: core

> Remove redundant TopN when its input's row count is provably at most one row.
> -
>
> Key: CALCITE-6038
> URL: https://issues.apache.org/jira/browse/CALCITE-6038
> Project: Calcite
>  Issue Type: New Feature
>  Components: core
>Reporter: LakeShen
>Assignee: LakeShen
>Priority: Major
>
> In Calcite , the TopN is represented by `Sort`,when a TopN's input source row 
> count is provably at most one row
> ,then we could remove the redundant TopN.
> For example,the sql:
> {code:java}
> SELECT count(*) FROM orders ORDER BY 1 LIMIT 10 {code}
> because the `SELECT count(*) FROM orders ` row count is 1, it is at most one 
> row,then we could remove `ORDER BY 1 LIMIT 10 `,after the optimization,the 
> sql is :
> {code:java}
> SELECT count(*) FROM orders  {code}
> Above logic are same as Presto/Trino's 
> [RemoveRedundantTopN|https://github.com/prestodb/presto/blob/21ab1ea2425e4bc65532ab156c60333e5a72dd09/presto-main/src/main/java/com/facebook/presto/sql/planner/iterative/rule/RemoveRedundantTopN.java#L27C1-L28C34]
>  rule:
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)