[jira] [Updated] (CALCITE-3130) Implement JSON_UNQUOTE, JSON_QUOTE function

2019-06-21 Thread Forward Xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Forward Xu updated CALCITE-3130:

Description: 
[{{JSON_QUOTE(_{{string_}})}}|https://dev.mysql.com/doc/refman/8.0/en/json-creation-functions.html#function_json-quote]

Quotes a string as a JSON value by wrapping it with double quote characters and 
escaping interior quote and other characters, then returning the result as a 
{{utf8mb4}}string. Returns {{NULL}} if the argument is {{NULL}}.

This function is typically used to produce a valid JSON string literal for 
inclusion within a JSON document.

SQL:
{code:java}
SELECT JSON_QUOTE('null'), JSON_QUOTE('"null"');
{code}
Result:
||Heading 1||Heading 2||
|"null"|"\"null\""|

[{{JSON_UNQUOTE(_{{json_val_}})}}|https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-unquote]

Unquotes JSON value and returns the result as a {{utf8mb4}} string. Returns 
{{NULL}} if the argument is {{NULL}}. An error occurs if the value starts and 
ends with double quotes but is not a valid JSON string literal.

Within a string, certain sequences have special meaning unless the 
[{{NO_BACKSLASH_ESCAPES}}|https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html#sqlmode_no_backslash_escapes]
 SQL mode is enabled. Each of these sequences begins with a backslash ({{}}), 
known as the _escape character_. MySQL recognizes the escape sequences shown in 
[Table 12.22, “JSON_UNQUOTE() Special Character Escape 
Sequences”|https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#json-unquote-character-escape-sequences].
 For all other escape sequences, backslash is ignored. That is, the escaped 
character is interpreted as if it was not escaped. For example, {{\x}} is just 
{{x}}. These sequences are case-sensitive. For example, {{\b}} is interpreted 
as a backspace, but {{\B}} is interpreted as {{B}}.

SQL:
{code:java}
SELECT JSON_UNQUOTE('"\\t\\u0032"');
{code}
Result:
||Heading 1||
|2|

Note:

[https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mode-setting]

 The JSON_UNQUOTE function needs to depend on the NO_BACKSLASH_ESCAPES setting 
of sql_mode.

 

  was:
[{{JSON_QUOTE(_{{string_}})}}|https://dev.mysql.com/doc/refman/8.0/en/json-creation-functions.html#function_json-quote]

Quotes a string as a JSON value by wrapping it with double quote characters and 
escaping interior quote and other characters, then returning the result as a 
{{utf8mb4}}string. Returns {{NULL}} if the argument is {{NULL}}.

This function is typically used to produce a valid JSON string literal for 
inclusion within a JSON document.

SQL:
{code:java}
SELECT JSON_QUOTE('null'), JSON_QUOTE('"null"');
{code}
Result:
||Heading 1||Heading 2||
|"null"|"\"null\""|

[{{JSON_UNQUOTE(_{{json_val_}})}}|https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-unquote]

Unquotes JSON value and returns the result as a {{utf8mb4}} string. Returns 
{{NULL}} if the argument is {{NULL}}. An error occurs if the value starts and 
ends with double quotes but is not a valid JSON string literal.

Within a string, certain sequences have special meaning unless the 
[{{NO_BACKSLASH_ESCAPES}}|https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html#sqlmode_no_backslash_escapes]
 SQL mode is enabled. Each of these sequences begins with a backslash ({{}}), 
known as the _escape character_. MySQL recognizes the escape sequences shown in 
[Table 12.22, “JSON_UNQUOTE() Special Character Escape 
Sequences”|https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#json-unquote-character-escape-sequences].
 For all other escape sequences, backslash is ignored. That is, the escaped 
character is interpreted as if it was not escaped. For example, {{\x}} is just 
{{x}}. These sequences are case-sensitive. For example, {{\b}} is interpreted 
as a backspace, but {{\B}} is interpreted as {{B}}.

SQL:
{code:java}
SELECT JSON_UNQUOTE('"\\t\\u0032"');
{code}
Result:
||Heading 1||
|2|

 


> Implement JSON_UNQUOTE, JSON_QUOTE function
> ---
>
> Key: CALCITE-3130
> URL: https://issues.apache.org/jira/browse/CALCITE-3130
> Project: Calcite
>  Issue Type: Sub-task
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> [{{JSON_QUOTE(_{{string_}})}}|https://dev.mysql.com/doc/refman/8.0/en/json-creation-functions.html#function_json-quote]
> Quotes a string as a JSON value by wrapping it with double quote characters 
> and escaping interior quote and other characters, then returning the result 
> as a {{utf8mb4}}string. Returns {{NULL}} if the argument is {{NULL}}.
> This function is typically used to produce a valid JSON string literal for 
> inclusion within a JSON 

[jira] [Updated] (CALCITE-3137) AssertionError when reconstructing STRUCTURED type fields

2019-06-21 Thread Haisheng Yuan (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haisheng Yuan updated CALCITE-3137:
---
Description: 
Add the following test to SqlToRelConverterTest.java.

{code:java}
@Test
  public void testRecontructStructFields() {
final String sql = "select HOME_ADDRESS from EMP_ADDRESS ";
sql(sql).convertsTo(
"LogicalProject(HOME_ADDRESS=[NEW($1.CITY, $1.ZIP, $1.STATE, 
$2.STREET):ObjectSqlType(ADDRESS) NOT NULL])\n"
+ "  LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]])"
);
  }
{code}

Error:

{code:java}
java.lang.AssertionError: wrong operand count 4 for NEW

at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
at 
org.apache.calcite.sql.SqlPrefixOperator.validRexOperands(SqlPrefixOperator.java:100)
at org.apache.calcite.rex.RexCall.(RexCall.java:86)
at 
org.apache.calcite.rex.RexBuilder.makeNewInvocation(RexBuilder.java:494)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructure(RelStructuredTypeFlattener.java:249)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:227)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:206)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:468)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:614)
{code}



  was:
Add the following test to SqlToRelConverterTest.java.

{code:java}
@Test
  public void testRecontructStructFields() {
final String sql = "select HOME_ADDRESS from EMP_ADDRESS ";
sql(sql).convertsTo(
"LogicalProject(HOME_ADDRESS=[NEW($1.CITY, $1.ZIP, $1.STATE, 
$2.STREET):ObjectSqlType(ADDRESS) NOT NULL])\n"
+ "  LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]])"
);
  }
{code}

Error:

{code:java}
java.lang.AssertionError: wrong operand count 4 for NEW

at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
at 
org.apache.calcite.sql.SqlPrefixOperator.validRexOperands(SqlPrefixOperator.java:100)
at org.apache.calcite.rex.RexCall.(RexCall.java:86)
at 
org.apache.calcite.rex.RexBuilder.makeNewInvocation(RexBuilder.java:494)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructure(RelStructuredTypeFlattener.java:249)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:227)
at 
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:206)
at 
org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:468)
at 
org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:614)
{code}

The assertion should be removed.



> AssertionError when reconstructing STRUCTURED type fields
> -
>
> Key: CALCITE-3137
> URL: https://issues.apache.org/jira/browse/CALCITE-3137
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> Add the following test to SqlToRelConverterTest.java.
> {code:java}
> @Test
>   public void testRecontructStructFields() {
> final String sql = "select HOME_ADDRESS from EMP_ADDRESS ";
> sql(sql).convertsTo(
> "LogicalProject(HOME_ADDRESS=[NEW($1.CITY, $1.ZIP, $1.STATE, 
> $2.STREET):ObjectSqlType(ADDRESS) NOT NULL])\n"
> + "  LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]])"
> );
>   }
> {code}
> Error:
> {code:java}
> java.lang.AssertionError: wrong operand count 4 for NEW
>   at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
>   at 
> org.apache.calcite.sql.SqlPrefixOperator.validRexOperands(SqlPrefixOperator.java:100)
>   at org.apache.calcite.rex.RexCall.(RexCall.java:86)
>   at 
> org.apache.calcite.rex.RexBuilder.makeNewInvocation(RexBuilder.java:494)
>   at 
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructure(RelStructuredTypeFlattener.java:249)
>   at 
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:227)
>   at 
> org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:206)
>   at 
> org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:468)
>   at 
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:614)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (CALCITE-3138) Restructuring ROW type fields is not supported

2019-06-21 Thread Igor Guzenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Guzenko reassigned CALCITE-3138:
-

Assignee: Igor Guzenko

> Restructuring ROW type fields is not supported
> --
>
> Key: CALCITE-3138
> URL: https://issues.apache.org/jira/browse/CALCITE-3138
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Haisheng Yuan
>Assignee: Igor Guzenko
>Priority: Major
> Attachments: ROW_repro.patch
>
>
> ROW type is not supported to restructure fields after flattening (see 
> RelStructuredTypeFlattener.restructureFields), which may cause wrong result. 
> Attached patch file can be used to reproduce the issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2624) Add a rule to copy a sort below a join operator

2019-06-21 Thread Stamatis Zampetakis (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869451#comment-16869451
 ] 

Stamatis Zampetakis commented on CALCITE-2624:
--

[~hyuan] I think I see your point. Concretely, what you are proposing is 
instead of adding the generic rule in this PR to rather modify 
Enumerable*Join*Rule(s) to change the traits of the inputs when the join has 
some collation trait associated with it (and when that is possible). Please 
correct me if I am wrong.

> Add a rule to copy a sort below a join operator
> ---
>
> Key: CALCITE-2624
> URL: https://issues.apache.org/jira/browse/CALCITE-2624
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.17.0
>Reporter: Stamatis Zampetakis
>Assignee: Khawla Mouhoubi
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently, the only rule that allows a sort to traverse a binary operator is 
> the SortJoinTransposeRule. The rule was introduced mainly to push limits in 
> the case of left and right outer joins (see CALCITE-831).
> I assume that the main reason that we don't have more rules is that sorts 
> with limits and offsets cannot be pushed safely below many types of join 
> operators. However, in many cases, it is possible and beneficial for 
> optimization purposes to just push the sort without the limit and offset. 
> Since we do not know in advance if the join operator preserves the order we 
> cannot remove (that is why I am saying copy and not transpose) the sort 
> operator on top of the join. The latter is not really a problem since the 
> SortRemoveRule can detect such cases and remove the sort if it is redundant.
> A few concrete examples where this optimization makes sense are outlined 
> below:
>  * allow the sort to be later absorbed by an index scan and disappear from 
> the plan (Sort + Tablescan => IndexScan with RelCollation);
>  * allow operators that require sorted inputs to be exploited more easily 
> (e.g., merge join);
>  * allow the sort to be performed on a possibly smaller result (assuming that 
> the physical binary operator that is going to be used preserves the order of 
> left/right input and the top sort operator can be removed entirely).
> I propose to add a new rule (e.g., SortCopyBelowJoinRule, 
> SortJoinCopyBelowRule) which allows a sort to be copied to the left or right 
> (or to both if it is rather easy to decompose the sort) of a join operator 
> (excluding the limit and offset attributes) if the respective inputs are not 
> already sorted. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3136) Fix the default rule description of ConverterRule.

2019-06-21 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3136:

Labels: pull-request-available  (was: )

> Fix the default rule description of ConverterRule.
> --
>
> Key: CALCITE-3136
> URL: https://issues.apache.org/jira/browse/CALCITE-3136
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>  Labels: pull-request-available
>
> The default rule description of ConvertRule does not obey the description 
> pattern of RelOptRule.
> {code:java}
> public RelOptRule(RelOptRuleOperand operand,
> RelBuilderFactory relBuilderFactory, String description) {
>   this.operand = Objects.requireNonNull(operand);
>   this.relBuilderFactory = Objects.requireNonNull(relBuilderFactory);
>   if (description == null) {
> description = guessDescription(getClass().getName());
>   }
>   if (!description.matches("[A-Za-z][-A-Za-z0-9_.():]*")) {
> throw new RuntimeException("Rule description '" + description
> + "' is not valid");
>   }
>   this.description = description;
>   this.operands = flattenOperands(operand);
>   assignSolveOrder();
> }
> {code}
> {code:java}
> public  ConverterRule(Class clazz,
> Predicate predicate, RelTrait in, RelTrait out,
> RelBuilderFactory relBuilderFactory, String description) {
>   super(convertOperand(clazz, predicate, in),
>   relBuilderFactory,
>   description == null
>   ? "ConverterRule"
>   : description);
>   this.inTrait = Objects.requireNonNull(in);
>   this.outTrait = Objects.requireNonNull(out);
>   // Source and target traits must have same type
>   assert in.getTraitDef() == out.getTraitDef();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-771) Use materialization for scan-project-sort query

2019-06-21 Thread Feng Zhu (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869221#comment-16869221
 ] 

Feng Zhu commented on CALCITE-771:
--

Hi, [~julianhyde], I opened a PR on this feature, do you have time to review?

> Use materialization for scan-project-sort query
> ---
>
> Key: CALCITE-771
> URL: https://issues.apache.org/jira/browse/CALCITE-771
> Project: Calcite
>  Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If a materialization is defined for scan-project-sort over another table, 
> then scan-project and scan-project-sort queries should be able to use it.
> For example, given the following schema (in pseudo-DDL):
> {code}
> CREATE TABLE Emp (empid, deptno, gender);
> CREATE MATERIALIZATION I_Emp_Deptno AS
>   SELECT deptno, empid FROM Emp ORDER BY deptno;
> {code}
> Calcite should rewrite the query
> {code}SELECT deptno FROM Emp ORDER BY deptno{code}
> to
> {code}
> Project(deptno)
>   TableScan(table=[I_Emp_Deptno])
> {code}
> Note that no sort is necessary.
> This rewrite is important for using secondary indexes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)