[jira] [Comment Edited] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100394#comment-17100394 ] Chunwei Lei edited comment on CALCITE-3972 at 5/6/20, 5:19 AM: --- Add the PR link: [https://github.com/apache/calcite/pull/1884|https://github.com/apache/calcite/pull/1884.] was (Author: chunwei lei): Add the PR link: [https://github.com/apache/calcite/pull/1884|https://github.com/apache/calcite/pull/1884.] > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100394#comment-17100394 ] Chunwei Lei edited comment on CALCITE-3972 at 5/6/20, 5:19 AM: --- Add the PR link: [https://github.com/apache/calcite/pull/1884|https://github.com/apache/calcite/pull/1884.] was (Author: chunwei lei): Add the PR link: [https://github.com/apache/calcite/pull/1884.] > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3963) Maintain logical properties at RelSet (equivalent group) instead of RelNode
[ https://issues.apache.org/jira/browse/CALCITE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiening Dai updated CALCITE-3963: - Summary: Maintain logical properties at RelSet (equivalent group) instead of RelNode (was: Maintains logical properties at RelSet (equivalent group) instead of RelNode) > Maintain logical properties at RelSet (equivalent group) instead of RelNode > --- > > Key: CALCITE-3963 > URL: https://issues.apache.org/jira/browse/CALCITE-3963 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > > Currently the logical properties (such as row count, distinct row count, etc) > are maintained at RelNode level. This creates a number of meta data > consistency problems, e.g. CALCITE-1048, CALCITE-2166. > In theory, all RelNodes in a RelSet should share the same logical properties > per definition of relational equivalence. So it makes more sense to keep > logical properties at RelSet level, rather than the RelNode. And such > properties shouldn't change when new sub set is created or subset's best is > changed. > Specifically I think below build in metadata should fall into the logical > properties category - > Selectivity > UniqueKeys > ColumnUniqueness > RowCount > MaxRowCount > MinRowCount > DistinctRowCount > Size (averageRowSize, averageColumnSize) > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3963) Maintains logical properties at RelSet (equivalent group) instead of RelNode
[ https://issues.apache.org/jira/browse/CALCITE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100463#comment-17100463 ] Jinpeng Wu commented on CALCITE-3963: - I think we all agree that RelNodes in a RelSet should share the same logical properties. The difference is how to do this. I agree with Julian that MetadataQuery is a good design to propagate logical properties for new RelNode. Storing a concrete value associate with a RelSet require complicated logic to maintain and invalidate the cached value. If some logic is considered flawed, it is a bug of metadata handler. It should be metadata handler's job to ensure logical properties across the RelSet is consistent. Haisheng mentioned that we have to decide when this value is used for logical space pruning. I think we can add a state field to RelSet, for example, EXPLORED or SUBSTITUTION_APPLIED. MetadataHandler can also leverage this value to decide its logic. This value requires invalidation when RelSets get merged. But it should be much simpler than storing a concrete metadata result. This strategy is somewhat like combining option one and option two. When new RelNode is registered into a RelSet, logical properties are recomputed as cache in RelMetadataQuery is invalidated. This value can not be used for logical space pruning until the RelSet is in a suitable state. And how to decide the state? It may be difficult now, but much simpler in top-down rule applying strategy. > Maintains logical properties at RelSet (equivalent group) instead of RelNode > > > Key: CALCITE-3963 > URL: https://issues.apache.org/jira/browse/CALCITE-3963 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > > Currently the logical properties (such as row count, distinct row count, etc) > are maintained at RelNode level. This creates a number of meta data > consistency problems, e.g. CALCITE-1048, CALCITE-2166. > In theory, all RelNodes in a RelSet should share the same logical properties > per definition of relational equivalence. So it makes more sense to keep > logical properties at RelSet level, rather than the RelNode. And such > properties shouldn't change when new sub set is created or subset's best is > changed. > Specifically I think below build in metadata should fall into the logical > properties category - > Selectivity > UniqueKeys > ColumnUniqueness > RowCount > MaxRowCount > MinRowCount > DistinctRowCount > Size (averageRowSize, averageColumnSize) > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3973) Writing SQL hints to a string results in SQL that cannot be parsed
[
https://issues.apache.org/jira/browse/CALCITE-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100436#comment-17100436
]
Danny Chen commented on CALCITE-3973:
-
Thanks [~alexbaden] for reporting this ~
You are right, in SqlSelectOperator, the leftPrec and rightPrec should both be
hard code to 0 [1], after fixing that, please also add a test case in
SqlParserTest#testQueryHint.
[1]
https://github.com/apache/calcite/blob/d1aae57fad71bf10942d4c53b339363537927c29/core/src/main/java/org/apache/calcite/sql/SqlSelectOperator.java#L150
> Writing SQL hints to a string results in SQL that cannot be parsed
> ---
>
> Key: CALCITE-3973
> URL: https://issues.apache.org/jira/browse/CALCITE-3973
> Project: Calcite
> Issue Type: Bug
>Reporter: Alex Baden
>Priority: Minor
>
> When using the new SQL hints feature:
>
> {code:java}
> select /*+ cpu */ x from test limit 2;
> {code}
> If one calls to SqlString() on the node:
>
>
> {code:java}
> node.toSqlString(CalciteSqlDialect.DEFAULT).toString()
> {code}
>
> We get:
>
> {code:java}
> 'SELECT /*+ ("cpu") */ "x" FROM "test" FETCH NEXT 2 ROWS ONLY'
> {code}
> {{which is not valid SQL for a hint. It looks like the precedence options
> need to be modified when writing out the hint in SqlSelectOperator.unparse}}
>
> We do some rewriting of the sql statement and then re-parse it, so this is
> preventing us from using hints. I am happy to submit a patch if I am headed
> in the right direction with the description above.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100431#comment-17100431
]
Xiening Dai commented on CALCITE-2970:
--
Also the RelBuilder is not immutable today. The stack can change after it's
passed into a function. So it's not a new problem...
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100427#comment-17100427
]
Xiening Dai commented on CALCITE-2970:
--
{quote}
It's not about the overhead of object creation.
{quote}
I mean with the adoptConvention() we create a new builder every time. It could
be problematic if this gets call frequently (such in high frequent rules).
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3970) Table-valued function TUMBLE uses non-standard syntax
[
https://issues.apache.org/jira/browse/CALCITE-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100402#comment-17100402
]
Danny Chen commented on CALCITE-3970:
-
[~amaliujia], previously we have supported a sub-query as the first argument of
the TUMBLE function, after we solve the CALCITE-3955, if we translate the
query/table argument of the table function, we should apply a explicit CURSOR
keyword for the query, that is CURSOR(query) <=> Table t.
> Table-valued function TUMBLE uses non-standard syntax
> -
>
> Key: CALCITE-3970
> URL: https://issues.apache.org/jira/browse/CALCITE-3970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Viliam Durina
>Priority: Major
>
> The currently supported syntax is this:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE my_table, ...
> {code}
> But the SQL standard specifies that {{my_table}} must be in parentheses, such
> as here:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE(my_table), ...
> {code}
> The second syntax is currently rejected with:
> {code:none}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered "(" at line 1, column 33.
> Was expecting one of:
> ...
> ...
> ...
> ...
> ...
> {code}
> I'm not sure if the currently supported syntax is optional, but I think it's
> not.
> I followed this document:
> [http://standards.iso.org/ittf/PubliclyAvailableStandards/c069776_ISO_IEC_TR_19075-7_2017.zip]
> The {{TABLE}} clause acts in both ways: (1) to convert a table value to a
> table object (when used in the {{FROM}} clause to convert the function
> result) and (2) to convert a table object to a table value (when used to
> convert arguments to a function).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100394#comment-17100394 ] Chunwei Lei commented on CALCITE-3972: -- Add the PR link: [https://github.com/apache/calcite/pull/1884.] > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3973) Writing SQL hints to a string results in SQL that cannot be parsed
Alex Baden created CALCITE-3973:
---
Summary: Writing SQL hints to a string results in SQL that cannot
be parsed
Key: CALCITE-3973
URL: https://issues.apache.org/jira/browse/CALCITE-3973
Project: Calcite
Issue Type: Bug
Reporter: Alex Baden
When using the new SQL hints feature:
{code:java}
select /*+ cpu */ x from test limit 2;
{code}
If one calls to SqlString() on the node:
{code:java}
node.toSqlString(CalciteSqlDialect.DEFAULT).toString()
{code}
We get:
{code:java}
'SELECT /*+ ("cpu") */ "x" FROM "test" FETCH NEXT 2 ROWS ONLY'
{code}
{{which is not valid SQL for a hint. It looks like the precedence options need
to be modified when writing out the hint in SqlSelectOperator.unparse}}
We do some rewriting of the sql statement and then re-parse it, so this is
preventing us from using hints. I am happy to submit a patch if I am headed in
the right direction with the description above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100341#comment-17100341 ] Haisheng Yuan commented on CALCITE-3972: [~julianhyde] [~zabetak] The new issue is here. > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100340#comment-17100340 ] Haisheng Yuan commented on CALCITE-3972: Spin out issue of CALCITE-2970. > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan resolved CALCITE-2970.
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/f17367e6ae4814c035e234568e212ba23080e9dd].
For physical RelBuilder, see https://issues.apache.org/jira/browse/CALCITE-3972.
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3972: --- Fix Version/s: 1.23.0 > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Fix For: 1.23.0 > > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3971) Support Calc in RelMdColumnOrigins
[
https://issues.apache.org/jira/browse/CALCITE-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan resolved CALCITE-3971.
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/d1aae57fad71bf10942d4c53b339363537927c29].
> Support Calc in RelMdColumnOrigins
> --
>
> Key: CALCITE-3971
> URL: https://issues.apache.org/jira/browse/CALCITE-3971
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Major
> Fix For: 1.23.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In my project, we often use `RelColumnOrigin` to find metadata information
> for columns.
> When i try to get origin column , and i got exception as below, the exception
> can be reproduced.
> {code:java}
> final String sql = "select name,deptno from dept";
> final RelNode relNode = convertSql(sql);
> final HepProgram program = new HepProgramBuilder().
> addRuleInstance(ProjectToCalcRule.INSTANCE).build();
> final HepPlanner planner = new HepPlanner(program);
> planner.setRoot(relNode);
> final RelNode calc = planner.findBestExp();
> final RelMetadataQuery mq = calc.getCluster().getMetadataQuery();
> final RelColumnOrigin nameColumn = mq.getColumnOrigin(calc, 0);
> assertThat(nameColumn.getOriginColumnOrdinal(), is(1));
> {code}
> java.lang.NullPointerException
> at
> org.apache.calcite.test.RelMetadataTest.testCalcColumnOriginsTable(RelMetadataTest.java:391)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3966) Trigger rules for existing RelSubset when it becomes delivered
[ https://issues.apache.org/jira/browse/CALCITE-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3966. Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/e7224147916d5a9bf43c322894c04b302b3a545d]. > Trigger rules for existing RelSubset when it becomes delivered > -- > > Key: CALCITE-3966 > URL: https://issues.apache.org/jira/browse/CALCITE-3966 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > Fix For: 1.23.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Existing RelSubset may be first created by parent trait request, so its state > is required. But when there is a new RelNode in the same RelSet can deliver > the same traitset, we mark the subset as derived/delivered too, in which > case, we should fire rule for the subset too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3718) Support Intersect and Minus in Bindables
[ https://issues.apache.org/jira/browse/CALCITE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3718. Fix Version/s: 1.23.0 Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/0cdb7a8a520eabdf103519a399c148b0df3ba20b]. > Support Intersect and Minus in Bindables > > > Key: CALCITE-3718 > URL: https://issues.apache.org/jira/browse/CALCITE-3718 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Intersect and Minus are not supported in Bindables, this pr try support > Intersect and Minus in Bindables. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3969) Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't cover collation or distribution keys
[
https://issues.apache.org/jira/browse/CALCITE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100281#comment-17100281
]
Haisheng Yuan commented on CALCITE-3969:
I don't think there is a need for \{{RelTrait.canApply(TargetMapping)}}. e.g.
RelCollation, the result is either another collation or not sorted anymore.
> Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't
> cover collation or distribution keys
> --
>
> Key: CALCITE-3969
> URL: https://issues.apache.org/jira/browse/CALCITE-3969
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Let's consider we have an input {{(id, name)}} ordered by {{id}} (i.e.
> collation == {{[0]}}). If we have a {{Project("name")}} on the top of this
> input and we apply project's mapping on the collation, we'll end up with
> exception:
> {noformat}
> java.lang.NullPointerException: at index 0
> at
> com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:239)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:230)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:225)
> at
> com.google.common.collect.ImmutableList.construct(ImmutableList.java:281)
> at
> com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:239)
> at org.apache.calcite.rel.RelCollations.of(RelCollations.java:69)
> at org.apache.calcite.rex.RexUtil.apply(RexUtil.java:1271)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:122)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:40)
> {noformat}
> This happens because the collation field {{id}} is not a part of the mapping.
> The same problem is with distribution trait when distribution keys are not
> covered by the mapping.
> Calcite should handle such situations gracefully. If it is not possible to
> deduce the collation/distribution after the mapping application, we should
> return:
> * {{EMPTY}} collation for {{RelCollation}} trait.
> * {{RANDOM_DISTRIBUTED}} distribution for {{RelDistribution}} trait.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
Xiening Dai created CALCITE-3972: Summary: Allow RelBuilder to create RelNode with convention and use it for trait convert Key: CALCITE-3972 URL: https://issues.apache.org/jira/browse/CALCITE-3972 Project: Calcite Issue Type: Bug Reporter: Xiening Dai 1. Provide Convention.transformRelBuilder() to transform an existing RelBuilder into one with specific convention. 2. RelBuilder provides withRelFactories() method to allow caller swap the underlying RelFactories and create a new builder. 3. Use the new interface in RelCollationTraitDef for converting into RelCollation traits We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiening Dai reassigned CALCITE-3972: Assignee: Xiening Dai > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100255#comment-17100255
]
Julian Hyde commented on CALCITE-2970:
--
It's not about the overhead of object creation.
Adding a 'set' method to a class that has no 'set' methods completely changes
the nature of the class and makes it more difficult to use safely.
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100223#comment-17100223
]
Xiening Dai commented on CALCITE-2970:
--
It would have been easier if Java supports "const T &" parameter like C++ does,
:(
I will forge ahead with this proposal. Later if I can quantify the overhead of
object creation, we can come back for discussion.
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3969) Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't cover collation or distribution keys
[
https://issues.apache.org/jira/browse/CALCITE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100211#comment-17100211
]
Julian Hyde commented on CALCITE-3969:
--
The new 'if' in
[getSourceOpt|https://github.com/apache/calcite/pull/1959/files#diff-fc0ae42ce6b0c7b3e645f7e65932b5d1R1249]
is too defensive. Target should never be out of range (greater or equal to
targetCount, or less than zero). If it is, the method should throw.
{{getSourceOpt}}, {{getTargetOpt}} need javadoc.
> Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't
> cover collation or distribution keys
> --
>
> Key: CALCITE-3969
> URL: https://issues.apache.org/jira/browse/CALCITE-3969
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Let's consider we have an input {{(id, name)}} ordered by {{id}} (i.e.
> collation == {{[0]}}). If we have a {{Project("name")}} on the top of this
> input and we apply project's mapping on the collation, we'll end up with
> exception:
> {noformat}
> java.lang.NullPointerException: at index 0
> at
> com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:239)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:230)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:225)
> at
> com.google.common.collect.ImmutableList.construct(ImmutableList.java:281)
> at
> com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:239)
> at org.apache.calcite.rel.RelCollations.of(RelCollations.java:69)
> at org.apache.calcite.rex.RexUtil.apply(RexUtil.java:1271)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:122)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:40)
> {noformat}
> This happens because the collation field {{id}} is not a part of the mapping.
> The same problem is with distribution trait when distribution keys are not
> covered by the mapping.
> Calcite should handle such situations gracefully. If it is not possible to
> deduce the collation/distribution after the mapping application, we should
> return:
> * {{EMPTY}} collation for {{RelCollation}} trait.
> * {{RANDOM_DISTRIBUTED}} distribution for {{RelDistribution}} trait.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3969) Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't cover collation or distribution keys
[
https://issues.apache.org/jira/browse/CALCITE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100209#comment-17100209
]
Julian Hyde commented on CALCITE-3969:
--
I see that you fixed some genuine bugs in Mappings, and added tests. Thanks.
Please modify the javadoc of {{RelTrait.apply}} to fully describe the behavior.
When does the method throw? When does it return null?
Should there be a method {{RelTrait.canApply(TargetMapping)}}?
Please also review methods {{RelDistribution.apply}},
{{RexUtil.apply(TargetMapping, FieldCollation)}}, and
{{RexUtil.applyFields(TargetMapping, FieldCollation)}}, their javadoc, and how
they are used. I see a couple of pieces of code that calls these methods and
does not handle a null return correctly. Wrap in {{Objects.requireNonNull}} if
a null return is impossible.
In {{RelDistributionTest}} and {{RelCollationTest}} it would be more
straightforward to pass {{MAPPING_SOURCE_COUNT}} as an argument to
{{mapping()}}. Assign it to a variable {{n}} so that the code is not too
verbose.
> Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't
> cover collation or distribution keys
> --
>
> Key: CALCITE-3969
> URL: https://issues.apache.org/jira/browse/CALCITE-3969
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Let's consider we have an input {{(id, name)}} ordered by {{id}} (i.e.
> collation == {{[0]}}). If we have a {{Project("name")}} on the top of this
> input and we apply project's mapping on the collation, we'll end up with
> exception:
> {noformat}
> java.lang.NullPointerException: at index 0
> at
> com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:239)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:230)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:225)
> at
> com.google.common.collect.ImmutableList.construct(ImmutableList.java:281)
> at
> com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:239)
> at org.apache.calcite.rel.RelCollations.of(RelCollations.java:69)
> at org.apache.calcite.rex.RexUtil.apply(RexUtil.java:1271)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:122)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:40)
> {noformat}
> This happens because the collation field {{id}} is not a part of the mapping.
> The same problem is with distribution trait when distribution keys are not
> covered by the mapping.
> Calcite should handle such situations gracefully. If it is not possible to
> deduce the collation/distribution after the mapping application, we should
> return:
> * {{EMPTY}} collation for {{RelCollation}} trait.
> * {{RANDOM_DISTRIBUTED}} distribution for {{RelDistribution}} trait.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3926) CannotPlanException when an empty LogicalValues requires a certain collation
[
https://issues.apache.org/jira/browse/CALCITE-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L reassigned CALCITE-3926:
--
Assignee: Ruben Q L
> CannotPlanException when an empty LogicalValues requires a certain collation
>
>
> Key: CALCITE-3926
> URL: https://issues.apache.org/jira/browse/CALCITE-3926
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> The issue can be reproduced with the following test (in RelBuilderTest.java):
> {code:java}
> @Test void testEmptyValues() throws Exception {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelNode root =
> builder
> .scan("DEPT")
> .filter(builder.literal(false)) // generates an empty
> LogicalValues
> .sort(
> builder.field("DNAME"),
> builder.field("DEPTNO"))
> .build();
> try (PreparedStatement preparedStatement = RelRunners.run(root)) {
> final String s =
> CalciteAssert.toString(preparedStatement.executeQuery());
> final String result = "";
> assertThat(s, is(result));
> }
> }
> {code}
> The test fails with the following message:
> {code:java}
> Caused by: org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There
> are not enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[1, 0].
> Missing conversion is LogicalValues[convention: NONE -> ENUMERABLE, sort:
> [[0, 1, 2], [1, 2], [2]] -> [1, 0]]
> There is 1 empty subset: rel#14:Subset#0.ENUMERABLE.[1, 0], the relevant part
> of the original plan is as follows
> 1:LogicalValues(tuples=[[]])
> {code}
> The issue seems to be caused by the empty values: Scan + Filter(false) =>
> LogicalValues(tuples=[[]]).
> This empty LogicalValues has as collation (in case of 3 fields): [[0, 1, 2],
> [1, 2], [2]]; and in the plan we are requesting as sort collation [1, 0].
> This could be achieved by: EnumerableSort(sort=[1,0]) +
> EnumerableValues(tuples=[[]]) ; but for some reason this plan is not
> generated.
> It seems the issue was somehow introduced by CALCITE-3886
>
> [https://github.com/apache/calcite/commit/08f4a9889728f94903ff424ed9c20d940b821af5]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3926) CannotPlanException when an empty LogicalValues requires a certain collation
[
https://issues.apache.org/jira/browse/CALCITE-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-3926:
-
Affects Version/s: (was: 1.23.0)
1.22.0
> CannotPlanException when an empty LogicalValues requires a certain collation
>
>
> Key: CALCITE-3926
> URL: https://issues.apache.org/jira/browse/CALCITE-3926
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Ruben Q L
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> The issue can be reproduced with the following test (in RelBuilderTest.java):
> {code:java}
> @Test void testEmptyValues() throws Exception {
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelNode root =
> builder
> .scan("DEPT")
> .filter(builder.literal(false)) // generates an empty
> LogicalValues
> .sort(
> builder.field("DNAME"),
> builder.field("DEPTNO"))
> .build();
> try (PreparedStatement preparedStatement = RelRunners.run(root)) {
> final String s =
> CalciteAssert.toString(preparedStatement.executeQuery());
> final String result = "";
> assertThat(s, is(result));
> }
> }
> {code}
> The test fails with the following message:
> {code:java}
> Caused by: org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There
> are not enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[1, 0].
> Missing conversion is LogicalValues[convention: NONE -> ENUMERABLE, sort:
> [[0, 1, 2], [1, 2], [2]] -> [1, 0]]
> There is 1 empty subset: rel#14:Subset#0.ENUMERABLE.[1, 0], the relevant part
> of the original plan is as follows
> 1:LogicalValues(tuples=[[]])
> {code}
> The issue seems to be caused by the empty values: Scan + Filter(false) =>
> LogicalValues(tuples=[[]]).
> This empty LogicalValues has as collation (in case of 3 fields): [[0, 1, 2],
> [1, 2], [2]]; and in the plan we are requesting as sort collation [1, 0].
> This could be achieved by: EnumerableSort(sort=[1,0]) +
> EnumerableValues(tuples=[[]]) ; but for some reason this plan is not
> generated.
> It seems the issue was somehow introduced by CALCITE-3886
>
> [https://github.com/apache/calcite/commit/08f4a9889728f94903ff424ed9c20d940b821af5]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvention
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099960#comment-17099960
]
Julian Hyde commented on CALCITE-2970:
--
Use fromContext. Context is straightforward and extensible.
The problem with mutating RelBuilder is that mutable objects are much more
complicated to reason about. For example, if you pass them as arguments to
methods you don’t know how they will behave afterwards.
> Performance issue when enabling abstract converter for EnumerableConvention
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3970) Table-valued function TUMBLE uses non-standard syntax
[
https://issues.apache.org/jira/browse/CALCITE-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099938#comment-17099938
]
Julian Hyde commented on CALCITE-3970:
--
If it’s in the standard we should do it. But can you please quote the relevant
section.
> Table-valued function TUMBLE uses non-standard syntax
> -
>
> Key: CALCITE-3970
> URL: https://issues.apache.org/jira/browse/CALCITE-3970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Viliam Durina
>Priority: Major
>
> The currently supported syntax is this:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE my_table, ...
> {code}
> But the SQL standard specifies that {{my_table}} must be in parentheses, such
> as here:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE(my_table), ...
> {code}
> The second syntax is currently rejected with:
> {code:none}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered "(" at line 1, column 33.
> Was expecting one of:
> ...
> ...
> ...
> ...
> ...
> {code}
> I'm not sure if the currently supported syntax is optional, but I think it's
> not.
> I followed this document:
> [http://standards.iso.org/ittf/PubliclyAvailableStandards/c069776_ISO_IEC_TR_19075-7_2017.zip]
> The {{TABLE}} clause acts in both ways: (1) to convert a table value to a
> table object (when used in the {{FROM}} clause to convert the function
> result) and (2) to convert a table object to a table value (when used to
> convert arguments to a function).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3969) Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't cover collation or distribution keys
[
https://issues.apache.org/jira/browse/CALCITE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099697#comment-17099697
]
Roman Kondakov commented on CALCITE-3969:
-
[~julianhyde], you are right. But this case is took into account: for relation
T, with columns x, y, sorted by x, y, if we have projects:
* 'create view V as select y as z from t', V will have an empty collation
* 'create view U as select x as z from t', U will have collation by z
And it looks correct.
I've added some tests in [PR|https://github.com/apache/calcite/pull/1959] to
{{RelCollationTest.testCollationMapping()}} and
{{RelDistributionTest.testRelDistributionMapping()}} with examples of valid
mappings. Please, take a look.
> Method RelTrait.apply(Mappings.Mapping) throws exception when mapping doesn't
> cover collation or distribution keys
> --
>
> Key: CALCITE-3969
> URL: https://issues.apache.org/jira/browse/CALCITE-3969
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Let's consider we have an input {{(id, name)}} ordered by {{id}} (i.e.
> collation == {{[0]}}). If we have a {{Project("name")}} on the top of this
> input and we apply project's mapping on the collation, we'll end up with
> exception:
> {noformat}
> java.lang.NullPointerException: at index 0
> at
> com.google.common.collect.ObjectArrays.checkElementNotNull(ObjectArrays.java:239)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:230)
> at
> com.google.common.collect.ObjectArrays.checkElementsNotNull(ObjectArrays.java:225)
> at
> com.google.common.collect.ImmutableList.construct(ImmutableList.java:281)
> at
> com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:239)
> at org.apache.calcite.rel.RelCollations.of(RelCollations.java:69)
> at org.apache.calcite.rex.RexUtil.apply(RexUtil.java:1271)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:122)
> at
> org.apache.calcite.rel.RelCollationImpl.apply(RelCollationImpl.java:40)
> {noformat}
> This happens because the collation field {{id}} is not a part of the mapping.
> The same problem is with distribution trait when distribution keys are not
> covered by the mapping.
> Calcite should handle such situations gracefully. If it is not possible to
> deduce the collation/distribution after the mapping application, we should
> return:
> * {{EMPTY}} collation for {{RelCollation}} trait.
> * {{RANDOM_DISTRIBUTED}} distribution for {{RelDistribution}} trait.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3951) Support different string comparison based on SqlCollation
[ https://issues.apache.org/jira/browse/CALCITE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099634#comment-17099634 ] Ruben Q L commented on CALCITE-3951: Thanks for you feedback [~zabetak]. bq. I am not sure if SqlCollation is the place to keep the comparison logic. Then, what could be the alternative? As I see it, SqlCollation already represents a (simplified) notion of what is described in "4.2.2 Comparison of character strings": - It stores a character set, and we do not allow the comparison of strings whose SqlCollation's charset are not the same (see SqlTypeUtil#isCharTypeComparable) - It stores the "Coercibility" value (EXPLICIT, IMPLICIT, COERCIBLE, NONE), which reflects the "collation derivation" rules in the standard (see SqCollation#getCoercibilityDyadic* methods). One thing that is missing is the actual string comparison process (the goal of the current ticket). I think that the easiest way to implement it with (hopefully) the minimum impact of the rest of Calcite classes and full backwards compatibility, was adding this information as a Collator in SqlCollation (as I mentioned in the PR, I would agree to annotate the feature as "Experimental" if we feel that this design might be reviewed in future versions). As you know ;) this is a quite urgent requirement for us, so it would be nice to have a functional implementation for this (even Experimental) in the next release. > Support different string comparison based on SqlCollation > - > > Key: CALCITE-3951 > URL: https://issues.apache.org/jira/browse/CALCITE-3951 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ruben Q L >Assignee: Ruben Q L >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > Currently SqlCollation defines concepts like Coercibility, Charset, Locale, > etc. However, we cannot specify on a certain collation that e.g. a string > field should use case insensitive comparison. The goal of this ticket is to > evolve SqlCollation to support that, and adapt the corresponding classes to > use that (optional) "non-standard" comparison. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3970) Table-valued function TUMBLE uses non-standard syntax
[
https://issues.apache.org/jira/browse/CALCITE-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099613#comment-17099613
]
Viliam Durina commented on CALCITE-3970:
The standard has this syntax for passing tables as arguments to functions:
{code:sql}
SELECT * FROM TABLE(TUMBLE(TABLE(my_table), ...))
{code}
This syntax is rejected by calcite, that's an issue I think. Maybe there's a
switch to enable it, I didn't find it. You can support non-standard syntax, but
the standard syntax should be supported too. The specification only mentions
this syntax.
> Table-valued function TUMBLE uses non-standard syntax
> -
>
> Key: CALCITE-3970
> URL: https://issues.apache.org/jira/browse/CALCITE-3970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Viliam Durina
>Priority: Major
>
> The currently supported syntax is this:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE my_table, ...
> {code}
> But the SQL standard specifies that {{my_table}} must be in parentheses, such
> as here:
> {code:java}
> SELECT * FROM TABLE(TUMBLE(TABLE(my_table), ...
> {code}
> The second syntax is currently rejected with:
> {code:none}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered "(" at line 1, column 33.
> Was expecting one of:
> ...
> ...
> ...
> ...
> ...
> {code}
> I'm not sure if the currently supported syntax is optional, but I think it's
> not.
> I followed this document:
> [http://standards.iso.org/ittf/PubliclyAvailableStandards/c069776_ISO_IEC_TR_19075-7_2017.zip]
> The {{TABLE}} clause acts in both ways: (1) to convert a table value to a
> table object (when used in the {{FROM}} clause to convert the function
> result) and (2) to convert a table object to a table value (when used to
> convert arguments to a function).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
