[jira] [Comment Edited] (CALCITE-4511) The number of distinct values for constant columns should be 1
[
https://issues.apache.org/jira/browse/CALCITE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289709#comment-17289709
]
Julian Hyde edited comment on CALCITE-4511 at 2/24/21, 6:57 AM:
Please change this case subject and the commit message to use Calcite's
terminology, not NDV. This enhancement applies to both PopulationSize and
DistinctRowCount. Can you please fix (and test) both?
was (Author: julianhyde):
Please change this case subject and the commit message to use Calcite's
terminology, not NDV.
> The number of distinct values for constant columns should be 1
> --
>
> Key: CALCITE-4511
> URL: https://issues.apache.org/jira/browse/CALCITE-4511
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Suppose we have a relation with 3 columns: {{(a, '1', '2')}}. The first
> column comes from an underlying table, while the other two columns are
> constants.
> For this relation, we should have {{NDV(1) = 1}}, since column 1 is a
> constant, which have only one possible value (suppose the column index starts
> from 0).
> Similarly, we should have {{NDV(1, 2) = 1}}, since both columns are constants.
> We should also have {{NDV(0, 1) == NDV(0)}}, because the number of distinct
> values depends on the non-const column, which is {{a}} from the underlying
> table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4511) The number of distinct values for constant columns should be 1
[
https://issues.apache.org/jira/browse/CALCITE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289709#comment-17289709
]
Julian Hyde commented on CALCITE-4511:
--
Please change this case subject and the commit message to use Calcite's
terminology, not NDV.
> The number of distinct values for constant columns should be 1
> --
>
> Key: CALCITE-4511
> URL: https://issues.apache.org/jira/browse/CALCITE-4511
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Suppose we have a relation with 3 columns: {{(a, '1', '2')}}. The first
> column comes from an underlying table, while the other two columns are
> constants.
> For this relation, we should have {{NDV(1) = 1}}, since column 1 is a
> constant, which have only one possible value (suppose the column index starts
> from 0).
> Similarly, we should have {{NDV(1, 2) = 1}}, since both columns are constants.
> We should also have {{NDV(0, 1) == NDV(0)}}, because the number of distinct
> values depends on the non-const column, which is {{a}} from the underlying
> table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4446) Implement three-valued logic for SEARCH operator
[
https://issues.apache.org/jira/browse/CALCITE-4446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4446:
Labels: pull-request-available (was: )
> Implement three-valued logic for SEARCH operator
> -
>
> Key: CALCITE-4446
> URL: https://issues.apache.org/jira/browse/CALCITE-4446
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Implement three-valued logic for SEARCH operator.
> Consider the expression {{x IN (10, 20)}}, which we might represent as
> {{SEARCH(x, Sarg(10, 20))}}. Suppose we invoke this with a value of {{NULL}}
> for {{x}}. Do we want it to return UNKNOWN, FALSE or TRUE? The answer is: all
> of the above.
> Here are the 3 variants:
> * {{Sarg(10, 20, UNKNOWN AS TRUE)}}: {{x IS NULL OR x IN (10, 20)}} →
> TRUE
> * {{Sarg(10, 20, UNKNOWN AS UNKNOWN)}}: {{x IN (10, 20)}} → UNKNOWN
> * {{Sarg(10, 20, UNKNOWN AS FALSE)}}: {{x IS NOT NULL AND (x IN (10, 20))}}
> → FALSE
> Currently {{class Sarg}} has a field {{boolean containsNull}} which deals
> with the first two cases. Changing {{boolean containsNull}} to {{RexUnknownAs
> unknownAs}} (which has 3 values) will allow us to represent the third. The
> new representation is symmetrical under negation, which de Morgan's law
> suggests is a good thing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4512) 'case when' project alias equals 'when condition column name or then column name' causes validation error
[ https://issues.apache.org/jira/browse/CALCITE-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanjing.wang updated CALCITE-4512: -- Environment: jvm: open-jdk8 > 'case when' project alias equals 'when condition column name or then column > name' causes validation error > - > > Key: CALCITE-4512 > URL: https://issues.apache.org/jira/browse/CALCITE-4512 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.26.0 > Environment: jvm: open-jdk8 >Reporter: yanjing.wang >Priority: Major > Fix For: 1.27.0 > > > String sql = "select case when ds > 20200103 then ds else id end as ds, > count(distinct id) from depts group by case when ds > 20200103 then ds else > id end"; > > 'case when ds > 20200103 then ds else id end as ds' has alias ds, and 'when > ds > 20200103' has column ds or ' then ds ' has ds, validation raises > exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4512) 'case when' project alias equals 'when condition column name or then column name' causes validation error
yanjing.wang created CALCITE-4512: - Summary: 'case when' project alias equals 'when condition column name or then column name' causes validation error Key: CALCITE-4512 URL: https://issues.apache.org/jira/browse/CALCITE-4512 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: yanjing.wang Fix For: 1.27.0 String sql = "select case when ds > 20200103 then ds else id end as ds, count(distinct id) from depts group by case when ds > 20200103 then ds else id end"; 'case when ds > 20200103 then ds else id end as ds' has alias ds, and 'when ds > 20200103' has column ds or ' then ds ' has ds, validation raises exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4510) Weird digests for literals with some user defined types
[
https://issues.apache.org/jira/browse/CALCITE-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4510:
Labels: pull-request-available (was: )
> Weird digests for literals with some user defined types
> ---
>
> Key: CALCITE-4510
> URL: https://issues.apache.org/jira/browse/CALCITE-4510
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We find weird literals for some user defined non-nullable types. Some
> investigation shows that the problem lies in the {{RexLiteral#toJavaString}}
> method.
> In particular, it checks the type string suffix with an 8-character string:
> {noformat}
> if (!fullTypeString.endsWith("NOT NULL")) {
> {noformat}
> However, it trims the last 9 characters from the end of the string:
> {noformat}
> sb.append(fullTypeString, 0, fullTypeString.length() - 9);
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4511) The number of distinct values for constant columns should be 1
[
https://issues.apache.org/jira/browse/CALCITE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-4511:
Labels: pull-request-available (was: )
> The number of distinct values for constant columns should be 1
> --
>
> Key: CALCITE-4511
> URL: https://issues.apache.org/jira/browse/CALCITE-4511
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Suppose we have a relation with 3 columns: {{(a, '1', '2')}}. The first
> column comes from an underlying table, while the other two columns are
> constants.
> For this relation, we should have {{NDV(1) = 1}}, since column 1 is a
> constant, which have only one possible value (suppose the column index starts
> from 0).
> Similarly, we should have {{NDV(1, 2) = 1}}, since both columns are constants.
> We should also have {{NDV(0, 1) == NDV(0)}}, because the number of distinct
> values depends on the non-const column, which is {{a}} from the underlying
> table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4507) 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
[
https://issues.apache.org/jira/browse/CALCITE-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde resolved CALCITE-4507.
--
Fix Version/s: 1.27.0
Resolution: Duplicate
> 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
> -
>
> Key: CALCITE-4507
> URL: https://issues.apache.org/jira/browse/CALCITE-4507
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Danny Chen
>Priority: Major
> Fix For: 1.27.0
>
>
> SQL statements:
> {code:sql}
> a = 0 or a is null -- returns true when a is null
> a in (0, null) -- returns null when a is null
> {code}
> have different semantics when {{a}} is null, we should not represent it as a
> sarg after {{RexSimplify}},
> check this test in {{RexProgramTest}}:
> {code:java}
> @Test void testSimplifyInOr() {
> or(
> gt(vInt(), literal(0)),
> isNull(vInt())),
> ">(?0.int0, 0)");
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4507) 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
[
https://issues.apache.org/jira/browse/CALCITE-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289050#comment-17289050
]
Danny Chen commented on CALCITE-4507:
-
Thanks for the explanation, i agree that CALCITE-4446 can cover this case.
> 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
> -
>
> Key: CALCITE-4507
> URL: https://issues.apache.org/jira/browse/CALCITE-4507
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Danny Chen
>Priority: Major
>
> SQL statements:
> {code:sql}
> a = 0 or a is null -- returns true when a is null
> a in (0, null) -- returns null when a is null
> {code}
> have different semantics when {{a}} is null, we should not represent it as a
> sarg after {{RexSimplify}},
> check this test in {{RexProgramTest}}:
> {code:java}
> @Test void testSimplifyInOr() {
> or(
> gt(vInt(), literal(0)),
> isNull(vInt())),
> ">(?0.int0, 0)");
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (CALCITE-4492) ExceptionInInitializerError caused by ImmutableBeans
[
https://issues.apache.org/jira/browse/CALCITE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L closed CALCITE-4492.
--
Resolution: Not A Problem
> ExceptionInInitializerError caused by ImmutableBeans
>
>
> Key: CALCITE-4492
> URL: https://issues.apache.org/jira/browse/CALCITE-4492
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
>
> While testing an application that uses Calcite, the following error is
> observed at start-up on a Websphere9 environment (IBM J9 VM, Java8, Ubuntu):
> {noformat}
> Caused by: java.lang.ExceptionInInitializerError
> at java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
> at
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
> ...
> Caused by: java.lang.RuntimeException: while binding method public default
> java.lang.Object
> com.onwbp.org.apache.calcite.plan.RelRule$Config.as(java.lang.Class)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.makeDef(ImmutableBeans.java:285)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.access$000(ImmutableBeans.java:51)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans$1.load(ImmutableBeans.java:64)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans$1.load(ImmutableBeans.java:61)
> at
> com.onwbp.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
> at com.onwbp.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
> at
> com.onwbp.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
> at
> com.onwbp.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.create_(ImmutableBeans.java:91)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.create(ImmutableBeans.java:72)
> at
> com.onwbp.org.apache.calcite.plan.RelRule$Config.(RelRule.java:121)
>
> ... 73 more
> Caused by: java.lang.IllegalAccessException:
> 'com.onwbp.org.apache.calcite.plan.RelRule$Config' no access to:
> 'com.onwbp.org.apache.calcite.plan.RelRule$Config.as:(Config,Class)Object/invokeSpecial'
> at
> java.lang.invoke.MethodHandles$Lookup.checkAccess(MethodHandles.java:333)
> at
> java.lang.invoke.MethodHandles$Lookup.unreflectSpecial(MethodHandles.java:970)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.makeDef(ImmutableBeans.java:283)
> ... 86 more
> {noformat}
> It would seem there is an issue around
> {{org.apache.calcite.util.ImmutableBeans}} and the reflection mechanisms that
> it uses, in the context of {{RelRule#Config}} (and specifically the default
> method {{RelRule$Config.as}}).
> Googling a bit, I found the following Flink ticket FLINK-19820, which has a
> very similar exception message. In that case it was seen with Java9 in a
> Debian environment, and the problem is also around {{ImmutableBeans}} in
> {{RelBuilder#Config}}, and again the issue seems generated by a default
> method ({{RelBuilder$Config.toBuilder}}).
> The Flink ticket was closed as "Won't Fix" just because Flink does not
> support Java9 (and apparently the issue did not occur with the
> Flink-supported Java versions), but the underlying Calcite problem was never
> tackled (and Calcite is supposed to work on Java9).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4492) ExceptionInInitializerError caused by ImmutableBeans
[
https://issues.apache.org/jira/browse/CALCITE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289014#comment-17289014
]
Ruben Q L commented on CALCITE-4492:
We have repeated the test with an upgraded version:
- WebSphere 9.0.5.6 (8.0.6.20 IBM Java)
and the exception no longer occurs, so we can conclude, as suspected, the issue
was on IBM's Java side.
I will close the current ticket, since no action is required on Calcite side.
> ExceptionInInitializerError caused by ImmutableBeans
>
>
> Key: CALCITE-4492
> URL: https://issues.apache.org/jira/browse/CALCITE-4492
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
>
> While testing an application that uses Calcite, the following error is
> observed at start-up on a Websphere9 environment (IBM J9 VM, Java8, Ubuntu):
> {noformat}
> Caused by: java.lang.ExceptionInInitializerError
> at java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
> at
> java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
> ...
> Caused by: java.lang.RuntimeException: while binding method public default
> java.lang.Object
> com.onwbp.org.apache.calcite.plan.RelRule$Config.as(java.lang.Class)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.makeDef(ImmutableBeans.java:285)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.access$000(ImmutableBeans.java:51)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans$1.load(ImmutableBeans.java:64)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans$1.load(ImmutableBeans.java:61)
> at
> com.onwbp.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
> at
> com.onwbp.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
> at com.onwbp.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
> at
> com.onwbp.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
> at
> com.onwbp.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.create_(ImmutableBeans.java:91)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.create(ImmutableBeans.java:72)
> at
> com.onwbp.org.apache.calcite.plan.RelRule$Config.(RelRule.java:121)
>
> ... 73 more
> Caused by: java.lang.IllegalAccessException:
> 'com.onwbp.org.apache.calcite.plan.RelRule$Config' no access to:
> 'com.onwbp.org.apache.calcite.plan.RelRule$Config.as:(Config,Class)Object/invokeSpecial'
> at
> java.lang.invoke.MethodHandles$Lookup.checkAccess(MethodHandles.java:333)
> at
> java.lang.invoke.MethodHandles$Lookup.unreflectSpecial(MethodHandles.java:970)
> at
> com.onwbp.org.apache.calcite.util.ImmutableBeans.makeDef(ImmutableBeans.java:283)
> ... 86 more
> {noformat}
> It would seem there is an issue around
> {{org.apache.calcite.util.ImmutableBeans}} and the reflection mechanisms that
> it uses, in the context of {{RelRule#Config}} (and specifically the default
> method {{RelRule$Config.as}}).
> Googling a bit, I found the following Flink ticket FLINK-19820, which has a
> very similar exception message. In that case it was seen with Java9 in a
> Debian environment, and the problem is also around {{ImmutableBeans}} in
> {{RelBuilder#Config}}, and again the issue seems generated by a default
> method ({{RelBuilder$Config.toBuilder}}).
> The Flink ticket was closed as "Won't Fix" just because Flink does not
> support Java9 (and apparently the issue did not occur with the
> Flink-supported Java versions), but the underlying Calcite problem was never
> tackled (and Calcite is supposed to work on Java9).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4507) 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
[
https://issues.apache.org/jira/browse/CALCITE-4507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288926#comment-17288926
]
Julian Hyde commented on CALCITE-4507:
--
The description in CALCITE-4446 is a bit imprecise.
But rest assured, we would not rewrite "x = 0 OR x IS NULL" to "x IN (0,
null)". We would rewrite it to "SEARCH(x, SARG(0, NULL AS TRUE))", which has
the desired semantics.
> 'a = 0 or a is null' should not be rewrite to a sarg 'a in (0, null)'
> -
>
> Key: CALCITE-4507
> URL: https://issues.apache.org/jira/browse/CALCITE-4507
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.26.0
>Reporter: Danny Chen
>Priority: Major
>
> SQL statements:
> {code:sql}
> a = 0 or a is null -- returns true when a is null
> a in (0, null) -- returns null when a is null
> {code}
> have different semantics when {{a}} is null, we should not represent it as a
> sarg after {{RexSimplify}},
> check this test in {{RexProgramTest}}:
> {code:java}
> @Test void testSimplifyInOr() {
> or(
> gt(vInt(), literal(0)),
> isNull(vInt())),
> ">(?0.int0, 0)");
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
