[jira] [Commented] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134593#comment-17134593
]
Xiening Dai commented on CALCITE-4049:
--
I should also mention that getShortestPath() is also used by materialized view
to determine if a materialization is used. It simply checks
getShortestPath(...) != null, which should have been getPath(...) != null.
https://github.com/apache/calcite/blob/24dd26640db01114c6931d6615b90a63969ffc42/core/src/main/java/org/apache/calcite/plan/RelOptMaterializations.java#L257
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3923) Refactor how planner rules are parameterized
[
https://issues.apache.org/jira/browse/CALCITE-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134592#comment-17134592
]
Julian Hyde commented on CALCITE-3923:
--
There's now a PR for review: [https://github.com/apache/calcite/pull/2024]
Here's the TL;DR:
Previously, it was not easy to customize, re-use or extend planner rules. If
you wished to customize a rule (i.e. create a new instance of a rule with
different properties) you would have to call the rule's constructor. Frequently
the required constructor did not exist, so we would have to add a new
constructor and deprecate the old one.
After this change, you start off from an instance of the rule, modify its
configuration, and call toRule() on the configuration. (Rule constructors are
now private, because only the configuration ever calls them.)
A good illustration of this is {{DruidRules}}, which used to contain many
sub-classes. Those sub-classes are no longer needed. Old code:
{code:java}
public static final DruidSortProjectTransposeRule SORT_PROJECT_TRANSPOSE =
new DruidSortProjectTransposeRule(RelFactories.LOGICAL_BUILDER);
public static class DruidSortProjectTransposeRule
extends SortProjectTransposeRule {
public DruidSortProjectTransposeRule(RelBuilderFactory relBuilderFactory)
{
super(
operand(Sort.class,
operand(Project.class, operand(DruidQuery.class, none(,
relBuilderFactory, null);
}
}
{code}
New code:
{code:java}
public static final SortProjectTransposeRule SORT_PROJECT_TRANSPOSE =
SortProjectTransposeRule.INSTANCE.config
.withOperandFor(Sort.class, Project.class, DruidQuery.class)
.toRule();
{code}
The change maintains backwards compatibility to a large degree. In a few
places, I had to change rule instances from type {{RelOptRule}} to
{{Supplier}}, to avoid deadlocks during class loading. For
instance, instead of writing {{FilterJoinRule.FILTER_ON_JOIN}} you must now
write {{FilterJoinRule.FILTER_ON_JOIN.get()}}.
> Refactor how planner rules are parameterized
>
>
> Key: CALCITE-3923
> URL: https://issues.apache.org/jira/browse/CALCITE-3923
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> People often want different variants of planner rules. An example is
> {{FilterJoinRule}}, which has a 'boolean smart’ parameter, a predicate (which
> returns whether to pull up filter conditions), operands (which determine the
> precise sub-classes of {{RelNode}} that the rule should match) and a
> {{RelBuilderFactory}} (which controls the type of {{RelNode}} created by this
> rule).
> Suppose you have an instance of {{FilterJoinRule}} and you want to change
> {{smart}} from true to false. The {{smart}} parameter is immutable (good!)
> but you can’t easily create a clone of the rule because you don’t know the
> values of the other parameters. Your instance might even be (unbeknownst to
> you) a sub-class with extra parameters and a private constructor.
> So, my proposal is to put all of the config information of a {{RelOptRule}}
> into a single {{config}} parameter that contains all relevant properties.
> Each sub-class of {{RelOptRule}} would have one constructor with just a
> ‘config’ parameter. Each config knows which sub-class of {{RelOptRule}} to
> create. Therefore it is easy to copy a config, change one or more properties,
> and create a new rule instance.
> Adding a property to a rule’s config does not require us to add or deprecate
> any constructors.
> The operands are part of the config, so if you have a rule that matches a
> {{EnumerableFilter}} on an {{EnumerableJoin}} and you want to make it match
> an {{EnumerableFilter}} on an {{EnumerableNestedLoopJoin}}, you can easily
> create one with one changed operand.
> The config is immutable and self-describing, so we can use it to
> automatically generate a unique description for each rule instance.
> (See the email thread [[DISCUSS] Refactor how planner rules are
> parameterized|https://lists.apache.org/thread.html/rfdf6f9b7821988bdd92b0377e3d293443a6376f4773c4c658c891cf9%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3923) Refactor how planner rules are parameterized
[
https://issues.apache.org/jira/browse/CALCITE-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-3923:
-
Fix Version/s: 1.24.0
> Refactor how planner rules are parameterized
>
>
> Key: CALCITE-3923
> URL: https://issues.apache.org/jira/browse/CALCITE-3923
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> People often want different variants of planner rules. An example is
> {{FilterJoinRule}}, which has a 'boolean smart’ parameter, a predicate (which
> returns whether to pull up filter conditions), operands (which determine the
> precise sub-classes of {{RelNode}} that the rule should match) and a
> {{RelBuilderFactory}} (which controls the type of {{RelNode}} created by this
> rule).
> Suppose you have an instance of {{FilterJoinRule}} and you want to change
> {{smart}} from true to false. The {{smart}} parameter is immutable (good!)
> but you can’t easily create a clone of the rule because you don’t know the
> values of the other parameters. Your instance might even be (unbeknownst to
> you) a sub-class with extra parameters and a private constructor.
> So, my proposal is to put all of the config information of a {{RelOptRule}}
> into a single {{config}} parameter that contains all relevant properties.
> Each sub-class of {{RelOptRule}} would have one constructor with just a
> ‘config’ parameter. Each config knows which sub-class of {{RelOptRule}} to
> create. Therefore it is easy to copy a config, change one or more properties,
> and create a new rule instance.
> Adding a property to a rule’s config does not require us to add or deprecate
> any constructors.
> The operands are part of the config, so if you have a rule that matches a
> {{EnumerableFilter}} on an {{EnumerableJoin}} and you want to make it match
> an {{EnumerableFilter}} on an {{EnumerableNestedLoopJoin}}, you can easily
> create one with one changed operand.
> The config is immutable and self-describing, so we can use it to
> automatically generate a unique description for each rule instance.
> (See the email thread [[DISCUSS] Refactor how planner rules are
> parameterized|https://lists.apache.org/thread.html/rfdf6f9b7821988bdd92b0377e3d293443a6376f4773c4c658c891cf9%40%3Cdev.calcite.apache.org%3E].)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134589#comment-17134589
]
Xiening Dai commented on CALCITE-4049:
--
After looking it deeper, I feel the shorted path might not needed at all.
Right now, the getShortestPath() is used only by
ConventionTraitDef.canConvert(). [1] But actually in
ConventionTraitDef.convert(), it doesn't use the shortest path for conversion,
instead it gets all possible paths and enumerate them one by one. [2] So it
looks to me that the shortest path is not used at all. And even though the
comment of Graph#getPaths() says it returns shortest path first, but actually
it uses a DFS to find the path, and doesn't grantee shortest first today. So I
believe the getShortedPath() can be removed.
[1]
https://github.com/apache/calcite/blob/52a57078ba081b24b9d086ed363c715485d1a519/core/src/main/java/org/apache/calcite/plan/ConventionTraitDef.java#L200
[2]
https://github.com/apache/calcite/blob/52a57078ba081b24b9d086ed363c715485d1a519/core/src/main/java/org/apache/calcite/plan/ConventionTraitDef.java#L139
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[ https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134579#comment-17134579 ] Julian Hyde commented on CALCITE-4061: -- Thanks for fixing, [~vlsi]! (Thanks for also fixing the bit-rot.) > Build should fail if Calcite code uses deprecated APIs > --- > > Key: CALCITE-4061 > URL: https://issues.apache.org/jira/browse/CALCITE-4061 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Vladimir Sitnikov >Priority: Major > Fix For: 1.24.0 > > > Calcite build should fail if Calcite code uses deprecated APIs. This includes > external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class > in the org.apache.calcite.util package). > This used to occur when the build was powered by Maven. > Compared to a policy where code is allowed to use deprecated APIs for a > 'grace period', this policy has a number of advantages. One is that Calcite > devs (or users) can upgrade dependencies at short notice, with no code > changes. > Another is that it forces people who are doing internal refactoring > (replacing one internal Calcite API with another) to finish the job. > [~vlsi] Can you take this please? I can't figure out how to set javac's > '-Xlint:deprecated -Werror' from Gradle. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134573#comment-17134573
]
Julian Hyde commented on CALCITE-4049:
--
The advantage of ConsList is that lists can share tails. For example, the paths
[a, d, e] and [a, b, c, d, e] share the tail [d, e]. Sharing tails saves some
memory (but I've not done any experiments to see whether there's a significant
advantage).
A single-linked list could in principle do the same, but java's LinkedList
class (or in fact any double-linked list) does not share links.
The original complaint in this bug is that when you get a path from a frozen
graph, calling List.size() is expensive. But when I look at the code, I see
that the "freeze" process ({{Graph.makeImmutable}}) heavily relies on
List.size() and therefore will perform badly.
A better implementation of "freeze" would perhaps use Floyd-Warshall algorithm
to compute and store the lengths of the shortest paths between any two nodes.
It would only store the length, not the path. The path can be reconstructed
fairly cheaply.
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134529#comment-17134529
]
Xiening Dai edited comment on CALCITE-4049 at 6/12/20, 8:44 PM:
I agree with [~julianhyde]. You cannot say there's no tradeoffs just because we
have more memory. There are OOM issues we have seen, such as CALCITE-3784.
In terms of the change itself, I wonder why we would need ConList in the first
place. If we just use a LinkedList, we can concact the head node and also have
O(1) complexity for size().
was (Author: xndai):
I agree with [~julianhyde]. You cannot say there's no tradeoffs just because we
have more memory. There are OOM issues we have seen, such as CALCITE-3784.
In terms of the change itself, I wonder why we would need ConList in the first
place. If we just use a LinkedList, we can contact the head node and also have
O(1) complexity for size().
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134529#comment-17134529
]
Xiening Dai commented on CALCITE-4049:
--
I agree with [~julianhyde]. You cannot say there's no tradeoffs just because we
have more memory. There are OOM issues we have seen, such as CALCITE-3784.
In terms of the change itself, I wonder why we would need ConList in the first
place. If we just use a LinkedList, we can contact the head node and also have
O(1) complexity for size().
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4060) TypeCoercionImpl#inOperationCoercion doesn't consider NOT_IN.
[ https://issues.apache.org/jira/browse/CALCITE-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134414#comment-17134414 ] Julian Hyde commented on CALCITE-4060: -- What's the test case? > TypeCoercionImpl#inOperationCoercion doesn't consider NOT_IN. > - > > Key: CALCITE-4060 > URL: https://issues.apache.org/jira/browse/CALCITE-4060 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Critical > Attachments: image-2020-06-12-10-22-49-218.png, > image-2020-06-12-10-43-52-584.png > > Time Spent: 20m > Remaining Estimate: 0h > > Current TypeCoercionImpl#inOperationCoercion only considers IN. > !image-2020-06-12-10-43-52-584.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Unnest an array of single-item structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Summary: Unnest an array of single-item structs causes ClassCastException
(was: Unnest an array of structs causes ClassCastException)
> Unnest an array of single-item structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 5:21 PM:
--
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- Ok
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- Error!
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- Ok
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
-- Ok
{code}
was (Author: rubenql):
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- Ok
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- Error!
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- Ok
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
-- Ok
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 5:21 PM:
--
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- Ok
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- Error!
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- Ok
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
-- Ok
{code}
was (Author: rubenql):
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- Error!
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 5:19 PM:
--
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- Error!
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
was (Author: rubenql):
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 5:19 PM:
--
You are right [~julianhyde].
The root cause seems to be a mismatch regarding what an UNNEST of a "simple"
ROW (just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception due to this mismatch.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW, so the problem is avoided:
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
was (Author: rubenql):
You are right [~julianhyde].
The root code seems to be a mismatch regarding what an UNNEST of a "simple" ROW
(just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW ,so the problem is avoided
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 5:18 PM:
--
You are right [~julianhyde].
The root code seems to be a mismatch regarding what an UNNEST of a "simple" ROW
(just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW ,so the problem is avoided
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
was (Author: rubenql):
The root code seems to be a mismatch regarding what an UNNEST of a "simple" ROW
(just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW ,so the problem is avoided
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134400#comment-17134400
]
Julian Hyde edited comment on CALCITE-4049 at 6/12/20, 5:18 PM:
You don't seem to realize that it is a trade-off. A change that improves some
things makes other things worse. It is glib of you to say 'computers have more
memory now, so there's no trade-off'.
Cons is a general-purpose data structure. You made it better for your purposes
but you have not considered other purposes.
The best practice with data structures is to be explicit about the cost for
various operations. In Cons, the running time of size is O\(n). People
understand and accept that.
was (Author: julianhyde):
You don't seem to realize that it is a trade-off. A change that improves some
things makes other things worse. It is glib of you to say 'computers have more
memory now, so there's no trade-off'.
Cons is a general-purpose data structure. You made it better for your purposes
but you have not considered other purposes.
The best practice with data structures is to be explicit about the cost for
various operations. In Cons, the running time of size is O(n). People
understand and accept that.
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4049) Reduce the time complexity of getting shortest distances
[
https://issues.apache.org/jira/browse/CALCITE-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134400#comment-17134400
]
Julian Hyde commented on CALCITE-4049:
--
You don't seem to realize that it is a trade-off. A change that improves some
things makes other things worse. It is glib of you to say 'computers have more
memory now, so there's no trade-off'.
Cons is a general-purpose data structure. You made it better for your purposes
but you have not considered other purposes.
The best practice with data structures is to be explicit about the cost for
various operations. In Cons, the running time of size is O(n). People
understand and accept that.
> Reduce the time complexity of getting shortest distances
>
>
> Key: CALCITE-4049
> URL: https://issues.apache.org/jira/browse/CALCITE-4049
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Assignee: Liya Fan
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, we have {{Graphs#makeImmutable}} to compute the shortest paths
> between all pairs of nodes. For many scenarios, however, we do not need the
> exact paths between nodes. Instead, we are only interested in the lengths of
> the shortest paths.
> To get the path length, we need to get the shortest path first, which is
> returned as a {{List}}, then we call the {{List#size()}} method. According to
> the current implementation, the returned list is of type {{ConsList}}. The
> time complexity of {{ConsList#size}} is O(p) (p is the number of vertices on
> the path), which is inefficient.
> In this issue, we revise the implementation of {{ConsList#size}} so that it
> takes O(1) time. In addition, we also give a utiltiy to get the shortest
> distances between nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Attachment: DynamicCode.java
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: DynamicCode.java
>
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134398#comment-17134398
]
Ruben Q L commented on CALCITE-4063:
The root code seems to be a mismatch regarding what an UNNEST of a "simple" ROW
(just 1 item) is expected to return as element type (integer), and what its
enumerable actually returns (List), see attachment.
If we run the UNNEST just by itself:
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
{code}
there is no problem. But if we try to chain it with other operations:
{code:sql}
{{select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as
T2(y);}}
{code}
we run into the exception.
Note that the problem does not occur if we UNNEST a "complex" ROW (more than 1
item), because in that case the element type and the enumerable return type are
aligned (List):
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- 'with ordinality' adds a second item to the ROW ,so the problem is avoided
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) WITH
ORDINALITY as T2(y, o)
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134392#comment-17134392
]
Julian Hyde commented on CALCITE-4063:
--
It looks as if, when dealing with a relation that has a single column, some
code expects rows to be represented as (singleton) lists, and some other code
expects just the raw values. I don't recall what is the right thing to do in
this particular piece of code.
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134302#comment-17134302
]
Ruben Q L edited comment on CALCITE-4063 at 6/12/20, 3:18 PM:
--
The issue might be on the UNNESTing of a simple ROW, which returns an array
instead of a value.
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3] => should it be y=3 ???
-- y=[4] => should it be y=4 ???
{code}
If we compare it with a similar query with a "complex" ROW, the problem does
not occur:
{code:sql}
select * from UNNEST(array[ROW(3, 5), ROW(4, 6)]) as T2(y, z)
-- y=3; z=5
-- y=4; z=6
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 5), ROW(4, 6)]) as
T2(y, z)
-- x=1; y=3; z=5
-- x=1; y=4; z=6
-- x=2; y=3; z=5
-- x=2; y=4; z=6
{code}
was (Author: rubenql):
The issue might be on the UNNESTing of a simple ROW, which returns an array
instead of a value.
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3] => should it be y=3 ???
-- y=[4] => should it be y=4 ???
{code}
If we compare it with a similar query with a "complex" ROW, the problem does
not occur:
{code:sql}
select * from UNNEST(array[ROW(3, 3), ROW(4, 4)]) as T2(y, z)
-- y=3; z=3
-- y=4; z=4
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 3), ROW(4, 4)]) as
T2(y, z)
-- x=1; y=3; z=3
-- x=1; y=4; z=4
-- x=2; y=3; z=3
-- x=2; y=4; z=4
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Description:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
-- Is this result ok? (see first comment of the current ticket)
{code}
However, if we try to combine them with a correlation with some other values,
as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
was:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
-- Is this result ok? (see first comment below)
{code}
However, if we try to combine them with a correlation with some other values,
as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment of the current ticket)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Description:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
-- Is this result ok? (see first comment below)
{code}
However, if we try to combine them with a correlation with some other values,
as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
was:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
{code}
However, if we try to combine them with a correlation with some other values,
as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> -- Is this result ok? (see first comment below)
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134302#comment-17134302
]
Ruben Q L commented on CALCITE-4063:
The issue might be on the UNNESTing of a simple ROW, which returns an array
instead of a value.
{code:sql}
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3] => should it be y=3 ???
-- y=[4] => should it be y=4 ???
{code}
If we compare it with a similar query with a "complex" ROW, the problem does
not occur:
{code:sql}
select * from UNNEST(array[ROW(3, 3), ROW(4, 4)]) as T2(y, z)
-- y=3; z=3
-- y=4; z=4
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3, 3), ROW(4, 4)]) as
T2(y, z)
-- x=1; y=3; z=3
-- x=1; y=4; z=4
-- x=2; y=3; z=3
-- x=2; y=4; z=4
{code}
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Summary: Unnest an array of structs causes ClassCastException (was:
Correlate with Unnest an array of structs causes ClassCastException)
> Unnest an array of structs causes ClassCastException
>
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4063) Correlate with Unnest an array of structs causes ClassCastException
Ruben Q L created CALCITE-4063:
--
Summary: Correlate with Unnest an array of structs causes
ClassCastException
Key: CALCITE-4063
URL: https://issues.apache.org/jira/browse/CALCITE-4063
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.23.0
Reporter: Ruben Q L
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
{code}
However, if we try to combine the them with a correlation with some other
values, as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4063) Correlate with Unnest an array of structs causes ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-4063:
---
Description:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
{code}
However, if we try to combine them with a correlation with some other values,
as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
was:
If we run the following queries with UNNEST operator, we get the expected
results:
{code:sql}
select * from UNNEST(array[3, 4]) as T2(y);
-- y=3
-- y=4
select * from UNNEST(array[array[3], array[4]]) as T2(y)
-- y=[3]
-- y=[4]
select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
-- y=[3]
-- y=[4]
{code}
However, if we try to combine the them with a correlation with some other
values, as we could do in more realistic examples:
{{select * from dept_nested as d, UNNEST(d.employees) e2}}
The first two return the expected results, but the last one throws an exception:
{code:sql}
select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
-- x=1; y=3
-- x=1; y=4
-- x=2; y=3
-- x=2; y=4
select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
T2(y);
-- x=1; y=[3]
-- x=1; y=[4]
-- x=2; y=[3]
-- x=2; y=[4]
select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
-- ERROR!!!
-- java.lang.ClassCastException: org.apache.calcite.runtime.FlatLists$Flat1List
cannot be cast to java.lang.Integer
{code}
> Correlate with Unnest an array of structs causes ClassCastException
> ---
>
> Key: CALCITE-4063
> URL: https://issues.apache.org/jira/browse/CALCITE-4063
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Priority: Major
>
> If we run the following queries with UNNEST operator, we get the expected
> results:
> {code:sql}
> select * from UNNEST(array[3, 4]) as T2(y);
> -- y=3
> -- y=4
> select * from UNNEST(array[array[3], array[4]]) as T2(y)
> -- y=[3]
> -- y=[4]
> select * from UNNEST(array[ROW(3), ROW(4)]) as T2(y)
> -- y=[3]
> -- y=[4]
> {code}
> However, if we try to combine them with a correlation with some other values,
> as we could do in more realistic examples:
> {{select * from dept_nested as d, UNNEST(d.employees) e2}}
> The first two return the expected results, but the last one throws an
> exception:
> {code:sql}
> select * from (values (1), (2)) T1(x), UNNEST(array[3, 4]) as T2(y);
> -- x=1; y=3
> -- x=1; y=4
> -- x=2; y=3
> -- x=2; y=4
> select * from (values (1), (2)) T1(x), UNNEST(array[array[3], array[4]]) as
> T2(y);
> -- x=1; y=[3]
> -- x=1; y=[4]
> -- x=2; y=[3]
> -- x=2; y=[4]
> select * from (values (1), (2)) T1(x), UNNEST(array[ROW(3), ROW(4)]) as T2(y);
> -- ERROR!!!
> -- java.lang.ClassCastException:
> org.apache.calcite.runtime.FlatLists$Flat1List cannot be cast to
> java.lang.Integer
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4013) Remove traitset derivation when creating logical operators
[ https://issues.apache.org/jira/browse/CALCITE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134228#comment-17134228 ] Haisheng Yuan commented on CALCITE-4013: LogicalFilter takes more: !image-2020-06-12-08-46-26-846.png! > Remove traitset derivation when creating logical operators > -- > > Key: CALCITE-4013 > URL: https://issues.apache.org/jira/browse/CALCITE-4013 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > Attachments: image-2020-06-12-08-46-26-846.png > > > Profiler shows traitset derivation accounts for 65% of the total runtime of > LoigcalProject.create(). Anyway, EnumerableProject will do it again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4013) Remove traitset derivation when creating logical operators
[ https://issues.apache.org/jira/browse/CALCITE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-4013: --- Attachment: image-2020-06-12-08-46-26-846.png > Remove traitset derivation when creating logical operators > -- > > Key: CALCITE-4013 > URL: https://issues.apache.org/jira/browse/CALCITE-4013 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > Attachments: image-2020-06-12-08-46-26-846.png > > > Profiler shows traitset derivation accounts for 65% of the total runtime of > LoigcalProject.create(). Anyway, EnumerableProject will do it again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[ https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134175#comment-17134175 ] Vladimir Sitnikov edited comment on CALCITE-4061 at 6/12/20, 1:11 PM: -- Fixed in https://github.com/apache/calcite/commit/bb22d4773cd2c24bb69acaaf548fb8d17ec99182 was (Author: vladimirsitnikov): Fixed in https://github.com/apache/calcite/commit/f09991959dea41b8805d8cdd18572803d42e3738 > Build should fail if Calcite code uses deprecated APIs > --- > > Key: CALCITE-4061 > URL: https://issues.apache.org/jira/browse/CALCITE-4061 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Vladimir Sitnikov >Priority: Major > Fix For: 1.24.0 > > > Calcite build should fail if Calcite code uses deprecated APIs. This includes > external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class > in the org.apache.calcite.util package). > This used to occur when the build was powered by Maven. > Compared to a policy where code is allowed to use deprecated APIs for a > 'grace period', this policy has a number of advantages. One is that Calcite > devs (or users) can upgrade dependencies at short notice, with no code > changes. > Another is that it forces people who are doing internal refactoring > (replacing one internal Calcite API with another) to finish the job. > [~vlsi] Can you take this please? I can't figure out how to set javac's > '-Xlint:deprecated -Werror' from Gradle. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[ https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Sitnikov resolved CALCITE-4061. Resolution: Fixed > Build should fail if Calcite code uses deprecated APIs > --- > > Key: CALCITE-4061 > URL: https://issues.apache.org/jira/browse/CALCITE-4061 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Vladimir Sitnikov >Priority: Major > Fix For: 1.24.0 > > > Calcite build should fail if Calcite code uses deprecated APIs. This includes > external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class > in the org.apache.calcite.util package). > This used to occur when the build was powered by Maven. > Compared to a policy where code is allowed to use deprecated APIs for a > 'grace period', this policy has a number of advantages. One is that Calcite > devs (or users) can upgrade dependencies at short notice, with no code > changes. > Another is that it forces people who are doing internal refactoring > (replacing one internal Calcite API with another) to finish the job. > [~vlsi] Can you take this please? I can't figure out how to set javac's > '-Xlint:deprecated -Werror' from Gradle. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[ https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134175#comment-17134175 ] Vladimir Sitnikov commented on CALCITE-4061: Fixed in https://github.com/apache/calcite/commit/f09991959dea41b8805d8cdd18572803d42e3738 > Build should fail if Calcite code uses deprecated APIs > --- > > Key: CALCITE-4061 > URL: https://issues.apache.org/jira/browse/CALCITE-4061 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Vladimir Sitnikov >Priority: Major > > Calcite build should fail if Calcite code uses deprecated APIs. This includes > external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class > in the org.apache.calcite.util package). > This used to occur when the build was powered by Maven. > Compared to a policy where code is allowed to use deprecated APIs for a > 'grace period', this policy has a number of advantages. One is that Calcite > devs (or users) can upgrade dependencies at short notice, with no code > changes. > Another is that it forces people who are doing internal refactoring > (replacing one internal Calcite API with another) to finish the job. > [~vlsi] Can you take this please? I can't figure out how to set javac's > '-Xlint:deprecated -Werror' from Gradle. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[ https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Sitnikov updated CALCITE-4061: --- Fix Version/s: 1.24.0 > Build should fail if Calcite code uses deprecated APIs > --- > > Key: CALCITE-4061 > URL: https://issues.apache.org/jira/browse/CALCITE-4061 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Vladimir Sitnikov >Priority: Major > Fix For: 1.24.0 > > > Calcite build should fail if Calcite code uses deprecated APIs. This includes > external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class > in the org.apache.calcite.util package). > This used to occur when the build was powered by Maven. > Compared to a policy where code is allowed to use deprecated APIs for a > 'grace period', this policy has a number of advantages. One is that Calcite > devs (or users) can upgrade dependencies at short notice, with no code > changes. > Another is that it forces people who are doing internal refactoring > (replacing one internal Calcite API with another) to finish the job. > [~vlsi] Can you take this please? I can't figure out how to set javac's > '-Xlint:deprecated -Werror' from Gradle. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4062) When deserialize UDF array type from json string, throw an exception
xzh_dz created CALCITE-4062:
---
Summary: When deserialize UDF array type from json string, throw
an exception
Key: CALCITE-4062
URL: https://issues.apache.org/jira/browse/CALCITE-4062
Project: Calcite
Issue Type: Wish
Reporter: xzh_dz
Function serialization and deserialization are often involved. When I run some
tests, I find that some functions have exceptions during deserialization.
The exception can be reproduced as below:
RelWriterTest:
{code:java}
@Test void test() {
final FrameworkConfig config = RelBuilderTest.config().build();
final RelBuilder builder = RelBuilder.create(config);
final RelNode rel = builder
.scan("EMP")
.project(
builder.call(new MockSqlOperatorTable.SplitFunction(),
builder.field("ENAME"), builder.literal(",")))
.build();
String relJson = RelOptUtil.dumpPlan("", rel,
SqlExplainFormat.JSON, SqlExplainLevel.EXPPLAN_ATTRIBUTES);
String s = deserializeAndDumpToTextFormat(getSchema(rel), relJson);
System.out.println(s);
}
MockSqlOperatorTable:
public static class SplitFunction extends SqlFunction {
public SplitFunction() {
super("split", new SqlIdentifier("split", SqlParserPos.ZERO),
SqlKind.OTHER_FUNCTION, null,
null, OperandTypes.family(SqlTypeFamily.STRING,
SqlTypeFamily.STRING), null,
SqlFunctionCategory.USER_DEFINED_FUNCTION);
}
@Override public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
final RelDataTypeFactory typeFactory =
opBinding.getTypeFactory();
return
typeFactory.createArrayType(typeFactory.createSqlType(SqlTypeName.VARCHAR), -1);
}
}
{code}
Exception：
{code:java}
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.AssertionError: use createArrayType() instead
at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:182)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:126)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:144)
at
org.apache.calcite.plan.RelWriterTest.deserializeAndDumpToTextFormat(RelWriterTest.java:838)
at
org.apache.calcite.plan.RelWriterTest.testArrayType(RelWriterTest.java:811)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(
[jira] [Comment Edited] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133842#comment-17133842 ] Jiatao Tao edited comment on CALCITE-4059 at 6/12/20, 11:07 AM: [~amaliujia] 👌, the desc is updated was (Author: aron.tao): [~amaliujia] 👌 > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Current equalSansNullability only considers the atomic type, not support > Array/Map(struct type can be supported by equalAsStructSansNullability). > This Jira mainly supports Array/Map type in > SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element > of the type will be discarded, the case is in testEqualSansNullability. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4059: Description: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can be supported by equalAsStructSansNullability). This Jira mainly supports Array/Map type in SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element of the type will be discarded, the case is in . was: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can be supported by equalAsStructSansNullability). This Jira mainly support Array/Map type in > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Current equalSansNullability only considers the atomic type, not support > Array/Map(struct type can be supported by equalAsStructSansNullability). > This Jira mainly supports Array/Map type in > SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element > of the type will be discarded, the case is in . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4059: Description: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can be supported by equalAsStructSansNullability). This Jira mainly supports Array/Map type in SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element of the type will be discarded, the case is in testEqualSansNullability. was: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can be supported by equalAsStructSansNullability). This Jira mainly supports Array/Map type in SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element of the type will be discarded, the case is in . > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Current equalSansNullability only considers the atomic type, not support > Array/Map(struct type can be supported by equalAsStructSansNullability). > This Jira mainly supports Array/Map type in > SqlTypeUtil#equalSansNullabilityI, the nullability of the type or the element > of the type will be discarded, the case is in testEqualSansNullability. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4059: Description: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can be supported by equalAsStructSansNullability). This Jira mainly support Array/Map type in was:Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can supported by ). > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Current equalSansNullability only considers the atomic type, not support > Array/Map(struct type can be supported by equalAsStructSansNullability). > This Jira mainly support Array/Map type in -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4059: Description: Current equalSansNullability only considers the atomic type, not support Array/Map(struct type can supported by ). (was: current equalSansNullability only considers ) > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Current equalSansNullability only considers the atomic type, not support > Array/Map(struct type can supported by ). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated CALCITE-4059: Description: current equalSansNullability only considers > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > current equalSansNullability only considers -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4060) TypeCoercionImpl#inOperationCoercion doesn't consider NOT_IN.
[ https://issues.apache.org/jira/browse/CALCITE-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133846#comment-17133846 ] Jiatao Tao edited comment on CALCITE-4060 at 6/12/20, 11:00 AM: Hi [~julianhyde], I add a ut, and debug find the op is NOT IN: org.apache.calcite.test.TypeCoercionConverterTest#testNotInOperation !image-2020-06-12-10-22-49-218.png|width=483,height=316! was (Author: aron.tao): Hi [~julianhyde], I add a ut, and debug find the op is NOT IN: org.apache.calcite.test.TypeCoercionConverterTest#testNotInOperation !image-2020-06-12-10-22-49-218.png! > TypeCoercionImpl#inOperationCoercion doesn't consider NOT_IN. > - > > Key: CALCITE-4060 > URL: https://issues.apache.org/jira/browse/CALCITE-4060 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Critical > Attachments: image-2020-06-12-10-22-49-218.png, > image-2020-06-12-10-43-52-584.png > > Time Spent: 20m > Remaining Estimate: 0h > > Current TypeCoercionImpl#inOperationCoercion only considers IN. > !image-2020-06-12-10-43-52-584.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3935) Materialization-Failed, when querying with LeftJoinWithFilter
[
https://issues.apache.org/jira/browse/CALCITE-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134120#comment-17134120
]
Jin Xing edited comment on CALCITE-3935 at 6/12/20, 10:46 AM:
--
Please refine the Jira title and give some basic analysis in the description.
was (Author: jinxing6...@126.com):
Refine the Jira title and give some basic analysis in the description.
> Materialization-Failed, when querying with LeftJoinWithFilter
> -
>
> Key: CALCITE-3935
> URL: https://issues.apache.org/jira/browse/CALCITE-3935
> Project: Calcite
> Issue Type: Bug
>Reporter: Xurenhe
>Priority: Major
> Attachments: Jietu20200417-200532.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> @Test public void testJoinOnLeftProjectWithFilterToJoin() {
> String mv = ""
> + "select \"emps\".\"empid\", \"depts\".\"name\",
> \"emps\".\"salary\"\n" +
> "from \"emps\"\n" +
> "left join \"depts\"\n" +
> "on \"emps\".\"deptno\" = \"depts\".\"deptno\"\n"
> + "where \"emps\".\"empid\" > 10";
> String query = ""
> + "select \"emps\".\"empid\", \"depts\".\"name\",
> \"emps\".\"salary\"\n" +
> "from \"emps\"\n" +
> "left join \"depts\"\n" +
> "on \"emps\".\"deptno\" = \"depts\".\"deptno\"\n"
> + "where \"emps\".\"empid\" > 40";
> sql(mv, query).withOnlyBySubstitution(true).ok();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3935) Materialization-Failed, when querying with LeftJoinWithFilter
[
https://issues.apache.org/jira/browse/CALCITE-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134120#comment-17134120
]
Jin Xing commented on CALCITE-3935:
---
Refine the Jira title and give some basic analysis in the description.
> Materialization-Failed, when querying with LeftJoinWithFilter
> -
>
> Key: CALCITE-3935
> URL: https://issues.apache.org/jira/browse/CALCITE-3935
> Project: Calcite
> Issue Type: Bug
>Reporter: Xurenhe
>Priority: Major
> Attachments: Jietu20200417-200532.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> @Test public void testJoinOnLeftProjectWithFilterToJoin() {
> String mv = ""
> + "select \"emps\".\"empid\", \"depts\".\"name\",
> \"emps\".\"salary\"\n" +
> "from \"emps\"\n" +
> "left join \"depts\"\n" +
> "on \"emps\".\"deptno\" = \"depts\".\"deptno\"\n"
> + "where \"emps\".\"empid\" > 10";
> String query = ""
> + "select \"emps\".\"empid\", \"depts\".\"name\",
> \"emps\".\"salary\"\n" +
> "from \"emps\"\n" +
> "left join \"depts\"\n" +
> "on \"emps\".\"deptno\" = \"depts\".\"deptno\"\n"
> + "where \"emps\".\"empid\" > 40";
> sql(mv, query).withOnlyBySubstitution(true).ok();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4040) Materialized View matching failed with NPE if aggregate function doesn't support roll up
[
https://issues.apache.org/jira/browse/CALCITE-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jin Xing updated CALCITE-4040:
--
Summary: Materialized View matching failed with NPE if aggregate function
doesn't support roll up (was: When an aggregate function does not support roll
up, materialization recognition should fail and return the original relnode
instead of throwing an exception.)
> Materialized View matching failed with NPE if aggregate function doesn't
> support roll up
>
>
> Key: CALCITE-4040
> URL: https://issues.apache.org/jira/browse/CALCITE-4040
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When i try to rollup some SqlAggFunctions in my project,I find something
> wrong.
> A case can be reproduced as below:
> MaterializationTest:
> {code:java}
> @Test public void testSqlAggFunctionRollup() {
> checkNoMaterialize(
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\", \"deptno\"",
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\"",
> HR_FKUK_MODEL);
> }
> {code}
> When an aggregate function does not support roll up, materialization
> recognition should fail and return the original relnode instead of throwing
> an exception.
> Exception:
> {code:java}
> java.sql.SQLException: Error while executing SQL "explain plan for select
> "empid", stddev_pop("deptno") from "emps" group by "empid"": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:528)
> Caused by: java.lang.NullPointerException
> at java.util.Objects.requireNonNull(Objects.java:203)
> at
> org.apache.calcite.rel.core.AggregateCall.(AggregateCall.java:98)
> at
> org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198)
> at
> org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1854)
> at
> org.apache.calcite.plan.SubstitutionVisitor$AggregateToAggregateUnifyRule.apply(SubstitutionVisitor.java:1545)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:544)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:478)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4061) Build should fail if Calcite code uses deprecated APIs
[
https://issues.apache.org/jira/browse/CALCITE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134086#comment-17134086
]
Vladimir Sitnikov commented on CALCITE-4061:
Here's a way to configure it:
{noformat}
diff --git a/build.gradle.kts b/build.gradle.kts
index af8e0ab45..751278328 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -523,6 +523,7 @@ allprojects {
configureEach {
options.encoding = "UTF-8"
+options.compilerArgs.addAll(listOf("-Xlint:deprecation",
"-Werror"))
}
configureEach {
useJUnitPlatform
{noformat}
> Build should fail if Calcite code uses deprecated APIs
> ---
>
> Key: CALCITE-4061
> URL: https://issues.apache.org/jira/browse/CALCITE-4061
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Vladimir Sitnikov
>Priority: Major
>
> Calcite build should fail if Calcite code uses deprecated APIs. This includes
> external APIs (e.g. Guava methods) and internal APIs (e.g. a deprecated class
> in the org.apache.calcite.util package).
> This used to occur when the build was powered by Maven.
> Compared to a policy where code is allowed to use deprecated APIs for a
> 'grace period', this policy has a number of advantages. One is that Calcite
> devs (or users) can upgrade dependencies at short notice, with no code
> changes.
> Another is that it forces people who are doing internal refactoring
> (replacing one internal Calcite API with another) to finish the job.
> [~vlsi] Can you take this please? I can't figure out how to set javac's
> '-Xlint:deprecated -Werror' from Gradle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134059#comment-17134059 ] Jiatao Tao edited comment on CALCITE-4059 at 6/12/20, 8:39 AM: --- [~danny0405] equalAsStructSansNullability only handles struct type, the case I added handle array/map. was (Author: aron.tao): [~danny0405] equalAsStructSansNullability handles struct type, the case I added handle array/map. > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4059) SqlTypeUtil#equalSansNullability doesn't consider Array/Map
[ https://issues.apache.org/jira/browse/CALCITE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134059#comment-17134059 ] Jiatao Tao commented on CALCITE-4059: - [~danny0405] equalAsStructSansNullability handles struct type, the case I added handle array/map. > SqlTypeUtil#equalSansNullability doesn't consider Array/Map > --- > > Key: CALCITE-4059 > URL: https://issues.apache.org/jira/browse/CALCITE-4059 > Project: Calcite > Issue Type: Bug >Reporter: Jiatao Tao >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4016) Implement trait propagation for EnumerableCalc
[ https://issues.apache.org/jira/browse/CALCITE-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-4016. -- Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/3c317b608344ab2ebd6d65b67f647bddf87376b3]. > Implement trait propagation for EnumerableCalc > -- > > Key: CALCITE-4016 > URL: https://issues.apache.org/jira/browse/CALCITE-4016 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Chunwei Lei >Priority: Major > Fix For: 1.24.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > It should be similar with EnumerableProject. Maybe same logic. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4055) RelFieldTrimmer loses hints
[
https://issues.apache.org/jira/browse/CALCITE-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-4055.
Fix Version/s: 1.24.0
Resolution: Fixed
Fixed via
https://github.com/apache/calcite/commit/7c2f6772ee6bbfe4fefc2cf35152d6c71cc7a361
Thanks [~danny0405] for your help!
> RelFieldTrimmer loses hints
> ---
>
> Key: CALCITE-4055
> URL: https://issues.apache.org/jira/browse/CALCITE-4055
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The trimmed plan generated by RelFieldTrimmer can lose the hints that might
> be present in the input plan.
> The issue can be reproduced with the following test (to be added to
> RelFieldTrimmerTest):
> {code:java}
> @Test void testJoinWithHints() {
> final RelHint noHashJoinHint =
> RelHint.builder("NO_HASH_JOIN").inheritPath(0).build();
> final RelBuilder builder = RelBuilder.create(config().build());
> final RelNode original =
> builder.scan("EMP")
> .scan("DEPT")
> .join(JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "DEPTNO")))
> .hints(noHashJoinHint)
> .project(
> builder.field("ENAME"),
> builder.field("DNAME"))
> .build();
> final RelFieldTrimmer fieldTrimmer = new RelFieldTrimmer(null, builder);
> final RelNode trimmed = fieldTrimmer.trim(original);
> final String expected = ""
> + "LogicalProject(ENAME=[$1], DNAME=[$4])\n"
> + " LogicalJoin(condition=[=($2, $3)], joinType=[inner])\n"
> + "LogicalProject(EMPNO=[$0], ENAME=[$1], DEPTNO=[$7])\n"
> + " LogicalTableScan(table=[[scott, EMP]])\n"
> + "LogicalProject(DEPTNO=[$0], DNAME=[$1])\n"
> + " LogicalTableScan(table=[[scott, DEPT]])\n";
> assertThat(trimmed, hasTree(expected));
> assertTrue(original.getInput(0) instanceof Join);
> final Join originalJoin = (Join) original.getInput(0);
> assertTrue(originalJoin.getHints().contains(noHashJoinHint));
> assertTrue(trimmed.getInput(0) instanceof Join);
> final Join join = (Join) trimmed.getInput(0);
> assertTrue(join.getHints().contains(noHashJoinHint));
> }
> {code}
> which fails in the last line:
> {{assertTrue(join.getHints().contains(noHashJoinHint));}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
