[jira] [Updated] (CALCITE-4505) SqlToRelConverter should support converting in condition in agg function
[ https://issues.apache.org/jira/browse/CALCITE-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanjing.wang updated CALCITE-4505: -- Environment: jvm: open-jdk8 > SqlToRelConverter should support converting in condition in agg function > > > Key: CALCITE-4505 > URL: https://issues.apache.org/jira/browse/CALCITE-4505 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.26.0 > Environment: jvm: open-jdk8 >Reporter: yanjing.wang >Priority: Major > Fix For: 1.27.0 > > > String sql = "select id, name, count(if(id in (1) ,1,null)) from test.depts b > where ds > 0 group by id,name"; > ... > sqlToRelConverter.convertQuery(validatedSqlNode, false, true) > > it raise npe exception. > Caused by: java.lang.NullPointerException at > java.util.Objects.requireNonNull(Objects.java:203) at > org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4914) > at > org.apache.calcite.sql2rel.StandardConvertletTable.convertExpressionList(StandardConvertletTable.java:839) > at > org.apache.calcite.sql2rel.StandardConvertletTable.convertFunction(StandardConvertletTable.java:659) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4505) SqlToRelConverter should support converting in condition in agg function
yanjing.wang created CALCITE-4505: - Summary: SqlToRelConverter should support converting in condition in agg function Key: CALCITE-4505 URL: https://issues.apache.org/jira/browse/CALCITE-4505 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.26.0 Reporter: yanjing.wang Fix For: 1.27.0 String sql = "select id, name, count(if(id in (1) ,1,null)) from test.depts b where ds > 0 group by id,name"; ... sqlToRelConverter.convertQuery(validatedSqlNode, false, true) it raise npe exception. Caused by: java.lang.NullPointerException at java.util.Objects.requireNonNull(Objects.java:203) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4914) at org.apache.calcite.sql2rel.StandardConvertletTable.convertExpressionList(StandardConvertletTable.java:839) at org.apache.calcite.sql2rel.StandardConvertletTable.convertFunction(StandardConvertletTable.java:659) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287182#comment-17287182
]
Ruben Q L edited comment on CALCITE-2000 at 2/19/21, 5:06 PM:
--
[~julianhyde], we discussed briefly in CALCITE-4064 (which I closed as a
duplicate of this one). I'd appreciate if you could take a look at the fix I
propose here.
was (Author: rubenql):
[~julianhyde], we discussed briefly in CALCITE-4064 (which I closed as a
duplicate of this one). I'd appreciate if you could take a look a look at the
fix I propose here.
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> new Employee(110, 10, "Theodore", 11500, 250,
> Collections.emptyList()),
> };
> public final Department[] depts = {
> new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
> new Location(-122, 38)),
> new Department(30, "Marketing", Collections.emptyList(),
> new Location(0, 52)),
> new Department(40, "HR", Collections.singletonList(emps[1]), null),
> };
> public final Dependent[] locations = {
> new Dependent(10, "San Francisco"),
> new Dependent(20, "San Diego"),
> };
> public QueryableTable foo(int count) {
> return Smalls.generateStrings(count);
> }
> public TranslatableTable view(String s) {
> return Smalls.view(s);
> }
> }
> public static class Employee {
> public final int empid;
> public final int deptno;
> public final String name;
> public final float salary;
> public final Integer commission;
> public final List dependents;
> public Employee(int empid, int deptno, String name, float salary,
[jira] [Commented] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287182#comment-17287182
]
Ruben Q L commented on CALCITE-2000:
[~julianhyde], we discussed briefly in CALCITE-4064 (which I closed as a
duplicate of this one). I'd appreciate if you could take a look a look at the
fix I propose here.
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> new Employee(110, 10, "Theodore", 11500, 250,
> Collections.emptyList()),
> };
> public final Department[] depts = {
> new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
> new Location(-122, 38)),
> new Department(30, "Marketing", Collections.emptyList(),
> new Location(0, 52)),
> new Department(40, "HR", Collections.singletonList(emps[1]), null),
> };
> public final Dependent[] locations = {
> new Dependent(10, "San Francisco"),
> new Dependent(20, "San Diego"),
> };
> public QueryableTable foo(int count) {
> return Smalls.generateStrings(count);
> }
> public TranslatableTable view(String s) {
> return Smalls.view(s);
> }
> }
> public static class Employee {
> public final int empid;
> public final int deptno;
> public final String name;
> public final float salary;
> public final Integer commission;
> public final List dependents;
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission) {
> this(empid, deptno, name, salary, commission,
> Collections.emptyList());
> }
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission, List dependents)
[jira] [Comment Edited] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287151#comment-17287151
]
Ruben Q L edited comment on CALCITE-2000 at 2/19/21, 4:13 PM:
--
I have been able to provide a fix for this issue, I admit it is a bit hacky,
but I do not see any other way to solve this situation.
Basically the idea is using in {{SqlFunctions$ProductComparableListEnumerator}}
an {{Object[]}} instead of {{E[]}}. Even if we need to return a
{{FlatLists.ComparableList}} we can exploit the fact that indeed (see
ComparableList javadoc):
{noformat}
You can create an instance whose type
parameter {@code T} does not extend {@link Comparable}, but you will get a
{@link ClassCastException} at runtime when you call
{@link #compareTo(Object)} if the elements of the list do not implement
{@code Comparable}.
{noformat}
This is precisely the case of the scenario presented here (or in CALCITE-4064),
where we will have an item (array or list) that is not a Comparable. But in
these cases, it is not a big problem, since we just need to create the UNNEST
result, we do not require to compare it. So with the change proposed in the PR,
we are avoiding the {{ArrayStoreException}} in the best case (e.g. in new unit
test {{simpleUnnestArrayOfRows4}}), or delaying the problem until {{compareTo}}
execution, in the worst case (if in fact a comparison-scenario arrives).
BTW, a similar Comparable-to-Object change was required in
{{SqlFunction#flatList}} introduced in 1.24 via CALCITE-4063, since we can also
reach the same situation were the item is not a Comparable, but e.g. an array
(see new unit test {{simpleUnnestArrayOfRows5}}).
was (Author: rubenql):
I have been able to provide a fix for this issue, I admit it is a bit hacky,
but I do not see any other way to solve this situation.
Basically the idea is using in {{SqlFunctions$ProductComparableListEnumerator}}
an {{Object[]}} instead of {{E[]}}. Even if we need to return a
{{FlatLists.ComparableList}} we can exploit the fact that indeed (see
ComparableList javadoc):
{noformat}
You can create an instance whose type does not extend {@link Comparable}, but
you will get a
{@link ClassCastException} at runtime when you call
{@link #compareTo(Object)} if the elements of the list do not implement
{@code Comparable}.
{noformat}
This is precisely the case of the scenario presented here (or in CALCITE-4064),
where we will have an item (array or list) that is not a Comparable. But in
these cases, it is not a big problem, since we just need to create the UNNEST
result, we do not require to compare it. So with the change proposed in the PR,
we are avoiding the {{ArrayStoreException}} in the best case (e.g. in new unit
test {{simpleUnnestArrayOfRows4}}), or delaying the problem until {{compareTo}}
execution, in the worst case (if in fact a comparison-scenario arrives).
BTW, a similar Comparable-to-Object change was required in
{{SqlFunction#flatList}} introduced in 1.24 via CALCITE-4063, since we can also
reach the same situation were the item is not a Comparable, but e.g. an array
(see new unit test {{simpleUnnestArrayOfRows5}}).
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.Ca
[jira] [Comment Edited] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287151#comment-17287151
]
Ruben Q L edited comment on CALCITE-2000 at 2/19/21, 4:12 PM:
--
I have been able to provide a fix for this issue, I admit it is a bit hacky,
but I do not see any other way to solve this situation.
Basically the idea is using in {{SqlFunctions$ProductComparableListEnumerator}}
an {{Object[]}} instead of {{E[]}}. Even if we need to return a
{{FlatLists.ComparableList}} we can exploit the fact that indeed (see
ComparableList javadoc):
{noformat}
You can create an instance whose type does not extend {@link Comparable}, but
you will get a
{@link ClassCastException} at runtime when you call
{@link #compareTo(Object)} if the elements of the list do not implement
{@code Comparable}.
{noformat}
This is precisely the case of the scenario presented here (or in CALCITE-4064),
where we will have an item (array or list) that is not a Comparable. But in
these cases, it is not a big problem, since we just need to create the UNNEST
result, we do not require to compare it. So with the change proposed in the PR,
we are avoiding the {{ArrayStoreException}} in the best case (e.g. in new unit
test {{simpleUnnestArrayOfRows4}}), or delaying the problem until {{compareTo}}
execution, in the worst case (if in fact a comparison-scenario arrives).
BTW, a similar Comparable-to-Object change was required in
{{SqlFunction#flatList}} introduced in 1.24 via CALCITE-4063, since we can also
reach the same situation were the item is not a Comparable, but e.g. an array
(see new unit test {{simpleUnnestArrayOfRows5}}).
was (Author: rubenql):
I have been able to provide a fix for this issue, I admit it is a bit hacky,
but I do not see any other way to solve this situation.
Basically the idea is using in {{SqlFunctions$ProductComparableListEnumerator}}
an {{Object[]}} instead of {{E[]}}. Even if we need to return a
{{FlatLists.ComparableList}} we can exploit the fact that indeed that (from
ComparableList javadoc)
{noformat}
You can create an instance whose type does not extend {@link Comparable}, but
you will get a {@link ClassCastException} at runtime when you call {@link
#compareTo(Object)} if the elements of the list do not implement {@code
Comparable}."
{noformat}
This is precisely the case of the scenario presented here (or in CALCITE-4064),
where we will have an item (array or list) that is not a Comparable. But in
these cases it is not a big problem, since we just need to create the UNNEST
result, we do not require to compare it. So with the change proposed in the PR,
we are avoiding the {{ArrayStoreException}} in the best case (e.g. in new unit
test {{simpleUnnestArrayOfRows4}}), or delaying the problem until {{compareTo}}
execution, in the worst case (if in fact a comparison-scenario arrives).
BTW, a similar Comparable-to-Object change was required in
{{SqlFunction#flatList}} introduced in 1.24 via CALCITE-4063, since we can also
reach the same situation were the item is not a Comparable, but e.g. an array
(see new unit test {{simpleUnnestArrayOfRows5}}).
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultS
[jira] [Commented] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287151#comment-17287151
]
Ruben Q L commented on CALCITE-2000:
I have been able to provide a fix for this issue, I admit it is a bit hacky,
but I do not see any other way to solve this situation.
Basically the idea is using in {{SqlFunctions$ProductComparableListEnumerator}}
an {{Object[]}} instead of {{E[]}}. Even if we need to return a
{{FlatLists.ComparableList}} we can exploit the fact that indeed that (from
ComparableList javadoc)
{noformat}
You can create an instance whose type does not extend {@link Comparable}, but
you will get a {@link ClassCastException} at runtime when you call {@link
#compareTo(Object)} if the elements of the list do not implement {@code
Comparable}."
{noformat}
This is precisely the case of the scenario presented here (or in CALCITE-4064),
where we will have an item (array or list) that is not a Comparable. But in
these cases it is not a big problem, since we just need to create the UNNEST
result, we do not require to compare it. So with the change proposed in the PR,
we are avoiding the {{ArrayStoreException}} in the best case (e.g. in new unit
test {{simpleUnnestArrayOfRows4}}), or delaying the problem until {{compareTo}}
execution, in the worst case (if in fact a comparison-scenario arrives).
BTW, a similar Comparable-to-Object change was required in
{{SqlFunction#flatList}} introduced in 1.24 via CALCITE-4063, since we can also
reach the same situation were the item is not a Comparable, but e.g. an array
(see new unit test {{simpleUnnestArrayOfRows5}}).
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> ne
[jira] [Updated] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-2000:
Labels: pull-request-available (was: )
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.27.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> new Employee(110, 10, "Theodore", 11500, 250,
> Collections.emptyList()),
> };
> public final Department[] depts = {
> new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
> new Location(-122, 38)),
> new Department(30, "Marketing", Collections.emptyList(),
> new Location(0, 52)),
> new Department(40, "HR", Collections.singletonList(emps[1]), null),
> };
> public final Dependent[] locations = {
> new Dependent(10, "San Francisco"),
> new Dependent(20, "San Diego"),
> };
> public QueryableTable foo(int count) {
> return Smalls.generateStrings(count);
> }
> public TranslatableTable view(String s) {
> return Smalls.view(s);
> }
> }
> public static class Employee {
> public final int empid;
> public final int deptno;
> public final String name;
> public final float salary;
> public final Integer commission;
> public final List dependents;
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission) {
> this(empid, deptno, name, salary, commission,
> Collections.emptyList());
> }
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission, List dependents) {
> this.empid = empid;
> this.deptno = deptno;
> this.name = name;
> this.salary = salary;
> this.commission = commission;
> this.dependents
[jira] [Updated] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L updated CALCITE-2000:
---
Fix Version/s: 1.27.0
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
> Fix For: 1.27.0
>
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> new Employee(110, 10, "Theodore", 11500, 250,
> Collections.emptyList()),
> };
> public final Department[] depts = {
> new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
> new Location(-122, 38)),
> new Department(30, "Marketing", Collections.emptyList(),
> new Location(0, 52)),
> new Department(40, "HR", Collections.singletonList(emps[1]), null),
> };
> public final Dependent[] locations = {
> new Dependent(10, "San Francisco"),
> new Dependent(20, "San Diego"),
> };
> public QueryableTable foo(int count) {
> return Smalls.generateStrings(count);
> }
> public TranslatableTable view(String s) {
> return Smalls.view(s);
> }
> }
> public static class Employee {
> public final int empid;
> public final int deptno;
> public final String name;
> public final float salary;
> public final Integer commission;
> public final List dependents;
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission) {
> this(empid, deptno, name, salary, commission,
> Collections.emptyList());
> }
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission, List dependents) {
> this.empid = empid;
> this.deptno = deptno;
> this.name = name;
> this.salary = salary;
> this.commission = commission;
> this.dependents = dependents;
> }
> @Override public String toString() {
> return "Employee [empid: " + empid + ", deptno: " + d
[jira] [Assigned] (CALCITE-2000) UNNEST a collection that has a field with nested data generates an Exception
[
https://issues.apache.org/jira/browse/CALCITE-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L reassigned CALCITE-2000:
--
Assignee: Ruben Q L
> UNNEST a collection that has a field with nested data generates an Exception
>
>
> Key: CALCITE-2000
> URL: https://issues.apache.org/jira/browse/CALCITE-2000
> Project: Calcite
> Issue Type: Bug
>Reporter: Luis Fernando Kauer
>Assignee: Ruben Q L
>Priority: Major
>
> When unnesting a column that has another level of nesting it generates the
> following exception:
> {noformat}
> Caused by: java.lang.ArrayStoreException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:2312)
> at
> org.apache.calcite.runtime.SqlFunctions$ProductComparableListEnumerator.current(SqlFunctions.java:1)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$17$1.current(EnumerableDefaults.java:1968)
> at
> org.apache.calcite.linq4j.EnumerableDefaults$11$1.moveNext(EnumerableDefaults.java:1225)
> at Baz$4$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:680)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:206)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:630)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:607)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 29 more
> {noformat}
> The problem is that Calcite uses FlatLists internally that expect items to
> implement Comparable.
> However, a column with a nested collection will have an array or a List as
> the column value, which do not implement Comparable, so it generates the
> Exception.
> To test this in JdbcTest, I created a field in Employee with a collection.
> For simplicity, I created a field called dependents in Employee class:
> {code:java}
> public static class HrSchema {
> @Override public String toString() {
> return "HrSchema";
> }
> public final Dependent[] dependents = {
> new Dependent(10, "Michael"),
> new Dependent(10, "Jane"),
> };
> public final Employee[] emps = {
> new Employee(100, 10, "Bill", 1, 1000, Arrays.asList(dependents)),
> new Employee(200, 20, "Eric", 8000, 500,
> Collections.emptyList()),
> new Employee(150, 10, "Sebastian", 7000, null,
> Collections.emptyList()),
> new Employee(110, 10, "Theodore", 11500, 250,
> Collections.emptyList()),
> };
> public final Department[] depts = {
> new Department(10, "Sales", Arrays.asList(emps[0], emps[2]),
> new Location(-122, 38)),
> new Department(30, "Marketing", Collections.emptyList(),
> new Location(0, 52)),
> new Department(40, "HR", Collections.singletonList(emps[1]), null),
> };
> public final Dependent[] locations = {
> new Dependent(10, "San Francisco"),
> new Dependent(20, "San Diego"),
> };
> public QueryableTable foo(int count) {
> return Smalls.generateStrings(count);
> }
> public TranslatableTable view(String s) {
> return Smalls.view(s);
> }
> }
> public static class Employee {
> public final int empid;
> public final int deptno;
> public final String name;
> public final float salary;
> public final Integer commission;
> public final List dependents;
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission) {
> this(empid, deptno, name, salary, commission,
> Collections.emptyList());
> }
> public Employee(int empid, int deptno, String name, float salary,
> Integer commission, List dependents) {
> this.empid = empid;
> this.deptno = deptno;
> this.name = name;
> this.salary = salary;
> this.commission = commission;
> this.dependents = dependents;
> }
> @Override public String toString() {
> return "Employee [empid: " + empid + ", deptno: " + deptno
> + ", name
