[jira] [Updated] (DRILL-6535) ClassCastException in Lateral Unnest queries when dealing with schema changed json data
[
https://issues.apache.org/jira/browse/DRILL-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6535:
-
Labels: ready-to-commit (was: )
> ClassCastException in Lateral Unnest queries when dealing with schema changed
> json data
> ---
>
> Key: DRILL-6535
> URL: https://issues.apache.org/jira/browse/DRILL-6535
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.14.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.14.0
>
>
> Found by [~kedarbcs16]
> ClassCastException in Lateral Unnest queries when dealing with schema changed
> json data
> {code:java}
> Query : SELECT customer.c_custkey,customer.c_acctbal,orders.o_orderkey,
> orders.o_totalprice,orders.o_orderdate FROM customer, LATERAL
> (SELECT O.ord.o_orderkey as o_orderkey, O.ord.o_totalprice as
> o_totalprice,O.ord.o_orderdate as o_orderdate FROM UNNEST(customer.c_orders)
> O(ord) WHERE year(O.ord.o_orderdate) <> 1998)orders;
> {code}
> The data is sf001 complex data in multi json format partitioned based on the
> year of the o_orderdata column.
> The last json file(for year 1998) has 2 schema changes for c_acctbal and
> o_shippriority .
> The logs are :-
> {code:java}
> [Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> ClassCastException: org.apache.drill.exec.vector.NullableFloat8Vector cannot
> be cast to org.apache.drill.exec.vector.NullableBigIntVector
> Fragment 0:0
> [Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:359)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:214)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:325)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.ClassCastException:
> org.apache.drill.exec.vector.NullableFloat8Vector cannot be cast to
> org.apache.drill.exec.vector.NullableBigIntVector
> at
> org.apache.drill.exec.vector.NullableBigIntVector.copyEntry(NullableBigIntVector.java:396)
> ~[vector-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.copyDataToOutputVectors(LateralJoinBatch.java:802)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.emitLeft(LateralJoinBatch.java:813)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.crossJoinAndOutputRecords(LateralJoinBatch.java:761)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.produceOutputBatch(LateralJoinBatch.java:479)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.innerNext(LateralJoinBatch.java:157)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java
[jira] [Updated] (DRILL-6535) ClassCastException in Lateral Unnest queries when dealing with schema changed json data
[
https://issues.apache.org/jira/browse/DRILL-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sorabh Hamirwasia updated DRILL-6535:
-
Description:
Found by [~kedarbcs16]
ClassCastException in Lateral Unnest queries when dealing with schema changed
json data
{code:java}
Query : SELECT customer.c_custkey,customer.c_acctbal,orders.o_orderkey,
orders.o_totalprice,orders.o_orderdate FROM customer, LATERAL
(SELECT O.ord.o_orderkey as o_orderkey, O.ord.o_totalprice as
o_totalprice,O.ord.o_orderdate as o_orderdate FROM UNNEST(customer.c_orders)
O(ord) WHERE year(O.ord.o_orderdate) <> 1998)orders;
{code}
The data is sf001 complex data in multi json format partitioned based on the
year of the o_orderdata column.
The last json file(for year 1998) has 2 schema changes for c_acctbal and
o_shippriority .
The logs are :-
{code:java}
[Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
ClassCastException: org.apache.drill.exec.vector.NullableFloat8Vector cannot be
cast to org.apache.drill.exec.vector.NullableBigIntVector
Fragment 0:0
[Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
~[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:359)
[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:214)
[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:325)
[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_161]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Caused by: java.lang.ClassCastException:
org.apache.drill.exec.vector.NullableFloat8Vector cannot be cast to
org.apache.drill.exec.vector.NullableBigIntVector
at
org.apache.drill.exec.vector.NullableBigIntVector.copyEntry(NullableBigIntVector.java:396)
~[vector-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.join.LateralJoinBatch.copyDataToOutputVectors(LateralJoinBatch.java:802)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.join.LateralJoinBatch.emitLeft(LateralJoinBatch.java:813)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.join.LateralJoinBatch.crossJoinAndOutputRecords(LateralJoinBatch.java:761)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.join.LateralJoinBatch.produceOutputBatch(LateralJoinBatch.java:479)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.join.LateralJoinBatch.innerNext(LateralJoinBatch.java:157)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:137)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.jav
[jira] [Updated] (DRILL-6535) ClassCastException in Lateral Unnest queries when dealing with schema changed json data
[
https://issues.apache.org/jira/browse/DRILL-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6535:
-
Reviewer: Parth Chandra
> ClassCastException in Lateral Unnest queries when dealing with schema changed
> json data
> ---
>
> Key: DRILL-6535
> URL: https://issues.apache.org/jira/browse/DRILL-6535
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.14.0
>Reporter: Sorabh Hamirwasia
>Assignee: Sorabh Hamirwasia
>Priority: Major
> Fix For: 1.14.0
>
>
> Found by [~kedarbcs16]
> ClassCastException in Lateral Unnest queries when dealing with schema changed
> json data
> {code:java}
> Query : SELECT customer.c_custkey,customer.c_acctbal,orders.o_orderkey,
> orders.o_totalprice,orders.o_orderdate FROM customer, LATERAL
> (SELECT O.ord.o_orderkey as o_orderkey, O.ord.o_totalprice as
> o_totalprice,O.ord.o_orderdate as o_orderdate FROM UNNEST(customer.c_orders)
> O(ord) WHERE year(O.ord.o_orderdate) <> 1998)orders;
> {code}
> The data is sf001 complex data in multi json format partitioned based on the
> year of the o_orderdata column.
> The last json file(for year 1998) has 2 schema changes for c_acctbal and
> o_shippriority .
> The logs are :-
> {code:java}
> [Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> ClassCastException: org.apache.drill.exec.vector.NullableFloat8Vector cannot
> be cast to org.apache.drill.exec.vector.NullableBigIntVector
> Fragment 0:0
> [Error Id: 6df4ceae-c989-4592-aeec-6d30b626f0ab on drill182:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:359)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:214)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:325)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.ClassCastException:
> org.apache.drill.exec.vector.NullableFloat8Vector cannot be cast to
> org.apache.drill.exec.vector.NullableBigIntVector
> at
> org.apache.drill.exec.vector.NullableBigIntVector.copyEntry(NullableBigIntVector.java:396)
> ~[vector-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.copyDataToOutputVectors(LateralJoinBatch.java:802)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.emitLeft(LateralJoinBatch.java:813)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.crossJoinAndOutputRecords(LateralJoinBatch.java:761)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.produceOutputBatch(LateralJoinBatch.java:479)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.join.LateralJoinBatch.innerNext(LateralJoinBatch.java:157)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:137)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.1
