Khurram Faraaz created DRILL-6254:
-------------------------------------
Summary: IllegalArgumentException: the requested size must be
non-negative
Key: DRILL-6254
URL: https://issues.apache.org/jira/browse/DRILL-6254
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 1.13.0
Reporter: Khurram Faraaz
Assignee: Padma Penumarthy
Attachments: genAllTypesJSN.py
Flatten query fails due to IllegalArgumentException: the requested size must be
non-negative.
Script to generate JSON data file is attached here.
{noformat}
0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_all_types_jsn_to_parquet AS
. . . . . . . . . . . . . . > SELECT
. . . . . . . . . . . . . . > CAST( col_int AS INT) col_int,
. . . . . . . . . . . . . . > CAST( col_bigint AS BIGINT) col_bigint,
. . . . . . . . . . . . . . > CAST( col_char AS CHAR(10)) col_char,
. . . . . . . . . . . . . . > CAST( col_fxdln_str AS VARCHAR(256))
col_fxdln_str,
. . . . . . . . . . . . . . > CAST( col_varln_str AS VARCHAR(256))
col_varln_str,
. . . . . . . . . . . . . . > CAST( col_float AS FLOAT) col_float,
. . . . . . . . . . . . . . > CAST( col_double AS DOUBLE PRECISION) col_double,
. . . . . . . . . . . . . . > CAST( col_date AS DATE) col_date,
. . . . . . . . . . . . . . > CAST( col_time AS TIME) col_time,
. . . . . . . . . . . . . . > CAST( col_tmstmp AS TIMESTAMP) col_tmstmp,
. . . . . . . . . . . . . . > CAST( col_boolean AS BOOLEAN) col_boolean,
. . . . . . . . . . . . . . > col_binary,
. . . . . . . . . . . . . . > array_of_ints from `all_supported_types.json`;
+-----------+----------------------------+
| Fragment | Number of records written |
+-----------+----------------------------+
| 0_0 | 9 |
+-----------+----------------------------+
1 row selected (0.29 seconds)
{noformat}
Reset all options and set slice_target=1
alter system reset all;
alter system set `planner.slice_target`=1;
output_batch_size was set to its default value
drill.exec.memory.operator.output_batch_size = 16777216
{noformat}
select *, flatten(array_of_ints) from tbl_all_types_jsn_to_parquet;
Error: SYSTEM ERROR: IllegalArgumentException: the requested size must be
non-negative
Fragment 0:0
[Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
(state=,code=0)
0: jdbc:drill:schema=dfs.tmp>
{noformat}
Stack trace from drillbit.log
{noformat}
2018-03-15 12:19:43,916 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
255538af-bcd5-98ee-32e0-68d98fc4a6fa: select *, flatten(array_of_ints) from
tbl_all_types_jsn_to_parquet
2018-03-15 12:19:43,952 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-03-15 12:19:43,953 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2018-03-15 12:19:43,966 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 0 ms to get file statuses
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1
using 1 threads. Time: 2ms total, 2.927366ms avg, 2ms max.
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1
using 1 threads. Earliest start: 2.829000 μs, Latest start: 2.829000 μs,
Average start: 2.829000 μs .
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 3 ms to read file metadata
2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0:
State to report: RUNNING
2018-03-15 12:19:44,905 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0:
State change requested RUNNING --> FAILED
2018-03-15 12:19:44,927 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0:
State change requested FAILED --> FINISHED
2018-03-15 12:19:44,928 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalArgumentException:
the requested size must be non-negative
Fragment 0:0
[Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalArgumentException: the requested size must be non-negative
Fragment 0:0
[Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:300)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_161]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Caused by: java.lang.IllegalArgumentException: the requested size must be
non-negative
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
~[guava-18.0.jar:na]
at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:224)
~[drill-memory-base-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:211)
~[drill-memory-base-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.vector.BigIntVector.allocateBytes(BigIntVector.java:239)
~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.vector.BigIntVector.allocateNew(BigIntVector.java:219)
~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.vector.RepeatedBigIntVector.allocateNew(RepeatedBigIntVector.java:272)
~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocatePrecomputedChildCount(AllocationHelper.java:41)
~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:66)
~[vector-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.RecordBatchSizer$ColumnSize.allocateVector(RecordBatchSizer.java:403)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.doAlloc(FlattenRecordBatch.java:276)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.handleRemainder(FlattenRecordBatch.java:240)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext(FlattenRecordBatch.java:165)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:233)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226)
~[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_161]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_161]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1707.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
... 4 common frames omitted
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
