Abhishek Girish created DRILL-1269:
--------------------------------------
Summary: Select * on parquet files containing nulls fail
Key: DRILL-1269
URL: https://issues.apache.org/jira/browse/DRILL-1269
Project: Apache Drill
Issue Type: Bug
Components: Storage - Parquet
Reporter: Abhishek Girish
Select * on a parquet file fails when some rows may contain null columns.
This is not seen when the nulls in the dataset are removed.
[dataset does not contain null]
> select * from promotion6 where p_promo_sk=5;
[Succeeds]
[dataset contains null]
[sample: ABC|DEF||JKL]
> select * from promotion5 where p_promo_sk=5;
[Fails with below error]
Query failed: Screen received stop request sent. null
[25a465b6-3acf-49fc-a6ea-0488439ec307]
Error: exception while executing query: Failure while trying to get next result
batch. (state=,code=0)
LOG:
2014-08-06 00:56:18,157 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.physical.impl.ScanBatch - Failed to read the batch. Stopping…
java.lang.ArrayIndexOutOfBoundsException: null
2014-08-06 00:56:18,158 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] ERROR
o.a.drill.exec.ops.FragmentContext - Fragment Context received failure. {}
java.lang.ArrayIndexOutOfBoundsException: null
2014-08-06 00:56:18,159 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] ERROR
o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error
25a465b6-3acf-49fc-a6ea-0488439ec307: Screen received stop request sent.
java.lang.ArrayIndexOutOfBoundsException: null
2014-08-06 00:56:18,161 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.drill.exec.ops.OperatorContext - Closing context for
org.apache.drill.exec.physical.config.Screen
2014-08-06 00:56:18,161 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.drill.exec.ops.OperatorContext - Closing context for
org.apache.drill.exec.physical.config.Project
2014-08-06 00:56:18,162 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.drill.exec.ops.OperatorContext - Closing context for
org.apache.drill.exec.physical.config.SelectionVectorRemover
2014-08-06 00:56:18,162 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.drill.exec.ops.OperatorContext - Closing context for
org.apache.drill.exec.physical.config.Filter
2014-08-06 00:56:18,162 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.drill.exec.ops.OperatorContext - Closing context for
org.apache.drill.exec.store.parquet.ParquetRowGroupScan
2014-08-06 00:56:18,163 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
0d7f9f82-ba3c-445a-a7ff-4731c282397f: Failure while running fragment.
java.lang.ArrayIndexOutOfBoundsException: null
2014-08-06 00:56:18,164 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.work.foreman.QueryManager - New fragment status was provided to
Foreman of profile {
state: FAILED
error {
error_id: "0d7f9f82-ba3c-445a-a7ff-4731c282397f"
endpoint {
address: "drillats3.qa.lab"
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: "Failure while running fragment. null
[0d7f9f82-ba3c-445a-a7ff-4731c282397f]"
}
minor_fragment_id: 0
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 4
operator_type: 21
setup_nanos: 0
process_nanos: 90838172
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 3
operator_type: 2
setup_nanos: 0
process_nanos: 3307
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 2
operator_type: 14
setup_nanos: 0
process_nanos: 2552
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 1
operator_type: 10
setup_nanos: 0
process_nanos: 3445
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 0
operator_type: 13
setup_nanos: 0
process_nanos: 996491
local_memory_allocated: 0
wait_nanos: 106634
}
start_time: 1407286578031
end_time: 1407286578162
memory_used: 0
}
handle {
query_id {
part1: -6293552266575722108
part2: -8769049111628070660
}
major_fragment_id: 0
minor_fragment_id: 0
}
2014-08-06 00:56:18,164 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.rpc.control.WorkEventBus - Removing framgent status listener for
queryId part1: -6293552266575722108
part2: -8769049111628070660
.
2014-08-06 00:56:18,165 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.work.foreman.QueryManager - New fragment status was provided to
Foreman of profile {
state: CANCELLED
minor_fragment_id: 0
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 4
operator_type: 21
setup_nanos: 0
process_nanos: 90838172
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 3
operator_type: 2
setup_nanos: 0
process_nanos: 3307
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 2
operator_type: 14
setup_nanos: 0
process_nanos: 2552
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 1
operator_type: 10
setup_nanos: 0
process_nanos: 3445
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 0
operator_type: 13
setup_nanos: 0
process_nanos: 996491
local_memory_allocated: 0
wait_nanos: 106634
}
start_time: 1407286578031
end_time: 1407286578164
memory_used: 0
}
handle {
query_id {
part1: -6293552266575722108
part2: -8769049111628070660
}
major_fragment_id: 0
minor_fragment_id: 0
}
2014-08-06 00:56:18,165 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.e.w.fragment.FragmentExecutor - Cancelled Fragment query_id {
part1: -6293552266575722108
part2: -8769049111628070660
}
major_fragment_id: 0
minor_fragment_id: 0
2014-08-06 00:56:18,167 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.e.w.fragment.FragmentExecutor - Caught exception while running fragment
java.lang.RuntimeException: Failure while accessing Zookeeper
at org.apache.drill.exec.store.sys.zk.ZkPStore.put(ZkPStore.java:96)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryStatus.updateCache(QueryStatus.java:133)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryStatus.update(QueryStatus.java:126)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.updateStatus(QueryManager.java:175)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.fail(QueryManager.java:199)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:161)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusHandler.statusChange(QueryManager.java:286)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:113)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:109)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:158)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:128)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.lang.InterruptedException: null
at java.lang.Object.wait(Native Method) ~[na:1.7.0_45]
at java.lang.Object.wait(Object.java:503) ~[na:1.7.0_45]
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)
~[curator-framework-2.5.0.jar:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
~[curator-client-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)
~[curator-framework-2.5.0.jar:na]
at org.apache.drill.exec.store.sys.zk.ZkPStore.put(ZkPStore.java:89)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
... 14 common frames omitted
2014-08-06 00:56:18,168 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
b746b426-1aa1-469e-88a6-a8dea5dc9365: Failure while running fragment.
java.lang.InterruptedException: null
at java.lang.Object.wait(Native Method) ~[na:1.7.0_45]
at java.lang.Object.wait(Object.java:503) ~[na:1.7.0_45]
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
~[zookeeper-3.4.6.jar:3.4.6-1569965]
at
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)
~[curator-framework-2.5.0.jar:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
~[curator-client-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)
~[curator-framework-2.5.0.jar:na]
at
org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)
~[curator-framework-2.5.0.jar:na]
at org.apache.drill.exec.store.sys.zk.ZkPStore.put(ZkPStore.java:89)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryStatus.updateCache(QueryStatus.java:133)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryStatus.update(QueryStatus.java:126)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.updateStatus(QueryManager.java:175)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.fail(QueryManager.java:199)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:161)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusHandler.statusChange(QueryManager.java:286)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:113)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:109)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:158)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:128)
~[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
[drill-java-exec-0.4.0-incubating-rebuffed.jar:0.4.0-incubating]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
2014-08-06 00:56:18,169 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.work.foreman.QueryManager - New fragment status was provided to
Foreman of profile {
state: FAILED
error {
error_id: "b746b426-1aa1-469e-88a6-a8dea5dc9365"
endpoint {
address: "drillats3.qa.lab"
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: "Failure while running fragment. Failure while accessing Zookeeper
[b746b426-1aa1-469e-88a6-a8dea5dc9365]"
}
minor_fragment_id: 0
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 4
operator_type: 21
setup_nanos: 0
process_nanos: 90838172
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 3
operator_type: 2
setup_nanos: 0
process_nanos: 3307
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 2
operator_type: 14
setup_nanos: 0
process_nanos: 2552
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 1
operator_type: 10
setup_nanos: 0
process_nanos: 3445
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 0
operator_type: 13
setup_nanos: 0
process_nanos: 996491
local_memory_allocated: 0
wait_nanos: 106634
}
start_time: 1407286578031
end_time: 1407286578167
memory_used: 0
}
handle {
query_id {
part1: -6293552266575722108
part2: -8769049111628070660
}
major_fragment_id: 0
minor_fragment_id: 0
}
2014-08-06 00:56:18,170 [a8a8cf8e-e766-4984-864e-144b1b7440fc:frag:0:0] DEBUG
o.a.d.exec.rpc.control.WorkEventBus - Removing framgent status listener for
queryId part1: -6293552266575722108
part2: -8769049111628070660
.
2014-08-06 00:56:18,189 [WorkManager-32] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
bd02bfe2-369f-49ff-8ec3-59ae3620edad: Failure while running fragment.
java.lang.ArrayIndexOutOfBoundsException: null
2014-08-06 00:56:18,189 [WorkManager-32] DEBUG
o.a.d.exec.work.foreman.QueryManager - New fragment status was provided to
Foreman of profile {
state: FAILED
error {
error_id: "bd02bfe2-369f-49ff-8ec3-59ae3620edad"
endpoint {
address: "drillats3.qa.lab"
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: "Failure while running fragment. null
[bd02bfe2-369f-49ff-8ec3-59ae3620edad]"
}
minor_fragment_id: 0
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 4
operator_type: 21
setup_nanos: 0
process_nanos: 90838172
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 3
operator_type: 2
setup_nanos: 0
process_nanos: 3307
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 2
operator_type: 14
setup_nanos: 0
process_nanos: 2552
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 1
operator_type: 10
setup_nanos: 0
process_nanos: 3445
local_memory_allocated: 0
wait_nanos: 0
}
operator_profile {
input_profile {
records: 0
batches: 0
schemas: 0
}
operator_id: 0
operator_type: 13
setup_nanos: 0
process_nanos: 996491
local_memory_allocated: 0
wait_nanos: 106634
}
start_time: 1407286578031
end_time: 1407286578188
memory_used: 0
}
handle {
query_id {
part1: -6293552266575722108
part2: -8769049111628070660
}
major_fragment_id: 0
minor_fragment_id: 0
}
2014-08-06 00:56:18,190 [WorkManager-32] DEBUG
o.a.d.exec.rpc.control.WorkEventBus - Removing framgent status listener for
queryId part1: -6293552266575722108
part2: -8769049111628070660
.
2014-08-06 00:56:18,214 [WorkManager-32] DEBUG
o.a.d.e.w.fragment.FragmentExecutor - Fragment runner complete. 0:0
--
This message was sent by Atlassian JIRA
(v6.2#6252)
