Matt Keranen created DRILL-8442:
-----------------------------------
Summary: NPE on DeltaRowGroupScan
Key: DRILL-8442
URL: https://issues.apache.org/jira/browse/DRILL-8442
Project: Apache Drill
Issue Type: Bug
Components: Storage - Other
Affects Versions: 1.21.1
Environment: pyspark 3.4.0
delta-spark 2.4.0
Ubuntu 22.04.2 LTS
Reporter: Matt Keranen
SELECT * on Delta table (Parquet) throws null pointer exception:
{noformat}
2023-06-20 18:58:19,058 [1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query with id
1b6e0933-dd1c-f16b-f6af-dd466d5d94f2 issued by mattk: ALTER SESSION SET
`exec.query.max_rows`=10002023-06-20 18:58:19,068
[1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING2023-06-20 18:58:19,068
[1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:0:0:
State to report: RUNNING2023-06-20 18:58:19,118
[1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:0:0:
State change requested RUNNING --> FINISHED2023-06-20 18:58:19,118
[1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 1b6e0933-dd1c-f16b-f6af-dd466d5d94f2:0:0:
State to report: FINISHED2023-06-20 18:58:19,137
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query with id
1b6e0933-c599-8d17-8971-5b0c2ecefac7 issued by mattk: select *from
table(delta.root.`Warehouse/dbo/DeltaTestTable` (type => 'delta'))limit
52023-06-20 18:58:23,037 [1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:1] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:1:
State change requested AWAITING_ALLOCATION --> FAILED2023-06-20 18:58:23,037
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:0:
State change requested AWAITING_ALLOCATION --> FAILED2023-06-20 18:58:23,037
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:1] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:1:
State change requested FAILED --> FINISHED2023-06-20 18:58:23,037
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:0:
State change requested FAILED --> FINISHED2023-06-20 18:58:23,038
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:3] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:3:
State change requested AWAITING_ALLOCATION --> FAILED2023-06-20 18:58:23,037
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:1] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
Fragment: 1:1
Please, refer to logs for more information.
[Error Id: c6b09027-199a-46e1-abb8-f37576c50382 on
vm-etl-01:31010]org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
NullPointerException
Fragment: 1:1
Please, refer to logs for more information.
[Error Id: c6b09027-199a-46e1-abb8-f37576c50382 on vm-etl-01:31010] at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:688)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:392)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:244)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:359)
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)Caused by:
com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot
construct instance of `org.apache.drill.exec.store.delta.DeltaRowGroupScan`,
problem: `java.lang.NullPointerException` at [Source: (String)"{ "pop" :
"single-sender", "@id" : 0, "receiver-major-fragment" : 0,
"receiver-minor-fragment" : 0, "child" : { "pop" :
"selection-vector-remover", "@id" : 1, "child" : { "pop" : "limit",
"@id" : 2, "child" : { "pop" : "delta-row-group-scan",
"@id" : 3, "userName" : "mattk", "formatPluginConfig" : {
"type" : "delta", "version" : null, "timestamp" : null
}, "rowGroupReadEntries"[truncated 18683 chars]; line: 467, column: 7]
(through reference chain:
org.apache.drill.exec.physical.config.SingleSender["child"]->org.apache.drill.exec.physical.config.SelectionVectorRemover["child"]->org.apache.drill.exec.physical.config.Limit["child"])
at
com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
at
com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:2052)
at
com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587)
at
com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610)
at
com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:293)
at
com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:288)
at
com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:519)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1371)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:218)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:144)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:110)
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263)
at
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:539)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:564)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1371)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:218)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:144)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:110)
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263)
at
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:539)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:564)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1371)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:218)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:144)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:110)
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263)
at
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:539)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:564)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1371)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:218)
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:144)
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:110)
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:263)
at
com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:74)
at
com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at
com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2105)
at
com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1546)
at
org.apache.drill.exec.planner.PhysicalPlanReader.readFragmentRoot(PhysicalPlanReader.java:103)
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:288)
... 4 common frames omittedCaused by: java.lang.NullPointerException: null
at
org.apache.drill.shaded.guava.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878)
at
org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.get(LocalCache.java:3950)
at
org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at
org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPluginByConfig(StoragePluginRegistryImpl.java:701)
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getFormatPluginByConfig(StoragePluginRegistryImpl.java:845)
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.resolveFormat(StoragePluginRegistryImpl.java:968)
at
org.apache.drill.exec.store.delta.DeltaRowGroupScan.<init>(DeltaRowGroupScan.java:65)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at
com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call(AnnotatedConstructor.java:128)
at
com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:291)
... 54 common frames omitted2023-06-20 18:58:23,038
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:3] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:3:
State change requested FAILED --> FINISHED2023-06-20 18:58:23,038
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:4] INFO
o.a.d.e.w.fragment.FragmentExecutor - 1b6e0933-c599-8d17-8971-5b0c2ecefac7:1:4:
State change requested AWAITING_ALLOCATION --> FAILED2023-06-20 18:58:23,037
[1b6e0933-c599-8d17-8971-5b0c2ecefac7:frag:1:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
Fragment: 1:0{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
