[
https://issues.apache.org/jira/browse/HIVE-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Walter Su updated HIVE-13747:
-----------------------------
Issue Type: Sub-task (was: Bug)
Parent: HIVE-7292
> NullPointerException thrown by Executors causes job can't be finished
> ---------------------------------------------------------------------
>
> Key: HIVE-13747
> URL: https://issues.apache.org/jira/browse/HIVE-13747
> Project: Hive
> Issue Type: Sub-task
> Reporter: Walter Su
>
> stderr log from one executor.
> {noformat}
> 16/05/12 15:56:51 INFO exec.MapJoinOperator: Initializing operator MAPJOIN[10]
> 16/05/12 15:56:51 INFO exec.CommonJoinOperator: JOIN
> struct<_col0:int,_col1:string,_col2:int,_col3:string> totalsz = 4
> 16/05/12 15:56:51 INFO spark.HashTableLoader: ******* Load from HashTable for
> input file: hdfs://test-cluster/user/hive/warehouse-store2/pokes/kv1.txt
> 16/05/12 15:56:51 INFO spark.HashTableLoader: Load back all hashtable
> files from tmp folder
> uri:hdfs://test-cluster/tmp/hive/hadoop/4062fcea-6759-4340-b4be-5e83181e68bf/hive_2016-05-12_15-56-50_196_4198620026582283764-1/-mr-10004/HashTable-Stage-1/MapJoin-mapfile11--.hashtable
> 16/05/12 15:56:51 INFO exec.MapJoinOperator: Exception loading hash tables.
> Clearing partially loaded hash table containers.
> 16/05/12 15:56:51 ERROR executor.Executor: Exception in task 0.0 in stage 3.0
> (TID 3)
> java.lang.RuntimeException: Map operator initialization failed:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:121)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:55)
> at
> org.apache.hadoop.hive.ql.exec.spark.HiveMapFunction.call(HiveMapFunction.java:30)
> at
> org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:192)
> at
> org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:192)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.mr.ObjectCache.retrieve(ObjectCache.java:57)
> at
> org.apache.hadoop.hive.ql.exec.mr.ObjectCache.retrieveAsync(ObjectCache.java:63)
> at
> org.apache.hadoop.hive.ql.exec.ObjectCacheWrapper.retrieveAsync(ObjectCacheWrapper.java:46)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:173)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:355)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:504)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:365)
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkMapRecordHandler.init(SparkMapRecordHandler.java:112)
> ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:151)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:299)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:180)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:176)
> at
> org.apache.hadoop.hive.ql.exec.mr.ObjectCache.retrieve(ObjectCache.java:55)
> ... 23 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.isDedicatedCluster(SparkUtilities.java:118)
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:158)
> at
> org.apache.hadoop.hive.ql.exec.spark.HashTableLoader.load(HashTableLoader.java:148)
> ... 27 more
> {noformat}
> The wiki
> https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started
> says:
> bq. Configure Spark-application configs for Hive. See:
> http://spark.apache.org/docs/latest/configuration.html. This can be done
> either by adding a file "spark-defaults.conf" with these properties to the
> Hive classpath, *or* by setting them on Hive configuration (hive-site.xml).
> For instance:
> So I only copy "spark-defaults.conf" to hive classpath. But as you can see
> from the hive source code:
> {code}
> public static boolean isDedicatedCluster(Configuration conf) {
> String master = conf.get("spark.master");
> return master.startsWith("yarn-") || master.startsWith("local");
> }
> {code}
> The conf value is got from {{Configuration}} which only read from
> hive-site.xml but not from spark-default.conf. That's where the NPE comes
> from.
> If I set spark.master in hive-site.xml, I don't hit the same NPE any more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
