[
https://issues.apache.org/jira/browse/HIVE-23502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiko Uchida reassigned HIVE-23502:
---------------------------------------
Assignee: (was: Toshihiko Uchida)
> 【hive on spark】 return code 3 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask
> ---------------------------------------------------------------------------------
>
> Key: HIVE-23502
> URL: https://issues.apache.org/jira/browse/HIVE-23502
> Project: Hive
> Issue Type: Bug
> Environment: hadoop 2.7.2 hive 1.2.1 sclala 2.9.x spark 1.3.1
> Reporter: tom
> Priority: Blocker
>
> Spark UI Log:
>
> 20/05/19 17:07:11 INFO exec.Utilities: No plan file found:
> hdfs://mycluster/tmp/hive/root/a3b20597-61d1-47a9-86b1-dde289fded78/hive_2020-05-19_17-06-53_394_4024151029162597012-1/-mr-10003/c586ae6a-eefb-49fd-92b6-7593e57f0a93/map.xml
> 20/05/19 17:07:11 ERROR executor.Executor: Exception in task 0.0 in stage 0.0
> (TID 0)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
> at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:64)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 20/05/19 17:07:11 INFO executor.CoarseGrainedExecutorBackend: Got assigned
> task 1
> 20/05/19 17:07:11 INFO executor.Executor: Running task 0.1 in stage 0.0 (TID
> 1)
> 20/05/19 17:07:11 INFO rdd.HadoopRDD: Input split:
> Paths:/user/hive/warehouse/orginfobig_fq/nd=2014/frcode=410503/fqdate=2014-01-01/part-m-00000:0+100InputFormatClass:
> org.apache.hadoop.mapred.TextInputFormat
> 20/05/19 17:07:11 INFO exec.Utilities: No plan file found:
> hdfs://mycluster/tmp/hive/root/a3b20597-61d1-47a9-86b1-dde289fded78/hive_2020-05-19_17-06-53_394_4024151029162597012-1/-mr-10003/c586ae6a-eefb-49fd-92b6-7593e57f0a93/map.xml
> 20/05/19 17:07:11 ERROR executor.Executor: Exception in task 0.1 in stage 0.0
> (TID 1)
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
> at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:64)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing broadcast 1
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing block broadcast_1
> 20/05/19 17:19:19 INFO storage.MemoryStore: Block broadcast_1 of size 189144
> dropped from memory (free 1665525606)
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing block broadcast_1_piece0
> 20/05/19 17:19:19 INFO storage.MemoryStore: Block broadcast_1_piece0 of size
> 55965 dropped from memory (free 1665581571)
> 20/05/19 17:19:19 INFO storage.BlockManagerMaster: Updated info of block
> broadcast_1_piece0
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing broadcast 0
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing block broadcast_0
> 20/05/19 17:19:19 INFO storage.MemoryStore: Block broadcast_0 of size 1349884
> dropped from memory (free 1666931455)
> 20/05/19 17:19:19 INFO storage.BlockManager: Removing block broadcast_0_piece0
> 20/05/19 17:19:19 INFO storage.MemoryStore: Block broadcast_0_piece0 of size
> 52726 dropped from memory (free 1666984181)
> 20/05/19 17:19:19 INFO storage.BlockManagerMaster: Updated info of block
> broadcast_0_piece0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
