java.lang.NullPointerException in reduce task
---------------------------------------------
Key: MAPREDUCE-2097
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2097
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Affects Versions: 0.20.2
Environment: RHEL 5.5
Reporter: vaibhav negi
While executing hive query "select count(*) from table " , i got this error
message
Ended Job = job_201009291356_0003 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
On investigating task tracker logs , i found out
2010-09-29 14:31:46,839 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
2010-09-29 14:31:46,862 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.job.tracker; Ignoring.
2010-09-29 14:31:46,943 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=SHUFFLE, sessionId=
2010-09-29 14:31:47,133 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
2010-09-29 14:31:47,133 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: dfs.data.dir; Ignoring.
2010-09-29 14:31:47,143 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.job.tracker; Ignoring.
2010-09-29 14:31:47,143 WARN org.apache.hadoop.conf.Configuration:
/home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: dfs.name.dir; Ignoring.
2010-09-29 14:31:47,209 INFO org.apache.hadoop.mapred.ReduceTask:
ShuffleRamManager: MemoryLimit=141387360, MaxSingleShuffleLimit=35346840
2010-09-29 14:31:47,219 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Thread started: Thread for merging on-disk
files
2010-09-29 14:31:47,220 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Thread waiting: Thread for merging on-disk
files
2010-09-29 14:31:47,222 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Thread started: Thread for merging in
memory files
2010-09-29 14:31:47,223 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Need another 17 map output(s) where 0 is
already in progress
2010-09-29 14:31:47,224 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Scheduled 0 outputs (0 slow hosts and0 dup
hosts)
2010-09-29 14:31:47,224 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201009291356_0003_r_000000_3 Thread started: Thread for polling Map
Completion Events
2010-09-29 14:31:47,233 FATAL org.apache.hadoop.mapred.TaskRunner:
attempt_201009291356_0003_r_000000_3 GetMapEventsThread Ignoring exception :
java.lang.NullPointerException
at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)
ReduceTask.java:2683 is : List<MapOutputLocation> loc = mapLocations.get(host)
where host is:-URI u = URI.create(event.getTaskTrackerHttp()); host =
u.getHost() ;
What's the fix??
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
