[
https://issues.apache.org/jira/browse/MAPREDUCE-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-5952:
----------------------------------------
Resolution: Fixed
Fix Version/s: 2.5.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Gera. Committed this to trunk, branch-2 and branch-2.5.
> LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single
> dir for mapOutIndex
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5952
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5952
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 2.3.0
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Blocker
> Fix For: 2.5.0
>
> Attachments: MAPREDUCE-5952.v01.patch, MAPREDUCE-5952.v02.patch,
> MAPREDUCE-5952.v03.patch, MAPREDUCE-5952.v04.patch
>
>
> The javadoc comment for {{renameMapOutputForReduce}} incorrectly refers to a
> single map output directory, whereas this depends on LOCAL_DIRS.
> mapOutIndex should be set to subMapOutputFile.getOutputIndexFile()
> {code}
> 2014-06-30 14:48:35,574 WARN [uber-SubtaskRunner]
> org.apache.hadoop.mapred.LocalContainerLauncher: Exception running local
> (uberized) 'child' : java.io.FileNotFoundException: File
> /Users/gshegalov/workspace/hadoop-common/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobs/org.apache.hadoop.mapreduce.v2.
>
> TestMRJobs-localDir-nm-2_3/usercache/gshegalov/appcache/application_1404164272885_0001/output/file.out.index
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:517)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:726)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:507)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
>
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
>
> at
> org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:334)
>
> at
> org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:504)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.renameMapOutputForReduce(LocalContainerLauncher.java:471)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:292)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:178)
> at
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:221)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
