[ https://issues.apache.org/jira/browse/MAPREDUCE-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe resolved MAPREDUCE-5661. ----------------------------------- Resolution: Not A Problem Closing as shufflehandler should continue to use the yarn property since it's part of the nodemanager process. > ShuffleHandler using yarn.nodemanager.local-dirs instead of > mapreduce.cluster.local.dir > --------------------------------------------------------------------------------------- > > Key: MAPREDUCE-5661 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5661 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 2.2.0 > Reporter: Eric Sirianni > Priority: Trivial > > While debugging an issue where a MapReduce job is failing due to running out > of disk space, I noticed that the {{ShuffleHandler}} uses > {{yarn.nodemanager.local-dirs}} for its {{LocalDirAllocator}} whereas all of > the other MapReduce classes use {{mapreduce.cluster.local.dir}}: > {noformat} > $ find hadoop-mapreduce-project/hadoop-mapreduce-client/*/src/main/java/ > -name "*.java" | xargs grep "new LocalDirAllocator(" > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java: > LocalDirAllocator lDirAlloc = new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnOutputFiles.java: > new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java: > new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/BackupStore.java: > this.lDirAlloc = new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MROutputFiles.java: > new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Merger.java: > new LocalDirAllocator(MRConfig.LOCAL_DIR); > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java: > this.lDirAlloc = new LocalDirAllocator(MRConfig.LOCAL_DIR); > *****hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java: > new LocalDirAllocator(YarnConfiguration.NM_LOCAL_DIRS); > {noformat} > This inconsistency feels like something that is likely to confuse admins. -- This message was sent by Atlassian JIRA (v6.1#6144)