JT and TT should prune invalid local dirs on startup
----------------------------------------------------
Key: MAPREDUCE-2049
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2049
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker, tasktracker
Affects Versions: 0.21.0, 0.22.0
Reporter: Eli Collins
Priority: Minor
Fix For: 0.21.1, 0.22.0
MRAsyncDiskService fails when it's given local dirs it can't create or write.
Per MAPREDUCE-1213 this conflicts with the previous behavior. The TTs will no
longer start on hosts where the config has localdirs that can not be created
(ie if some hosts don't have the same local mounts).
The TT and JT should prune out invalid local dirs (ones that fail
checkLocalDirs ie that it can not create) before giving them to
MRAsyncDiskService and using them otherwise.
Also mapred-default.xml states that mapred.local.dir that do not exist are
ignored, this should be updated as directories that don't exist are created (if
possible), they're only ignored if they can not be created.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
