[
https://issues.apache.org/jira/browse/MAPREDUCE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sriramadasu updated MAPREDUCE-1186:
-----------------------------------------------
Attachment: patch-1186-3-ydist.txt
Patch renaming uniqueSubDir. I think the log will be useful in debugging, so
leaving it as is.
Verified that chmod is not done on baseDir, by manually verifying logs.
Reproduced the issue by putting about 1Lakh files in base dir, and running a
job using 5 files on distributed cache.
Verified that the same job, under same environment, finishes faster with the
patch.
Verified that streaming and pipes jobs, which use executables from distributed
cache, work fine (with both DefaultTaskController and LinuxTaskController).
Thanks Karthikeyan for helping in testing.
test-patch and ant test passed on local machine, except TestHdfsProxy.
> While localizing a DistributedCache file, TT sets permissions recursively on
> the whole base-dir
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1186
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1186
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-1186-3-ydist.txt, patch-1186-ydist.txt,
> patch-1186-ydist.txt, patch-1186.txt
>
>
> This is a performance problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
