[
https://issues.apache.org/jira/browse/MAPREDUCE-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842892#action_12842892
]
Amar Kamat commented on MAPREDUCE-1422:
---------------------------------------
bq. Please change the method name buildTaskControllerPathDeletionContexts() to
buildTaskControllerTaskPathDeletionContexts()
I kept it that way to avoid unnecessary diffs in my patch. Let me try to change
it in this patch.
bq. AFAIK, task work directory is always cleaned up, but task directory is not
cleaned up sometimes until a job cleanup happens
Can you tell me a scenario where this can happen. But it better we add files to
task dir too.
bq. Also, just so to avoid false positives concerning this kind of loop
Just to add more confidence to this patch, I added log statements and checked
it. I will add checks to make sure that atleast one file gets written. I see
similar code in TestTaskTrackerLocalization#validateRemoveFiles(). I will add
checks there too. Thoughts?
> Changing permissions of files/dirs under job-work-dir may be needed sothat
> cleaning up of job-dir in all mapred-local-directories succeeds always
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1422
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1422
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Ravi Gummadi
> Assignee: Amar Kamat
> Attachments: mapreduce-1422-test-v1.0.patch,
> mapreduce-1422-v1.4.2.patch, mapreduce-1422-v1.5.2.patch
>
>
> After MAPREDUCE-896, if LinuxTaskController is set in config, task-controller
> binary is launched for changing permissions of taskAttemptDir and taskWorkDir
> before cleaning up of these directories sothat cleanup will be succeeded even
> if user had created files/dirs under taskAttemptDir or taskWorkDir with
> non-writable permissions. Users can't create files/dirs under job-dir
> directly as we set 2570 for job-dir. But as job-work-dir has 2770 permissions
> and user can create files/dirs under job-work-dir with non-writable
> permissions, Changing permissions of files/dirs under job-work-dir may be
> needed sothat cleaning up of job-dir in all mapred-local-directories succeeds
> always.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
