[
https://issues.apache.org/jira/browse/MAPREDUCE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785200#action_12785200
]
Hemanth Yamijala commented on MAPREDUCE-896:
--------------------------------------------
Started looking at this patch. I need to still see the C changes, but have the
following comments so far:
- I think we should not pass the full path for deletion to the task-controller
for security purposes. We should use the pattern we have used for other
commands like this:
task-controller user-name cmd mapred-local-dir job-id task-id is-work-dir
- PathDeletionContext can have APIs like jobId, taskId, isWorkDir, which it can
construct based on the path passed to it.
- Can we call the API and commands in TaskController as cleanupTask, just so
the names are consistent like initializeTask, etc.
- The new addToQueue API can be package private.
- We should probably take care that if user or taskController is not null, the
other is non-null as well and throw an IllegalArgumentException if it is not.
- After enabling path for deletion using taskcontroller, we should probably use
context.fs.delete(context.path, true). This way the deletion logic remains
same. Also, the deletePath implementation can be simplified as follows:
{code}
if (context.taskController != null) {
context.taskController.cleanupTask(context);
}
return context.fs.delete(context.path, true);
{code}
- I don't think LinuxTaskController must do context.fs.delete in case arguments
in pathdeletioncontext are incorrect. This can hide errors and result in the
undeleted directories.
- There is a change in JVMManager where the workdir for the last task was being
deleted inline, but now we delete it asynchronously. Is this OK ? It seems like
it should be because there is no reuse of directories after this point and
hence, a little delay in cleaning up the workdir shouldn't have any effect.
- The change in setupWorkDir does not seem to be creating a workDir after
deletion.
> Users can set non-writable permissions on temporary files for TT and can
> abuse disk usage.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: MR-896.patch, MR-896.v1.patch
>
>
> As of now, irrespective of the TaskController in use, TT itself does a full
> delete on local files created by itself or job tasks. This step, depending
> upon TT's umask and the permissions set by files by the user, for e.g in
> job-work/task-work or child.tmp directories, may or may not go through
> successful completion fully. Thus is left an opportunity for abusing disk
> space usage either accidentally or intentionally by TT/users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
