[
https://issues.apache.org/jira/browse/HADOOP-4491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod K V updated HADOOP-4491:
------------------------------
Attachment: HADOOP-4491-20090703-common.txt
HADOOP-4491-20090703.txt
Adding patches for quick review. Still a work in progress. Incorporated most of
Devaraj's comments.
Two major changes from the previous patch:
- Pulled _taskTracker/jobcache/attemp-id/work_ to
_taskTracker/jobcache/task-work/attempt-id/work_. This is done to solve issues
with jvm reuse. The work dir needs to be shared across tasks with jvm reuse and
so cannot be finalized to be owned back by the TT on task finish. Pulling this
out makes things clean, the original attempt-dir is finalized, and the
task-work directory is cleaned up on jvm exit. TODO: task-work can have better
name, perhaps jvmcache.
- _log-dir/userlogs/attempt-id_ will still have 777 permissions with all the
files inside created by linuxTaskController but owned by the user. This is done
because
-- attempt-id dir has to be writable by the child for log.tmp which is done
periodically in synclogs.
-- attempt-dir has to be readable by TT for log serving and writable by TT
also by cleanup.
Long time, the correct solution is for this directory to be owned by the child,
but cleaned up by TT using a LinuxTaskController binary lauch to swith user.
Pending items from Devaraj's comments:
- It will be nice to combine the APIs for creating files/directories and
setting appropriate permissions all in one API
- DiskChecker.java has lot of code to do with permissions handling to make it
generic but not everything would be actually used. In fact, the approach taken
in making some APIs generic is debatable. We might as well keep it simple for
now and extend those APIs as and when required.
- LocalDirAllocator.getSecureLocalPathForWrite could be renamed as
getPrivateLocalPathForEWrite
Extra:
- Test TaskTracker.initializeJobDirs
- Extra tests in taskcontroller.c
- Test various scenarios of finalizeTaskDirs
- Finish tests for jvm reuse
There are some minor TODO's also left to do.
> Per-job local data on the TaskTracker node should have right access-control
> ---------------------------------------------------------------------------
>
> Key: HADOOP-4491
> URL: https://issues.apache.org/jira/browse/HADOOP-4491
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: mapred, security
> Reporter: Arun C Murthy
> Assignee: Vinod K V
> Attachments: HADOOP-4491-20090623-common.1.txt,
> HADOOP-4491-20090623-mapred.1.txt, HADOOP-4491-20090703-common.txt,
> HADOOP-4491-20090703.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
