[
https://issues.apache.org/jira/browse/MAPREDUCE-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901782#action_12901782
]
Ravi Gummadi commented on MAPREDUCE-1664:
-----------------------------------------
Already attached 4 patches form the combined patch for Yahoo security branch
20.1xx.
At a higher level, the above 4 patches did the following:
(1) Removed the flag mapreduce.cluster.job-authorization-enabled and let the
flag mapred.acls.enabled drive both queue acls checks and job acls checks. All
users will be able to do queue level and job level operations if
mapred.acls.enabled is set to false. If mapred.acls.enabled is set to true,
then the UNION of users part of queue ACLs and job ACLs will be able to do
operation if that operation is driven by both queue ACLs and job ACLs --- like
killJob, killTask and setJobPriority.
(2) JobHistory is changed to have "*" as value for the job acls' keys VIEW_JOB
and MODIFY_JOB in the line of job submission in history file when
mapred.acls.enabled is false. This would allow all users to view job-history of
this job even when we restart the cluster with mapred.acls.enabled set to true.
(3) JobHistory is changed to have job queue name in the job submission line in
history file. This is used to allow queue admins of that queue to view history
details when serving job history based queries.
(4) Created new enum "Operation" that contains the actual operations like
SUBMIT_JOB, KILL_TASK, FAIL_TASK, KILL_JOB,SET_JOB_PRIORITY, VIEW_TASK_LOGS,
VIEW_JOB_DETAILS, VIEW_JOB_COUNTERS.
(5) Added new class ACLsManager that is responsible for authorization of all
queue level and job level operations.
ACLsManager internally maintains MROwner(user who started mapreduce cluster),
cluster administrators(configured by MROwner via
mapreduce.cluster.permissions.supergroup). ACLsManager uses QueueManager and
JobACLsManager for checking queue acls and job acls respectively for
authorizing users to do various operations.
(6) Operation SUBMIT_JOB is solely driven by the queue acl acl-submit-job. The
operations VIEW_JOB_DETAILS, VIEW_JOB_COUNTERS, VIEW_TASK_LOGS, FAIL_TASK,
KILL_TASK, KILL_JOB, SET_JOB_PRIORITY are driven by both job acls(either
acl-view-job OR acl-modify-job based on the operation) and queue acl
acl-administer-jobs. If a user is part of job acl that drives the specific
operation OR part of the queue acl acl-administer-jobs, then he/she is an
authorized user to perform the operation. Irrespective of queue acls and job
acls configured, any operation can be done by MROwner and cluster
administrators(members of supergroup).
(7) Default values for the queue acls is changed from "*"(all users allowed by
default) to " "(no user is allowed by default). This is to make the default
values of queue acls consistent with job acls.
(8) Since TaskTracker will not have job acls once the job finishes execution,
the job acls are persisted to userlogs/$jobid/job-acls.xml so that single file
per job is maintained. This file contains job's acl-view-job, job queue name,
job owner name, queue admins acl of the queue to which this job was submitted
to. These info will be used by TaskTracker to do authorization of users for
accessing task logs(in TaskLogServlet).
(9) Made LinuxTaskController to set permissions for job acls file also when
permissions are being set for job userlog dir.
(10) For TaskTrackers to write queue admins of specific job queue to
job-acls.xml file, the queue admins acl is written to job conf by job client.
Made JobClient to get queue admins(of the queue to which the job is being
submitted to) from JobTracker and set this ACL in job conf. Tasktracker reads
this queue admins ACL from job conf and writes to job-acls.xml file at the time
of job localization.
(11) TaskLogServlet checks the queue admins ACL explicitly(as there is no
Queuemanager on TaskTracker) before checking for other things like jobOwner,
cluster admins and view-job-acl. The queue admins ACL and view-job-acl are
obtained from job-acls.xml file.
(12) Changed readCounters() in CompletedJobStatusStore so that queue name is
obtained from job profile and thus allowing job-view of that job for the queue
admins of that queue.
(13) Since the newly added method getQueueAdmins() in ClientProtocol returns
AccessControlList object directly, creation of AccessControlList object
through Reflection utils is needed. This needed a new constructor for
AccessControlList and that to be registered in WritableFactories.
> Job Acls affect Queue Acls
> --------------------------
>
> Key: MAPREDUCE-1664
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1664
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1664.20S.3.4.patch, 1664.qAdminsJobView.20S.v1.6.patch,
> M1664y20s-testfix.patch, mr-1664-20-bugfix.patch
>
>
> MAPREDUCE-1307 introduced job ACLs for securing job level operations. So in
> current trunk, queue ACLs and job ACLs are checked(with AND for both acls)
> for allowing job level operations. So for doing operations like killJob,
> killTask and setJobPriority user should be part of both
> mapred.queue.{queuename}.acl-administer-jobs and in
> mapreduce.job.acl-modify-job. This needs to change so that users who are part
> of mapred.queue.{queuename}.acl-administer-jobs will be able to do
> killJob,killTask,setJobPriority and users part of
> mapreduce.job.acl-modify-job will be able to do
> killJob,killTask,setJobPriority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
