[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842618#action_12842618 ] Hudson commented on MAPREDUCE-1493: --- Integrated in Hadoop-Mapreduce-trunk-Commit #267 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/267/]) . Authorization for job-history pages. Contributed by Vinod Kumar Vavilapalli. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt, MAPREDUCE-1493-20100304.1.txt, > MAPREDUCE-1493-20100304.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842588#action_12842588 ] Ravi Gummadi commented on MAPREDUCE-1493: - Patch looks good. +1 > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt, MAPREDUCE-1493-20100304.1.txt, > MAPREDUCE-1493-20100304.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842566#action_12842566 ] Hadoop QA commented on MAPREDUCE-1493: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438161/MAPREDUCE-1493-20100304.1.txt against trunk revision 919645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/505/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/505/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/505/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/505/console This message is automatically generated. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt, MAPREDUCE-1493-20100304.1.txt, > MAPREDUCE-1493-20100304.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12841807#action_12841807 ] Hadoop QA commented on MAPREDUCE-1493: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437853/MAPREDUCE-1493-20100304.txt against trunk revision 919277. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/502/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/502/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/502/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/502/console This message is automatically generated. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt, MAPREDUCE-1493-20100304.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840063#action_12840063 ] Hadoop QA commented on MAPREDUCE-1493: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437473/MAPREDUCE-1493-20100301.1.txt against trunk revision 916823. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/494/console This message is automatically generated. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12840055#action_12840055 ] Ravi Gummadi commented on MAPREDUCE-1493: - Latest patch looks fine. +1 > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt, > MAPREDUCE-1493-20100227.2-ydist.txt, MAPREDUCE-1493-20100227.3-ydist.txt, > MAPREDUCE-1493-20100301.1.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1283#action_1283 ] Ravi Gummadi commented on MAPREDUCE-1493: - Patch looks good. +1 > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt, > MAPREDUCE-1493-20100225.2.txt, MAPREDUCE-1493-20100226.1.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837114#action_12837114 ] Ravi Gummadi commented on MAPREDUCE-1493: - Some more comments: (6) JobACLs.java In comment, change "specify" to "specified". Also it says 'superuser/supergroup of the "jobTracker"' --- superuser/supergroup will not be specific to JT after 1455 and it is cluster level. (7) Error messages displayed when authorization fails are not having the job-view-ACLs configured for the job. Please add the message obtained from Acces sControlException in the error message. (8) In TestWebUIAuthorization.java, the name of existing method testWebUIAuthorization() is changed to TestWebUIAuthorization(). May be you did it for your testing sothat the method won't run. Please change the name back. A cooment refers to "jobdetails.jsp" --- should instead be "jobdetailshistory.jsp". props.setProperty(JSPUtil.PRIVATE_ACTIONS_KEY, "true"); is not needed for testing history related stuff. (9) Don't we want to support viewing of older history files ? With older history files(where job ACLs are not there), JSPUtil.getJobInfo() gets NPE because JobSubmittedEvent.getJobACLs() gets NPE. Should we allow viewing of older history files assuming that view access exists for those jobs for all users ? (10) taskstatshistory.jsp, taskdetailshistory.jsp and TaskLogServlet can be validated in the testcase. (11) In rumen, in Job20LineHistoryEventEmitter.java, from the parsed line of history file, jobACLs are not read but empty ACLs are written to. I guess we need to build ACLs from the parsedLine of history. (12) In all history related jsps, unnecessary import of "org.apache.hadoop.security.UserGroupInformation" can be removed. Also unnecesary line "String user = request.getRemoteUser();" can be removed. (13) Very minor: Indentation of line "JSPUtil.checkAccess" in jobdetailshistory.jsp (14) We wanted to remove the variable "conf"(which is doing just new Configuration()) from JSPUtil.java in this patch. Can you do it here ? We can get the CACHE_SIZE using jobTracker.conf.get(JT_JOBHISTORY_CACHE_SIZE) in getJobInfo() ? (15) Am not sure if passing of rumen tests is good enough for this patch. Do we need any additional testing ? > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836831#action_12836831 ] Ravi Gummadi commented on MAPREDUCE-1493: - Didn't review the whole patch. Patch looks good functionally, when I tested this patch on web UI. Some comments: (1) Though it doesn't seem to be a security hole, taskstatshistory.jsp can build taskid from attemptid instead of taking both as parameters. (2) Similar to jobhistory.jsp, jobtracker.jsp's Retired Jobs need to take only logFile as parameter and need not take jobId as parameter. (3) All jsps modified in MAPREDUCE-1455 have parameter names as "tipid" and "taskid" to refer to tip and attempt. But in history related jsps, name "taskid" is used to refer to tip sometimes and attempt in some other places. We could follow the same names as all of jsps of MAPREDUCE-1455 are following. For eg, links of task logs and task counters in taskdetailshistory.jsp. (4) jobdetailshistory.jsp can display the job ACLs similar to jobdetails.jsp. (5) Irrespective of this patch, search on the jobhistory page seem to be taking only till underscore(excluding underscore) in the username. For eg, if I search for user name "ravi_tmp", it gives No Jobs even though I launched jobs as ravi_tmp. It gives job s of ravi_tmp if I give username as ravi. Is this a known issue ? > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V >Assignee: Vinod K V > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1493-20100222.1.txt > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836397#action_12836397 ] Devaraj Das commented on MAPREDUCE-1493: I think it makes more sense to do only the JobHistory lookup for the ACL (there should be a standard place in the history file where the job information is logged), and avoid looking up the jobconf.. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V > Fix For: 0.22.0 > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835123#action_12835123 ] Vinod K V commented on MAPREDUCE-1493: -- Regarding putting the ACLs in JobHistory I've checked with [~jothipn] to see how easy/difficult it is to put it in there. From what I understand from that conversation, it is non-trivial to do it and will involve changes in our JobHistory APIs and possibly in other places like Vaidya, Rumen which consume JobHistory. Further, he was of the opinion that ACLs are simply a configuration and may not exactly fit into the history file, though we agreed the line is really blurred as to what should go into the history file and what should not. Given that, I am leaning towards going with the conf file itself where the ACLs are already present. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V > Fix For: 0.22.0 > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833900#action_12833900 ] Devaraj Das commented on MAPREDUCE-1493: I think we anyway should log the job ACLs in the history file for the job. So in that sense, the first option seems to make more sense to me. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V > Fix For: 0.22.0 > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833721#action_12833721 ] Vinod K V commented on MAPREDUCE-1493: -- For getting the ACLs for retired jobs, we actually have two options: - *Log the ACLs also into job-history itself* during job-submission and retrieve the same for access control. -- This avoids reading job-conf file for every request of a job specific page. -- Needs a change to JobHistory submisison event. - *Read the job-conf file on history also* to get the acls. -- Introduces extra round trip to DFS -- No change to JobHistory. I think the former is better performance wise. Thoughts? > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V > Fix For: 0.22.0 > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1493) Authorization for job-history pages
[ https://issues.apache.org/jira/browse/MAPREDUCE-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833703#action_12833703 ] Vinod K V commented on MAPREDUCE-1493: -- History pages are served by the JobTracker directly off the job-history files on the DFS. To facilitate the authorization of these pages, we will need the job ACLs that are introduced by MAPREDUCE-1307 and used subsequently in MAPREDUCE-1455. Along with JobHistory files, job configuration files are also stored on DFS, so we can directly read the ACLs from conf files and use them for the sake of authorization. > Authorization for job-history pages > --- > > Key: MAPREDUCE-1493 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1493 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobtracker, security >Reporter: Vinod K V > Fix For: 0.22.0 > > > MAPREDUCE-1455 introduces authorization for most of the Map/Reduce jsp pages > and servlets, but left history pages. This JIRA will make sure that > authorization checks are made while accessing job-history pages also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
