[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135751#comment-13135751 ] Hudson commented on MAPREDUCE-2858: --- Integrated in Hadoop-Hdfs-0.23-Commit #54 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/54/]) Merge -c 1189036 from trunk to branch-0.23 to fix for MAPREDUCE-2858. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189037 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/start-all.sh * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/stop-all.sh * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/StringHelper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/View.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/ErrorPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnCssLayout.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnLayout.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttempt.java * /had
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135753#comment-13135753 ] Hudson commented on MAPREDUCE-2858: --- Integrated in Hadoop-Common-0.23-Commit #53 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/53/]) Merge -c 1189036 from trunk to branch-0.23 to fix for MAPREDUCE-2858. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189037 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/start-all.sh * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/stop-all.sh * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/StringHelper.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/View.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/ErrorPage.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnCssLayout.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnLayout.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttempt.java *
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135747#comment-13135747 ] Hudson commented on MAPREDUCE-2858: --- Integrated in Hadoop-Common-trunk-Commit #1157 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1157/]) MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189036 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/start-all.sh * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/stop-all.sh * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/StringHelper.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/View.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/ErrorPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnCssLayout.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnLayout.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTrans
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Robert! > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135746#comment-13135746 ] Hudson commented on MAPREDUCE-2858: --- Integrated in Hadoop-Hdfs-trunk-Commit #1235 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1235/]) MAPREDUCE-2858. Added a WebApp Proxy for applications. Contributed by Robert Evans. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189036 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/NotRunningJob.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/start-all.sh * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/stop-all.sh * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/StringHelper.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/View.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/ErrorPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnCssLayout.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/TwoColumnLayout.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttempt.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitio
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135745#comment-13135745 ] Mahadev konar commented on MAPREDUCE-2858: -- +1, looks good to me. We can go ahead and commit this for now. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135738#comment-13135738 ] Hadoop QA commented on MAPREDUCE-2858: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500824/MAPREDUCE-2858.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 170 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-web-proxy.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1149//console This message is automatically generated. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135733#comment-13135733 ] Hadoop QA commented on MAPREDUCE-2858: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500823/MAPREDUCE-2858.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1148//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1148//console This message is automatically generated. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Status: Patch Available (was: Open) > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Attachment: MAPREDUCE-2858.patch Missed a change. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MAPREDUCE-2858.patch, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Status: Open (was: Patch Available) > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Status: Patch Available (was: Open) Kicking hudson. With my limited knowledge of this stuff this looks fine. I'll commit unless I hear objections - we can iterate more with a couple of follow on jiras, this one is too big to keep up-to-date. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Target Version/s: 0.23.0 (was: 0.24.0, 0.23.0) Affects Version/s: (was: 0.24.0) Status: Open (was: Patch Available) Need to rebase. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2858: - Attachment: MAPREDUCE-2858.patch Rebased. Minor nit - changed names from Proxy* to WebAppProxy* to ensure we don't confuse new devs with java's proxy etc. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2858.patch, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858.txt, MR-2858.txt, MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135713#comment-13135713 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Mapreduce-0.23-Commit #52 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/52/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) - Merging r1189023 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189024 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3240: - Status: Open (was: Patch Available) Need to investigate test failure. > NM should send a SIGKILL for completed containers also > -- > > Key: MAPREDUCE-3240 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, > MR-3240.wip.patch > > > This is to address the containers which exit properly after spawning > sub-processes themselves. We don't want to leave these sub-process-tree or > else they can pillage the NM's resources. > Today, we already have code to send SIGKILL to the whole process-trees > (because of single sessionId resulting from setsid) when the container is > alive. We need to obtain the PID of the containers when they start and use > that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135709#comment-13135709 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1169 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1169/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189023 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135703#comment-13135703 ] Hadoop QA commented on MAPREDUCE-3240: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500815/MR-3240.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 22 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 166 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1147//console This message is automatically generated. > NM should send a SIGKILL for completed containers also > -- > > Key: MAPREDUCE-3240 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, > MR-3240.wip.patch > > > This is to address the containers which exit properly after spawning > sub-processes themselves. We don't want to leave these sub-process-tree or > else they can pillage the NM's resources. > Today, we already have code to send SIGKILL to the whole process-trees > (because of single sessionId resulting from setsid) when the container is > alive. We need to obtain the PID of the containers when they start and use > that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135700#comment-13135700 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Common-0.23-Commit #52 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/52/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) - Merging r1189023 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189024 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135699#comment-13135699 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Hdfs-0.23-Commit #53 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/53/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) - Merging r1189023 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189024 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135695#comment-13135695 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Common-trunk-Commit #1155 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1155/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189023 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3250: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Also, ran dos2unix on TestClientServiceDelegate.java when committing. Thanks Vinod. > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135693#comment-13135693 ] Hudson commented on MAPREDUCE-3250: --- Integrated in Hadoop-Hdfs-trunk-Commit #1233 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1233/]) MAPREDUCE-3250. When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. (vinodkv via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189023 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientServiceDelegate.java > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3175: - Status: Open (was: Patch Available) Seems ok, one nit - move ApplicationACLsManager to yarn-common rather than add module deps for yarn-server-common. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3240: --- Status: Open (was: Patch Available) > NM should send a SIGKILL for completed containers also > -- > > Key: MAPREDUCE-3240 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, > MR-3240.wip.patch > > > This is to address the containers which exit properly after spawning > sub-processes themselves. We don't want to leave these sub-process-tree or > else they can pillage the NM's resources. > Today, we already have code to send SIGKILL to the whole process-trees > (because of single sessionId resulting from setsid) when the container is > alive. We need to obtain the PID of the containers when they start and use > that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3240: --- Status: Patch Available (was: Open) > NM should send a SIGKILL for completed containers also > -- > > Key: MAPREDUCE-3240 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, > MR-3240.wip.patch > > > This is to address the containers which exit properly after spawning > sub-processes themselves. We don't want to leave these sub-process-tree or > else they can pillage the NM's resources. > Today, we already have code to send SIGKILL to the whole process-trees > (because of single sessionId resulting from setsid) when the container is > alive. We need to obtain the PID of the containers when they start and use > that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3240) NM should send a SIGKILL for completed containers also
[ https://issues.apache.org/jira/browse/MAPREDUCE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3240: --- Attachment: MR-3240.3.patch Re-submitting same patch to get pre-commit build to run > NM should send a SIGKILL for completed containers also > -- > > Key: MAPREDUCE-3240 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3240 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Hitesh Shah > Fix For: 0.23.0 > > Attachments: MR-3240.1.patch, MR-3240.2.patch, MR-3240.3.patch, > MR-3240.wip.patch > > > This is to address the containers which exit properly after spawning > sub-processes themselves. We don't want to leave these sub-process-tree or > else they can pillage the NM's resources. > Today, we already have code to send SIGKILL to the whole process-trees > (because of single sessionId resulting from setsid) when the container is > alive. We need to obtain the PID of the containers when they start and use > that PID to send signal for completed containers' case also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135685#comment-13135685 ] Hadoop QA commented on MAPREDUCE-3250: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500812/MAPREDUCE-3250.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 166 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1146//console This message is automatically generated. > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3250: - Status: Patch Available (was: Open) Still running some manual tests to verify. > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3250: - Attachment: MAPREDUCE-3250.patch Updated patch for trunk > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3250) When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3250: - Status: Open (was: Patch Available) Looks like the patch doesnt apply. > When AM restarts, client keeps reconnecting to the new AM and prints a lots > of logs. > > > Key: MAPREDUCE-3250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3250 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3250-20111024.txt, > MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250-20111025.txt, MAPREDUCE-3250.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135680#comment-13135680 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1168 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1168/]) MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189012 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services/org.apache.hadoop.security.token.TokenRenewer > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Projec
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135679#comment-13135679 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Mapreduce-0.23-Commit #51 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/51/]) Merge -c 1189012 from trunk to branch-0.23 to complete fix for MAPREDUCE-2977. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189013 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resource
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135674#comment-13135674 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Common-trunk-Commit #1154 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1154/]) MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189012 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services/org.apache.hadoop.security.token.TokenRenewer > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Had
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135673#comment-13135673 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Common-0.23-Commit #51 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/51/]) Merge -c 1189012 from trunk to branch-0.23 to complete fix for MAPREDUCE-2977. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189013 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanage
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135672#comment-13135672 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Hdfs-trunk-Commit #1232 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1232/]) MAPREDUCE-2977. Fix ResourceManager to renew HDFS delegation tokens for applications. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189012 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services/org.apache.hadoop.security.token.TokenRenewer > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135671#comment-13135671 ] Hudson commented on MAPREDUCE-2977: --- Integrated in Hadoop-Hdfs-0.23-Commit #52 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/52/]) Merge -c 1189012 from trunk to branch-0.23 to complete fix for MAPREDUCE-2977. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189013 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/META-INF/services * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/sr
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135669#comment-13135669
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Mapreduce-0.23-Commit #50 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/50/])
Merge -c 1188999 from trunk to branch-0.23 to complete fix for
MAPREDUCE-3269.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch, > MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135668#comment-13135668 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Mapreduce-0.23-Commit #50 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/50/]) Merge -c 1188997 from trunk to branch-0.23 to complete fix for MAPREDUCE-3263. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188998 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135666#comment-13135666
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Mapreduce-trunk-Commit #1167 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1167/])
MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for
JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135665#comment-13135665 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1167 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1167/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3028) Support job end notification in .next /0.23
[ https://issues.apache.org/jira/browse/MAPREDUCE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135667#comment-13135667 ] Hudson commented on MAPREDUCE-3028: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1167 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1167/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > Support job end notification in .next /0.23 > --- > > Key: MAPREDUCE-3028 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3028 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Mohammad Kamrul Islam >Assignee: Ravi Prakash >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3028.branch-0.23.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, RMContainerAllocator.diff > > > Oozie primarily depends on the job end notification to determine when the > job finishes. In the current version, job end notification is implemented in > job tracker. Since job tracker will be removed in the upcoming hadoop release > (.next), we wander where this support will move. I think this best effort > notification could be implemented in the new Application Manager as one of > the last step of job completion. > Whatever implementation will it be, Oozie badly needs this feature to be > continued in next releases as well. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2858) MRv2 WebApp Security
[ https://issues.apache.org/jira/browse/MAPREDUCE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135663#comment-13135663 ] Robert Joseph Evans commented on MAPREDUCE-2858: It looks like Trunk was broken when I submitted the patch. https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1138/console If someone could kick the build again I would appreciate it. If not I will resubmit tomorrow morning. > MRv2 WebApp Security > > > Key: MAPREDUCE-2858 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2858 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, security >Affects Versions: 0.23.0, 0.24.0 >Reporter: Luke Lu >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, > MR-2858-branch-0.23.txt, MR-2858-branch-0.23.txt, MR-2858.txt, MR-2858.txt, > MR-2858.txt, MR-2858.txt > > > In MRv2, while the system servers (ResourceManager (RM), NodeManager (NM) and > NameNode (NN)) run as "trusted" > system users, the application masters (AM) run as users who submit the > application. While this offers great flexibility > to run multiple version of mapreduce frameworks (including their UI) on the > same Hadoop cluster, it has significant > implication for the security of webapps (Please do not discuss company > specific vulnerabilities here). > Requirements: > # Secure authentication for AM (for app/job level ACLs). > # Webapp security should be optional via site configuration. > # Support existing pluggable single sign on mechanisms. > # Should not require per app/user configuration for deployment. > # Should not require special site-wide DNS configuration for deployment. > This the top jira for webapp security. A design doc/notes of threat-modeling > and counter measures will be posted on the wiki. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135655#comment-13135655 ] Hadoop QA commented on MAPREDUCE-2977: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500804/MAPREDUCE-2977.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 25 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 1700 javac compiler warnings (more than the trunk's current 1699 warnings). -1 findbugs. The patch appears to introduce 166 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1145//console This message is automatically generated. > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch, > MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135650#comment-13135650 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Common-0.23-Commit #50 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/50/]) Merge -c 1188997 from trunk to branch-0.23 to complete fix for MAPREDUCE-3263. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188998 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135651#comment-13135651
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Common-0.23-Commit #50 (See
[https://builds.apache.org/job/Hadoop-Common-0.23-Commit/50/])
Merge -c 1188999 from trunk to branch-0.23 to complete fix for
MAPREDUCE-3269.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3028) Support job end notification in .next /0.23
[ https://issues.apache.org/jira/browse/MAPREDUCE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135647#comment-13135647 ] Hudson commented on MAPREDUCE-3028: --- Integrated in Hadoop-Hdfs-trunk-Commit #1231 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1231/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > Support job end notification in .next /0.23 > --- > > Key: MAPREDUCE-3028 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3028 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Mohammad Kamrul Islam >Assignee: Ravi Prakash >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3028.branch-0.23.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, RMContainerAllocator.diff > > > Oozie primarily depends on the job end notification to determine when the > job finishes. In the current version, job end notification is implemented in > job tracker. Since job tracker will be removed in the upcoming hadoop release > (.next), we wander where this support will move. I think this best effort > notification could be implemented in the new Application Manager as one of > the last step of job completion. > Whatever implementation will it be, Oozie badly needs this feature to be > continued in next releases as well. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135646#comment-13135646
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Hdfs-trunk-Commit #1231 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1231/])
MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for
JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135643#comment-13135643
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Common-trunk-Commit #1153 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1153/])
MAPREDUCE-3269. Fixed log4j properties to correctly set logging options for
JobHistoryServer vis-a-vis JobSummary logs. Contributed by Mahadev Konar.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188999
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3028) Support job end notification in .next /0.23
[ https://issues.apache.org/jira/browse/MAPREDUCE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135644#comment-13135644 ] Hudson commented on MAPREDUCE-3028: --- Integrated in Hadoop-Common-trunk-Commit #1153 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1153/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > Support job end notification in .next /0.23 > --- > > Key: MAPREDUCE-3028 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3028 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Mohammad Kamrul Islam >Assignee: Ravi Prakash >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3028.branch-0.23.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, MAPREDUCE-3028.patch, > MAPREDUCE-3028.patch, RMContainerAllocator.diff > > > Oozie primarily depends on the job end notification to determine when the > job finishes. In the current version, job end notification is implemented in > job tracker. Since job tracker will be removed in the upcoming hadoop release > (.next), we wander where this support will move. I think this best effort > notification could be implemented in the new Application Manager as one of > the last step of job completion. > Whatever implementation will it be, Oozie badly needs this feature to be > continued in next releases as well. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135645#comment-13135645 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Hdfs-trunk-Commit #1231 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1231/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135642#comment-13135642 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Common-trunk-Commit #1153 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1153/]) MAPREDUCE-3263. Fixed the MAPREDUCE-3028 commit which broke MR1. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188997 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135640#comment-13135640 ] Hudson commented on MAPREDUCE-3263: --- Integrated in Hadoop-Hdfs-0.23-Commit #51 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/51/]) Merge -c 1188997 from trunk to branch-0.23 to complete fix for MAPREDUCE-3263. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188998 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/NotificationTestCase.java > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135641#comment-13135641
]
Hudson commented on MAPREDUCE-3269:
---
Integrated in Hadoop-Hdfs-0.23-Commit #51 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/51/])
Merge -c 1188999 from trunk to branch-0.23 to complete fix for
MAPREDUCE-3269.
acmurthy :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189000
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn-daemon.sh
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3264) mapreduce.job.user.name needs to be set automatically
[ https://issues.apache.org/jira/browse/MAPREDUCE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3264: - Attachment: MAPREDUCE-3264.patch One possible, simple fix. > mapreduce.job.user.name needs to be set automatically > - > > Key: MAPREDUCE-3264 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3264 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Priority: Blocker > Attachments: MAPREDUCE-3264.patch > > > Currently in MR2 I have to manually specify mapreduce.job.user.name for each > job. It's not picking it up from the security infrastructure, at least when > running with DefaultContainerExecutor. This is obviously incorrect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3264) mapreduce.job.user.name needs to be set automatically
[ https://issues.apache.org/jira/browse/MAPREDUCE-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135638#comment-13135638 ] Arun C Murthy commented on MAPREDUCE-3264: -- Vinod - can we do better? > mapreduce.job.user.name needs to be set automatically > - > > Key: MAPREDUCE-3264 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3264 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Priority: Blocker > Attachments: MAPREDUCE-3264.patch > > > Currently in MR2 I have to manually specify mapreduce.job.user.name for each > job. It's not picking it up from the security infrastructure, at least when > running with DefaultContainerExecutor. This is obviously incorrect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3257) Authorization checks needed for AM->RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3257: - Status: Open (was: Patch Available) Need to investigate unit test failures. > Authorization checks needed for AM->RM protocol > --- > > Key: MAPREDUCE-3257 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3257 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3257-20111025.2.txt, > MAPREDUCE-3257-20111025.txt > > > This is like MAPREDUCE-3256, but for AM->RM protocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-3269:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Mahadev!
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3263: - Resolution: Fixed Fix Version/s: 0.23.0 Status: Resolved (was: Patch Available) I just committed this. Thanks Hitesh! > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Attachment: MAPREDUCE-2977.patch Fixed test and javadoc warnings. The javac one is due to the test which isn't legit. > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch, > MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Status: Patch Available (was: Open) > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch, > MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Status: Open (was: Patch Available) > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135626#comment-13135626 ] Jonathan Eagles commented on MAPREDUCE-3175: javadoc warnings are the same as last build. findbugs seems to be getting confused > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135616#comment-13135616 ] Jonathan Eagles commented on MAPREDUCE-3175: -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 8 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce -9 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2736) Remove unused contrib components dependent on MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135613#comment-13135613 ] Eli Collins commented on MAPREDUCE-2736: Anyone object to the script/delta? If not I'll commit this to both trunk and 23. > Remove unused contrib components dependent on MR1 > - > > Key: MAPREDUCE-2736 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2736 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: jobtracker, tasktracker >Reporter: Eli Collins >Assignee: Eli Collins > Fix For: 0.23.0 > > Attachments: mapreduce-2736-1.patch, mapreduce-2736-2.patch, > mapreduce-2736-2.patch, mapreduce-2736-3.patch, mapreduce-2736-delta-1.patch, > rm-mr1.sh > > > As discussed on mapreduce-dev@ and general@, let's remove or disable MR1. > 1. > http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-dev/201107.mbox/%3ccapn_vtsdiiqfcb2g0hfsor3w_4pkoocpctf2vb93y3mzrzr...@mail.gmail.com%3E > 2. > http://mail-archives.apache.org/mod_mbox/hadoop-general/201107.mbox/%3ccapn_vtsjcertt35rskcyky2m+qbk3w-zemqkpd+wnrayo7t...@mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3254) Streaming jobs failing with PipeMapRunner ClassNotFoundException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned MAPREDUCE-3254: -- Assignee: Arun C Murthy (was: Hitesh Shah) > Streaming jobs failing with PipeMapRunner ClassNotFoundException > > > Key: MAPREDUCE-3254 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3254 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Arun C Murthy >Priority: Blocker > > ClassNotFoundException: org.apache.hadoop.streaming.PipeMapRunner encountered > while running streaming jobs. Stack trace in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135612#comment-13135612 ] Hitesh Shah commented on MAPREDUCE-3263: Javadoc warnings are unrelated. Seem to be due to a private class: Tag @link: reference not found: ResourceLocalizationService.LocalizerTracker Will take a look separately. > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3262) Handle missing events in failure scenarios
[ https://issues.apache.org/jira/browse/MAPREDUCE-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned MAPREDUCE-3262: -- Assignee: Hitesh Shah > Handle missing events in failure scenarios > -- > > Key: MAPREDUCE-3262 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah > Fix For: 0.23.0 > > > Need to handle kill container event in localization failed state. > Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3273) mapreduce.cluster.adminstrators should be included in mapred-default.xml
mapreduce.cluster.adminstrators should be included in mapred-default.xml Key: MAPREDUCE-3273 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3273 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation Affects Versions: 0.20.205.0 Reporter: Eli Collins Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135594#comment-13135594 ] Ramya Sunil commented on MAPREDUCE-2977: Tested the patch by setting a delegation token renewal time < duration to complete a app. RM renewed the DT periodically and the app completed successfully: Renewing HDFS_DELEGATION_TOKEN token for org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer: Renewing delegation-token for:; new expiration; After the job completed, the delegation token was cancelled org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer: Canceling token > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135586#comment-13135586 ] Hadoop QA commented on MAPREDUCE-3263: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500784/MR-3263.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 8 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 165 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1144//console This message is automatically generated. > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135580#comment-13135580 ] Hadoop QA commented on MAPREDUCE-2977: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500781/MAPREDUCE-2977.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 25 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 9 warning messages. -1 javac. The applied patch generated 1700 javac compiler warnings (more than the trunk's current 1699 warnings). -1 findbugs. The patch appears to introduce 166 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestAppManager +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1143//console This message is automatically generated. > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3263: --- Attachment: MR-3263.1.patch > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135575#comment-13135575
]
Hadoop QA commented on MAPREDUCE-3269:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12500777/MAPREDUCE-3269.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 javadoc. The javadoc tool appears to have generated 8 warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 165 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1142//console
This message is automatically generated.
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3263: --- Status: Patch Available (was: Open) Attached patch to fix errors in build due to change in fields in MRJobConfig. > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > Attachments: MR-3263.1.patch > > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3263) compile-mapred-test target fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned MAPREDUCE-3263: -- Assignee: Hitesh Shah > compile-mapred-test target fails > > > Key: MAPREDUCE-3263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Hitesh Shah >Priority: Blocker > > Compile mapred test target is broken due to which the builds are not > archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Attachment: MAPREDUCE-2977.patch Updated patch, includes a test case. > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2977) ResourceManager needs to renew and cancel tokens associated with a job
[ https://issues.apache.org/jira/browse/MAPREDUCE-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2977: - Status: Patch Available (was: Open) > ResourceManager needs to renew and cancel tokens associated with a job > -- > > Key: MAPREDUCE-2977 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2977 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, resourcemanager, security >Affects Versions: 0.23.0 >Reporter: Owen O'Malley >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2977.patch, MAPREDUCE-2977.patch > > > The JobTracker currently manages tokens for the applications and the resource > manager needs the same functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3272) Lost NMs fail to rejoin
Lost NMs fail to rejoin
---
Key: MAPREDUCE-3272
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3272
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
Lost nodemanagers fail to join back.
When the NM is lost, RM log reads
{noformat}
INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: Expired:
Timed out after 600 secs
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
Processing of type EXPIRE
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Removed
Node
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
Node Transitioned from RUNNING to LOST
{noformat}
When the NM joins back, RM log reads
{noformat}
INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: Node
not found rebooting
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3270) Decommissioned node not removed from active NM list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3270: - Priority: Major (was: Blocker) > Decommissioned node not removed from active NM list > --- > > Key: MAPREDUCE-3270 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3270 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil > > A decommissioned node is not being removed from the "Total nodes" list and is > not added to the "Decommissioned nodes" list. > The list of nodes to decommission is added in a file defined by > "yarn.resourcemanager.nodes.exclude-path" and excluded via refreshNodes CLI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3270) Decommissioned node not removed from active NM list
[ https://issues.apache.org/jira/browse/MAPREDUCE-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135565#comment-13135565 ] Arun C Murthy commented on MAPREDUCE-3270: -- MAPREDUCE-2775 should fix this. > Decommissioned node not removed from active NM list > --- > > Key: MAPREDUCE-3270 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3270 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Priority: Blocker > > A decommissioned node is not being removed from the "Total nodes" list and is > not added to the "Decommissioned nodes" list. > The list of nodes to decommission is added in a file defined by > "yarn.resourcemanager.nodes.exclude-path" and excluded via refreshNodes CLI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3271) Lost nodes list and count not updated
Lost nodes list and count not updated - Key: MAPREDUCE-3271 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3271 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 When nodemanagers are lost, the "Lost Nodes" list and the count is not incremented. Either we, 1. Fix the lost nodes list when a nodemanager is lost - The problem with tracking lost nodes is, if the nodemanager joins back, there would be duplicate entries in active and lost nodes with different port numbers. 2. Do not track lost nodemanagers -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3269:
-
Fix Version/s: 0.23.0
Status: Patch Available (was: Open)
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3269:
-
Attachment: MAPREDUCE-3269.patch
Tested the fix. This works.
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3269.patch, MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3269:
-
Attachment: MAPREDUCE-3269.patch
Minor fixes to set default to INFO, JSA in yarn-daemon.sh. Doing some testing.
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Priority: Blocker
> Attachments: MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
[
https://issues.apache.org/jira/browse/MAPREDUCE-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar reassigned MAPREDUCE-3269:
Assignee: Mahadev konar
> Jobsummary logs not being moved to a separate file
> --
>
> Key: MAPREDUCE-3269
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Attachments: MAPREDUCE-3269.patch
>
>
> The jobsummary logs are not being moved to a separate file. Below is the
> configuration in log4j.properties:
> {noformat}
> mapred.jobsummary.logger=INFO,console
> log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
> log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
> log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
> log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
> log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> log4j.appender.JSA.DatePattern=.-MM-dd
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2775) [MR-279] Decommissioned node does not shutdown
[ https://issues.apache.org/jira/browse/MAPREDUCE-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2775: - Status: Open (was: Patch Available) Devaraj, the patch looks good. But you seemed to have missed some files (ClusterMetrics.java) etc. Also, please move NodeAction to yarn-server-common, it shouldn't be in yarn-api since it's internal. It should be in ./hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/proto/yarn_server_common_protos.proto > [MR-279] Decommissioned node does not shutdown > -- > > Key: MAPREDUCE-2775 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2775 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2775-1.patch, MAPREDUCE-2775-2.patch, > MAPREDUCE-2775-3.patch, MAPREDUCE-2775-4.patch, MAPREDUCE-2775-5.patch, > MAPREDUCE-2775.patch, MAPREDUCE-2775.patch > > > A Nodemanager which is decommissioned by an admin via refreshnodes does not > automatically shutdown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3251) JobClient should have an option to only to talk to RM+HistoryServer to get job status
[ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135500#comment-13135500 ] Milind Bhandarkar commented on MAPREDUCE-3251: -- Got it. Thanks. (I know of a few dashboards that might break if non-gateway access to JT was completely disabled ;-) > JobClient should have an option to only to talk to RM+HistoryServer to get > job status > - > > Key: MAPREDUCE-3251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Anupam Seth >Priority: Blocker > Fix For: 0.23.0 > > > In 0.20.xxx, the JobClient while polling goes to JT to get the job status. > With YARN, AM can be launched on any port and the client will have to have > ACL open to that port to talk to AM and get the job status. When the client > is within the same grid network access to AM is not a problem. But some > applications may have one installation per set of clusters and may launch > jobs even across such sets (on job trackers in another set of clusters). For > that to work only the JT port needs to be open currently. In case of YARN, > all ports will have to be opened up for things to work. That would be a > security no-no. > There are two possible solutions: > 1) Make the job client only talk to RM (as an option) to get the job > status. > 2) Limit the range of ports AM can listen on. > Option 2) may not be favorable as there is no direct OS API to find a free > port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2328) memory-related configurations missing from mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135502#comment-13135502 ] Konstantin Shvachko commented on MAPREDUCE-2328: Sorry to come even later. This does apply to 0.22 and would be good to have it in the release. But a patch for trunk is needed. > memory-related configurations missing from mapred-default.xml > - > > Key: MAPREDUCE-2328 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2328 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Harsh J > Labels: newbie > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2328.r1.diff, MAPREDUCE-2328.r2.diff, > MAPREDUCE-2328.r3.diff > > > HADOOP-5881 added new configuration parameters for memory-based scheduling, > but they weren't added to mapred-default.xml -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3251) JobClient should have an option to only to talk to RM+HistoryServer to get job status
[ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135487#comment-13135487 ] Arun C Murthy commented on MAPREDUCE-3251: -- Milind, this option is only for jobclients who cannot talk to AM directly, they get to choose. By default we assume JobClient can talk to AM to get full status etc. As described in the description of the jira it's for a few special cases. > JobClient should have an option to only to talk to RM+HistoryServer to get > job status > - > > Key: MAPREDUCE-3251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Anupam Seth >Priority: Blocker > Fix For: 0.23.0 > > > In 0.20.xxx, the JobClient while polling goes to JT to get the job status. > With YARN, AM can be launched on any port and the client will have to have > ACL open to that port to talk to AM and get the job status. When the client > is within the same grid network access to AM is not a problem. But some > applications may have one installation per set of clusters and may launch > jobs even across such sets (on job trackers in another set of clusters). For > that to work only the JT port needs to be open currently. In case of YARN, > all ports will have to be opened up for things to work. That would be a > security no-no. > There are two possible solutions: > 1) Make the job client only talk to RM (as an option) to get the job > status. > 2) Limit the range of ports AM can listen on. > Option 2) may not be favorable as there is no direct OS API to find a free > port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135488#comment-13135488 ] Jonathan Eagles commented on MAPREDUCE-3175: Patch needs HADOOP-7764 to compile. Will post test results from my box. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135485#comment-13135485 ] Hadoop QA commented on MAPREDUCE-3175: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500529/MAPREDUCE-3175.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1141//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1141//console This message is automatically generated. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3251) JobClient should have an option to only to talk to RM+HistoryServer to get job status
[ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135481#comment-13135481 ] Milind Bhandarkar commented on MAPREDUCE-3251: -- Arun, sorry to keep harping on this, but with this approach, the current individual task completion status will never be conveyed to the job client, right ? And since ApplicationReport is part of the public API, there is no way to include MR-specific details (even as a blob) there. As a concrete case, I do not see a way for the RM to report individual map progress, and reduce progress, for example, if we stop opening the JT port for job client. > JobClient should have an option to only to talk to RM+HistoryServer to get > job status > - > > Key: MAPREDUCE-3251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Anupam Seth >Priority: Blocker > Fix For: 0.23.0 > > > In 0.20.xxx, the JobClient while polling goes to JT to get the job status. > With YARN, AM can be launched on any port and the client will have to have > ACL open to that port to talk to AM and get the job status. When the client > is within the same grid network access to AM is not a problem. But some > applications may have one installation per set of clusters and may launch > jobs even across such sets (on job trackers in another set of clusters). For > that to work only the JT port needs to be open currently. In case of YARN, > all ports will have to be opened up for things to work. That would be a > security no-no. > There are two possible solutions: > 1) Make the job client only talk to RM (as an option) to get the job > status. > 2) Limit the range of ports AM can listen on. > Option 2) may not be favorable as there is no direct OS API to find a free > port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135467#comment-13135467 ] Jonathan Eagles commented on MAPREDUCE-3175: Will file another bug to address ease of use for non-MR apps. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135463#comment-13135463 ] Jonathan Eagles commented on MAPREDUCE-3175: Since this change only connected the ACLs to the HttpServer, I have added no tests since classes inheriting from WebApp are not very testable. Manual tests confirm /stack /metrics /logs /logLevel /conf and /jmx are not accessible by other users from RM, NM, HS, and AM. > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3175) Yarn httpservers not created with access Control lists
[ https://issues.apache.org/jira/browse/MAPREDUCE-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-3175: --- Status: Patch Available (was: Open) > Yarn httpservers not created with access Control lists > -- > > Key: MAPREDUCE-3175 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3175 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Jonathan Eagles >Priority: Blocker > Attachments: MAPREDUCE-3175.patch, MAPREDUCE-3175.patch, > MAPREDUCE-3175.patch > > > RM, NM, job history, and application master httpservers are not created with > access Control lists. I believe this means that anyone can access any of the > standard servlets that check to see if the user has administrator access - > like /jmx, /stacks, etc and ops has no way to restrict access to these things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3253) ContextFactory throw NoSuchFieldException
[ https://issues.apache.org/jira/browse/MAPREDUCE-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135453#comment-13135453 ] Hudson commented on MAPREDUCE-3253: --- Integrated in Hadoop-Hdfs-trunk #843 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/843/]) MAPREDUCE-3253. Fixed ContextFactory to clone JobContext correctly. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188842 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ContextFactory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestContextFactory.java > ContextFactory throw NoSuchFieldException > - > > Key: MAPREDUCE-3253 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3253 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Daniel Dai >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3253.patch > > > I see exceptions from ContextFactory when I am running Pig unit test: > Caused by: java.lang.IllegalArgumentException: Can't find field > at > org.apache.hadoop.mapreduce.ContextFactory.(ContextFactory.java:139) > Caused by: java.lang.NoSuchFieldException: reporter > at java.lang.Class.getDeclaredField(Class.java:1882) > at > org.apache.hadoop.mapreduce.ContextFactory.(ContextFactory.java:126) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3186) User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135447#comment-13135447 ] Eric Payne commented on MAPREDUCE-3186: --- Thanks Sid! That did the trick. One more question... Any thoughts on the best way to unit test this? > User jobs are getting hanged if the Resource manager process goes down and > comes up while job is getting executed. > -- > > Key: MAPREDUCE-3186 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3186 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 > Environment: linux >Reporter: Ramgopal N >Assignee: Eric Payne > Labels: test > > If the resource manager is restarted while the job execution is in progress, > the job is getting hanged. > UI shows the job as running. > In the RM log, it is throwing an error "ERROR > org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: > AppAttemptId doesnt exist in cache appattempt_1318579738195_0004_01" > In the console MRAppMaster and Runjar processes are not getting killed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3251) JobClient should have an option to only to talk to RM+HistoryServer to get job status
[ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135445#comment-13135445 ] Arun C Murthy commented on MAPREDUCE-3251: -- In case of running jobs, if this option is enabled, we should never talk to AM and instead just get progress from ApplicationReport to display it. Once the job completes, the job-client should get counters etc. from the MR JobHistoryServer. Makes sense? > JobClient should have an option to only to talk to RM+HistoryServer to get > job status > - > > Key: MAPREDUCE-3251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Anupam Seth >Priority: Blocker > Fix For: 0.23.0 > > > In 0.20.xxx, the JobClient while polling goes to JT to get the job status. > With YARN, AM can be launched on any port and the client will have to have > ACL open to that port to talk to AM and get the job status. When the client > is within the same grid network access to AM is not a problem. But some > applications may have one installation per set of clusters and may launch > jobs even across such sets (on job trackers in another set of clusters). For > that to work only the JT port needs to be open currently. In case of YARN, > all ports will have to be opened up for things to work. That would be a > security no-no. > There are two possible solutions: > 1) Make the job client only talk to RM (as an option) to get the job > status. > 2) Limit the range of ports AM can listen on. > Option 2) may not be favorable as there is no direct OS API to find a free > port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
