[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116236#comment-13116236 ] Abhijit Suresh Shingate commented on MAPREDUCE-2843: Thanks a lot vinod!!! :-) > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2702) [MR-279] OutputCommitter changes for MR Application Master recovery
[ https://issues.apache.org/jira/browse/MAPREDUCE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116228#comment-13116228 ] Sharad Agarwal commented on MAPREDUCE-2702: --- can be done as separate jira for old api. anyway if the outputcommitter doesn't support recovery (the default), the MR AM falls back to rerun the job from beginning. > [MR-279] OutputCommitter changes for MR Application Master recovery > --- > > Key: MAPREDUCE-2702 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2702 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: mrv2 >Reporter: Sharad Agarwal >Assignee: Sharad Agarwal >Priority: Blocker > Attachments: mr2702_v1.patch, mr2702_v2.patch, mr2702_v3.patch, > mr2702_v4.patch > > > In MR AM recovers from a crash, it only reruns the non completed tasks. The > completed tasks (along with their output, if any) needs to be recovered from > the previous life. This would require some changes in OutputCommitter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116224#comment-13116224
]
Hudson commented on MAPREDUCE-3110:
---
Integrated in Hadoop-Common-trunk-Commit #975 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/975/])
MAPREDUCE-3110. Fixed TestRPC failure. Contributed by Vinod Kumar
Vavilapalli.
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176739
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116221#comment-13116221
]
Hudson commented on MAPREDUCE-3110:
---
Integrated in Hadoop-Hdfs-trunk-Commit #1053 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1053/])
MAPREDUCE-3110. Fixed TestRPC failure. Contributed by Vinod Kumar
Vavilapalli.
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176739
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116215#comment-13116215 ] Hudson commented on MAPREDUCE-2843: --- Integrated in Hadoop-Mapreduce-trunk-Commit #995 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/995/]) MAPREDUCE-2843. Fixed the node-table to be completely displayed and making node entries on RM UI to be sortable. Contributed by Abhijit Suresh Shingate. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176730 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116212#comment-13116212 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3021: Great test coverage again, Thomas! > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3078) Application's progress isn't updated from AM to RM.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116210#comment-13116210 ] Arun C Murthy commented on MAPREDUCE-3078: -- Testing on a cluster now, I'll commit after. > Application's progress isn't updated from AM to RM. > --- > > Key: MAPREDUCE-3078 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3078 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster, mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3078-20110927.txt > > > It helps to be able to monitor the application-progress from the RM UI itself. > Bits of it is already there, even the AM-RM API (in AllocateRequest). We just > need to make sure the progress is produced and consumed properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3110:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Trivial test-fix. I just committed this to trunk and branch-0.23.
Devaraj K/Mahadev, can you see if it works for you now? Thanks!
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3078) Application's progress isn't updated from AM to RM.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116206#comment-13116206 ] Arun C Murthy commented on MAPREDUCE-3078: -- +1 > Application's progress isn't updated from AM to RM. > --- > > Key: MAPREDUCE-3078 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3078 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster, mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3078-20110927.txt > > > It helps to be able to monitor the application-progress from the RM UI itself. > Bits of it is already there, even the AM-RM API (in AllocateRequest). We just > need to make sure the progress is produced and consumed properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3088) Clover 2.4.3 breaks build for 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116205#comment-13116205 ] Hudson commented on MAPREDUCE-3088: --- Integrated in Hadoop-Hdfs-22-branch #92 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/92/]) MAPREDUCE-3088. Turn off Clover 2.4.3 until upgraded. Contributed by Konstantin Shvachko. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176723 Files : * /hadoop/nightly/hudsonBuildHadoopNightly.sh > Clover 2.4.3 breaks build for 0.22 branch > - > > Key: MAPREDUCE-3088 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3088 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: nightly.patch > > > Due to known bug in Clover 2.4.3 build for 0.22 branch is broken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116201#comment-13116201 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3098: oh, and let us call this FinalApplicationStatus or something similar mainly to not confuse it with any other state. > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116197#comment-13116197 ] Hudson commented on MAPREDUCE-2843: --- Integrated in Hadoop-Common-trunk-Commit #974 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/974/]) MAPREDUCE-2843. Fixed the node-table to be completely displayed and making node entries on RM UI to be sortable. Contributed by Abhijit Suresh Shingate. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176730 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116198#comment-13116198 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3098: Okay I thought about it a bit more. All my other comments regarding ApplicationState and JobState withstanding, I am fine with an enum for the JobState. After all, there are only so many final states a job can have, even if it is a DAG inside - SUCCEEDED, FAILED, KILLED and may be ERROR/ABORTED. > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116194#comment-13116194 ] Hudson commented on MAPREDUCE-2843: --- Integrated in Hadoop-Hdfs-trunk-Commit #1052 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1052/]) MAPREDUCE-2843. Fixed the node-table to be completely displayed and making node entries on RM UI to be sortable. Contributed by Abhijit Suresh Shingate. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176730 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116192#comment-13116192
]
Hadoop QA commented on MAPREDUCE-3110:
--
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496844/MAPREDUCE-3110-20110928.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/878//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/878//console
This message is automatically generated.
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3088) Clover 2.4.3 breaks build for 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-3088: --- Component/s: build > Clover 2.4.3 breaks build for 0.22 branch > - > > Key: MAPREDUCE-3088 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3088 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: nightly.patch > > > Due to known bug in Clover 2.4.3 build for 0.22 branch is broken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2762) [MR-279] - Cleanup staging dir after job completion
[
https://issues.apache.org/jira/browse/MAPREDUCE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116188#comment-13116188
]
Mahadev konar commented on MAPREDUCE-2762:
--
Preliminary patch that fixes the issue. Doing a little more manual testing and
adding test cases.
> [MR-279] - Cleanup staging dir after job completion
> ---
>
> Key: MAPREDUCE-2762
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2762
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2762.patch
>
>
> The files created under the staging dir have to be deleted after job
> completion. Currently, all job.* files remain forever in the
> ${yarn.apps.stagingDir}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2762) [MR-279] - Cleanup staging dir after job completion
[
https://issues.apache.org/jira/browse/MAPREDUCE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-2762:
-
Attachment: MAPREDUCE-2762.patch
> [MR-279] - Cleanup staging dir after job completion
> ---
>
> Key: MAPREDUCE-2762
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2762
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2762.patch
>
>
> The files created under the staging dir have to be deleted after job
> completion. Currently, all job.* files remain forever in the
> ${yarn.apps.stagingDir}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3088) Clover 2.4.3 breaks build for 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116187#comment-13116187 ] Hudson commented on MAPREDUCE-3088: --- Integrated in Hadoop-Mapreduce-trunk-Commit #994 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/994/]) MAPREDUCE-3088. Turn off Clover 2.4.3 until upgraded. Contributed by Konstantin Shvachko. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176723 Files : * /hadoop/nightly/hudsonBuildHadoopNightly.sh > Clover 2.4.3 breaks build for 0.22 branch > - > > Key: MAPREDUCE-3088 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3088 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: nightly.patch > > > Due to known bug in Clover 2.4.3 build for 0.22 branch is broken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3112) Calling hadoop cli inside mapreduce job leads to errors
Calling hadoop cli inside mapreduce job leads to errors
---
Key: MAPREDUCE-3112
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3112
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.205.0
Environment: Java, Linux
Reporter: Eric Yang
Assignee: Eric Yang
Fix For: 0.20.205.0
When running a streaming job with mapper
bin/hadoop --config /etc/hadoop/ jar
contrib/streaming/hadoop-streaming-0.20.205.0.jar -mapper "hadoop --config
/etc/hadoop/ dfs -help" -reducer NONE -input "/tmp/input.txt" -output NONE
Task log shows:
{noformat}
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:57)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
Caused by: org.apache.commons.logging.LogConfigurationException: User-specified
log class 'org.apache.commons.logging.impl.Log4JLogger' cannot be found or is
not useable.
at
org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:874)
at
org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
at org.apache.hadoop.conf.Configuration.(Configuration.java:142)
... 3 more
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed
with code 1
at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:255)
{noformat}
Upon inspection, there are two problems in the inherited from environment which
prevent the logger initialization to work properly. In hadoop-env.sh, the
HADOOP_OPTS is inherited from the parent process. This configuration was
requested by user to have a way to override HADOOP environment in the
configuration template:
{noformat}
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_OPTS"
{noformat}
-Dhadoop.log.dir=$HADOOP_LOG_DIR/task_tracker_user is injected into HADOOP_OPTS
in the tasktracker environment. Hence, the running task would inherit the
wrong logging directory, which the end user might not have sufficient access to
write. Second, $HADOOP_ROOT_LOGGER is override to:
-Dhadoop.root.logger=INFO,TLA by the task controller, therefore, the bin/hadoop
script will attempt to use hadoop.root.logger=INFO,TLA, but fail to initialize.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2843: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this to trunk and branch-0.23. Thanks Abhijit! > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116180#comment-13116180 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-2843: +1. Looks good to me too. Verified that it is working on a single node. > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3088) Clover 2.4.3 breaks build for 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116174#comment-13116174 ] Hudson commented on MAPREDUCE-3088: --- Integrated in Hadoop-Common-trunk-Commit #973 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/973/]) MAPREDUCE-3088. Turn off Clover 2.4.3 until upgraded. Contributed by Konstantin Shvachko. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176723 Files : * /hadoop/nightly/hudsonBuildHadoopNightly.sh > Clover 2.4.3 breaks build for 0.22 branch > - > > Key: MAPREDUCE-3088 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3088 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: nightly.patch > > > Due to known bug in Clover 2.4.3 build for 0.22 branch is broken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3088) Clover 2.4.3 breaks build for 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116170#comment-13116170 ] Hudson commented on MAPREDUCE-3088: --- Integrated in Hadoop-Hdfs-trunk-Commit #1051 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1051/]) MAPREDUCE-3088. Turn off Clover 2.4.3 until upgraded. Contributed by Konstantin Shvachko. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176723 Files : * /hadoop/nightly/hudsonBuildHadoopNightly.sh > Clover 2.4.3 breaks build for 0.22 branch > - > > Key: MAPREDUCE-3088 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3088 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 0.22.0 > > Attachments: nightly.patch > > > Due to known bug in Clover 2.4.3 build for 0.22 branch is broken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3110:
---
Status: Patch Available (was: Reopened)
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3110:
---
Attachment: MAPREDUCE-3110-20110928.txt
Attaching patch that should fix this. Using a pattern for matching the
exception message.
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3110-20110928.txt
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116152#comment-13116152 ] Hudson commented on MAPREDUCE-3041: --- Integrated in Hadoop-Mapreduce-trunk-Commit #992 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/992/]) MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource capabilities along-with new ApplicationId for application submission. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176715 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116145#comment-13116145 ] Arun C Murthy commented on MAPREDUCE-3098: -- bq. Apologies if I am giving an impression of creating a storm. But honestly all of this was thought through earlier end-to-end and I firmly belive that the current states and records represent the correct abstraction. Healthy back & forth is good. Helps everyone - don't worry about it. Seems to me that there are 2 states: a) State of the AM _attempt_ b) State of the application itself. Oh, we also have 'diagnostics', more on this later. So, the crux is: forcing application writers to come up with Strings to describe their state seems a poor api/abstraction. Can we do better? > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116142#comment-13116142 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Mapreduce-trunk-Commit #991 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/991/]) MAPREDUCE-3021. Adding additional files needed for new web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176708 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node/.keep > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[
https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116138#comment-13116138
]
Mahadev konar commented on MAPREDUCE-3054:
--
Thanks Vinod, that would have been a circular dependency :).
> Unable to kill submitted jobs
> -
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Siddharth Seth
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly
> killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3110) TestRPC.testUnknownCall() is failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3110:
---
Component/s: test
Summary: TestRPC.testUnknownCall() is failing (was:
Hadoop-Mapreduce-0.23-Build - Build # 29 - Still Failing)
> TestRPC.testUnknownCall() is failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAPREDUCE-3110) Hadoop-Mapreduce-0.23-Build - Build # 29 - Still Failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli reopened MAPREDUCE-3110:
Assignee: Vinod Kumar Vavilapalli (was: Devaraj K)
It is failing on Jenkins now, as was observed with the patch at MAPREDUCE-3054.
The exception trace seems to be dependent on the jvm used, I'll fix this to be
a pattern.
> Hadoop-Mapreduce-0.23-Build - Build # 29 - Still Failing
>
>
> Key: MAPREDUCE-3110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> {code:xml}
> Failed tests:
> testUnknownCall(org.apache.hadoop.yarn.TestRPC): null
> expected:<...icationId called on []org.apache.hadoop.ya...> but
> was:<...icationId called on [interface ]org.apache.hadoop.ya...>
> Tests run: 65, Failures: 1, Errors: 0, Skipped: 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[
https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116136#comment-13116136
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
I meant via MAPREDUCE-3110.
> Unable to kill submitted jobs
> -
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Siddharth Seth
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly
> killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116133#comment-13116133 ] Hudson commented on MAPREDUCE-3041: --- Integrated in Hadoop-Hdfs-trunk-Commit #1049 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1049/]) MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource capabilities along-with new ApplicationId for application submission. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176715 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3041: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Hitesh! > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116131#comment-13116131 ] Hudson commented on MAPREDUCE-3041: --- Integrated in Hadoop-Common-trunk-Commit #971 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/971/]) MAPREDUCE-3041. Fixed ClientRMProtocol to provide min/max resource capabilities along-with new ApplicationId for application submission. Contributed by Hitesh Shah. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176715 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationIdResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNewApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationIdResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNewApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This messag
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[
https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116130#comment-13116130
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
bq. -1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
The patch broke TestRPC on Jenkins. The exception trace seems to be dependant
on the jvm used. I'll fix it via MAPREDUCE-3053.
> Unable to kill submitted jobs
> -
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Siddharth Seth
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly
> killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116126#comment-13116126 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3098: bq. True however what this actually represents is the state of the current attempt's AM. That's not true and which I am trying to impress upon. ApplicationState.New and ApplicationState.SUBMITTED don't even represent any state of the AM. ApplicationState.RUNNING means multiple things: it may be that an AM is running, or it may also mean that AM is down and in the process of restarting. ApplicationState.FAILED may actually mean that the platform is not even able to launch the AM for some reason. To be semantically correct, RMAppAttemptState represents the AM state, ApplicationState is much more than RMAppAttemptState, it subsumes the whole application including any and all app-attempts. bq. A submitted/running/completed AM only really implies the state of the AM and it would be better if we changed this from ApplicationState to ApplicationMasterState to avoid confusion as to what it really implies. Ditto. Like I pointed above, ApplicationState.Running doesn't map directly to a running AM, neither does ApplicationState.FAILED represent actual job-failure. bq. From the perspective of a client, a client would prefer to assume that the application 'finish' state is whether my "job" completed successfully or not. Not sure if you worked with other resource-managers which give you an ability to run your own code. In torque, for example, the corresponding state for ApplicationState.FINISHED only means the app has finished. To know what happened to your actual job, you need a separate state. bq. The RM does not need to act on it today so a string should work. However, the question is whether using a flexible string/blob based approach is the way to go? I, in particular, do not have any preference over string/enum. I prefer the blob approach as having an enum is stricter and obviously we cannot have an enum representing final-states for all possible frameworks. bq. A client would monitoring AppMasterState for completion ( succeeded/failed/killed ) and if succeeded, look at the ApplicationFinishState to actually verify that the work was done successfully. Given above, a client would be instead monitoring for ApplicationState, if it is one of the final states SUCCEEDED, FAILED or KILLED, it will look at the corresponding ApplicationReport.FinalFrameworkState to see what actually happened to the job. Apologies if I am giving an impression of creating a storm. But honestly all of this was thought through earlier end-to-end and I firmly belive that the current states and records represent the correct abstraction. > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116128#comment-13116128 ] Arun C Murthy commented on MAPREDUCE-3041: -- +1 lgtm. The tests failed because I had missed committing some files during MAPREDUCE-3021. > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116127#comment-13116127 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3098: Oh BTW, I do sense that we should rename ApplicationState.SUCCEEDED to ApplicationState.FINISHED. It sure is one source of confusion. > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116120#comment-13116120 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Hdfs-trunk-Commit #1048 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1048/]) MAPREDUCE-3021. Adding additional files needed for new web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176708 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node/.keep > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116119#comment-13116119 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Common-trunk-Commit #970 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/970/]) MAPREDUCE-3021. Adding additional files needed for new web-ui. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176708 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/cluster/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/jobhistory/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/mapreduce/.keep * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/node/.keep > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116114#comment-13116114 ] Arun C Murthy commented on MAPREDUCE-3021: -- I just added the empty .keep files - this should fix the failing tests. > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3039) Make mapreduce use same version of avro as HBase
[ https://issues.apache.org/jira/browse/MAPREDUCE-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-3039: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I just committed this. Thank you Joep. > Make mapreduce use same version of avro as HBase > > > Key: MAPREDUCE-3039 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3039 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/capacity-sched, contrib/fair-share, > contrib/gridmix, contrib/mrunit, contrib/mumak, contrib/raid, > contrib/streaming, jobhistoryserver >Affects Versions: 0.22.0 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 0.22.0 > > Attachments: MAPREDUCE-3039-branch-0.22-shv.patch, > MAPREDUCE-3039-branch-0.22.patch > > > HBase depends on avro 1.5.3 whereas hadoop-common depends on 1.3.2. > When building HBase on top of hadoop, this should be consistent. > Moreover, this should be consistent between common, hdfs, and mapreduce. > Contribs seem to have declared a dependency on avro but are not in fact > depending on it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3039) Make mapreduce use same version of avro as HBase
[ https://issues.apache.org/jira/browse/MAPREDUCE-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116092#comment-13116092 ] Hadoop QA commented on MAPREDUCE-3039: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496835/MAPREDUCE-3039-branch-0.22-shv.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/877//console This message is automatically generated. > Make mapreduce use same version of avro as HBase > > > Key: MAPREDUCE-3039 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3039 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/capacity-sched, contrib/fair-share, > contrib/gridmix, contrib/mrunit, contrib/mumak, contrib/raid, > contrib/streaming, jobhistoryserver >Affects Versions: 0.22.0 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 0.22.0 > > Attachments: MAPREDUCE-3039-branch-0.22-shv.patch, > MAPREDUCE-3039-branch-0.22.patch > > > HBase depends on avro 1.5.3 whereas hadoop-common depends on 1.3.2. > When building HBase on top of hadoop, this should be consistent. > Moreover, this should be consistent between common, hdfs, and mapreduce. > Contribs seem to have declared a dependency on avro but are not in fact > depending on it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3039) Make mapreduce use same version of avro as HBase
[ https://issues.apache.org/jira/browse/MAPREDUCE-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-3039: --- Attachment: MAPREDUCE-3039-branch-0.22-shv.patch A slight modification to avoid avro dependency on packaged jars in Hadoop. > Make mapreduce use same version of avro as HBase > > > Key: MAPREDUCE-3039 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3039 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/capacity-sched, contrib/fair-share, > contrib/gridmix, contrib/mrunit, contrib/mumak, contrib/raid, > contrib/streaming, jobhistoryserver >Affects Versions: 0.22.0 >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 0.22.0 > > Attachments: MAPREDUCE-3039-branch-0.22-shv.patch, > MAPREDUCE-3039-branch-0.22.patch > > > HBase depends on avro 1.5.3 whereas hadoop-common depends on 1.3.2. > When building HBase on top of hadoop, this should be consistent. > Moreover, this should be consistent between common, hdfs, and mapreduce. > Contribs seem to have declared a dependency on avro but are not in fact > depending on it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116085#comment-13116085 ] Hadoop QA commented on MAPREDUCE-3041: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496832/MR-3041.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/876//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/876//console This message is automatically generated. > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3041: --- Status: Patch Available (was: Open) > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3041: --- Attachment: MR-3041.3.patch Rebased patch. > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch, MR-3041.3.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-1207) Allow admins to set java options for map/reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-1207:
-
Fix Version/s: 0.23.0
> Allow admins to set java options for map/reduce tasks
> -
>
> Key: MAPREDUCE-1207
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1207
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client, mrv2
>Reporter: Arun C Murthy
>Assignee: Arun C Murthy
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: mapred-1207.patch
>
>
> It will be useful for allow cluster-admins to set some java options for child
> map/reduce tasks.
> E.g. We've had to ask users to set -Djava.net.preferIPv4Stack=true in their
> jobs, it would be nice to do it for all users in such scenarios even when
> people override mapred.child.{map|reduce}.java.opts but forget to add this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116070#comment-13116070 ] Thomas Graves commented on MAPREDUCE-3021: -- thanks Arun! > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116068#comment-13116068 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Mapreduce-trunk-Commit #989 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/989/]) MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176681 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/NavBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/WebServer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116066#comment-13116066 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Common-trunk-Commit #968 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/968/]) MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176681 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/NavBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/WebServer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116064#comment-13116064 ] Hudson commented on MAPREDUCE-3021: --- Integrated in Hadoop-Hdfs-trunk-Commit #1046 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1046/]) MAPREDUCE-3021. Change base urls for RM web-ui. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176681 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/NavBlock.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/WebServer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116065#comment-13116065 ] Arun C Murthy commented on MAPREDUCE-2791: -- Oh, you need to rebase too... > [MR-279] Missing/incorrect info on job -status CLI > --- > > Key: MAPREDUCE-2791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2791-1.patch, MAPREDUCE-2791-1.patch, > MAPREDUCE-2791-2.patch, MAPREDUCE-2791-3.patch, MAPREDUCE-2791.patch > > > There are a couple of details missing/incorrect on the job -status command > line output for completed jobs: > 1. Incorrect job file > 2. map() completion is always 0 > 3. reduce() completion is always set to 0 > 4. history URL is empty > 5. Missing launched map tasks > 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2791: - Fix Version/s: (was: 0.24.0) Status: Open (was: Patch Available) Devaraj, the patch looks good. Thanks! Can you please check why the patch can't be applied by hudson? Maybe it's got to do with ^M? If not, can you please run: # mvn clean install to run all tests (incl. integration tests) and paste output? # AlPlease run old tests i.e ant test to make sure. Also, can you pls paste the latest output? Thanks! Meanwhile I'll run this on a secure cluster to test. > [MR-279] Missing/incorrect info on job -status CLI > --- > > Key: MAPREDUCE-2791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2791-1.patch, MAPREDUCE-2791-1.patch, > MAPREDUCE-2791-2.patch, MAPREDUCE-2791-3.patch, MAPREDUCE-2791.patch > > > There are a couple of details missing/incorrect on the job -status command > line output for completed jobs: > 1. Incorrect job file > 2. map() completion is always 0 > 3. reduce() completion is always set to 0 > 4. history URL is empty > 5. Missing launched map tasks > 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3021: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Thomas! > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3041: - Status: Open (was: Patch Available) Hitesh, can you pls rebase? Can't apply the patch. Also, could you please run 'mvn install' to make sure the integration tests pass? Thanks. > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2892) Improvements to AM apis
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned MAPREDUCE-2892: -- Assignee: Hitesh Shah (was: Anupam Seth) > Improvements to AM apis > --- > > Key: MAPREDUCE-2892 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Some api changes before we declare yarn apis as stable: > # FinishApplicationMasterRequest doesn't need to send out 'tracking url'. > # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's > got to be an enum and we need to use that enum appropriately in the RM's > state-machines. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2986) Multiple node managers support for the MiniYARNCluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anupam Seth updated MAPREDUCE-2986: --- Attachment: MAPREDUCE-2986-branch-0_23.patch > Multiple node managers support for the MiniYARNCluster > -- > > Key: MAPREDUCE-2986 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2986 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Anupam Seth >Assignee: Anupam Seth >Priority: Critical > Attachments: BZ-4805926-branch-0_23.patch, > MAPREDUCE-2986-branch-0_23.patch, MAPREDUCE-2986-branch-0_23.patch, > MAPREDUCE-2986-branch-0_23.patch > > > The current MiniYARNCluster can only support 1 node manager, which is not > enough for the full test purposes. > Would like to have a simulator that can support multiple node managers as the > real scenario. This might be beneficial for hadoop users, testers and > developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115937#comment-13115937 ] Hudson commented on MAPREDUCE-3099: --- Integrated in Hadoop-Mapreduce-trunk-Commit #987 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/987/]) MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176607 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/SingleCluster.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/index.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/site.xml > Add docs for setting up a single node MRv2 cluster. > --- > > Key: MAPREDUCE-3099 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3099 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Assignee: Mahadev konar > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3099.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
[ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115936#comment-13115936 ] Hitesh Shah commented on MAPREDUCE-3098: bq. The ApplicationState returned to the client via ApplicationReport is really the state of the application as known to RM. True however what this actually represents is the state of the current attempt's AM. A submitted/running/completed AM only really implies the state of the AM and it would be better if we changed this from ApplicationState to ApplicationMasterState to avoid confusion as to what it really implies. From the perspective of a client, a client would prefer to assume that the application 'finish' state is whether my "job" completed successfully or not. Regd. the JobState, I agree. It is app-specific and has no relation to the state of the application but it should be exposed to the client as part of the report. The RM does not need to act on it today so a string should work. However, the question is whether using a flexible string/blob based approach is the way to go? I, in particular, do not have any preference over string/enum. Having an enum to define a set of end states which most applications will end up in might be a more stricter but clear approach. The diagnostics string could be used to provide the additional app specific info if needed. So my proposed changes are: Change ApplicationState to ApplicationMasterState to improve clarity on what the state implies. Add ApplicationFinishState to ApplicationReport as an enum. A client would monitoring AppMasterState for completion ( succeeded/failed/killed ) and if succeeded, look at the ApplicationFinishState to actually verify that the work was done successfully. Comments? > Report Application status as well as ApplicationMaster status in > GetApplicationReportResponse > -- > > Key: MAPREDUCE-3098 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > > Currently, an application report received by the client from the RM/ASM for a > given application returns the status of the application master. It does not > return the status of the application i.e. whether that particular job > succeeded or failed. > The AM status would be one of FINISHED (SUCCEEDED should be renamed to > FINISHED as AM state does not indicate overall success/failure), FAILED or > KILLED. > The final state sent by the AM to the RM in the > FinishApplicationMasterRequest should be exposed to the client as > ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115915#comment-13115915 ] Hudson commented on MAPREDUCE-3099: --- Integrated in Hadoop-Hdfs-trunk-Commit #1044 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1044/]) MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176607 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/SingleCluster.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/index.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/site.xml > Add docs for setting up a single node MRv2 cluster. > --- > > Key: MAPREDUCE-3099 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3099 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Assignee: Mahadev konar > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3099.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115912#comment-13115912 ] Hudson commented on MAPREDUCE-3054: --- Integrated in Hadoop-Mapreduce-trunk-Commit #986 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/986/]) MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Unable to kill submitted jobs > - > > Key: MAPREDUCE-3054 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.
[jira] [Commented] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115914#comment-13115914 ] Hudson commented on MAPREDUCE-3099: --- Integrated in Hadoop-Common-trunk-Commit #966 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/966/]) MAPREDUCE-3099. Add docs for setting up a single node MRv2 cluster. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176607 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/SingleCluster.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/apt/index.apt.vm * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/src/site/site.xml > Add docs for setting up a single node MRv2 cluster. > --- > > Key: MAPREDUCE-3099 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3099 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Assignee: Mahadev konar > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3099.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3099: - Resolution: Fixed Status: Resolved (was: Patch Available) Just committed this to trunk and 0.23. > Add docs for setting up a single node MRv2 cluster. > --- > > Key: MAPREDUCE-3099 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3099 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Assignee: Mahadev konar > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3099.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
[
https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3054:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Just committed this. Thanks Arun and Vinod for a detailed review.
> Unable to kill submitted jobs
> -
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Siddharth Seth
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch,
> MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly
> killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> job_1316203984216_0002 PREP1316204924937 hadoopqadefault NORMAL
> job_1316203984216_0001 PREP1316204031206 hadoopqadefault NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115892#comment-13115892 ] Hudson commented on MAPREDUCE-3054: --- Integrated in Hadoop-Hdfs-trunk-Commit #1043 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1043/]) MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Unable to kill submitted jobs > - > > Key: MAPREDUCE-3054 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115891#comment-13115891 ] Hudson commented on MAPREDUCE-3054: --- Integrated in Hadoop-Common-trunk-Commit #965 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/965/]) MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Unable to kill submitted jobs > - > > Key: MAPREDUCE-3054 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >
[jira] [Commented] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115882#comment-13115882 ] Hudson commented on MAPREDUCE-3095: --- Integrated in Hadoop-Mapreduce-trunk-Commit #985 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/985/]) MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs. (John George via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176566 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/fairscheduler/ivy.xml > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2747) [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources
[ https://issues.apache.org/jira/browse/MAPREDUCE-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans reassigned MAPREDUCE-2747: -- Assignee: Robert Joseph Evans > [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources > - > > Key: MAPREDUCE-2747 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2747 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, nodemanager, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0 > > > There are a lot of references to the old task-controller nomenclature still, > job/task refs instead of app/container. > Also the configuration file is named as taskcontroller.cfg and the configured > variables are also from the mapred world (mrv1). These SHOULD be fixed > before we make a release. Marking this as blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3081) Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh
[
https://issues.apache.org/jira/browse/MAPREDUCE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115866#comment-13115866
]
Hudson commented on MAPREDUCE-3081:
---
Integrated in Hadoop-Mapreduce-trunk-Commit #984 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/984/])
MAPREDUCE-3081. Fix vaidya startup script. Contributed by Suhas
gkesavan :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176550
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/vaidya/src/java/org/apache/hadoop/vaidya/vaidya.sh
> Change the name format for hadoop core and vaidya jar to be
> hadoop-{core/vaidya}-{version}.jar in vaidya.sh
> ---
>
> Key: MAPREDUCE-3081
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3081
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/vaidya
>Affects Versions: 0.20.1, 0.20.205.0, 0.20.206.0
>Reporter: vitthal (Suhas) Gogate
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-3081-trunk.patch, MAPREDUCE-3081-2.patch
>
>
> Vaidya script is broken due to change in the naming convention for hadoop
> core jar and vaidya jar.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115858#comment-13115858 ] Hudson commented on MAPREDUCE-3095: --- Integrated in Hadoop-Hdfs-trunk-Commit #1042 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1042/]) MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs. (John George via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176566 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/fairscheduler/ivy.xml > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115856#comment-13115856 ] Hudson commented on MAPREDUCE-3095: --- Integrated in Hadoop-Common-trunk-Commit #964 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/964/]) MAPREDUCE-3095. fairscheduler ivy including wrong version for hdfs. (John George via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176566 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/fairscheduler/ivy.xml > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3095: - Resolution: Fixed Target Version/s: 0.23.0 (was: 0.24.0, 0.23.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Just committed this. Thanks John! > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115848#comment-13115848 ] John George commented on MAPREDUCE-3095: Thanks Mahadev. > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3081) Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh
[
https://issues.apache.org/jira/browse/MAPREDUCE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated MAPREDUCE-3081:
--
Resolution: Fixed
Fix Version/s: 0.23.0
0.20.205.0
Status: Resolved (was: Patch Available)
committed to trunk and 23 branch
> Change the name format for hadoop core and vaidya jar to be
> hadoop-{core/vaidya}-{version}.jar in vaidya.sh
> ---
>
> Key: MAPREDUCE-3081
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3081
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/vaidya
>Affects Versions: 0.20.1, 0.20.205.0, 0.20.206.0
>Reporter: vitthal (Suhas) Gogate
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-3081-trunk.patch, MAPREDUCE-3081-2.patch
>
>
> Vaidya script is broken due to change in the naming convention for hadoop
> core jar and vaidya jar.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3081) Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh
[
https://issues.apache.org/jira/browse/MAPREDUCE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115844#comment-13115844
]
Hudson commented on MAPREDUCE-3081:
---
Integrated in Hadoop-Hdfs-trunk-Commit #1041 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1041/])
MAPREDUCE-3081. Fix vaidya startup script. Contributed by Suhas
gkesavan :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176550
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/vaidya/src/java/org/apache/hadoop/vaidya/vaidya.sh
> Change the name format for hadoop core and vaidya jar to be
> hadoop-{core/vaidya}-{version}.jar in vaidya.sh
> ---
>
> Key: MAPREDUCE-3081
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3081
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/vaidya
>Affects Versions: 0.20.1, 0.20.205.0, 0.20.206.0
>Reporter: vitthal (Suhas) Gogate
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-3081-trunk.patch, MAPREDUCE-3081-2.patch
>
>
> Vaidya script is broken due to change in the naming convention for hadoop
> core jar and vaidya jar.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3095) fairscheduler ivy including wrong version for hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3095: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) > fairscheduler ivy including wrong version for hdfs > -- > > Key: MAPREDUCE-3095 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3095 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: John George >Assignee: John George > Fix For: 0.23.0 > > Attachments: mapreduce-3095.patch, mapreduce-branch-0.23.patch > > > fairscheduler ivy.xml includes the common version for hdfs dependency. This > could break builds that have different common and hdfs version numbers. The > reason we dont see it on the jenkins build is because we use the same version > number for common and hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3081) Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh
[
https://issues.apache.org/jira/browse/MAPREDUCE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115843#comment-13115843
]
Hudson commented on MAPREDUCE-3081:
---
Integrated in Hadoop-Common-trunk-Commit #963 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/963/])
MAPREDUCE-3081. Fix vaidya startup script. Contributed by Suhas
gkesavan :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176550
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/vaidya/src/java/org/apache/hadoop/vaidya/vaidya.sh
> Change the name format for hadoop core and vaidya jar to be
> hadoop-{core/vaidya}-{version}.jar in vaidya.sh
> ---
>
> Key: MAPREDUCE-3081
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3081
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/vaidya
>Affects Versions: 0.20.1, 0.20.205.0, 0.20.206.0
>Reporter: vitthal (Suhas) Gogate
> Attachments: HADOOP-3081-trunk.patch, MAPREDUCE-3081-2.patch
>
>
> Vaidya script is broken due to change in the naming convention for hadoop
> core jar and vaidya jar.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115823#comment-13115823 ] Hadoop QA commented on MAPREDUCE-3041: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496783/MR-3041.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/874//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/874//console This message is automatically generated. > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2980) Fetch failures and other related issues in Jetty 6.1.26
[ https://issues.apache.org/jira/browse/MAPREDUCE-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115814#comment-13115814 ] Todd Lipcon commented on MAPREDUCE-2980: Shall I ping the dev list to see what the opinions are? I agree that (a) it's unfortunate to ship a patched Jetty, and (b) that the current fetch failure rate is unacceptable in 20x. Unfortunately to fix (b) we need to do (a)... as of yet, no word on Jetty 6.1.27 release timeline either. It seems to be low on their priority list (sort of like if someone asked us to release Hadoop 0.18.4 or something!) > Fetch failures and other related issues in Jetty 6.1.26 > --- > > Key: MAPREDUCE-2980 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2980 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.205.0, 0.23.0 >Reporter: Todd Lipcon >Priority: Critical > > Since upgrading Jetty from 6.1.14 to 6.1.26 we've had a ton of HTTP-related > issues, including: > - Much higher incidence of fetch failures > - A few strange file-descriptor related bugs (eg MAPREDUCE-2389) > - A few unexplained issues where long "fsck"s on the NameNode drop out > halfway through with a ClosedChannelException > Stress tests with 1Map x 1Reduce sleep jobs reliably reproduce fetch > failures at a rate of about 1 per million on a 25 node test cluster. These > problems are all new since the upgrade from 6.1.14. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3041: --- Status: Patch Available (was: Open) > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol
[ https://issues.apache.org/jira/browse/MAPREDUCE-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3041: --- Attachment: MR-3041.2.patch Addressed code review comments. s/ClientRMProtocol#getNewApplicationId/ClientRMProtocol#getNewApplication/ s/GetNewApplicationId[Request|Response]/GetNewApplication[Request|Response]/ > Enhance YARN Client-RM protocol to provide access to information such as > cluster's Min/Max Resource capabilities similar to that of AM-RM protocol > -- > > Key: MAPREDUCE-3041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Hitesh Shah >Assignee: Hitesh Shah >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MR-3041.1.patch, MR-3041.2.patch > > > To request a container to launch an application master, the client needs to > know the min/max resource capabilities so as to be able to make a proper > resource request when submitting a new application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAPREDUCE-3064) 27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar reopened MAPREDUCE-3064: -- Reopening the issue, we are still having issues with the build: https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/983/ Venu, can you please take a look? > 27 unit test failures with Invalid "mapreduce.jobtracker.address" > configuration value for JobTracker: "local" > -- > > Key: MAPREDUCE-3064 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3064 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Venu Gopala Rao >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3064.1.patch, MAPREDUCE-3064.patch > > > unit test failure here: > https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/946/ > Test Result (27 failures / +27) > org.apache.hadoop.mapred.TestCollect.testCollect > org.apache.hadoop.mapred.TestComparators.testDefaultMRComparator > org.apache.hadoop.mapred.TestComparators.testUserMRComparator > org.apache.hadoop.mapred.TestComparators.testUserValueGroupingComparator > org.apache.hadoop.mapred.TestComparators.testAllUserComparators > org.apache.hadoop.mapred.TestFileOutputFormat.testCustomFile > org.apache.hadoop.mapred.TestJavaSerialization.testMapReduceJob > org.apache.hadoop.mapred.TestJavaSerialization.testWriteToSequencefile > org.apache.hadoop.mapred.TestMapOutputType.testKeyMismatch > org.apache.hadoop.mapred.TestMapOutputType.testValueMismatch > org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch > org.apache.hadoop.mapred.TestMapRed.testMapred > org.apache.hadoop.mapred.TestMapRed.testNullKeys > org.apache.hadoop.mapred.TestMapRed.testCompression > org.apache.hadoop.mapred.TestMapRed.testSmallInput > org.apache.hadoop.mapred.TestMapRed.testBiggerInput > org.apache.hadoop.mapreduce.TestMapCollection.testValLastByte > org.apache.hadoop.mapreduce.TestMapCollection.testLargeRecords > org.apache.hadoop.mapreduce.TestMapCollection.testSpillPer2B > org.apache.hadoop.mapreduce.TestMapCollection.testZeroVal > org.apache.hadoop.mapreduce.TestMapCollection.testSingleRecord > org.apache.hadoop.mapreduce.TestMapCollection.testLowSpill > org.apache.hadoop.mapreduce.TestMapCollection.testSplitMetaSpill > org.apache.hadoop.mapreduce.TestMapCollection.testPostSpillMeta > org.apache.hadoop.mapreduce.TestMapCollection.testLargeRecConcurrent > org.apache.hadoop.mapreduce.TestMapCollection.testRandom > org.apache.hadoop.mapreduce.TestMapCollection.testRandomCompress > All of them have similar stack traces of: > java.io.IOException: Invalid "mapreduce.jobtracker.address" configuration > value for JobTracker: "local" > at > org.apache.hadoop.mapred.JobTrackerClientProtocolProvider.create(JobTrackerClientProtocolProvider.java:47) > at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:74) > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:459) > at org.apache.hadoop.mapred.JobClient.(JobClient.java:438) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:777) > at > org.apache.hadoop.mapred.TestCollect.testCollect(TestCollect.java:133) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3021) all yarn webapps use same base name of "yarn/"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115757#comment-13115757 ] Thomas Graves commented on MAPREDUCE-3021: -- Manual testing done since no unit test framework for the links. Tested all the yarn webapp: resource manager, node manager, job history, and app master ui. For each verified that all the links in the tables and in the menus properly worked and went to the new uri which would be: resource maanger: host:port/cluster node manager: host:port/node job history: host:port/jobhistory app master: host:port/mapreduce Note that with MAPREDUCE-2999 host:port/ gets redirected now to host:port/. I tested with auth filtering on (hadoop.http.filter.initializers) with an auth filter that allowed you to bypass authentication on pages as configured. Tested this with the new naming conventions and verified that the pages could be bypassed with the new names without collision between components. resource manager has links: - nodes page has links to node manager. - links to containers logs on node manager for the application - link to job history for finished apps - link to app master for running apps app master page has menu in upper left corner back to the resource manager. app master has link to container logs on node manager Verify all the links within a component still work. > all yarn webapps use same base name of "yarn/" > -- > > Key: MAPREDUCE-3021 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3021 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-3021.patch, MAPREDUCE-3021.patch > > > All of the yarn webapps (resource manager, node manager, app master, job > history) use the same base url of /yarn/. This doesn't lend itself very well > to filters be able to differentiate them to say allow some to be not > authenticated and other to be authenticated. Perhaps we should rename them > based on component. > There are also things in the code that hardcode paths to "/yarn" that should > be fixed up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-376) Add serialization for Thrift
[ https://issues.apache.org/jira/browse/MAPREDUCE-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Bloniarz updated MAPREDUCE-376: - Attachment: ThriftSerialization.java As the previous patch has a bug and is based off an old version of thrift, I'm attaching a Serialization implementation which works better. This contains only a standalone org.apache.hadoop.io.serializer.Serialization implementation (the original patch has testcases etc). Tested against Thrift 0.7 and Hadoop 0.20.1. > Add serialization for Thrift > > > Key: MAPREDUCE-376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-376 > Project: Hadoop Map/Reduce > Issue Type: Wish >Reporter: Tom White > Attachments: ThriftSerialization.java, hadoop-3787.patch, > libthrift.jar > > > Thrift (http://incubator.apache.org/thrift/) is cross-language serialization > and RPC framework. This issue is to write a ThriftSerialization to support > using Thrift types in MapReduce programs, including an example program. This > should probably go into contrib. > (There is a prototype implementation in > https://issues.apache.org/jira/secure/attachment/12370464/hadoop-serializer-v2.tar.gz) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115735#comment-13115735 ] Hadoop QA commented on MAPREDUCE-2791: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496768/MAPREDUCE-2791-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/873//console This message is automatically generated. > [MR-279] Missing/incorrect info on job -status CLI > --- > > Key: MAPREDUCE-2791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2791-1.patch, MAPREDUCE-2791-1.patch, > MAPREDUCE-2791-2.patch, MAPREDUCE-2791-3.patch, MAPREDUCE-2791.patch > > > There are a couple of details missing/incorrect on the job -status command > line output for completed jobs: > 1. Incorrect job file > 2. map() completion is always 0 > 3. reduce() completion is always set to 0 > 4. history URL is empty > 5. Missing launched map tasks > 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115724#comment-13115724 ] Hudson commented on MAPREDUCE-2999: --- Integrated in Hadoop-Mapreduce-trunk-Commit #982 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/982/]) MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet paths. Contributed by Thomas Graves. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176469 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/Dispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/TestWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-2791: - Status: Patch Available (was: Open) I updated the patch as per the above suggestion. > [MR-279] Missing/incorrect info on job -status CLI > --- > > Key: MAPREDUCE-2791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2791-1.patch, MAPREDUCE-2791-1.patch, > MAPREDUCE-2791-2.patch, MAPREDUCE-2791-3.patch, MAPREDUCE-2791.patch > > > There are a couple of details missing/incorrect on the job -status command > line output for completed jobs: > 1. Incorrect job file > 2. map() completion is always 0 > 3. reduce() completion is always set to 0 > 4. history URL is empty > 5. Missing launched map tasks > 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-2791: - Attachment: MAPREDUCE-2791-3.patch > [MR-279] Missing/incorrect info on job -status CLI > --- > > Key: MAPREDUCE-2791 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Devaraj K >Priority: Blocker > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2791-1.patch, MAPREDUCE-2791-1.patch, > MAPREDUCE-2791-2.patch, MAPREDUCE-2791-3.patch, MAPREDUCE-2791.patch > > > There are a couple of details missing/incorrect on the job -status command > line output for completed jobs: > 1. Incorrect job file > 2. map() completion is always 0 > 3. reduce() completion is always set to 0 > 4. history URL is empty > 5. Missing launched map tasks > 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115716#comment-13115716 ] Hudson commented on MAPREDUCE-2999: --- Integrated in Hadoop-Common-trunk-Commit #962 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/962/]) MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet paths. Contributed by Thomas Graves. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176469 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/Dispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/TestWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115715#comment-13115715 ] Hudson commented on MAPREDUCE-2999: --- Integrated in Hadoop-Hdfs-trunk-Commit #1040 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1040/]) MAPREDUCE-2999. Fix YARN webapp framework to properly filter servlet paths. Contributed by Thomas Graves. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176469 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/Dispatcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/WebApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/TestWebApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2999: --- Resolution: Fixed Status: Resolved (was: Patch Available) Great coverage of the manual tests, good job! I just committed this to trunk and branch-0.23. Thanks Thomas! > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115702#comment-13115702 ] Hadoop QA commented on MAPREDUCE-2999: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496763/MAPREDUCE-2999.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/872//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/872//console This message is automatically generated. > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3001) Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115701#comment-13115701 ] Hadoop QA commented on MAPREDUCE-3001: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496717/MR-3001.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/871//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/871//console This message is automatically generated. > Map Reduce JobHistory and AppMaster UI should have ability to display task > specific counters. > - > > Key: MAPREDUCE-3001 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3001 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver, mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans >Priority: Blocker > Fix For: 0.23.0, 0.24.0 > > Attachments: MR-3001.txt > > > Map Reduce JobHistory and AppMaster UI should have ability to display task > specific counters. I think the best way to do this is to include in the Nav > Block a task specific section with task links when a task is selected. > Counters is already set up to deal with a task passed in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115695#comment-13115695 ] Thomas Graves commented on MAPREDUCE-2999: -- Manual testing done: Test with and without an auth filter (config settinghadoop.http.filter.initializers). When filter setting is set verified that the load of each page properly went through the filter specified. When going through the filter the page properly showed the current User in the upper right hand corner. Without the filter it shows the default hadoop user of "dr.who". Modified my auth filter to be able to choose which pages to actually authenticate and which not to. When it didn't authenticate pages verified that anyone could go there and the upper right corner was empty (didn't show the "Logged in as:"). Tested that all the yarn web pages and links from yarn web pages to other pages worked. For instance RM page has link to node manager, links from app to job history and to app master. Verified the links from app master back to RM work, etc... and that when the filter is there they properly go through the filter and ask for authentication. Verify that each yarn web app allows the user to get there from: - host:port/ -> This redirects you to host:port/yarn so that the filter can differentiate and chose to authenticate or not - host:port/yarn -> goes to default page specified by webapp - host:port/yarn/* -> goes to page specified by webapp Verify that all of the default applications registered in HttpServer still work. (note that HADOOP-7662 is needed for the /logs to work). - /logs - /static - /stacks - /logLevel - /metrics - /jmx - /conf > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3092) Remove JOB_ID_COMPARATOR usage in JobHistory.java
[ https://issues.apache.org/jira/browse/MAPREDUCE-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115691#comment-13115691 ] Hudson commented on MAPREDUCE-3092: --- Integrated in Hadoop-Mapreduce-trunk-Commit #981 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/981/]) MAPREDUCE-3092. Removed a special comparator for JobIDs in JobHistory as JobIDs are already comparable. Contributed by Devaraj K. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176453 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java > Remove JOB_ID_COMPARATOR usage in JobHistory.java > - > > Key: MAPREDUCE-3092 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3092 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.24.0 >Reporter: Devaraj K >Assignee: Devaraj K >Priority: Minor > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-3092.patch > > > As part of the defect MAPREDUCE-2965, JobId.compareTo() has been implemented. > Usage of JOB_ID_COMPARATOR in JobHistory.java can be removed because > comparison is handling by JobId itself. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2999) hadoop.http.filter.initializers not working properly on yarn UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-2999: - Attachment: MAPREDUCE-2999.patch reattaching same patch to kick jenkins. > hadoop.http.filter.initializers not working properly on yarn UI > --- > > Key: MAPREDUCE-2999 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2999 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, MAPREDUCE-2999.patch, > MAPREDUCE-2999.patch > > > Currently httpserver only has *.html", "*.jsp as user facing urls when you > add a filter. For the new web framework in yarn, the pages no longer have the > *.html or *.jsp and thus they are not properly being filtered. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115686#comment-13115686 ] Abhijit Suresh Shingate commented on MAPREDUCE-2843: Hi lu, Thanks for the review & feedback!! Regards, Abhijit > [MR-279] Node entries on the RM UI are not sortable > --- > > Key: MAPREDUCE-2843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 0.24.0 >Reporter: Ramya Sunil >Assignee: Abhijit Suresh Shingate > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2843- scr shot.jpg, MAPREDUCE-2843.1.patch, > MAPREDUCE-2843.2.patch, MAPREDUCE-2843.3.patch, MAPREDUCE-2843.patch, > MAPREDUCE-2843.patch > > > The nodemanager entries on the RM UI is not sortable unlike the other web > pages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
