[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149751#comment-13149751 ]
jirapos...@reviews.apache.org commented on MAPREDUCE-3309: ---------------------------------------------------------- ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2822/ ----------------------------------------------------------- Review request for Tom Graves, Robert Evans and Mark Holderbaugh. Summary ------- Make provision to report the AM hostname of an application in the RM/JHS UI. It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder. ------------------ The big idea is to add the application master to the jobhistory ui so that can be sort and increase search capabilities. Currently, adding anything to the jobhistory apps page means to add to the job history file name since that is parsed to get job meta data. This is done for performance reasons to prevent reading all jobhistory files. This addresses bug MAPREDUCE-3309. http://issues.apache.org/jira/browse/MAPREDUCE-3309 Diffs ----- hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java c4ef938 hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java a27ba15 hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/DefaultSchedulerPage.java 8db4caf hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java 700fdb3 hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java 7c175d1 hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java 1e9215f hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsList.java 57e695c hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java ca4ab18 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java d412a63 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java 0a6b969 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java ddb904d hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobIndexInfo.java 3c0f3bb hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java 3a35045 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java 0d0271c hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java ae886cf hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/Job.java 4abcd34 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java dd19ed0 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java 5a67576 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java f82c1d5 Diff: https://reviews.apache.org/r/2822/diff Testing ------- Manual test done: 1. Verify correct application master on job history apps page with jobhistory files using new format 2. Verify old job history files are still usable to retrieve meta data minus the application master 3. Verify all fields are searchable using field search as well and general search Thanks, Jonathan > Report the AM of an application in the UI > ----------------------------------------- > > Key: MAPREDUCE-3309 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 > Affects Versions: 0.23.0 > Reporter: Ramya Sunil > Assignee: Jonathan Eagles > Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, > MAPREDUCE-3309.patch, MAPREDUCE-3309.patch > > > Make provision to report the AM hostname of an application in the RM/JHS UI. > It is difficult to trace back the AM on which an app ran when there are 100+ > jobs in history. Digging through the logs is an option but since there is no > consistency maintained in naming of apps in UI/logs/local dirs > (MAPREDUCE-2793), debugging is all the more harder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira