[
https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149751#comment-13149751
]
[email protected] commented on MAPREDUCE-3309:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2822/
-----------------------------------------------------------
Review request for Tom Graves, Robert Evans and Mark Holderbaugh.
Summary
-------
Make provision to report the AM hostname of an application in the RM/JHS UI.
It is difficult to trace back the AM on which an app ran when there are 100+
jobs in history. Digging through the logs is an option but since there is no
consistency maintained in naming of apps in UI/logs/local dirs
(MAPREDUCE-2793), debugging is all the more harder.
------------------
The big idea is to add the application master to the jobhistory ui so that can
be sort and increase search capabilities. Currently, adding anything to the
jobhistory apps page means to add to the job history file name since that is
parsed to get job meta data. This is done for performance reasons to prevent
reading all jobhistory files.
This addresses bug MAPREDUCE-3309.
http://issues.apache.org/jira/browse/MAPREDUCE-3309
Diffs
-----
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
c4ef938
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
a27ba15
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/DefaultSchedulerPage.java
8db4caf
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java
700fdb3
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java
7c175d1
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
1e9215f
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsList.java
57e695c
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
ca4ab18
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
d412a63
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java
0a6b969
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
ddb904d
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobIndexInfo.java
3c0f3bb
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
3a35045
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
0d0271c
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
ae886cf
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/Job.java
4abcd34
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
dd19ed0
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
5a67576
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
f82c1d5
Diff: https://reviews.apache.org/r/2822/diff
Testing
-------
Manual test done:
1. Verify correct application master on job history apps page with jobhistory
files using new format
2. Verify old job history files are still usable to retrieve meta data minus
the application master
3. Verify all fields are searchable using field search as well and general
search
Thanks,
Jonathan
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch,
> MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+
> jobs in history. Digging through the logs is an option but since there is no
> consistency maintained in naming of apps in UI/logs/local dirs
> (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
