[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Attachment: MAPREDUCE-6376.003.patch - Fix initializing corner cases. Should fix unit tests issues. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch, > MAPREDUCE-6376.003.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581414#comment-14581414 ] Ray Chiang commented on MAPREDUCE-6376: --- Ugh, put up wrong patch. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581415#comment-14581415 ] Ray Chiang commented on MAPREDUCE-6376: --- Ugh, put up wrong patch. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Attachment: (was: MAPREDUCE-6376.003.patch) > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Attachment: MAPREDUCE-6376.003.patch - Fix one area of propagation for skipCounters > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch, > MAPREDUCE-6376.003.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5003) AM recovery should recreate records for attempts that were incomplete
[
https://issues.apache.org/jira/browse/MAPREDUCE-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581380#comment-14581380
]
Hadoop QA commented on MAPREDUCE-5003:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 21s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 43s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 17s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 9s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 33s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 46m 18s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-mapreduce-client-app |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738915/MAPREDUCE-5003.1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5795/artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5795/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5795/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5795/console |
This message was automatically generated.
> AM recovery should recreate records for attempts that were incomplete
> -
>
> Key: MAPREDUCE-5003
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5003
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am
>Reporter: Jason Lowe
>Assignee: Chang Li
> Attachments: MAPREDUCE-5003.1.patch
>
>
> As discussed in MAPREDUCE-4992, it would be nice if the AM recovered task
> attempt entries for *all* task attempts launched by the prior app attempt
> even if those task attempts did not complete. The attempts would have to be
> marked as killed or something similar to indicate it is no longer running.
> Having records for the task attempts enables the user to see what nodes were
> associated with the attempts and potentially access their logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[
https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581373#comment-14581373
]
Hadoop QA commented on MAPREDUCE-6376:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 19m 8s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 54s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 3s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 44s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 0s | Tests failed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | mapreduce tests | 0m 47s | Tests passed in
hadoop-mapreduce-client-common. |
| {color:green}+1{color} | mapreduce tests | 1m 42s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 56m 34s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapreduce.v2.app.TestRecovery |
| | hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler |
| | hadoop.mapreduce.v2.app.TestMRApp |
| | hadoop.mapreduce.v2.app.TestFetchFailure |
| | hadoop.mapreduce.v2.app.TestStagingCleanup |
| | hadoop.mapreduce.v2.app.TestAMInfos |
| | hadoop.mapreduce.v2.app.TestMRAppComponentDependencies |
| | hadoop.mapreduce.jobhistory.TestEvents |
| | hadoop.mapreduce.v2.app.TestMRAppMaster |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738929/MAPREDUCE-6376.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5793/console |
This message was automatically generated.
> Fix long load times of .jhist file in JobHistoryServer
> --
>
> Key: MAPREDUCE-6376
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Labels: supportability
> Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch
>
>
> When you click on a Job link in the JHS Web UI, it loads the .jhist file.
> For jobs which have a large number of tasks, the load time can break UI
> responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[
https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581374#comment-14581374
]
Hadoop QA commented on MAPREDUCE-6394:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 5s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 16 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 49s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 50s | The applied patch generated 1
new checkstyle issues (total was 430, now 430). |
| {color:green}+1{color} | whitespace | 0m 7s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 5s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 10m 4s | Tests failed in
hadoop-mapreduce-client-app. |
| {color:red}-1{color} | mapreduce tests | 5m 50s | Tests failed in
hadoop-mapreduce-client-hs. |
| | | 56m 14s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-mapreduce-client-app |
| Failed unit tests | hadoop.mapreduce.v2.app.webapp.TestAMWebServicesTasks |
| | hadoop.mapreduce.v2.hs.webapp.TestBlocks |
| | hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738931/MAPREDUCE-6394.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5794/console |
This message was automatically generated.
> Speed up Task processing loop in HsTasksBlock#render()
> --
>
> Key: MAPREDUCE-6394
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Labels: supportability
> Attachments: MAPREDUCE-6394.001.patch
>
>
> In HsTasksBlock#render(), there is a loop to create a Javascript table which
> slows down immensely on jobs with a large number of tasks (200k or more).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5003) AM recovery should recreate records for attempts that were incomplete
[ https://issues.apache.org/jira/browse/MAPREDUCE-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-5003: Status: Patch Available (was: Open) > AM recovery should recreate records for attempts that were incomplete > - > > Key: MAPREDUCE-5003 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5003 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mr-am >Reporter: Jason Lowe >Assignee: Chang Li > Attachments: MAPREDUCE-5003.1.patch > > > As discussed in MAPREDUCE-4992, it would be nice if the AM recovered task > attempt entries for *all* task attempts launched by the prior app attempt > even if those task attempts did not complete. The attempts would have to be > marked as killed or something similar to indicate it is no longer running. > Having records for the task attempts enables the user to see what nodes were > associated with the attempts and potentially access their logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581341#comment-14581341 ] Ray Chiang commented on MAPREDUCE-6376: --- RE: findbugs Getting a lot of these empty html messages. Filed HADOOP-12083 for investigation. RE: failed unit tests All failed tests pass in my tree. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[
https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581232#comment-14581232
]
Hadoop QA commented on MAPREDUCE-6394:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 3s | Findbugs (version 3.0.0)
appears to be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 16 new or modified test files. |
| {color:green}+1{color} | javac | 7m 37s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 7s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 59s | The patch appears to introduce 2
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 46s | Tests failed in
hadoop-mapreduce-client-app. |
| {color:red}-1{color} | mapreduce tests | 5m 48s | Tests failed in
hadoop-mapreduce-client-hs. |
| | | 54m 17s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-mapreduce-client-app |
| Failed unit tests | hadoop.mapreduce.v2.app.webapp.TestAMWebServicesTasks |
| | hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks |
| | hadoop.mapreduce.v2.hs.TestJobHistoryParsing |
| | hadoop.mapreduce.v2.hs.webapp.dao.TestJobInfo |
| | hadoop.mapreduce.v2.hs.TestJobHistoryEntities |
| | hadoop.mapreduce.v2.hs.webapp.TestBlocks |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738931/MAPREDUCE-6394.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5792/artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5792/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5792/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5792/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5792/console |
This message was automatically generated.
> Speed up Task processing loop in HsTasksBlock#render()
> --
>
> Key: MAPREDUCE-6394
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Labels: supportability
> Attachments: MAPREDUCE-6394.001.patch
>
>
> In HsTasksBlock#render(), there is a loop to create a Javascript table which
> slows down immensely on jobs with a large number of tasks (200k or more).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[
https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581194#comment-14581194
]
Hadoop QA commented on MAPREDUCE-6376:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 35s | Findbugs (version 3.0.0)
appears to be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 44s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 42s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 48s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 20s | Tests failed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | mapreduce tests | 0m 46s | Tests passed in
hadoop-mapreduce-client-common. |
| {color:green}+1{color} | mapreduce tests | 1m 40s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 55m 0s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-mapreduce-client-app |
| Failed unit tests | hadoop.mapreduce.v2.app.TestRecovery |
| | hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler |
| | hadoop.mapreduce.v2.app.TestMRApp |
| | hadoop.mapreduce.v2.app.TestFetchFailure |
| | hadoop.mapreduce.v2.app.TestStagingCleanup |
| | hadoop.mapreduce.v2.app.TestAMInfos |
| | hadoop.mapreduce.v2.app.TestMRAppComponentDependencies |
| | hadoop.mapreduce.jobhistory.TestEvents |
| | hadoop.mapreduce.v2.app.TestMRAppMaster |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738929/MAPREDUCE-6376.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5791/console |
This message was automatically generated.
> Fix long load times of .jhist file in JobHistoryServer
> --
>
> Key: MAPREDUCE-6376
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Labels: supportability
> Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch
>
>
> When you click on a Job link in the JHS Web UI, it loads the .jhist file.
> For jobs which have a large number of tasks, the load time can break UI
> responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Status: Patch Available (was: Open) Submit for testing. > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6394.001.patch > > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely on jobs with a large number of tasks (200k or more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Attachment: MAPREDUCE-6394.001.patch Initial version to remove copying YARN counters to TaskReports. > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6394.001.patch > > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely on jobs with a large number of tasks (200k or more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581158#comment-14581158 ] Ray Chiang commented on MAPREDUCE-6394: --- In doing some profiling, it looks as though the bulk of the runtime occurs copying the YARN counters into the TaskReport. Since this value isn't used in the web display for the tasks page, turn it off for webapp rendering. For 404k map tasks, this reduces the loop from about 50 seconds down to about 5 seconds. > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6394.001.patch > > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely on jobs with a large number of tasks (200k or more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6222) Add server-side pagination to JobHistoryServer tasks page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6222: -- Labels: BB2015-05-TBR supportability (was: BB2015-05-TBR) > Add server-side pagination to JobHistoryServer tasks page > - > > Key: MAPREDUCE-6222 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6222 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Andrew Johnson >Assignee: Ray Chiang > Labels: BB2015-05-TBR, supportability > Attachments: JHS New Display Top.png, JHS Original Display Top.png, > MAPREDUCE-6222.001.patch, MAPREDUCE-6222.002.patch, MAPREDUCE-6222.003.patch, > MAPREDUCE-6222.005.patch, MAPREDUCE-6222.006.patch, MAPREDUCE-6222.007.patch, > MAPREDUCE-6222.008.patch, MAPREDUCE-6222.009.patch, Screen Shot 2015-05-20 at > 11.16.25 AM.png, head.jhist, historyserver_jstack.txt > > > I'm encountering an issue with the Mapreduce HistoryServer processing the > history files for large jobs. This has come up several times with for jobs > with around 6 total tasks. When the HistoryServer loads the .jhist file > from HDFS for a job of that size (which is usually around 500 Mb), the > HistoryServer's CPU usage spiked and the UI became unresponsive. After about > 10 minutes I restarted the HistoryServer and it was behaving normally again. > The cluster is running CDH 5.3 (2.5.0-cdh5.3.0). I've attached the output of > jstack from a time this was occurring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6222) Add server-side pagination to JobHistoryServer tasks page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6222: -- Component/s: jobhistoryserver > Add server-side pagination to JobHistoryServer tasks page > - > > Key: MAPREDUCE-6222 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6222 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Andrew Johnson >Assignee: Ray Chiang > Labels: BB2015-05-TBR, supportability > Attachments: JHS New Display Top.png, JHS Original Display Top.png, > MAPREDUCE-6222.001.patch, MAPREDUCE-6222.002.patch, MAPREDUCE-6222.003.patch, > MAPREDUCE-6222.005.patch, MAPREDUCE-6222.006.patch, MAPREDUCE-6222.007.patch, > MAPREDUCE-6222.008.patch, MAPREDUCE-6222.009.patch, Screen Shot 2015-05-20 at > 11.16.25 AM.png, head.jhist, historyserver_jstack.txt > > > I'm encountering an issue with the Mapreduce HistoryServer processing the > history files for large jobs. This has come up several times with for jobs > with around 6 total tasks. When the HistoryServer loads the .jhist file > from HDFS for a job of that size (which is usually around 500 Mb), the > HistoryServer's CPU usage spiked and the UI became unresponsive. After about > 10 minutes I restarted the HistoryServer and it was behaving normally again. > The cluster is running CDH 5.3 (2.5.0-cdh5.3.0). I've attached the output of > jstack from a time this was occurring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6222) Add server-side pagination to JobHistoryServer tasks page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6222: -- Affects Version/s: 2.7.0 > Add server-side pagination to JobHistoryServer tasks page > - > > Key: MAPREDUCE-6222 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6222 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Andrew Johnson >Assignee: Ray Chiang > Labels: BB2015-05-TBR, supportability > Attachments: JHS New Display Top.png, JHS Original Display Top.png, > MAPREDUCE-6222.001.patch, MAPREDUCE-6222.002.patch, MAPREDUCE-6222.003.patch, > MAPREDUCE-6222.005.patch, MAPREDUCE-6222.006.patch, MAPREDUCE-6222.007.patch, > MAPREDUCE-6222.008.patch, MAPREDUCE-6222.009.patch, Screen Shot 2015-05-20 at > 11.16.25 AM.png, head.jhist, historyserver_jstack.txt > > > I'm encountering an issue with the Mapreduce HistoryServer processing the > history files for large jobs. This has come up several times with for jobs > with around 6 total tasks. When the HistoryServer loads the .jhist file > from HDFS for a job of that size (which is usually around 500 Mb), the > HistoryServer's CPU usage spiked and the UI became unresponsive. After about > 10 minutes I restarted the HistoryServer and it was behaving normally again. > The cluster is running CDH 5.3 (2.5.0-cdh5.3.0). I've attached the output of > jstack from a time this was occurring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Labels: supportability (was: ) > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Labels: supportability (was: ) > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely on jobs with a large number of tasks (200k or more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Description: In HsTasksBlock#render(), there is a loop to create a Javascript table which slows down immensely on jobs with a large number of tasks (200k or more). (was: In HsTasksBlock#render(), there is a loop to create a Javascript table which slows down immensely.) > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely on jobs with a large number of tasks (200k or more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Attachment: MAPREDUCE-6376.002.patch Update to apply cleanly after MAPREDUCE-6350. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[
https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581140#comment-14581140
]
Hadoop QA commented on MAPREDUCE-6376:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738679/MAPREDUCE-6376.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a7a7768 |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5790/console |
This message was automatically generated.
> Fix long load times of .jhist file in JobHistoryServer
> --
>
> Key: MAPREDUCE-6376
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: jobhistoryserver
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
> Attachments: MAPREDUCE-6376.001.patch
>
>
> When you click on a Job link in the JHS Web UI, it loads the .jhist file.
> For jobs which have a large number of tasks, the load time can break UI
> responsiveness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Status: Patch Available (was: Open) Submit patch for testing > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Attachments: MAPREDUCE-6376.001.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6222) Add server-side pagination to JobHistoryServer tasks page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581134#comment-14581134 ] Ray Chiang commented on MAPREDUCE-6222: --- For 2) above, MAPREDUCE-6394 has been created. Even with MAPREDUCE-6376 and MAPREDUCE-6394, it turns out that the Javascript UI is pretty slow on jobs with a really large number of tasks (~10 seconds to sort 404k tasks, ~20 seconds to sort 751k tasks). So, this JIRA could still be desirable for some. > Add server-side pagination to JobHistoryServer tasks page > - > > Key: MAPREDUCE-6222 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6222 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Andrew Johnson >Assignee: Ray Chiang > Labels: BB2015-05-TBR > Attachments: JHS New Display Top.png, JHS Original Display Top.png, > MAPREDUCE-6222.001.patch, MAPREDUCE-6222.002.patch, MAPREDUCE-6222.003.patch, > MAPREDUCE-6222.005.patch, MAPREDUCE-6222.006.patch, MAPREDUCE-6222.007.patch, > MAPREDUCE-6222.008.patch, MAPREDUCE-6222.009.patch, Screen Shot 2015-05-20 at > 11.16.25 AM.png, head.jhist, historyserver_jstack.txt > > > I'm encountering an issue with the Mapreduce HistoryServer processing the > history files for large jobs. This has come up several times with for jobs > with around 6 total tasks. When the HistoryServer loads the .jhist file > from HDFS for a job of that size (which is usually around 500 Mb), the > HistoryServer's CPU usage spiked and the UI became unresponsive. After about > 10 minutes I restarted the HistoryServer and it was behaving normally again. > The cluster is running CDH 5.3 (2.5.0-cdh5.3.0). I've attached the output of > jstack from a time this was occurring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Summary: Speed up Task processing loop in HsTasksBlock#render() (was: Avoid copying counters in task reports for Job History Server) > Speed up Task processing loop in HsTasksBlock#render() > -- > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6394) Avoid copying counters in task reports for Job History Server
[ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6394: -- Issue Type: Sub-task (was: Improvement) Parent: MAPREDUCE-3973 > Avoid copying counters in task reports for Job History Server > - > > Key: MAPREDUCE-6394 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > > In HsTasksBlock#render(), there is a loop to create a Javascript table which > slows down immensely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6394) Avoid copying counters in task reports for Job History Server
Ray Chiang created MAPREDUCE-6394: - Summary: Avoid copying counters in task reports for Job History Server Key: MAPREDUCE-6394 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Affects Versions: 2.7.0 Reporter: Ray Chiang Assignee: Ray Chiang In HsTasksBlock#render(), there is a loop to create a Javascript table which slows down immensely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6222) Add server-side pagination to JobHistoryServer tasks page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6222: -- Issue Type: Sub-task (was: Bug) Parent: MAPREDUCE-3973 > Add server-side pagination to JobHistoryServer tasks page > - > > Key: MAPREDUCE-6222 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6222 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Andrew Johnson >Assignee: Ray Chiang > Labels: BB2015-05-TBR > Attachments: JHS New Display Top.png, JHS Original Display Top.png, > MAPREDUCE-6222.001.patch, MAPREDUCE-6222.002.patch, MAPREDUCE-6222.003.patch, > MAPREDUCE-6222.005.patch, MAPREDUCE-6222.006.patch, MAPREDUCE-6222.007.patch, > MAPREDUCE-6222.008.patch, MAPREDUCE-6222.009.patch, Screen Shot 2015-05-20 at > 11.16.25 AM.png, head.jhist, historyserver_jstack.txt > > > I'm encountering an issue with the Mapreduce HistoryServer processing the > history files for large jobs. This has come up several times with for jobs > with around 6 total tasks. When the HistoryServer loads the .jhist file > from HDFS for a job of that size (which is usually around 500 Mb), the > HistoryServer's CPU usage spiked and the UI became unresponsive. After about > 10 minutes I restarted the HistoryServer and it was behaving normally again. > The cluster is running CDH 5.3 (2.5.0-cdh5.3.0). I've attached the output of > jstack from a time this was occurring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated MAPREDUCE-6376: -- Issue Type: Sub-task (was: Bug) Parent: MAPREDUCE-3973 > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Attachments: MAPREDUCE-6376.001.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5003) AM recovery should recreate records for attempts that were incomplete
[ https://issues.apache.org/jira/browse/MAPREDUCE-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-5003: Attachment: MAPREDUCE-5003.1.patch the AM hanging issue mentioned in MAPREDUCE-4992 no longer exists in current recover implementation. Moreover, current recovery implementation for task attempts will recover incomplete task attempts to killed state. So we no longer need to remove those incomplete task attempts for recovery. > AM recovery should recreate records for attempts that were incomplete > - > > Key: MAPREDUCE-5003 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5003 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mr-am >Reporter: Jason Lowe >Assignee: Chang Li > Attachments: MAPREDUCE-5003.1.patch > > > As discussed in MAPREDUCE-4992, it would be nice if the AM recovered task > attempt entries for *all* task attempts launched by the prior app attempt > even if those task attempts did not complete. The attempts would have to be > marked as killed or something similar to indicate it is no longer running. > Having records for the task attempts enables the user to see what nodes were > associated with the attempts and potentially access their logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-5003) AM recovery should recreate records for attempts that were incomplete
[ https://issues.apache.org/jira/browse/MAPREDUCE-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li reassigned MAPREDUCE-5003: --- Assignee: Chang Li > AM recovery should recreate records for attempts that were incomplete > - > > Key: MAPREDUCE-5003 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5003 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mr-am >Reporter: Jason Lowe >Assignee: Chang Li > > As discussed in MAPREDUCE-4992, it would be nice if the AM recovered task > attempt entries for *all* task attempts launched by the prior app attempt > even if those task attempts did not complete. The attempts would have to be > marked as killed or something similar to indicate it is no longer running. > Having records for the task attempts enables the user to see what nodes were > associated with the attempts and potentially access their logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5948) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well
[
https://issues.apache.org/jira/browse/MAPREDUCE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580875#comment-14580875
]
Hadoop QA commented on MAPREDUCE-5948:
--
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 35s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 46s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests | 23m 18s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | mapreduce tests | 1m 43s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 67m 24s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738857/MAPREDUCE-5948.003.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c7729ef |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5789/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5789/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5789/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5789/console |
This message was automatically generated.
> org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record
> delimiters well
> --
>
> Key: MAPREDUCE-5948
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5948
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 0.20.2, 0.23.9, 2.2.0
> Environment: CDH3U2 Redhat linux 5.7
>Reporter: Kris Geusebroek
>Assignee: Akira AJISAKA
>Priority: Critical
> Attachments: HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch,
> HADOOP-9867.patch, MAPREDUCE-5948.002.patch, MAPREDUCE-5948.003.patch
>
>
> Having defined a recorddelimiter of multiple bytes in a new InputFileFormat
> sometimes has the effect of skipping records from the input.
> This happens when the input splits are split off just after a
> recordseparator. Starting point for the next split would be non zero and
> skipFirstLine would be true. A seek into the file is done to start - 1 and
> the text until the first recorddelimiter is ignored (due to the presumption
> that this record is already handled by the previous maptask). Since the re
> ord delimiter is multibyte the seek only got the last byte of the delimiter
> into scope and its not recognized as a full delimiter. So the text is skipped
> until the next delimiter (ignoring a full record!!)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580836#comment-14580836
]
Brahma Reddy Battula commented on MAPREDUCE-6389:
-
Thanks a lot [~devaraj.k] for review and commit!!!
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5948) org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well
[ https://issues.apache.org/jira/browse/MAPREDUCE-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-5948: - Attachment: MAPREDUCE-5948.003.patch v3 patch * Refactored the code. * Added comments for the code. > org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record > delimiters well > -- > > Key: MAPREDUCE-5948 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5948 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.20.2, 0.23.9, 2.2.0 > Environment: CDH3U2 Redhat linux 5.7 >Reporter: Kris Geusebroek >Assignee: Akira AJISAKA >Priority: Critical > Attachments: HADOOP-9867.patch, HADOOP-9867.patch, HADOOP-9867.patch, > HADOOP-9867.patch, MAPREDUCE-5948.002.patch, MAPREDUCE-5948.003.patch > > > Having defined a recorddelimiter of multiple bytes in a new InputFileFormat > sometimes has the effect of skipping records from the input. > This happens when the input splits are split off just after a > recordseparator. Starting point for the next split would be non zero and > skipFirstLine would be true. A seek into the file is done to start - 1 and > the text until the first recorddelimiter is ignored (due to the presumption > that this record is already handled by the previous maptask). Since the re > ord delimiter is multibyte the seek only got the last byte of the delimiter > into scope and its not recognized as a full delimiter. So the text is skipped > until the next delimiter (ignoring a full record!!) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580674#comment-14580674
]
Hudson commented on MAPREDUCE-6389:
---
SUCCESS: Integrated in Hadoop-trunk-Commit #8002 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8002/])
MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. Contributed by
(devaraj: rev c7729efee8727b59f2c78cd5a3ad23fa84139068)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-6389:
-
Resolution: Fixed
Fix Version/s: 2.8.0
Status: Resolved (was: Patch Available)
Thanks [~brahmareddy] for your contribution.
Committed to trunk and branch-2.
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580647#comment-14580647 ] Hudson commented on MAPREDUCE-6350: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2170 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2170/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-6389:
-
Hadoop Flags: Reviewed
+1, looks good to me, will commit it shortly.
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Trivial
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-6389:
-
Priority: Trivial (was: Major)
Target Version/s: 2.8.0
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Trivial
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580622#comment-14580622 ] Hudson commented on MAPREDUCE-6350: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #222 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/222/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580593#comment-14580593 ] Hudson commented on MAPREDUCE-6350: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #213 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/213/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580578#comment-14580578 ] Hudson commented on MAPREDUCE-6350: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2152 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2152/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/CHANGES.txt > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6389) Fix BaileyBorweinPlouffe CLI usage message
[
https://issues.apache.org/jira/browse/MAPREDUCE-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580441#comment-14580441
]
Brahma Reddy Battula commented on MAPREDUCE-6389:
-
[~xgong] kindly look into this issue..
> Fix BaileyBorweinPlouffe CLI usage message
> ---
>
> Key: MAPREDUCE-6389
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6389
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-6389.patch
>
>
> The usage message from code does not match with the command helpline option.
> *usage from command line*
> {noformat}
> hduser@canberra:~/work/software/cloudera/hadoop-2.0.0-cdh4.0.0$ hadoop jar
> src/hadoop-mapreduce-project/hadoop-mapreduce-examples/target/hadoop-mapreduce-examples-2.0.0-cdh4.0.0.jar
>
> An example program must be given as the first argument.
> Valid program names are:
> aggregatewordcount: An Aggregate based map/reduce program that counts the
> words in the input files.
> aggregatewordhist: An Aggregate based map/reduce program that computes the
> histogram of the words in the input files.
> bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact
> digits of Pi.
> dbcount: An example job that count the pageview counts from a database.
> distbbp: A map/reduce program that uses a BBP-type formula to compute exact
> bits of Pi.
> grep: A map/reduce program that counts the matches of a regex in the input.
> {noformat}
> *usage from code*
> {code}
> public int run(String[] args) throws IOException {
> if (args.length != 4) {
> System.err.println("Usage: java " + getClass().getName()
> + "");
> ToolRunner.printGenericCommandUsage(System.err);
> return -1;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6332) Provide facility to users for writting custom MergeManager implementation when custom shuffleconsumerPluggin is used
[ https://issues.apache.org/jira/browse/MAPREDUCE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-6332: - Status: Open (was: Patch Available) > Provide facility to users for writting custom MergeManager implementation > when custom shuffleconsumerPluggin is used > > > Key: MAPREDUCE-6332 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6332 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Reporter: Rohith >Assignee: Rohith > Attachments: 0001-MAPREDUCE-6332.patch, 0002-MAPREDUCE-6332.patch, > 0003-MAPREDUCE-6332.patch, 0004-MAPREDUCE-6332.patch > > > MR provides ability to the user for plugin custom ShuffleConsumerPlugin using > *mapreduce.job.reduce.shuffle.consumer.plugin.class*. When the user is > allowed to use this configuration as plugin, user also interest in > implementing his own MergeManagerImpl. > But now , user is forced to use MR provided MergeManagerImpl instead of > custom MergeManagerImpl when user is using shuffle.consumer.plugin class. > There should be well defined API's in MergeManager that can be used for any > implementation without much effort to user for custom implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580381#comment-14580381 ] Hudson commented on MAPREDUCE-6350: --- FAILURE: Integrated in Hadoop-Yarn-trunk #954 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/954/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6332) Provide facility to users for writting custom MergeManager implementation when custom shuffleconsumerPluggin is used
[
https://issues.apache.org/jira/browse/MAPREDUCE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580385#comment-14580385
]
Devaraj K commented on MAPREDUCE-6332:
--
Thanks [~rohithsharma] for the patch. Here are some of my comments,
1. InMemoryReader still uses MergeManagerImpl, can you update the references to
MergeManager.
2. I don't think it is a good idea to have static inner class in the exposing
interface. Can we move CompressAwarePath as a separate class instead of having
it in MergeManager?
3.
{code}
void closeInMemoryFile(InMemoryMapOutput mapOutput);
{code}
Here we are providing API with parameter type as InMemoryMapOutput, but
InMemoryMapOutput marked as private class and also the visibility is default
which cannot be accessed outside of the package.
4. {code:xml}
void closeOnDiskFile(CompressAwarePath compressAwarePath);
{code}
CompressAwarePath is used mostly inside the class MergeManagerImpl except one
place in OnDiskMapOutput. Can we avoid giving the argument type as
CompressAwarePath here?
5. Can we have detailed java doc for MergeManager at class level, how to
implement it for custom merge manager? And also please add detailed description
for the newly added API's in MergeManager.
6. Can we have a test to demonstrate how do we configure and use the custom
merge manager?
7. And also there are tests using MergeManagerImpl type for references, please
update them as well.
> Provide facility to users for writting custom MergeManager implementation
> when custom shuffleconsumerPluggin is used
>
>
> Key: MAPREDUCE-6332
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6332
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
>Reporter: Rohith
>Assignee: Rohith
> Attachments: 0001-MAPREDUCE-6332.patch, 0002-MAPREDUCE-6332.patch,
> 0003-MAPREDUCE-6332.patch, 0004-MAPREDUCE-6332.patch
>
>
> MR provides ability to the user for plugin custom ShuffleConsumerPlugin using
> *mapreduce.job.reduce.shuffle.consumer.plugin.class*. When the user is
> allowed to use this configuration as plugin, user also interest in
> implementing his own MergeManagerImpl.
> But now , user is forced to use MR provided MergeManagerImpl instead of
> custom MergeManagerImpl when user is using shuffle.consumer.plugin class.
> There should be well defined API's in MergeManager that can be used for any
> implementation without much effort to user for custom implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580374#comment-14580374 ] Hudson commented on MAPREDUCE-6350: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #224 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/224/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java * hadoop-mapreduce-project/CHANGES.txt > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580166#comment-14580166 ] Hudson commented on MAPREDUCE-6350: --- FAILURE: Integrated in Hadoop-trunk-Commit #8001 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8001/]) MAPREDUCE-6350. JobHistory doesn't support fully-functional search. (devaraj: rev 6785661e554114a4613b5fe7dabec9bfa80c41d9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-6350: - Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Thanks [~l201514]. Committed to trunk and branch-2. > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6350) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/MAPREDUCE-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-6350: - Hadoop Flags: Reviewed +1, will commit it shortly. > JobHistory doesn't support fully-functional search > -- > > Key: MAPREDUCE-6350 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6350 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Siqi Li >Assignee: Siqi Li >Priority: Critical > Attachments: MAPREDUCE-6350-branch-2.patch, MAPREDUCE-6350.v1.patch, > YARN-1614.v1.patch, YARN-1614.v2.patch, YARN-1614.v3.patch, > YARN-1614.v4.patch, YARN-1614.v5.patch, YARN-1614.v6.patch > > > job history server will only output the first 50 characters of the job names > in webUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
