[jira] [Commented] (MAPREDUCE-5002) AM could potentially allocate a reduce container to a map attempt
[
https://issues.apache.org/jira/browse/MAPREDUCE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594924#comment-14594924
]
Hadoop QA commented on MAPREDUCE-5002:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 9s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 48s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 6s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 4s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 48m 0s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740565/MAPREDUCE-5002.2.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c7d022b |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/console |
This message was automatically generated.
> AM could potentially allocate a reduce container to a map attempt
> -
>
> Key: MAPREDUCE-5002
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5002
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
>Affects Versions: 2.0.3-alpha, 0.23.7, 2.7.0
>Reporter: Jason Lowe
>Assignee: Chang Li
> Attachments: MAPREDUCE-5002.1.patch, MAPREDUCE-5002.2.patch,
> MAPREDUCE-5002.2.patch
>
>
> As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically
> possible for the AM to accidentally assign a reducer container to a map
> attempt if the AM doesn't find a reduce attempt actively looking for the
> container (e.g.: the RM accidentally allocated too many reducer containers).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5002) AM could potentially allocate a reduce container to a map attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-5002: Affects Version/s: 2.7.0 > AM could potentially allocate a reduce container to a map attempt > - > > Key: MAPREDUCE-5002 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5002 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.0.3-alpha, 0.23.7, 2.7.0 >Reporter: Jason Lowe >Assignee: Chang Li > Attachments: MAPREDUCE-5002.1.patch, MAPREDUCE-5002.2.patch, > MAPREDUCE-5002.2.patch > > > As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically > possible for the AM to accidentally assign a reducer container to a map > attempt if the AM doesn't find a reduce attempt actively looking for the > container (e.g.: the RM accidentally allocated too many reducer containers). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6410) Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594845#comment-14594845
]
Hadoop QA commented on MAPREDUCE-6410:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 50s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 10m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 23s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 27s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 33s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 46s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 41s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 9s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 6m 26s | Tests passed in
hadoop-mapreduce-client-hs. |
| | | 52m 47s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740836/YARN-3779.03.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c7d022b |
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5820/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5820/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5820/console |
This message was automatically generated.
> Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings
> in secure cluster
> --
>
> Key: MAPREDUCE-6410
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6410
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: mrV2, secure mode
>Reporter: Zhang Wei
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-3779.01.patch, YARN-3779.02.patch,
> YARN-3779.03.patch, log_aggr_deletion_on_refresh_error.log,
> log_aggr_deletion_on_refresh_fix.log
>
>
> {{GSSException}} is thrown everytime log aggregation deletion is attempted
> after executing bin/mapred hsadmin -refreshLogRetentionSettings in a secure
> cluster.
> The problem can be reproduced by following steps:
> 1. startup historyserver in secure cluster.
> 2. Log deletion happens as per expectation.
> 3. execute {{mapred hsadmin -refreshLogRetentionSettings}} command to refresh
> the configuration value.
> 4. All the subsequent attempts of log deletion fail with {{GSSException}}
> Following exception can be found in historyserver's log if log deletion is
> enabled.
> {noformat}
> 2015-06-04 14:14:40,070 | ERROR | Timer-3 | Error reading root log dir this
> deletion attempt is being aborted | AggregatedLogDeletionService.java:127
> java.io.IOException: Failed on local exception: java.io.IOException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]; Host Details : local host is: "vm-31/9.91.12.31";
> destination host is: "vm-33":25000;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
> at org.apache.hadoop.ipc.Client.call(Client.java:1414)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getListing(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:519)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
[jira] [Commented] (MAPREDUCE-6403) Fix typo in the usage of NNBench
[
https://issues.apache.org/jira/browse/MAPREDUCE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594829#comment-14594829
]
Hadoop QA commented on MAPREDUCE-6403:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 52s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 30s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 48s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 107m 47s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 124m 56s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestMiniMRClientCluster |
| | hadoop.mapred.TestReduceFetch |
| | hadoop.mapred.TestReduceFetchFromPartialMem |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740827/MAPREDUCE-6403-00.patch
|
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 63d40d5 |
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5819/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5819/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5819/console |
This message was automatically generated.
> Fix typo in the usage of NNBench
>
>
> Key: MAPREDUCE-6403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6403-00.patch
>
>
> Fix typo 'becnhmarks'.
> {code:title=NNBench.java}
> "\t-baseDir"This is not mandatory>\n" +
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594824#comment-14594824 ] Hudson commented on MAPREDUCE-6405: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2180 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2180/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/CHANGES.txt > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594825#comment-14594825 ] Hudson commented on MAPREDUCE-6316: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2180 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2180/]) MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) (gera: rev 63d40d528654707b3f56619a4eb20e21cdeadc7e) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestBlocks.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsAttemptsPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AttemptsPage.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestBlocks.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/yarn.dt.plugins.js > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594819#comment-14594819 ] Ray Chiang commented on MAPREDUCE-6376: --- Just a quick follow up. I wrote up a quick program to convert old .jhist files into the new binary form and loaded them into a new VM. The speedups are decent at a high # of tasks, but closer to 2x (versus the 3x at <50k tasks). 751k Mappers/10 Reducers (json) - Read Time: 163.8 seconds - Event Time: 16.6 seconds 751k Mappers/10 Reducers (binary) - Read Time: 86.7 seconds - Event Time: 17.4 seconds 404k Mappers/4 Reducers (json) - Read Time: 93 seconds - Event Time: 8.9 seconds 404k Mappers/4 Reducers (binary) - Read Time: 49.1 seconds - Event Time: 9.2 seconds I'm probably not going to get to a random event generator this week, but this gives us at least two more data points. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch, > MAPREDUCE-6376.003.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594806#comment-14594806 ] Hudson commented on MAPREDUCE-6405: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #232 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/232/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/CHANGES.txt > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594807#comment-14594807 ] Hudson commented on MAPREDUCE-6316: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #232 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/232/]) MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) (gera: rev 63d40d528654707b3f56619a4eb20e21cdeadc7e) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsAttemptsPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AttemptsPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestBlocks.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestBlocks.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/yarn.dt.plugins.js > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (MAPREDUCE-6410) Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena moved YARN-3779 to MAPREDUCE-6410:
---
Tags: jobhistory
Target Version/s: 2.7.1 (was: 2.7.1)
Affects Version/s: (was: 2.7.0)
2.7.0
Key: MAPREDUCE-6410 (was: YARN-3779)
Project: Hadoop Map/Reduce (was: Hadoop YARN)
> Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings
> in secure cluster
> --
>
> Key: MAPREDUCE-6410
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6410
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: mrV2, secure mode
>Reporter: Zhang Wei
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-3779.01.patch, YARN-3779.02.patch,
> YARN-3779.03.patch, log_aggr_deletion_on_refresh_error.log,
> log_aggr_deletion_on_refresh_fix.log
>
>
> {{GSSException}} is thrown everytime log aggregation deletion is attempted
> after executing bin/mapred hsadmin -refreshLogRetentionSettings in a secure
> cluster.
> The problem can be reproduced by following steps:
> 1. startup historyserver in secure cluster.
> 2. Log deletion happens as per expectation.
> 3. execute {{mapred hsadmin -refreshLogRetentionSettings}} command to refresh
> the configuration value.
> 4. All the subsequent attempts of log deletion fail with {{GSSException}}
> Following exception can be found in historyserver's log if log deletion is
> enabled.
> {noformat}
> 2015-06-04 14:14:40,070 | ERROR | Timer-3 | Error reading root log dir this
> deletion attempt is being aborted | AggregatedLogDeletionService.java:127
> java.io.IOException: Failed on local exception: java.io.IOException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]; Host Details : local host is: "vm-31/9.91.12.31";
> destination host is: "vm-33":25000;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
> at org.apache.hadoop.ipc.Client.call(Client.java:1414)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getListing(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:519)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy10.getListing(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1767)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1750)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:691)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:102)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:753)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:749)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:749)
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogDeletionService$LogDeletionTask.run(AggregatedLogDeletionService.java:68)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
> Caused by: java.io.IOException: javax.security.sasl.SaslException: GSS
> initiate failed [Caused by GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)]
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:677)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1641)
> at
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:640)
> at
>
[jira] [Commented] (MAPREDUCE-6410) Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594799#comment-14594799
]
Varun Saxena commented on MAPREDUCE-6410:
-
Moved this to MAPREDUCE as code change is in MAPREDUCE code
> Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings
> in secure cluster
> --
>
> Key: MAPREDUCE-6410
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6410
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: mrV2, secure mode
>Reporter: Zhang Wei
>Assignee: Varun Saxena
>Priority: Critical
> Attachments: YARN-3779.01.patch, YARN-3779.02.patch,
> YARN-3779.03.patch, log_aggr_deletion_on_refresh_error.log,
> log_aggr_deletion_on_refresh_fix.log
>
>
> {{GSSException}} is thrown everytime log aggregation deletion is attempted
> after executing bin/mapred hsadmin -refreshLogRetentionSettings in a secure
> cluster.
> The problem can be reproduced by following steps:
> 1. startup historyserver in secure cluster.
> 2. Log deletion happens as per expectation.
> 3. execute {{mapred hsadmin -refreshLogRetentionSettings}} command to refresh
> the configuration value.
> 4. All the subsequent attempts of log deletion fail with {{GSSException}}
> Following exception can be found in historyserver's log if log deletion is
> enabled.
> {noformat}
> 2015-06-04 14:14:40,070 | ERROR | Timer-3 | Error reading root log dir this
> deletion attempt is being aborted | AggregatedLogDeletionService.java:127
> java.io.IOException: Failed on local exception: java.io.IOException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]; Host Details : local host is: "vm-31/9.91.12.31";
> destination host is: "vm-33":25000;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
> at org.apache.hadoop.ipc.Client.call(Client.java:1414)
> at org.apache.hadoop.ipc.Client.call(Client.java:1363)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getListing(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:519)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy10.getListing(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1767)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1750)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:691)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:102)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:753)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:749)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:749)
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogDeletionService$LogDeletionTask.run(AggregatedLogDeletionService.java:68)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
> Caused by: java.io.IOException: javax.security.sasl.SaslException: GSS
> initiate failed [Caused by GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)]
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:677)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1641)
> at
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:640)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367)
> a
[jira] [Commented] (MAPREDUCE-6305) AM/Task log page should be able to link back to the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594796#comment-14594796 ] Hudson commented on MAPREDUCE-6305: --- FAILURE: Integrated in Hadoop-trunk-Commit #8042 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8042/]) MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) (gera: rev 055cd5a9a11bd3e544a8a67c6c44b53efcb3a9ed) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsLogsPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsSingleCounterPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsCountersPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsController.java * hadoop-mapreduce-project/CHANGES.txt > AM/Task log page should be able to link back to the job > --- > > Key: MAPREDUCE-6305 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6305 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6305.v1.patch, MAPREDUCE-6305.v2.patch, > MAPREDUCE-6305.v3.patch, MAPREDUCE-6305.v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6305) AM/Task log page should be able to link back to the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-6305: - Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks [~l201514] for the contribution. Committed to trunk and branch-2 > AM/Task log page should be able to link back to the job > --- > > Key: MAPREDUCE-6305 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6305 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6305.v1.patch, MAPREDUCE-6305.v2.patch, > MAPREDUCE-6305.v3.patch, MAPREDUCE-6305.v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6305) AM/Task log page should be able to link back to the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594750#comment-14594750 ] Gera Shegalov commented on MAPREDUCE-6305: -- +1 for v4 . Checkstyle warning is false. > AM/Task log page should be able to link back to the job > --- > > Key: MAPREDUCE-6305 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6305 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6305.v1.patch, MAPREDUCE-6305.v2.patch, > MAPREDUCE-6305.v3.patch, MAPREDUCE-6305.v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594744#comment-14594744 ] Hudson commented on MAPREDUCE-6316: --- FAILURE: Integrated in Hadoop-trunk-Commit #8041 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8041/]) MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) (gera: rev 63d40d528654707b3f56619a4eb20e21cdeadc7e) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestBlocks.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsAttemptsPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestBlocks.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AttemptsPage.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/yarn.dt.plugins.js > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594740#comment-14594740 ] Hudson commented on MAPREDUCE-6405: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #223 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/223/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java * hadoop-mapreduce-project/CHANGES.txt > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-6316: - Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks [~l201514] for the contribution! Committed to trunk and branch-2. > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Fix For: 2.8.0 > > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594729#comment-14594729 ] Gera Shegalov commented on MAPREDUCE-6316: -- TestBlocks failure was expected and, succeeds now that MAPREDUCE-6405 is in. Checkstyle warning is out of scope for this JIRA > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594722#comment-14594722 ] Hudson commented on MAPREDUCE-6405: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #2162 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2162/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594683#comment-14594683 ] Hudson commented on MAPREDUCE-6405: --- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #234 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/234/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594675#comment-14594675 ] Hudson commented on MAPREDUCE-6405: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #964 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/964/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6406) Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT to match mapred-default.xml
[
https://issues.apache.org/jira/browse/MAPREDUCE-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594610#comment-14594610
]
Hadoop QA commented on MAPREDUCE-6406:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 2s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 48s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 48s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 23s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 36s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 39m 42s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740460/MAPREDUCE-6406.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 20c03c9 |
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5818/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5818/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5818/console |
This message was automatically generated.
> Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT to
> match mapred-default.xml
>
>
> Key: MAPREDUCE-6406
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6406
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 2.7.0
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Minor
> Labels: newbie, supportability
> Attachments: MAPREDUCE-6406.001.patch
>
>
> MAPREDUCE-6336 updated the default version for the property
> mapreduce.fileoutputcommitter.algorithm.version to 2. Should the
> FileOutputCommitter class default be updated to match?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6403) Fix typo in the usage of NNBench
[
https://issues.apache.org/jira/browse/MAPREDUCE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagadesh Kiran N updated MAPREDUCE-6403:
Status: Patch Available (was: Open)
> Fix typo in the usage of NNBench
>
>
> Key: MAPREDUCE-6403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6403-00.patch
>
>
> Fix typo 'becnhmarks'.
> {code:title=NNBench.java}
> "\t-baseDir"This is not mandatory>\n" +
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594548#comment-14594548 ] Hudson commented on MAPREDUCE-6405: --- FAILURE: Integrated in Hadoop-trunk-Commit #8040 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8040/]) MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by (devaraj: rev 311a4179cc53a2754d1185b1cb10e6adb1b95d3d) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6405.002.patch, MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6408) Queue name and user name should be printed on the job page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594541#comment-14594541
]
Hadoop QA commented on MAPREDUCE-6408:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 9s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 46s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 1
new checkstyle issues (total was 64, now 65). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 7s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 8s | Tests failed in
hadoop-mapreduce-client-app. |
| | | 47m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740750/MAPREDUCE-6408.v1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 20c03c9 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5817/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5817/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5817/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5817/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5817/console |
This message was automatically generated.
> Queue name and user name should be printed on the job page
> --
>
> Key: MAPREDUCE-6408
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6408
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: MAPREDUCE-6408.v1.patch
>
>
> Sometimes, users just give us a job link, and from there we don't which user
> submits this job and which queue was used. Currently we have go through the
> confs to find that out.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6409) NM restarts could lead to app failures
[
https://issues.apache.org/jira/browse/MAPREDUCE-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594520#comment-14594520
]
Hadoop QA commented on MAPREDUCE-6409:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 51s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 45s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 53s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 4
new checkstyle issues (total was 261, now 264). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 7s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 12s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 47m 0s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740777/MAPREDUCE-6409.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 20c03c9 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5816/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5816/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5816/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5816/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5816/console |
This message was automatically generated.
> NM restarts could lead to app failures
> --
>
> Key: MAPREDUCE-6409
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6409
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Karthik Kambatla
>Assignee: Robert Kanter
>Priority: Critical
> Attachments: MAPREDUCE-6409.001.patch
>
>
> Consider the following scenario:
> 1. RM assigns a container on node N to an app A.
> 2. Node N is restarted
> 3. A tries to launch container on node N.
> 3 could lead to an NMNotYetReadyException depending on whether NM N has
> registered with the RM. In MR, this is considered a task attempt failure. A
> few of these could lead to a task/job failure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6403) Fix typo in the usage of NNBench
[
https://issues.apache.org/jira/browse/MAPREDUCE-6403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagadesh Kiran N updated MAPREDUCE-6403:
Attachment: MAPREDUCE-6403-00.patch
Thanks [~ajisakaa] for reporting the same, I have fixed the same please find
the attached patch ,review the same
> Fix typo in the usage of NNBench
>
>
> Key: MAPREDUCE-6403
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6403
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
>Priority: Trivial
> Labels: newbie
> Attachments: MAPREDUCE-6403-00.patch
>
>
> Fix typo 'becnhmarks'.
> {code:title=NNBench.java}
> "\t-baseDir"This is not mandatory>\n" +
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6407) Migrate MAPREDUCE native build to new CMake framework
[ https://issues.apache.org/jira/browse/MAPREDUCE-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Burlison updated MAPREDUCE-6407: - Attachment: MAPREDUCE-6407.001.patch Patch to migrate MAPREDUCE over to the new CMake infrastructure. Requires HADOOP-12036 > Migrate MAPREDUCE native build to new CMake framework > - > > Key: MAPREDUCE-6407 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6407 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: build >Affects Versions: 2.7.0 >Reporter: Alan Burlison >Assignee: Alan Burlison > Attachments: MAPREDUCE-6407.001.patch > > > As per HADOOP-12036, the CMake infrastructure should be refactored and made > common across all Hadoop components. This bug covers the migration of > MAPREDUCE to the new CMake infrastructure. This change will also add support > for building MAPREDUCE Native components on Solaris. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6390) Improve Solaris support in Hadoop Map/Reduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Burlison updated MAPREDUCE-6390: - Attachment: (was: MAPREDUCE-6390.001.patch) > Improve Solaris support in Hadoop Map/Reduce > > > Key: MAPREDUCE-6390 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6390 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: build > Environment: Solaris x86, Solaris sparc >Reporter: Alan Burlison >Assignee: Alan Burlison > > At present the Hadoop Map/Reduce native components aren't fully supported on > Solaris primarily due to differences between Linux and Solaris. This > top-level task will be used to group together both existing and new issues > related to this work. A second goal is to improve Hadoop Map/Reduce > performance on Solaris wherever possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6390) Improve Solaris support in Hadoop Map/Reduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Burlison updated MAPREDUCE-6390: - Attachment: MAPREDUCE-6390.001.patch Patch to migrate Map/Reduce over to the new CMake infrastructure. Requires HADOOP-12036 > Improve Solaris support in Hadoop Map/Reduce > > > Key: MAPREDUCE-6390 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6390 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: build > Environment: Solaris x86, Solaris sparc >Reporter: Alan Burlison >Assignee: Alan Burlison > Attachments: MAPREDUCE-6390.001.patch > > > At present the Hadoop Map/Reduce native components aren't fully supported on > Solaris primarily due to differences between Linux and Solaris. This > top-level task will be used to group together both existing and new issues > related to this work. A second goal is to improve Hadoop Map/Reduce > performance on Solaris wherever possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
